diff --git a/CHANGELOG.md b/CHANGELOG.md index a5bc7a8c..1ab925bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,9 @@ - **LPCM** (BD and DVD) is decoded to interleaved big-endian PCM in WAVE_FORMAT_EXTENSIBLE channel order (BD pad channel removed, LFE/surround remapped, DVD 20/24-bit groups unpacked) at the source depth: 16-bit stays 16-bit, 20- and 24-bit sources output 24-bit; MKV `BitDepth` matches, and M2TS/MPG output keeps the depth. BD 20-bit packets are kept (24-bit container); reserved header codes are dropped. On M2TS output LPCM is repacked to the BD layout; LPCM BD cannot carry (e.g. 44.1 kHz) is dropped and reported. - **M2TS:** reordered video (H.264, HEVC, MPEG-2, VC-1) now carries a DTS (H.222.0 2.7.5); PTS round to the nearest 90 kHz tick (a PTS can be 1 tick later than 1.7.7), the first frame lands one second above zero, PCR never decreases with B-frames; output always declares `bd-ts` in its FMKV header. PGS `.sup` ticks use the same rounding. - **MKV:** DTS, DTS-HD HRA and DTS-HD MA are all `A_DTS`; streams with no Matroska CodecID (text subtitles, unknown codecs) are omitted rather than declared under another codec; SDR BT.2020 video is tagged with the BT.2020 transfer (14), not PQ; the timeline origin is the earliest sample of any selected track (audio and subtitles before the first video keyframe are kept at their true offsets; video starts at its first keyframe), and Blu-ray clip IN applies only to BD-TS sources, whose frames before the IN are dropped and counted; VobSub cues without a stop command end at the next subtitle (10 s cap) instead of 30 s. **MP4:** a track that starts after the earliest sample keeps its offset through an empty edit, and video before its first keyframe is not stored. +- **One decryption stage for every input:** `mpg://`, `m2ts://`, `mkv://`, `mp4://`, `network://` and `stdio://` pass the same content-detected stage as disc/`iso://`/`dir://`. CSS packs are cracked and descrambled, AACS units are decrypted with `InputOptions.keys`, and clear containers pass untouched. An encrypted loose `.m2ts` is decrypted (keys proven on arrival) or refused with E7022 before any output instead of being muxed as ciphertext; `--raw` passes ciphertext. A clear `.m2ts` whose CPI bits were left set still opens, also when a unit has damaged packets (half of them synced suffices). Encrypted content under a clear container's scheme is refused (E7023/E7022). A file whose start is zero-filled is judged from its first written sector; a file judged clear still refuses a scrambled pack (E7023). On `m2ts://` an unflagged unit with no TS structure is blanked and counted in `errors()`. +- **One CSS pack test for disc and file:** `css::is_scrambled_pack` (so the crack scan, `descramble_sector` and `descramble_region`) reads the scramble bits from the pack's first PES header: past pack stuffing and any system header, map, padding or nav packet, and only from MPEG-2 PES flags. Every DVD-Video pack is judged as before; a stuffed, map-first or MPEG-1-PES pack whose byte 0x14 merely looks flagged is no longer descrambled on the disc path, and a stuffed scrambled pack is descrambled by its own flags. +- **A flagged partial AACS unit that ends the source** (a truncated copy) is blanked and counted as damage instead of failing the read. - **Decrypted BD-TS units have the Copy_permission_indicator cleared** (top 2 bits of each TP_extra_header), so decrypted output differs from 1.7.7 in those bits. HD DVD and units left as ciphertext are untouched. - **`json://`** audio `sample_rate` is a number in Hz (`null` when unknown), not a string such as `"48kHz"`; a `sample_rates` array lists every rate a stream carries. - **FMKV (`network://`, `stdio://`) header version 2** is used when a track carries decoder timing (Opus CodecDelay/SeekPreRoll) and adds per-frame DiscardPadding; streams without timing stay version 1. Headers now also carry display aspect, CICP, stream purpose/qualifier, chapters and content format, and PIDs actually written by M2TS. A header declaring more than 256 streams is refused with E9008. An older receiver omits the MKV Channels element for the new channel-layout strings; keep both ends on one version. @@ -32,15 +35,16 @@ ### Added -- `mpg://` as a source and a destination: a DVD-Video / MPEG-2 program stream file reads through the DVD sector pipeline (a scrambled `.vob` is cracked, `--raw` keeps the ciphertext; a clear MPEG-1 or stuffed MPEG-2 stream is no longer read as scrambled, and a scrambled pack with no key fails E7023). `MpgSink` writes an H.222.0 program stream with P-STD pack timing, system header and PSM; `Error::MpgNoVideoTrack` (E9074), `Error::MpgUnpacketized` (E9075). `FileSectorSource::open_padded` zero-pads a partial tail sector. -- `keys` module: `ResolvedKeySet::resolve(disc, scope, sources, opts)` (`KeyScope::{None, Titles, WholeDisc}`, `ResolveKeysOptions`, `KeyResolution`) asks the key sources once before any output and proves each held key against the disc's ciphertext per stream file; keys and the Volume ID stay in memory (`Debug` redacts). Also `title_reader`, `whole_disc_reader`, `decrypt_status`, `check_decryptable`, `KeySetStatus`, `ForensicState`, `DecryptStatus`, `ProofCache`. A piece that cannot be proven up front is proven on first read; a unit no held key opens stops with E7022 (title) or E7032 (image). `ResolveCtx::halt()` / `progress()` give sources the Stop token; key-service retries back off 1 s to 8 s. `MediaKeyVariantError::code()` returns E7100-E7108. +- `mpg://` as a source and a destination: a DVD-Video / MPEG-2 program stream file reads through the decryption stage and the DVD sector pipeline (a scrambled `.vob` is cracked once, `--raw` keeps the ciphertext; a clear MPEG-1 or stuffed MPEG-2 stream is no longer read as scrambled, and a scrambled pack with no key fails E7023). `MpgSink` writes an H.222.0 program stream with P-STD pack timing, system header and PSM; `Error::MpgNoVideoTrack` (E9074), `Error::MpgUnpacketized` (E9075). `FileSectorSource::open_padded` zero-pads a partial tail sector. +- `keys` module: `ResolvedKeySet::resolve(disc, scope, sources, opts)` (`KeyScope::{None, Titles, WholeDisc}`, `ResolveKeysOptions`, `KeyResolution`) asks the key sources once before any output and proves each held key against the disc's ciphertext per stream file; keys and the Volume ID stay in memory (`Debug` redacts). Also `title_reader`, `whole_disc_reader`, `decrypt_status`, `check_decryptable`, `KeySetStatus`, `ForensicState`, `DecryptStatus`, `ProofCache`. A piece that cannot be proven up front is proven on first read; a unit no held key opens stops a title rip with E7022; in an image or folder it is blanked, warned and counted. `ResolveCtx::halt()` / `progress()` give sources the Stop token; key-service retries back off 1 s to 8 s. `MediaKeyVariantError::code()` returns E7100-E7108. +- `disc_root_of(path)`: the AACS disc folder a loose Blu-ray clip sits in (`/BDMV/STREAM/x.m2ts` with `/AACS`), so a caller can resolve the clip's keys on `dir://`. - `mux::fit_report` / `FitReport` / `SkipReason` (`Mp2Extension`, `NoStreamId`, `UnmappableSubtitle`): a container-neutral pre-mux plan of which streams a destination carries. `mp4_fit_report` is unchanged. - `DiscSession::{open_with, scan_with, finish, attach_progress, token, from_drive, source_mut}` and `session::Finish::{Release, Unlock, Eject}`: a session runs under the caller's Stop token and ends on the handle it holds. `Drive::{attach, detach, attach_progress, halt, token}`: every CDB is checked against the op token. - `halt`: waits re-check every `WAIT_SLICE` (20 ms); `Progress`, `StallTimer`, `join_within`, `spawn_drive_holder`. `halt` is available with only the `scsi` feature. - `NetworkStream::listen_with_halt` / `accept_from_with_halt`; `MuxEvents::on_flush_progress(bytes_durable, bytes_total)`; `io::{ArtifactLock, durable_sync_file, FlushProgress}` (`.lock` sidecar; E9073 `TimedOut` after 30 s with no change); `Pipeline::{set_op_token, progress}`, `Sink::close_stopped` (a consumer closing after Stop keeps `*.partial`); `ISO_MUX_BATCH_SECTORS` is public. - MPEG-2 multichannel extension audio (DVD `0xD0|n`) is a dependent track (`AudioStream::is_mp2_extension`) carried in the IR and over FMKV; sinks with no mapping list it in `MuxOutcome::undelivered_streams`, `mp4://` skips it (`Mp4SkipReason::Mp2Extension`). MPEG-1/2 audio (PES `0xC0|n`) is demuxed on DVD. - Errors: E9085 `MuxBatchSectorsZero`, E6020 `ImageEndsBeforeRead`, E6021 `BusStreamUnmapped`, E6022 `ImageScoped`, E6023 XPL too large, E7031 `AacsKeyFileUnreadable`, E7032 `WholeDiscKeyMissing`, E7033 `AacsNoUsableHostCert`, E7034 `AacsVidNeedsDisc`, E7100-E7108 MKB variant faults, E9071 `StreamClosed`, E9072 `StreamHeaderWritten`, E9073 `TimedOut { op }`, E9074, E9075, and the remux/preflight set E9077 `RemuxVerifyFailed { kind, path }` (`E9077: `, or `runtime-mismatch /`), E9078 `MuxIncomplete`, E9079 `RemuxStagingInvalid`, E9080 `StagedCopySizeMismatch`, E9081 `WorkerLost { op }`, E9082 `MultipassRequiresRaw`, E9083 `StreamLanguageUnknown`, E9084 `RemuxTargetExists`. -- `whole_disc::{whole_disc_reader, WholeDiscReader}`: the decrypting reader for whole-disc / image to ISO copies; it keys every stream file no title plays (probed at its first unit and up to 32 across it) and refuses with E7032 before any output when ciphertext no held key opens remains. `ScanOptions::raw_copy` scans on past an unreadable `Unit_Key_RO.inf` (E7031 recorded, every key refused). +- `whole_disc::{whole_disc_reader, WholeDiscReader}`: the decrypting reader for whole-disc / image to ISO copies; it keys every stream file no title plays (probed at its first unit and up to 32 across it) and blanks, warns and counts the units of a stream file no held key opens (E7032 only when no file opens at all, a source faults, or no key is held). `ScanOptions::raw_copy` is kept but has no effect: an unreadable `Unit_Key_RO.inf` never fails the scan. - `sector::bus_removal::{UnmappedStreamFile, ensure_image_debussable}`, `BusMap::unmapped`, `SectorSource::unmapped_stream_files`, `Disc::{mkv_staging_ranges, stream_content_ranges, inputs_with_samples}`, `DecryptingSectorSource::{blanked_units, clear_unit_base}`, `DiscPresence` / `disc_presence(path)`, `Playlist::duration_ticks`. - `probe_mkv` / `probe_mkv_with_cues` read a Matroska file's Info and Tracks (and last Cues time) without reading a cluster; `parse_freemkv_version`. - `mkv://` read-back carries display shape (aspect, PixelCrop), colour, frame rate, `LanguageBCP47` and chapters and undoes content encodings (header stripping, zlib); `mkv://` writes `MaxBlockAdditionID` for MVC 3D. `Mp4Reader` reports colour (CICP, HDR10, HLG, Dolby Vision), frame rate, duration, QuickTime v1/v2 and ISO v1 audio entries, `dac3`/`dec3` layouts and DTS `ddts` rate. @@ -50,10 +54,10 @@ ### Changed -- **AACS damage policy (E7013):** a flagged AACS unit no key opens (no TS sync, zero-filled or garbled head, lone FMTS verify failure, off-grid) is zero-filled, warned and counted on every path (ISO mux, live mux, extract, whole-disc reader) instead of failing with E7013, with an end-of-rip "N damaged AACS units blanked" log. A wrong key still fails: E7013 at resolve (or when two or more FMTS units fail under one key with none verifying), E7022 / E7032 on arrival. -- A BD-TS mux with no AACS key set fails E7022 on the first AACS-flagged unit instead of writing ciphertext; an AACS HD DVD session with no set fails `NoDiscKey`. A key-set stop (E7022, E7032) is never shrunk, recovered or skipped. +- **AACS damage policy (E7013):** a flagged AACS unit no key opens (no TS sync, zero-filled or garbled head, lone FMTS verify failure, off-grid) is zero-filled, warned and counted on every path (ISO mux, live mux, extract, whole-disc reader) instead of failing with E7013, with an end-of-rip "N damaged AACS units blanked" log. A wrong key still fails: E7013 at resolve (or when two or more FMTS units fail under one key with none verifying), E7022 on arrival in a title rip (an image or folder blanks the unit instead). +- A BD-TS mux with no AACS key set fails E7022 on the first AACS-flagged unit instead of writing ciphertext; an AACS HD DVD session with no set fails `NoDiscKey`. A key-set stop (E7022) is never shrunk, recovered or skipped. - A Stop during a mux is always `Halted` / `completed = false`; a stopped read is a stop, not a skipped sector. A read error repeats on every later `read()` of the highway stream. A halted `PrefetchedSectorSource` close is `Err(Halted)`, not EOF. -- `Disc::scan` follows the standard AACS order (UDF, `Unit_Key_RO.inf`, certificate, MKB by plain reads, then the handshake, only on a disc with an AACS directory), no longer reads the MKB from the drive, and passes `None` to `KeySource::host_certs`. A live AACS disc whose `Unit_Key_RO.inf` is missing or unreadable fails E7031; image and folder scans record it and continue. A handshake transport fault fails with `Drive::init`'s error and a Stop returns `Halted`. `DiscSession::scan` returns the key sources on failure so a retry keeps them. +- `Disc::scan` follows the standard AACS order (UDF, `Unit_Key_RO.inf`, certificate, MKB by plain reads, then the handshake, only on a disc with an AACS directory), no longer reads the MKB from the drive, and passes `None` to `KeySource::host_certs`. A live AACS disc whose `Unit_Key_RO.inf` is missing or unreadable is warned and scanned on, with E7031 recorded in `Disc::aacs_error` and every key refused (titles that need keys then fail E7022 on the normal key path); image and folder scans record it the same way. A handshake transport fault fails with `Drive::init`'s error and a Stop returns `Halted`. `DiscSession::scan` returns the key sources on failure so a retry keeps them. - Host-key (cert-route) bus-encrypted discs: every `/BDMV/STREAM` file is de-bussed (BEE flag and per-unit CPI honoured; CPI 0 passes through). A stream file whose extents cannot be read is re-read with FUA (bounded, 5 s pause a Stop ends), then recorded; `ensure_image_debussable` refuses a whole-disc image with E6021 naming each file and cause, `extract_tree` counts such files lost whole, and `stream_content_ranges` fails E6021. - `DecryptingSectorSource` has no default AACS unit base: a content read before `set_unit_base` fails `DecryptFailed`, as does a flagged BD-TS unit lacking TS sync at byte 4. - `read_mkb_from_drive` returns `AacsKeyRead` for a short, over-long or header-only pack; record 0x07 is no longer read as a cvalue table; a content certificate type other than 0x00 / 0x10 no longer parses. AACS 2.1 tries every device key before declaring the MKB underivable; an HD DVD VTKF is parsed by its magic. A CSS crack that read no sector is `Unreadable`, never `Unencrypted`. @@ -62,12 +66,13 @@ - `list_drives` on Linux without sysfs keeps an sg node unless INQUIRY says not optical, or open fails with ENOENT/ENXIO/ENODEV/EACCES/EPERM. Dropping a `Drive` or transport unlocks the tray only if it locked it. `DriveId::from_drive` reads firmware date and serial from GET CONFIGURATION features and returns `Halted` on Stop. - macOS: opening a drive is cancellable, each CDB uses the caller's timeout (`timeout_ms` 0 is 60 s), and `find_drive` releases an empty drive before the next candidate. The shim is built against `MACOSX_DEPLOYMENT_TARGET` (11.0 aarch64, 10.12 x86_64). - `Pipeline::finish_with_halt` waits on consumer progress (the 600 s join is a stall window). `sync_all` on a `WritebackFile` fails E9056 only after 60 s without flush progress and returns `Halted` on cancel; `WritebackFile` flushes in chunks (64 MiB, down to 4 MiB). Linux `sync_file_range` errors are latched; `write_image` fsyncs the parent directory; reads past EOF of an image return `ImageTruncated`. +- `network://` send and receive accept LAN targets: loopback, private (RFC 1918), link-local, ULA and CGNAT addresses connect. Only unspecified, multicast, broadcast and reserved addresses are refused (E9022); a caller that takes targets from untrusted input must vet them itself. - `network://`: send connects to each vetted address in turn (10 s timeout); both ends enable TCP keepalive (a rejected option only warns); receive no longer blocks on a vanished sender or a Stop. `stdio://` input fails E9008 on a non-FMKV or damaged header once bytes were consumed. - Container headers (MKV, fragmented MP4, M2TS, network) wait (5 s of source PTS, 2048 frames or half the header buffer) for the AAC AudioSpecificConfig and BD LPCM layout byte; a later config is back-patched into a seekable MKV. `MuxEvents::on_output_opened` lists only the streams the sink writes. `DiscStream::new` fails on `batch_sectors == 0`. - DVD: titles map through `VTS_PTT_SRPT` (TTN 0 rejected), one angle counted, an unusable IFO falls back to its BUP; MP2 `Channels` follows CRC-verified Layer II frames; the live path shares the file-backed path's access-unit assembly and H.264 second-field merge; unmapped PS streams warn once. - Codec framing: ADTS and MPEG audio resync only on a header confirmed by the next, so a few KB of garbage no longer poisons a track; MPEG-2 field order follows picture_structure; H.264 1080i I/P field pairs are keyframes; HEVC arms CRA-to-BLA on gaps. TS demux skips duplicate packets, resyncs after a byte slip, honours transport_error_indicator. - `mp4://` DTS follows ETSI TS 102 114 Annex E (`ddts` carries the stream's maximum rate; 96 kHz DTS-HD MA reads back as 96 kHz); video CTS is rounded to the nearest tick; `Mp4Reader` labels MPEG-audio `mp4a` as Mp3/Mp2. -- UDF: a corrupt directory FID returns `DiscRead` for the root and lists a subdirectory empty with a warning; AD chains are validated (`UdfAdChainTooLong`); fragmented Metadata Files are mapped. Directory images report `DirImagePlacement` / `DirImageFanout`. MPLS: truncated play-item list is `MplsParse`; CLPI needs 60 bytes. bdnav / dvdnav VM follows libbluray / libdvdnav. +- UDF: a corrupt directory FID is warned and stops that directory listing (the root keeps the entries before it, a subdirectory lists empty); AD chains are validated (`UdfAdChainTooLong`); fragmented Metadata Files are mapped. Directory images report `DirImagePlacement` / `DirImageFanout`. MPLS: a truncated play-item list keeps the items before the damage (warned; `MplsParse` only when none remain); a CLPI cut short of its 60-byte head still parses with an unknown (0) packet count, sized from the clip's extents. bdnav / dvdnav VM follows libbluray / libdvdnav. - Labels: BD-J detection reads each jar once under a shared inflate budget and abstains on unreadable evidence; `clpi_audit` keys streams by (clip, PID). - HD DVD: a title naming an absent or truncated clip is dropped; the `X!` AACS directory gates identify and scan and is stripped from extractions; oversized XPL is E6023. - `Error::DirNameCollision` and `StreamLanguageUnknown` display with control characters escaped. @@ -86,6 +91,10 @@ - UDF stale cache window after a failed read; Stop is honoured in image scan, the FE re-read pause and READ CAPACITY retries; a `/dev/null` mapfile uses a private per-process temp dir; `extract` sanitises reserved file names and checks extent LBA arithmetic. - `GET CONFIGURATION` honours Data/Additional Length and feature code; MODE SELECT list length is MDL+2; Windows SPTI timeout/alignment/bounce-buffer fixes; macOS DiskArbitration claim leak. - Labels: Paramount "feature" matching, Criterion duplicate mappings, B/T language match, bdmt setNumber. Diagnostic opening capture no longer aliases track index 256 and above. +- `m2ts://` output now carries a PAT and an HDMV-registered PMT (PID 0x0100, repeated every 100 ms) declaring each track's stream_type, so FFmpeg and players identify every track: BD LPCM (0x80) decodes as `pcm_bluray` at the source depth instead of "unknown"/`mp3`. AAC is re-framed as ADTS from its AudioSpecificConfig (stream_type 0x0F; a config ADTS cannot signal is dropped and reported), MP2/MP3 are 0x03, and both use the MPEG-audio PES stream_id 0xC0. +- `m2ts://` output now carries a program clock: PCR packets on PID 0x1001 (the PMT's PCR_PID, BD-ROM convention) at most 100 ms apart, and every packet's TP_extra_header arrival time stamp on the same 27 MHz clock, at most 1 s ahead of its PES's DTS (it was 0). A timeline jump of more than 10 s restarts the clock with the discontinuity_indicator; PID 0x1001 is never given to a stream. +- `m2ts://` output keeps every raw_data_block of an AAC access unit taken from a multi-block ADTS frame: the ADTS header declares the block count (from the frame's duration) instead of one, which made decoders play only the first 1024 samples. +- Loose `m2ts://` input keeps AAC, MP2 and MP3 that other m2ts muxers declare as PES private data (stream_type 0x06): an MPEG-audio stream_id plus the ES sync names the codec. `mp4://` carries AAC, MP2 and MP3 (`mp4a` + `esds`) instead of skipping them. ## [1.7.7] — 2026-09-26 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 74e123d6..2457e2ad 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,6 +16,17 @@ cargo build cargo test ``` +### Parity goldens + +`parity_*` tests pin what every input-to-output path produces today (output hashes, loss +counters, refusal codes) in `tests/goldens/*.golden`. A change that moves one must say why. +Re-bless on purpose, then review the diff: + +```bash +FREEMKV_BLESS_GOLDENS=1 cargo test --lib parity_ +git diff tests/goldens +``` + ## Code style - Follow Rust conventions (`cargo fmt`, `cargo clippy`) diff --git a/src/clpi.rs b/src/clpi.rs index d464b40f..3a185782 100644 --- a/src/clpi.rs +++ b/src/clpi.rs @@ -34,14 +34,11 @@ pub(crate) struct ClpiStream { pub language: String, } -/// Parse a CLPI file from raw bytes. +/// Parse a CLPI file from raw bytes. A file too short to hold the packet count still +/// parses (count 0); only a bad magic or a head under 16 bytes errors. pub fn parse(data: &[u8]) -> Result { - // 60 bytes: the ClipInfo section through source_packet_count at [56..60]. - if data.len() < 60 { - return Err(Error::ClpiParse); - } - - if &data[0..4] != b"HDMV" { + // 16 bytes reach the section offsets; below that, nothing usable. + if data.len() < 16 || &data[0..4] != b"HDMV" { return Err(Error::ClpiParse); } @@ -50,7 +47,10 @@ pub fn parse(data: &[u8]) -> Result { // ClipInfo section at offset 40 // source_packet_count at offset 40 + 4(len) + 2(reserved) + 1(stream_type) + 1(app_type) + 4(reserved) + 4(ts_rate) - let source_packet_count = u32::from_be_bytes([data[56], data[57], data[58], data[59]]); + // A file cut before [56..60] keeps its streams and reports 0 (unknown) packets. + let source_packet_count = data + .get(56..60) + .map_or(0, |b| u32::from_be_bytes([b[0], b[1], b[2], b[3]])); // Parse ProgramInfo (per-stream language + codec), best-effort: malformed // program_info doesn't fail the parse, just yields an empty streams list. @@ -168,17 +168,13 @@ mod tests { } #[test] - fn parse_truncated_clipinfo_no_panic() { - // 57/58/59-byte CLPI with valid magic: passes the data.len() < 40 - // guard but data[56..60] needs 60 bytes. Must error, not panic. - for len in 40..60usize { - let mut data = vec![0u8; len]; - data[0..4].copy_from_slice(b"HDMV"); - if len >= 8 { - data[4..8].copy_from_slice(b"0200"); - } - assert!(parse(&data).is_err(), "len {len} lacks source_packet_count"); + fn parse_truncated_clipinfo_parses_with_unknown_count() { + let full = build_clpi(1000, None); + for len in 16..60usize { + let c = parse(&full[..len]).unwrap_or_else(|e| panic!("len {len}: {e:?}")); + assert_eq!(c.source_packet_count, 0, "len {len}"); } + assert_eq!(parse(&full[..60]).expect("60").source_packet_count, 1000); } #[test] @@ -259,11 +255,10 @@ mod tests { assert!(parse(&data).is_err()); } - /// Input shorter than the 60-byte ClipInfo head is rejected before any - /// field read (`data.len() < 60` guard). + /// Input shorter than the 16-byte header is rejected before any field read. #[test] - fn under_40_bytes_rejected() { - assert!(parse(&[0u8; 39]).is_err()); + fn under_16_bytes_rejected() { + assert!(parse(&[0u8; 15]).is_err()); assert!(parse(b"HDMV0200").is_err()); assert!(parse(&[]).is_err()); } diff --git a/src/consts.rs b/src/consts.rs index 654563d7..c134e8b4 100644 --- a/src/consts.rs +++ b/src/consts.rs @@ -107,6 +107,8 @@ pub mod coding_type { pub mod pes_stream_id { /// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`). pub const VIDEO: u8 = 0xE0; + /// MPEG audio stream (`110x xxxx`; 11172-3/13818-3/13818-7, freemkv emits `0xC0`). + pub const MPEG_AUDIO: u8 = 0xC0; /// system_header start code — the MPEG-PS `00 00 01 BB` structural header /// (rate/bound bounds), never an elementary stream. On a DVD NAV pack it /// follows the pack header, so it lands at sector offset 0x11. diff --git a/src/css/mod.rs b/src/css/mod.rs index f6182c5c..768e6241 100644 --- a/src/css/mod.rs +++ b/src/css/mod.rs @@ -135,12 +135,10 @@ pub fn crack_key_outcome( batch_sectors: u16, halt: Option<&crate::halt::Halt>, ) -> CrackOutcome { - crack_key_scan_with(reader, extents, batch_sectors, halt, |s| { - is_scrambled_pack(s).then_some(0x14) - }) + crack_key_scan(reader, extents, batch_sectors, halt) } -// The SINGLE place every DVD read path obtains a title key when the caller supplied none. A +// A DVD title's key when the caller supplied none: [`crack_title_key`] over its extents. A // scrambled-but-uncrackable title is a hard, skippable per-title CssKeyMissing (another VTS may // still crack). pub(crate) fn resolve_dvd_title_key( @@ -161,30 +159,35 @@ pub(crate) fn resolve_dvd_title_key( if matches!(keys, crate::decrypt::DecryptKeys::None) && format == crate::disc::ContentFormat::MpegPs { - // `halt` threads the caller's cancellation token so /api/stop can - // interrupt a long crack scan (the old scan-time crack honored it too). - match crack_key_outcome(reader, extents, batch_sectors, halt) { - CrackOutcome::Cracked(state) => { - *keys = crate::decrypt::DecryptKeys::Css { - title_key: state.title_key, - }; - } - CrackOutcome::ScrambledUncracked => { - return Err(crate::error::Error::CssKeyMissing.into()); - } - CrackOutcome::Unreadable(e) => return Err(e.into()), - // A cancelled crack is a TRUNCATED scan, not a verdict. - CrackOutcome::Halted => return Err(crate::error::Error::Halted.into()), - CrackOutcome::Unencrypted => {} - } + *keys = crack_title_key(reader, extents, batch_sectors, halt)?; } Ok(()) } +// The SINGLE place every read path (disc, image, loose PS file) cracks a CSS title key: the +// keys, or `None` for clear content. `halt` lets /api/stop interrupt a long scan. +pub(crate) fn crack_title_key( + reader: &mut dyn SectorSource, + extents: &[Extent], + batch_sectors: u16, + halt: Option<&crate::halt::Halt>, +) -> std::io::Result { + match crack_key_scan(reader, extents, batch_sectors, halt) { + CrackOutcome::Cracked(state) => Ok(crate::decrypt::DecryptKeys::Css { + title_key: state.title_key, + }), + CrackOutcome::ScrambledUncracked => Err(crate::error::Error::CssKeyMissing.into()), + CrackOutcome::Unreadable(e) => Err(e.into()), + // A cancelled crack is a TRUNCATED scan, not a verdict. + CrackOutcome::Halted => Err(crate::error::Error::Halted.into()), + CrackOutcome::Unencrypted => Ok(crate::decrypt::DecryptKeys::None), + } +} + /// Where a 13818-1 pack's first PES header flags byte sits, `Some` only when that PES is -/// scrambled: the `mpg://` file's test (m1, B-1). Leading packets with no PES header (a +/// scrambled: the ONE CSS pack test, disc and file alike. Leading packets with no PES header (a /// system header, a map, padding) are walked by their length; only the first PES is judged. -pub(crate) fn ps_scrambled_at(sector: &[u8]) -> Option { +pub(crate) fn scrambled_at(sector: &[u8]) -> Option { if sector.len() < 2048 || sector[..4] != PACK_START || sector[4] >> 6 != 0b01 { return None; } @@ -211,11 +214,6 @@ pub(crate) fn ps_scrambled_at(sector: &[u8]) -> Option { } } -/// [`ps_scrambled_at`] as a test. -pub(crate) fn is_scrambled_ps_pack(sector: &[u8]) -> bool { - ps_scrambled_at(sector).is_some() -} - // Run `f` on `sector` with its flags byte `at` presented at 0x14, where the LFSR and the // keyless crack read it; both bytes are put back after (a stuffed pack, m1/m2). fn with_flags_at_0x14(sector: &mut [u8], at: usize, f: impl FnOnce(&mut [u8]) -> T) -> T { @@ -230,39 +228,18 @@ fn with_flags_at_0x14(sector: &mut [u8], at: usize, f: impl FnOnce(&mut [u8]) out } -/// The `mpg://` file's crack (design §4 step 2.1, D3): raw = false, judging each pack by -/// [`ps_scrambled_at`]; the resolved keys, `DecryptKeys::None` when the file is clear. -pub(crate) fn resolve_ps_file_title_key( - reader: &mut dyn SectorSource, - extents: &[Extent], - batch_sectors: u16, - halt: Option<&crate::halt::Halt>, -) -> std::io::Result { - match crack_key_scan_with(reader, extents, batch_sectors, halt, ps_scrambled_at) { - CrackOutcome::Cracked(state) => Ok(crate::decrypt::DecryptKeys::Css { - title_key: state.title_key, - }), - CrackOutcome::Unencrypted => Ok(crate::decrypt::DecryptKeys::None), - CrackOutcome::ScrambledUncracked => Err(crate::error::Error::CssKeyMissing.into()), - CrackOutcome::Unreadable(e) => Err(e.into()), - CrackOutcome::Halted => Err(crate::error::Error::Halted.into()), - } -} - #[cfg(test)] thread_local! { /// Crack scans run on this thread (test-only: the mpg:// path must scan once). pub(crate) static CRACK_SCANS: std::cell::Cell = const { std::cell::Cell::new(0) }; } -// The crack scan with its test: `scrambled_at` gives a scrambled pack's flags offset (always -// 0x14 on the disc path; `ps_scrambled_at` for mpg:// files). -fn crack_key_scan_with( +// The crack scan over `extents`, judging each sector with [`scrambled_at`]. +fn crack_key_scan( reader: &mut dyn SectorSource, extents: &[Extent], batch_sectors: u16, halt: Option<&crate::halt::Halt>, - scrambled_at: fn(&[u8]) -> Option, ) -> CrackOutcome { #[cfg(test)] CRACK_SCANS.with(|n| n.set(n.get() + 1)); @@ -348,7 +325,7 @@ fn crack_key_scan_with( // or an unencrypted title falsely reports E7023. if let Some(at) = scrambled_at(sect) { saw_scrambled = true; - // An mpg:// pack's flags may sit past 0x14 (m1); the disc's never do. + // A stuffed pack's flags sit past 0x14 (m1); a DVD-Video pack's never do. let key = if at == 0x14 { keyless::crack_title_key(sect) } else { @@ -404,15 +381,14 @@ fn crack_key_scan_with( /// Descramble a single CSS-encrypted sector in place. /// -/// A no-op unless the sector is a scrambled MPEG-2 PS PACK: the pack start code is checked, not -/// just the byte 0x14 flag bits, since 0x14 alone is unreliable outside a pack (see -/// [`is_scrambled_pack`]). Making the guard part of the function, rather than something each -/// caller must remember, is what keeps the safe path the easy one. +/// A no-op unless the sector is a scrambled MPEG-2 PS PACK ([`is_scrambled_pack`]): byte 0x14 +/// alone is unreliable outside a pack. Making the guard part of the function, rather than +/// something each caller must remember, is what keeps the safe path the easy one. pub fn descramble_sector(state: &CssState, sector: &mut [u8]) { - if !is_scrambled_pack(sector) { + let Some(at) = scrambled_at(sector) else { return; - } - lfsr::descramble_sector(&state.title_key, sector); + }; + with_flags_at_0x14(sector, at, |s| lfsr::descramble_sector(&state.title_key, s)); } // How many mismatches a failed re-crack's "false positive" verdict covers before the next retry — @@ -428,28 +404,12 @@ const RECRACK_RETRY_EVERY: u32 = 16; /// Never returns `Err` and always returns `Ok(0)` (CSS drops no sectors); `Result` only /// matches the decrypt seam this is dispatched from (see [`crate::decrypt::decrypt_sectors`]). pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result { - // `is_scrambled_pack`, NOT the looser `is_scrambled`: this sees arbitrary - // regions (IFO/UDF/ISO 9660) where raw byte 0x14 isn't a reliable flag. - // Measured: an IFO misread this way was destroyed, dropping titles 38→10. - descramble_region_with(buf, title_key, |c| is_scrambled_pack(c).then_some(0x14)) -} - -/// [`descramble_region`] for an `mpg://` file: each pack is judged, and its flags cleared, -/// where its own pack stuffing puts them ([`ps_scrambled_at`], m2). -pub(crate) fn descramble_ps_region(buf: &mut [u8], title_key: &mut [u8; 5]) { - let _ = descramble_region_with(buf, title_key, ps_scrambled_at); -} - -// `scrambled_at(chunk)`: the offset of a scrambled pack's flags byte, else `None`. -fn descramble_region_with( - buf: &mut [u8], - title_key: &mut [u8; 5], - scrambled_at: fn(&[u8]) -> Option, -) -> crate::error::Result { // Consecutive crib mismatches since the last re-crack attempt (0 = none // pending). Reset by a validated cache hit; a fresh run always attempts // on its first mismatch, then at most once every RECRACK_RETRY_EVERY. let mut mismatches_since_attempt: u32 = 0; + // The pack test, NOT raw byte 0x14: this sees arbitrary regions (IFO/UDF/ISO 9660). + // Measured: an IFO misread by 0x14 alone was destroyed, dropping titles 38→10. for chunk in buf.chunks_mut(2048) { let Some(at) = (chunk.len() >= 2048).then(|| scrambled_at(chunk)).flatten() else { continue; @@ -525,25 +485,25 @@ pub fn has_scramble_flag_bits(sector: &[u8]) -> bool { /// untouched, so this signature survives scrambling. pub(crate) const PACK_START: [u8; 4] = [0x00, 0x00, 0x01, 0xBA]; -/// Check if a sector is a CSS-scrambled DVD **video pack** — the HARDENED gate the -/// crack scan's `saw_scrambled` flag and [`descramble_sector`]/[`descramble_region`] share. -/// -/// Requires BOTH the MPEG-PS pack-start code AND the 0x14 scramble bits -/// ([`has_scramble_flag_bits`] alone is not enough), and excludes structural/nav `stream_id`s -/// at 0x11 (`0xBB`/`0xBE`/`0xBF`) that CSS never scrambles so a decrypted HD-DVD's RDI nav -/// packs can't falsely trip it. An 11172-1 pack cannot be CSS; pack stuffing is not checked -/// here (libdvdcss reads 0x14 regardless), only by the `mpg://` file scan. +/// Check if a sector is a CSS-scrambled pack: a 13818-1 pack whose first PES header (past any +/// pack stuffing and packets with no PES header: system header, map, padding, nav) is MPEG-2 +/// and carries scramble bits. The one test the crack scan and every descramble share, disc and +/// file alike. An IFO/UDF sector, an 11172-1 pack and an MPEG-1 PES are never scrambled. pub fn is_scrambled_pack(sector: &[u8]) -> bool { - use crate::consts::pes_stream_id::{PADDING_STREAM, PRIVATE_STREAM_2, SYSTEM_HEADER}; - // B2: a 13818-1 pack ('01'); an 11172-1 pack's 0x11/0x14 are a packet length and PTS. - sector.len() >= 2048 - && sector[0x00..0x04] == PACK_START - && sector[0x04] >> 6 == 0b01 - && !matches!( - sector[0x11], - SYSTEM_HEADER | PADDING_STREAM | PRIVATE_STREAM_2 - ) - && (sector[0x14] >> 4) & 0x03 != 0 + scrambled_at(sector).is_some() +} + +/// Test fixture: the DVD-Video pack layout every real VOB sector has (13818-1 pack header, no +/// stuffing, a `stream_id` PES at 0x0E with MPEG-2 flags at 0x14), keeping its scramble bits. +#[cfg(test)] +pub(crate) fn dvd_pack_header(sector: &mut [u8], stream_id: u8) { + sector[..4].copy_from_slice(&PACK_START); + sector[4] = 0x44; + sector[0x0D] = 0xF8; + sector[0x0E..0x11].copy_from_slice(&[0, 0, 1]); + sector[0x11] = stream_id; + sector[0x12..0x14].copy_from_slice(&0x07ECu16.to_be_bytes()); + sector[0x14] = 0x80 | (sector[0x14] & 0x30); } #[cfg(test)] @@ -615,10 +575,8 @@ mod tests { /// so `descramble_region` actually enters the mismatch branch. fn false_positive_scrambled_pack() -> [u8; 2048] { let mut sector = [0u8; 2048]; - sector[0x00..0x04].copy_from_slice(&PACK_START); - sector[4] = 0x44; // '01': a 13818-1 pack - sector[0x11] = crate::consts::pes_stream_id::PRIVATE_STREAM_1; sector[0x14] = 0x30; + dvd_pack_header(&mut sector, crate::consts::pes_stream_id::PRIVATE_STREAM_1); for (i, b) in sector.iter_mut().enumerate().take(0x80).skip(0x20) { *b = (i % 4) as u8; } @@ -742,8 +700,7 @@ mod tests { let pack = false_positive_scrambled_pack(); for len in [0, 1, 0x14, 0x15, 1024, 2047] { assert!(!is_scrambled_pack(&pack[..len]), "len {len}"); - assert!(ps_scrambled_at(&pack[..len]).is_none(), "len {len}"); - assert!(!is_scrambled_ps_pack(&pack[..len]), "len {len}"); + assert!(scrambled_at(&pack[..len]).is_none(), "len {len}"); } } @@ -764,7 +721,6 @@ mod tests { let mut short = pack[..1500].to_vec(); let before = short.clone(); descramble_region(&mut short, &mut key).expect("never fails"); - descramble_ps_region(&mut short, &mut key); assert_eq!(short, before); } @@ -834,7 +790,10 @@ mod tests { s[at..at + 4].copy_from_slice(&[0, 0, 1, 0xE0]); s[at + 4..at + 6].copy_from_slice(&0x07EBu16.to_be_bytes()); s[at + 6] = 0x81; - assert!(!is_scrambled_ps_pack(&s), "pack_stuffing_length {stuffing}"); + assert!( + scrambled_at(&s).is_none(), + "pack_stuffing_length {stuffing}" + ); } } @@ -859,37 +818,42 @@ mod tests { // A PSM whose byte at 0x14 (current_next_indicator, version) reads as "scrambled". let psm = [0, 0, 1, 0xBC, 0, 10, 0xE0, 0xFF, 0, 0, 0, 0, 0, 0, 0, 0]; let (_, clear) = pack_after(&psm, 0x81); - assert_eq!(ps_scrambled_at(&clear), None, "clear"); + assert_eq!(scrambled_at(&clear), None, "clear"); let (at, scrambled) = pack_after(&psm, 0x91); - assert_eq!(ps_scrambled_at(&scrambled), Some(at), "scrambled"); + assert_eq!(scrambled_at(&scrambled), Some(at), "scrambled"); // Every stream_id with no PES header (MS-31), and the system header, is walked past. for id in [0xBB, 0xBC, 0xBE, 0xBF, 0xF0, 0xF1, 0xF2, 0xF8, 0xFF] { let lead = [0, 0, 1, id, 0, 4, 0xB0, 0xB0, 0xB0, 0xB0]; - assert_eq!(ps_scrambled_at(&pack_after(&lead, 0x81).1), None, "{id:#x}"); + assert_eq!(scrambled_at(&pack_after(&lead, 0x81).1), None, "{id:#x}"); } } - // D3: on the disc/ISO path a scrambled 13818-1 pack is detected and descrambled whatever - // its pack_stuffing_length (libdvdcss reads 0x14 regardless); only mpg:// files are strict. + // One pack test on every path: a stuffed 13818-1 pack (DVD-Video never stuffs) is judged + // by the PES its stuffing points at, and descrambled with that PES's own flags. #[test] - fn a_stuffed_scrambled_pack_is_still_descrambled_on_the_disc_path() { - let mut pack = vec![0u8; 2048]; - for (i, b) in pack.iter_mut().enumerate() { - *b = (i as u8).wrapping_mul(29).wrapping_add(3); - } - pack[0x00..0x04].copy_from_slice(&PACK_START); - pack[4] = 0x44; // '01': a 13818-1 pack + fn a_stuffed_scrambled_pack_is_descrambled_by_its_own_flags() { + let mut pack: Vec = (0..2048u32) + .map(|i| (i as u8).wrapping_mul(29).wrapping_add(3)) + .collect(); + dvd_pack_header(&mut pack, 0xE0); pack[0x0D] = 0xF8 | 1; // pack_stuffing_length 1 - pack[0x14] = 0x30; - assert!(is_scrambled_pack(&pack), "stuffed but scrambled"); + pack[0x0E] = 0xFF; + pack[0x0F..0x13].copy_from_slice(&[0, 0, 1, 0xE0]); + pack[0x13..0x15].copy_from_slice(&0x07EBu16.to_be_bytes()); + pack[0x15] = 0xB0; + assert_eq!(scrambled_at(&pack), Some(0x15)); let before = pack.clone(); - let mut key = [0x42, 0x13, 0x37, 0xBE, 0xEF]; - descramble_region(&mut pack, &mut key).expect("region descramble"); + descramble_region(&mut pack, &mut [0x42, 0x13, 0x37, 0xBE, 0xEF]).unwrap(); assert_ne!( pack[0x80..], before[0x80..], "descrambled, not passed through" ); + assert_eq!( + (pack[0x14], pack[0x15]), + (before[0x14], 0x80), + "its own flags cleared" + ); } // Fix 3 hardening: `is_scrambled_pack` requires BOTH the pack-start code @@ -915,43 +879,29 @@ mod tests { "a wrong pack-start byte must not qualify" ); // The real signature flips it to a scrambled pack. - s[0x00..0x04].copy_from_slice(&PACK_START); - s[4] = 0x44; // '01': a 13818-1 pack + dvd_pack_header(&mut s, 0xE0); assert!( is_scrambled_pack(&s), "valid pack-start + 0x14 bits → scrambled pack" ); } - // Detection fix: `is_scrambled_pack` must EXCLUDE structural/nav stream_ids - // (0xBB/0xBE/0xBF) at 0x11, the exact decrypted-HD-DVD defect (an `.evo` RDI pack is 0xBF). + // Detection fix: a pack led by a system header, padding or nav packet (an `.evo` RDI pack + // is 0xBF) is not scramble evidence, whatever its 0x14 holds; an ES pack still is. #[test] fn is_scrambled_pack_excludes_nav_and_structural_stream_ids() { use crate::consts::pes_stream_id::{ PADDING_STREAM, PRIVATE_STREAM_1, PRIVATE_STREAM_2, SYSTEM_HEADER, VIDEO, VIDEO_MAX, }; - // A pack-start pack with 0x14 scramble bits set, varying only 0x11. let mut s = vec![0u8; 2048]; - s[0x00..0x04].copy_from_slice(&PACK_START); - s[4] = 0x44; // '01': a 13818-1 pack s[0x14] = 0x30; for excluded in [SYSTEM_HEADER, PADDING_STREAM, PRIVATE_STREAM_2] { - s[0x11] = excluded; - assert!( - !is_scrambled_pack(&s), - "stream_id {excluded:#04x} is a structural/nav pack CSS never scrambles — \ - it must NOT count as scramble evidence (else a decrypted HD-DVD RDI pack → E7023)" - ); + dvd_pack_header(&mut s, excluded); + assert!(!is_scrambled_pack(&s), "stream_id {excluded:#04x}"); } - // A genuinely scramblable elementary-stream pack must STILL register — - // proving the exclusion did not weaken real-DVD CSS detection. for scramblable in [VIDEO, VIDEO_MAX, PRIVATE_STREAM_1, 0xE2] { - s[0x11] = scramblable; - assert!( - is_scrambled_pack(&s), - "stream_id {scramblable:#04x} is a scramblable ES pack — a real CSS DVD's \ - scrambled sector must still be counted, never passed through as plaintext" - ); + dvd_pack_header(&mut s, scramblable); + assert!(is_scrambled_pack(&s), "stream_id {scramblable:#04x}"); } } @@ -991,7 +941,7 @@ mod tests { locked: Vec::new(), crackable: None, short_read: None, - stream_id: 0x00, + stream_id: 0xE0, } } } @@ -1003,9 +953,8 @@ mod tests { const RUN_START: usize = 0x59; const SEED_OFFSET: usize = 0x54; let mut plaintext = vec![0u8; 2048]; - plaintext[0x00..0x04].copy_from_slice(&PACK_START); // valid DVD pack header - plaintext[4] = 0x44; // '01': a 13818-1 pack plaintext[0x14] = 0x10; // scramble flag + dvd_pack_header(&mut plaintext, 0xE0); let pat: Vec = (0..period) .map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A) .collect(); @@ -1063,13 +1012,10 @@ mod tests { buf[base..base + 2048].copy_from_slice(sector); } _ => { - // `is_scrambled_pack` requires the MPEG-PS pack-start code - // before trusting 0x14; `stream_id` defaults to 0x00 - // (scramblable) vs. an RDI nav pack's excluded 0xBF. - buf[base..base + 4].copy_from_slice(&PACK_START); - buf[base + 4] = 0x44; // '01': a 13818-1 pack - buf[base + 0x11] = self.stream_id; + // A DVD-Video pack; `stream_id` defaults to video (scramblable) + // vs. an RDI nav pack's 0xBF, which CSS never scrambles. buf[base + 0x14] = self.flag_byte; + dvd_pack_header(&mut buf[base..base + 2048], self.stream_id); } } } @@ -1264,10 +1210,8 @@ mod tests { for (i, b) in pack.iter_mut().enumerate() { *b = (i as u8).wrapping_mul(29).wrapping_add(3); } - pack[0x00..0x04].copy_from_slice(&PACK_START); - pack[4] = 0x44; // '01': a 13818-1 pack - pack[0x0D] = 0xF8; // pack_stuffing_length 0 pack[0x14] = 0x30; + dvd_pack_header(&mut pack, 0xE0); assert!(is_scrambled_pack(&pack)); let scrambled = pack.clone(); diff --git a/src/decrypt.rs b/src/decrypt.rs index 9cac447e..e1ee6815 100644 --- a/src/decrypt.rs +++ b/src/decrypt.rs @@ -644,15 +644,16 @@ const WRONG_KEY_FAILURES: usize = 2; /// Blank the damaged BD-TS units of `buf` (read at `base_lba` on the caller's unit grid): /// zero-fill them, like a sweep's unread sector, and return how many. Damaged: no TS sync at /// byte 4, flagged (CPI) or not; unflagged only when not clean TS either (a zeroed head over -/// clear TS is a sweep hole). No key opens them: read damage, never a key verdict or E7013, -/// alone, clustered, or off the grid (1.7.7 muxed through all three). Only units `covered` -/// (keyed or proven on arrival) are judged. A flagged garbage seed that keeps 0x47 at byte 4 -/// (~1 in 256) is not caught here: it decrypts as garbage, or on arrival reads as unopened. +/// clear TS is a sweep hole), or a flagged partial unit ending the source (`tail_at_end`). +/// No key opens them: read damage, never a key verdict or E7013, alone, clustered, or off the +/// grid (1.7.7 muxed through all three). Only units `covered` (keyed or proven on arrival) are +/// judged. A flagged garbage seed keeping 0x47 at byte 4 (~1 in 256) is not caught here. pub(crate) fn blank_damaged_units( buf: &mut [u8], base_lba: u32, format: crate::disc::ContentFormat, covered: &dyn Fn(u32) -> bool, + tail_at_end: bool, ) -> usize { if format != crate::disc::ContentFormat::BdTs { return 0; @@ -678,10 +679,23 @@ pub(crate) fn blank_damaged_units( damaged.push(i); } } + // A truncated copy's flagged last partial unit cannot be opened as a unit (KS-3); a partial + // read mid-content is a caller bug, left for the decrypt to refuse. + let whole = buf.len() / unit_len; + let tail = &buf[whole * unit_len..]; + let at = base_lba.saturating_add(whole as u32 * aacs::content::ALIGNED_UNIT_SECTORS); + if tail_at_end + && !tail.is_empty() + && covered(at) + && aacs::content::aacs_unit_seed_encrypted(tail, format) + { + damaged.push(whole); + } // KS-3 [BD] §3.10.1: "A new CBC cipher chain is started for each Aligned Unit", so the // loss is this unit alone. for &i in &damaged { - buf[i * unit_len..(i + 1) * unit_len].fill(0); + let end = ((i + 1) * unit_len).min(buf.len()); + buf[i * unit_len..end].fill(0); } if let Some(&first) = damaged.first() { tracing::warn!( @@ -873,6 +887,7 @@ mod tests { plaintext[0x00..0x04].copy_from_slice(&css::PACK_START); plaintext[4] = 0x44; // '01': a 13818-1 pack plaintext[0x14] = 0x10; // CSS scramble flag (DVD-Video sector header) + crate::css::dvd_pack_header(&mut plaintext, 0xE0); let pat: Vec = (0..PERIOD) .map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A) .collect(); @@ -940,6 +955,7 @@ mod tests { plaintext[0x00..0x04].copy_from_slice(&css::PACK_START); plaintext[4] = 0x44; // '01': a 13818-1 pack plaintext[0x14] = 0x10; // scramble flag + crate::css::dvd_pack_header(&mut plaintext, 0xE0); let pat: Vec = (0..PERIOD) .map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A) .collect(); @@ -1084,6 +1100,7 @@ mod tests { sector[4] = 0x44; // '01': a 13818-1 pack sector[0x0D] = 0xF8; // pack_stuffing_length 0 sector[0x14] = 0x30; // scramble flag (bits 4-5) + crate::css::dvd_pack_header(&mut sector, 0xE0); sector[0x54..0x59].copy_from_slice(seed); let plaintext = sector.clone(); css::lfsr::scramble_sector(title_key, &mut sector); @@ -1152,6 +1169,7 @@ mod tests { plaintext[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); plaintext[4] = 0x44; // '01': a 13818-1 pack plaintext[0x14] = 0x10; // scramble flag + crate::css::dvd_pack_header(&mut plaintext, 0xE0); // Periodic run from 0x59 (just above the seed) through 0x80 and on into // the encrypted region; phase anchored to offset 0 so it is continuous // across the 0x80 boundary. @@ -2071,7 +2089,7 @@ mod tests { crate::test_util::damage_unit_seed(&mut garbage); garbage[0] &= 0x3F; let mut buf = [garbage, clear_ts_unit(), vec![0u8; ul]].concat(); - let n = blank_damaged_units(&mut buf, 0, ContentFormat::BdTs, &|_| true); + let n = blank_damaged_units(&mut buf, 0, ContentFormat::BdTs, &|_| true, false); assert_eq!(n, 1); assert!(buf[..ul].iter().all(|&b| b == 0)); assert_eq!(&buf[ul..2 * ul], &clear_ts_unit()[..]); @@ -2370,4 +2388,24 @@ mod spec_guards { decrypt_sectors_mapped(&mut kept, &keys, 0, &map).expect("a verified unit decrypts"); check("kept Verify", &kept, plain); } + // A flagged partial unit is blanked only when it ends the source (a truncated copy); a + // partial read mid-content is left for the decrypt to refuse. + #[test] + fn a_flagged_partial_unit_is_blanked_only_at_the_end() { + use crate::disc::ContentFormat; + let mut tail = vec![0x5Au8; 4096]; + tail[0] |= 0xC0; + tail[4] = 0x47; + let mut mid = tail.clone(); + assert_eq!( + blank_damaged_units(&mut mid, 0, ContentFormat::BdTs, &|_| true, false), + 0 + ); + assert_eq!(mid, tail); + assert_eq!( + blank_damaged_units(&mut tail, 0, ContentFormat::BdTs, &|_| true, true), + 1 + ); + assert!(tail.iter().all(|&b| b == 0)); + } } diff --git a/src/disc/bluray.rs b/src/disc/bluray.rs index 22e6b2d1..96c20036 100644 --- a/src/disc/bluray.rs +++ b/src/disc/bluray.rs @@ -282,6 +282,10 @@ impl Disc { push_extents(file_exts, &mut extents, &mut feed_pos); if feed_pos > span_start { spans.insert(play_item.clip_id.clone(), (span_start, feed_pos)); + // A damaged CLPI reports no packet count: size the clip by its extents. + if pkt_count == 0 { + total_size += feed_pos - span_start; + } } } } @@ -1101,6 +1105,27 @@ mod tests { assert_eq!(t.clips[1].clip_id, "00001"); } + // A CLPI with no packet count (cut short) still sizes the clip, from its extents. + #[test] + fn parse_playlist_unknown_clpi_count_sizes_from_extents() { + let mut disc = MemDisc::new(); + let udf = make_bdmv_fs(&mut disc, &[("00001", 1000, 0, 5000)]); + let mpls = build_mpls( + &[PiSpec { + clip_id: *b"00001", + in_time: 0, + out_time: 60 * 45000, + }], + (0, 0, 0, 0, 0, 0, 0, 0), + &[], + &[], + ); + let t = Disc::parse_playlist(&mut disc, &udf, "00001.mpls", &mpls) + .expect("scan") + .expect("title"); + assert_eq!(t.size_bytes, 1000 * 2048); + } + /// Distinct clips each contribute their own extent and bytes, in /// PlayItem order (mux relies on extent order). #[test] diff --git a/src/disc/encrypt.rs b/src/disc/encrypt.rs index 46223fb8..cfd542fd 100644 --- a/src/disc/encrypt.rs +++ b/src/disc/encrypt.rs @@ -159,12 +159,11 @@ pub(super) struct AacsCapture { pub(super) enum CaptureFrom { /// Image or folder: the key-file error is recorded as read (E7000/E6000). Image, - /// Live drive: E7031, fatal unless `raw_copy` (then recorded, keys refused). - Live { raw_copy: bool }, + /// Live drive: E7031 is recorded, warned and the scan goes on (keys refused). + Live, } -// Reads the AACS key files (plain READs, no AACS command). `Err` for a Stop, and for a -// live key-file failure without `raw_copy`, returned before any other read. +// Reads the AACS key files (plain READs, no AACS command). `Err` only for a Stop. pub(super) fn capture( reader: &mut dyn SectorSource, udf_fs: &udf::UdfFs, @@ -177,11 +176,13 @@ pub(super) fn capture( }); match (&uk_ro, from) { (Err(Error::Halted), _) => return Err(Error::Halted), - (Err(e), CaptureFrom::Live { raw_copy }) => { - tracing::debug!(target: "freemkv::scan", phase = "aacs_capture", error_code = e.code(), raw_copy); - if !raw_copy { - return Err(Error::AacsKeyFileUnreadable); - } + (Err(e), CaptureFrom::Live) => { + tracing::warn!( + target: "freemkv::scan", + phase = "aacs_capture", + error_code = e.code(), + "Unit_Key_RO.inf unreadable; scanning on, titles needing keys will not be keyed." + ); uk_ro = Err(Error::AacsKeyFileUnreadable); } _ => {} @@ -739,7 +740,7 @@ mod tests { fn from_for(bus: &BusOutcome) -> CaptureFrom { match bus { BusOutcome::FileOrIso => CaptureFrom::Image, - _ => CaptureFrom::Live { raw_copy: false }, + _ => CaptureFrom::Live, } } diff --git a/src/disc/extract.rs b/src/disc/extract.rs index b267a403..f064dd57 100644 --- a/src/disc/extract.rs +++ b/src/disc/extract.rs @@ -1630,6 +1630,7 @@ mod tests { plain[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); plain[4] = 0x44; // '01': a 13818-1 pack plain[0x14] = 0x10; // scramble flag + crate::css::dvd_pack_header(&mut plain, 0xE0); let pat: Vec = (0..8) .map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A) .collect(); @@ -1670,7 +1671,7 @@ mod tests { // Descrambled output matches the plaintext, with the scramble flag // cleared by the descrambler. let mut expect = plain.clone(); - expect[0x14] = 0x00; + expect[0x14] = 0x80; assert_eq!(got, expect, "VOB descrambled to plaintext"); assert!(res.complete); } @@ -2213,6 +2214,7 @@ mod tests { plain[0x00..0x04].copy_from_slice(&crate::css::PACK_START); plain[4] = 0x44; // '01': a 13818-1 pack plain[0x14] = 0x10; // scramble flag + crate::css::dvd_pack_header(&mut plain, 0xE0); let pat: Vec = (0..8) .map(|k| (0xA0u8.wrapping_add(k as u8) ^ marker) ^ 0x5A) .collect(); @@ -2263,9 +2265,9 @@ mod tests { let got_1 = read_out(out.path(), "VIDEO_TS/VTS_01_1.VOB").expect("vts01 vob"); let got_2 = read_out(out.path(), "VIDEO_TS/VTS_02_1.VOB").expect("vts02 vob"); let mut expect_1 = plain_1.clone(); - expect_1[0x14] = 0x00; + expect_1[0x14] = 0x80; let mut expect_2 = plain_2.clone(); - expect_2[0x14] = 0x00; + expect_2[0x14] = 0x80; assert_eq!( got_1, expect_1, "VTS_01 must descramble under its OWN cracked key, not VTS_02's" @@ -2288,6 +2290,7 @@ mod tests { plain[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); plain[4] = 0x44; // '01': a 13818-1 pack plain[0x14] = 0x10; + crate::css::dvd_pack_header(&mut plain, 0xE0); let pat: Vec = (0..8) .map(|k| (0xA0u8.wrapping_add(k as u8) ^ 0x01) ^ 0x5A) .collect(); @@ -2306,6 +2309,7 @@ mod tests { sect[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); sect[4] = 0x44; // '01': a 13818-1 pack sect[0x14] = 0x10; + crate::css::dvd_pack_header(&mut sect, 0xE0); for (i, b) in sect.iter_mut().enumerate().skip(0x59) { // Non-repeating, so no run of any period survives to 0x80. *b = (i as u8).wrapping_mul(37).wrapping_add(11); @@ -3000,6 +3004,7 @@ mod tests { sect[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); sect[4] = 0x44; // '01': a 13818-1 pack sect[0x14] = 0x10; + crate::css::dvd_pack_header(&mut sect, 0xE0); for (i, b) in sect.iter_mut().enumerate().skip(0x59) { *b = (i as u8).wrapping_mul(37).wrapping_add(11); } @@ -3099,6 +3104,7 @@ mod tests { sect[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); sect[4] = 0x44; // '01': a 13818-1 pack sect[0x14] = 0x10; + crate::css::dvd_pack_header(&mut sect, 0xE0); for (i, b) in sect.iter_mut().enumerate().skip(0x59) { *b = (i as u8).wrapping_mul(37).wrapping_add(11); } diff --git a/src/disc/mod.rs b/src/disc/mod.rs index 11e6f51b..d81dab8e 100644 --- a/src/disc/mod.rs +++ b/src/disc/mod.rs @@ -1749,10 +1749,8 @@ pub struct ScanOptions { /// scanned title (e.g. `freemkv info`) opt in. The rip path leaves it off: /// the muxer detects forced during muxing without a second read. pub probe_forced_subtitles: bool, - /// The caller copies sectors raw and never decrypts or accepts keys (a raw - /// disc→ISO copy). A live AACS disc whose `Unit_Key_RO.inf` is unreadable then - /// still scans, with [`Error::AacsKeyFileUnreadable`] recorded and every key refused; - /// otherwise `Disc::scan` returns that error. + /// The caller copies sectors raw (a raw disc→ISO copy). No effect on the scan since + /// an unreadable `Unit_Key_RO.inf` never fails it; kept for source compatibility. pub raw_copy: bool, } @@ -1933,9 +1931,9 @@ impl Disc { /// capacity + UDF, (3) the AACS files (`Unit_Key_RO.inf`, content cert, MKB) with /// plain READs, (4) the AACS handshake if AACS, (5) titles + labels. /// - /// A live AACS disc whose `Unit_Key_RO.inf` cannot be read fails with - /// [`Error::AacsKeyFileUnreadable`] before any AACS command (see - /// [`ScanOptions::raw_copy`]). A dead bus during a handshake aborts the scan. + /// A live AACS disc whose `Unit_Key_RO.inf` cannot be read is warned and scanned on, + /// with [`Error::AacsKeyFileUnreadable`] recorded and every key refused. A dead bus + /// during a handshake aborts the scan. /// `opts.halt` is the scan's op token unless the drive has one attached (§2.2). pub fn scan(session: &mut Drive, opts: &ScanOptions) -> Result { let mut session = session.alias(opts.halt.as_ref()); @@ -1969,9 +1967,7 @@ impl Disc { let from = if dvd { encrypt::CaptureFrom::Image } else { - encrypt::CaptureFrom::Live { - raw_copy: opts.raw_copy, - } + encrypt::CaptureFrom::Live }; let cap = encrypt::capture(&mut buffered, &udf_fs, from)?; let bus = if !dvd { @@ -6560,6 +6556,7 @@ mod tests { sec[0x00..0x04].copy_from_slice(&crate::css::PACK_START); sec[4] = 0x44; // '01': a 13818-1 pack sec[0x14] = 0x10; // scramble flag + crate::css::dvd_pack_header(&mut sec, 0xE0); for (i, b) in sec.iter_mut().enumerate().skip(RUN_START) { *b = (0xA0u8.wrapping_add((i % PERIOD) as u8)) ^ 0x5A; } @@ -8397,7 +8394,7 @@ mod tests { udf: &udf::UdfFs, rdk: Option<[u8; 16]>, ) -> Option<[u8; 16]> { - let from = encrypt::CaptureFrom::Live { raw_copy: true }; + let from = encrypt::CaptureFrom::Live; let cap = encrypt::capture(mem, udf, from).expect("capture"); let bus = encrypt::BusOutcome::Handshake(encrypt::HandshakeResult { volume_id: [0x11; 16], @@ -8507,7 +8504,7 @@ mod tests { let (mem, _) = bus_fixture_with(Some(cert), wire.clone()); let mut d = Drive::from_transport_for_test(Box::new(MemTransport(mem))); let (capacity, mut buffered, udf) = Disc::read_udf(&mut d).expect("udf"); - let from = encrypt::CaptureFrom::Live { raw_copy: true }; + let from = encrypt::CaptureFrom::Live; let cap = encrypt::capture(&mut buffered, &udf, from).expect("capture"); let bus = encrypt::BusOutcome::Handshake(handshake()); Disc::live_finish( @@ -8540,11 +8537,7 @@ mod tests { ) -> Result<(Drive, Disc)> { let mut d = Drive::from_transport_for_test(Box::new(MemTransport(mem))); let (capacity, mut buffered, udf) = Disc::read_udf(&mut d)?; - let cap = encrypt::capture( - &mut buffered, - &udf, - encrypt::CaptureFrom::Live { raw_copy: true }, - )?; + let cap = encrypt::capture(&mut buffered, &udf, encrypt::CaptureFrom::Live)?; let bus = encrypt::BusOutcome::Handshake(encrypt::HandshakeResult { volume_id: [0x11; 16], read_data_key: rdk, diff --git a/src/disc/scan_order_tests.rs b/src/disc/scan_order_tests.rs index ff56e769..a3a78b91 100644 --- a/src/disc/scan_order_tests.rs +++ b/src/disc/scan_order_tests.rs @@ -359,7 +359,7 @@ fn unencrypted_bd_issues_no_aacs_scsi() { } #[test] -fn live_unreadable_uk_ro_fails_scan_before_any_aacs_scsi() { +fn live_unreadable_uk_ro_is_recorded_scan_continues() { for bee in [true, false] { // Both copies absent: the file is simply not on the disc. let mut mem = bd_disc(Some(bee)); @@ -377,13 +377,16 @@ fn live_unreadable_uk_ro_fails_scan_before_any_aacs_scsi() { }, ); let mut rig = Rig::new(mem, |_| {}); - let r = Disc::scan(&mut rig.drive, &with_hc()); + let d = Disc::scan(&mut rig.drive, &with_hc()).expect("absent: scan goes on"); + assert!( + d.aacs.is_none() && !d.titles.is_empty(), + "absent, bee={bee}" + ); assert_eq!( - r.err().map(|e| e.code()), + d.aacs_error.as_ref().map(|e| e.code()), Some(KEY_FILE_CODE), "absent, bee={bee}" ); - assert_eq!(rig.count(is_aacs_cdb), 0, "absent, bee={bee}"); // Present but unreadable. let mut rig = Rig::new(bd_disc(Some(bee)), |t| { @@ -393,34 +396,19 @@ fn live_unreadable_uk_ro_fails_scan_before_any_aacs_scsi() { .then(|| check(3, 0x11, 0, 0x28)) })); }); - let r = Disc::scan(&mut rig.drive, &with_hc()); + let d = Disc::scan(&mut rig.drive, &with_hc()).expect("read error: scan goes on"); + assert!( + d.aacs.is_none() && !d.titles.is_empty(), + "read error, bee={bee}" + ); assert_eq!( - r.err().map(|e| e.code()), + d.aacs_error.as_ref().map(|e| e.code()), Some(KEY_FILE_CODE), "read error, bee={bee}" ); - assert_eq!(rig.count(is_aacs_cdb), 0, "read error, bee={bee}"); } } -#[test] -fn raw_copy_scan_records_unreadable_uk_ro() { - let mut rig = Rig::new(bd_disc(Some(false)), |t| { - t.fail_read = Some(Box::new(|lba, n, _| { - (lba..lba + n as u32) - .contains(&UK_LBA) - .then(|| check(3, 0x11, 0, 0x28)) - })); - }); - let opts = ScanOptions { - raw_copy: true, - ..with_hc() - }; - let d = Disc::scan(&mut rig.drive, &opts).expect("raw copy scans on"); - assert!(d.aacs.is_none() && !d.titles.is_empty()); - assert_eq!(d.aacs_error.as_ref().map(|e| e.code()), Some(KEY_FILE_CODE)); -} - #[test] fn failed_handshake_cannot_poison_uk_ro_read() { let mut rig = Rig::new(bd_disc(Some(false)), |t| { @@ -750,11 +738,7 @@ impl crate::sector::SectorSource for HaltAt { fn capture_propagates_a_stop_from_every_key_file_read() { use super::encrypt::{CaptureFrom, capture}; for lba in [UK_LBA, CERT_LBA, MKB_LBA] { - for from in [ - CaptureFrom::Image, - CaptureFrom::Live { raw_copy: true }, - CaptureFrom::Live { raw_copy: false }, - ] { + for from in [CaptureFrom::Image, CaptureFrom::Live] { let mut src = HaltAt { mem: bd_disc(Some(true)), lba, diff --git a/src/error.rs b/src/error.rs index 241875ab..74e83b4c 100644 --- a/src/error.rs +++ b/src/error.rs @@ -240,9 +240,8 @@ pub const E_STREAM_CLOSED: u16 = 9071; /// Per-track metadata was set after the stream header was already written. pub const E_STREAM_HEADER_WRITTEN: u16 = 9072; pub const E_M2TS_PACKET_MALFORMED: u16 = 9021; -/// A `network://` output target resolved to no address that is safe to -/// connect to (every resolved IP was loopback / private / link-local / -/// multicast / unspecified). Closes the DNS-rebinding SSRF window. +/// A `network://` output target resolved to no connectable address (every +/// resolved IP was unspecified / multicast / broadcast / reserved). pub const E_NETWORK_ADDR_BLOCKED: u16 = 9022; /// A muxer's `finish()` was called after zero frames were emitted — the /// output would be a header-only container with no media. Surfaced so a @@ -674,7 +673,7 @@ pub enum Error { /// scans, where no bus-key handshake runs. AacsBusKeyUnavailable, /// A live AACS disc's `Unit_Key_RO.inf` is missing or unreadable (both copies). - /// `Disc::scan` returns it; a `raw_copy` scan records it and refuses every key. + /// Recorded in `Disc::aacs_error` (the scan goes on); every key is refused. AacsKeyFileUnreadable, /// A decrypted whole-disc image would keep encrypted pieces: a stream file /// no title plays is encrypted and no held key opens it. See @@ -730,9 +729,8 @@ pub enum Error { addr: String, }, /// A `network://` output host resolved to no connectable address — - /// every resolved IP was loopback / private / link-local / multicast / - /// unspecified. Carries the offending `host:port`. Re-checked at - /// connect time to close the DNS-rebinding TOCTOU. + /// every resolved IP was unspecified / multicast / broadcast / reserved. + /// Carries the offending `host:port`. NetworkAddrBlocked { addr: String, }, @@ -1448,7 +1446,7 @@ impl From for std::io::Error { // a non-188-byte packet — treat as invalid data. 9021 => std::io::ErrorKind::InvalidData, // 9022 NetworkAddrBlocked: the output host resolved only to - // blocked (loopback/private/link-local) addresses — refuse. + // invalid (unspecified/multicast/broadcast/reserved) addresses. E_NETWORK_ADDR_BLOCKED => std::io::ErrorKind::PermissionDenied, // 9023 MuxEmpty: finish() reached with zero frames — the output // would be a header-only container. Treat as invalid output. diff --git a/src/keys/arrival.rs b/src/keys/arrival.rs index 4d6131d5..c2c950d4 100644 --- a/src/keys/arrival.rs +++ b/src/keys/arrival.rs @@ -5,9 +5,10 @@ //! backward and one forward side read (`recovery = false`, never retried, failures swallowed). //! A held key opening the unit and a partner is `Proven`; else `Provisional` until another //! readable unit confirms it. A unit the key does not open is damage, blanked and counted, -//! when a held key opens a partner or no partner is readable; it stops E7022 / E7032 when +//! when a held key opens a partner or no partner is readable; it stops E7022 when //! another held key opens it, when partners are readable and none opens, or when a second //! partnerless unit (another LBA) opens under no key before any proof; never in a clear piece. +//! An image or folder (`StopKind::Image`) never stops: those units are blanked and counted. use super::{Proof, ProofCache, ResolvedKeySet, StopKind}; use crate::aacs::content::{ @@ -100,6 +101,15 @@ impl Arrival { && aacs_unit_on_grid(unit, self.format) } + // An unopenable unit stops a title rip (E7022); in an image it is blanked and counted. + fn refuse(&self, lba: u32, why: &'static str) -> Result<()> { + if matches!(self.stop, StopKind::Image) { + tracing::warn!(target: "freemkv::keys", lba, why, "no held key opens the unit: blanking it in the image"); + return Ok(()); + } + Err(self.stop(lba, why)) + } + fn stop(&self, lba: u32, why: &'static str) -> Error { let err = self.stop.error(); tracing::error!( @@ -205,11 +215,11 @@ impl Arrival { let unit = &buf[u * ALIGNED_UNIT_LEN..(u + 1) * ALIGNED_UNIT_LEN]; let opens = |p: &[u8], s: usize| self.opens(p, &unit_keys[s].1); if self.held(candidate).any(|s| opens(unit, s)) { - return Err(self.stop(at, "another held key opens the unit")); + return self.refuse(at, "another held key opens the unit"); } // No key held for a piece not resolved clear (a mux given no set): nothing to prove. if self.base == 0 && !self.pieces[span.3].2 { - return Err(self.stop(at, "no key held")); + return self.refuse(at, "no key held"); } // A clear piece has no partners and no key to be wrong: a flagged unit is damage. if self.pieces[span.3].2 { @@ -225,12 +235,12 @@ impl Arrival { return Ok(()); } if !batch.is_empty() || !side.is_empty() { - return Err(self.stop(at, "no held key opens the unit or a partner")); + return self.refuse(at, "no held key opens the unit or a partner"); } let mut strikes = self.strikes.lock().unwrap_or_else(|e| e.into_inner()); match strikes.get_mut(span.3) { Some(Some(prev)) if *prev != at => { - Err(self.stop(at, "a second partnerless unit no held key opens")) + self.refuse(at, "a second partnerless unit no held key opens") } Some(slot) => { *slot = Some(at); diff --git a/src/keys/mod.rs b/src/keys/mod.rs index da2719c0..221e7394 100644 --- a/src/keys/mod.rs +++ b/src/keys/mod.rs @@ -240,8 +240,6 @@ pub(crate) struct Inner { pub(crate) spans: Vec, // `resolve` found no stream file (no `/BDMV/STREAM` or `/HVDVD_TS`, or an empty one). pub(crate) no_stream_files: bool, - // A Lazy piece has intact encrypted probes and no held key opens any (KU §2.3 step 9.2). - pub(crate) lazy_unopened: bool, pub(crate) arrival: Vec, pub(crate) lazy: Vec<(u32, u32)>, pub(crate) proven: Vec, @@ -273,7 +271,6 @@ impl Inner { map: Arc::new(AacsKeyMap::from_ranges(Vec::new())), spans: Vec::new(), no_stream_files: false, - lazy_unopened: false, arrival: Vec::new(), lazy: Vec::new(), proven: Vec::new(), @@ -698,6 +695,39 @@ impl ResolvedKeySet { ResolvedKeySet(Arc::new(i)) } + // A loose clip file's set (no CPS-unit map): `held`'s base keys, proven on arrival over the + // file's one piece. Title scope, so a unit no held key opens stops E7022 (KU §2.4). + pub(crate) fn loose_file(held: Option<&ResolvedKeySet>, capacity: u32) -> Self { + let mut i = Inner::empty(); + i.aacs = true; + i.scope = KeyScope::Titles(vec![0]); + i.capacity = capacity; + if let Some(h) = held.filter(|h| h.is_aacs()) { + i.pool = h.0.pool.clone(); + i.disc_hash = h.0.disc_hash.clone(); + } + let piece = resolve::loose_file_piece(capacity); + i.spans = piece.spans.clone(); + i.lazy = piece + .spans + .iter() + .map(|&(s, n, _)| (s, s.saturating_add(n))) + .collect(); + i.arrival.push(piece); + Self(Arc::new(i)) + } + + // A set holding `keys` as base keys and no disc: a loose file's key set in tests. + #[cfg(test)] + pub(crate) fn held_for_test(keys: &[[u8; 16]]) -> Self { + let mut i = Inner::empty(); + i.aacs = true; + i.scope = KeyScope::WholeDisc; + i.pool = keys.to_vec(); + i.proven = (0..keys.len()).collect(); + ResolvedKeySet(Arc::new(i)) + } + /// The decrypting reader for title `idx` of `disc` over the raw, random-access `inner`: /// keyed pieces through the set's map, the rest proven on arrival (a readable unit no /// held key opens stops with E7022). E7013 if the set is not for `disc`, does not cover @@ -737,7 +767,7 @@ impl ResolvedKeySet { /// The whole-disc decrypting reader for a decrypted image or folder copy over the raw, /// random-access `inner`, on each stream file's own unit grid. A readable unit no held - /// key opens stops with E7032. E7013 if the set is not for `disc`, does not cover the + /// key opens is blanked and counted. E7013 if the set is not for `disc`, does not cover the /// whole disc, or `inner` cannot seek; E7026 if forensic keys are Pending. pub fn whole_disc_reader( &self, @@ -774,11 +804,6 @@ impl ResolvedKeySet { }; return Err(Error::UdfNotFound { path: path.into() }); } - if self.0.lazy_unopened { - // KU §2.1 (6) "Refuse first": a sweep reads that unit, and no held key opens it. - tracing::error!(target: "freemkv::keys", code = crate::error::E_WHOLE_DISC_KEY_MISSING, "a stream file holds a unit no held key opens: refusing the sweep before any output"); - return Err(Error::WholeDiscKeyMissing); - } let mut dec = self.decrypting(inner, None, StopKind::Image, false)?; content.extend(self.0.spans.iter().map(|&(s, n, _)| (s, n))); let content = crate::whole_disc::merge_ranges(content); diff --git a/src/keys/resolve.rs b/src/keys/resolve.rs index bab1572f..f723e26d 100644 --- a/src/keys/resolve.rs +++ b/src/keys/resolve.rs @@ -410,6 +410,26 @@ fn pieces(disc: &Disc, files: &[Vec<(u32, u32)>], sel: &[usize], whole: bool) -> out } +// A loose clip file's one piece, in the shape `pieces` builds for a disc: every aligned unit +// of the file (KS-1) on its own grid from byte 0, title 0, proven on arrival. +pub(crate) fn loose_file_piece(capacity: u32) -> super::ArrivalPiece { + let p = Piece { + spans: vec![(0, capacity, 0)], + titles: vec![0], + rank: 0, + units: 0, + enc: Vec::new(), + faults: 0, + verdict: Verdict::Lazy(None), + }; + super::ArrivalPiece { + id: p.id(), + spans: p.spans, + candidate: None, + clear: false, + } +} + // Each whole-disc piece's unit spans, for the unit-grid guards in `whole_disc_tests`. #[cfg(test)] pub(crate) fn whole_disc_pieces(disc: &Disc, files: &[Vec<(u32, u32)>]) -> Vec> { @@ -818,6 +838,18 @@ fn resolve_bd( }; } apply_single_unit_rule(run, &mut ps, n_decl)?; + // A whole-disc copy blanks a piece no held key opens (counted on arrival) while any + // other piece is usable; a source fault, an empty pool or no usable piece still refuses. + if *scope == KeyScope::WholeDisc + && !run.pool.is_empty() + && run.first_failure.is_none() + && ps.iter().any(|p| p.verdict != Verdict::Missing) + { + for p in ps.iter_mut().filter(|p| p.verdict == Verdict::Missing) { + tracing::warn!(target: "freemkv::keys", lba = p.id(), "no held key opens this stream file: it will be blanked in the image"); + p.verdict = Verdict::Lazy(None); + } + } if let Some(i) = ps.iter().position(|p| p.verdict == Verdict::Missing) { let lba = ps[i].id(); let failure = run.first_failure.take(); @@ -832,16 +864,6 @@ fn resolve_bd( } let mut inner = aacs_inner(run); inner.no_stream_files = files.is_empty(); - // Refused up front only if no probe of the piece opens: arrival blanks a unit whose batch - // partner opens (damage), and a damaged seed opens under no key (E7013 option A). - inner.lazy_unopened = ps.iter().any(|p| { - let opened = |u: &Vec| (0..run.pool.len()).any(|s| run.opens(u, s)); - matches!(p.verdict, Verdict::Lazy(_)) - && !p.enc.iter().any(opened) - && p.enc - .iter() - .any(|u| crate::aacs::content::aacs_unit_on_grid(u, format)) - }); // Step 6 (continued): forensic keys, reused from the seed or anchored once. let forensic = match &layout { None => None, diff --git a/src/keys/tests.rs b/src/keys/tests.rs index f1b8251f..c1f4ab80 100644 --- a/src/keys/tests.rs +++ b/src/keys/tests.rs @@ -1079,16 +1079,16 @@ fn on_arrival_proof() { ); assert_eq!(set.proof_cache().get(b), Some(Proof::Proven(1))); - // (b) no pool key opens it → E7022 (title) / E7032 (image); nothing asked. + // (b) no pool key opens it → E7022 (title) / blanked and counted (image); nothing asked. let (fx, set, src) = lazy_b(&[K1]); let mut r = set.title_reader(&fx.disc, 0, src.clone()).unwrap(); assert_eq!(code(read(&mut r, &fx, 1, 0, 4)), E7022); let mut w = set.whole_disc_reader(&fx.disc, src, None).unwrap(); let mut buf = vec![0u8; ALIGNED_UNIT_LEN]; - assert_eq!( - code(w.read_sectors(fx.unit(1, 0), 3, &mut buf, true)), - E7032 - ); + w.read_sectors(fx.unit(1, 0), 3, &mut buf, true) + .expect("an image blanks the unit, never stops"); + assert!(buf.iter().all(|&b| b == 0)); + assert_eq!(w.blanked_units(), 1); // (c) the piece's LAST unit: no forward units; the backward side read finds a partner. let (fx, set, src) = lazy_b(&[K1, K2]); @@ -1785,22 +1785,27 @@ fn on_arrival_proof_skips_forensic_segment_units() { // ── §3.1 surfaces: extract, input, mux, session ───────────────────────────── -/// LK11 (K-2). KS-1 [BD] §3.10.1: "encryption is applied to every Aligned Unit in the -/// file"; KS-10. A decrypted folder never keys a file with a key that does not open it: -/// `resolve(WholeDisc)` refuses up front (E7032), and a file left Lazy whose readable unit -/// no held key opens stops the extract (E7032), never written. +/// Nothing usable remains: a whole-disc copy where no stream file opens under any held key +/// still refuses E7032 before any output. #[test] -fn extract_tree_refuses_a_file_no_key_opens() { +fn whole_disc_refuses_when_no_file_opens() { let fx = two_units(); let calls = Calls::default(); - assert_eq!( - code(resolve( - &fx, - KeyScope::WholeDisc, - &[Spec::keydb(&[K1], &calls)] - )), - E7032 - ); + let r = resolve(&fx, KeyScope::WholeDisc, &[Spec::keydb(&[K3], &calls)]); + assert_eq!(code(r), E7032); +} + +/// LK11 (K-2). KS-1 [BD] §3.10.1: "encryption is applied to every Aligned Unit in the +/// file"; KS-10. A decrypted folder never keys a file with a key that does not open it: +/// `resolve(WholeDisc)` leaves it Lazy and its units are blanked and counted, never E7032. +#[test] +fn extract_tree_blanks_a_file_no_key_opens() { + let fx = two_units(); + let calls = Calls::default(); + let set = resolve(&fx, KeyScope::WholeDisc, &[Spec::keydb(&[K1], &calls)]) + .expect("one keyed file keeps the copy usable"); + assert_eq!(set.status().keyed, 1); + assert_eq!(set.lazy().len(), 1); let (fx, set, mut src) = lazy_b(&[K1]); let dest = tempfile::tempdir().unwrap(); @@ -1808,12 +1813,10 @@ fn extract_tree_refuses_a_file_no_key_opens() { keys: Some(&set), ..Default::default() }; - let r = fx.disc.extract_tree(&mut src, dest.path(), &opts); - assert_eq!(code(r), E7032); - assert!( - !dest.path().join("BDMV/STREAM/00002.m2ts").exists(), - "B is never written" - ); + let res = fx.disc.extract_tree(&mut src, dest.path(), &opts).unwrap(); + let got = std::fs::read(dest.path().join("BDMV/STREAM/00002.m2ts")).unwrap(); + assert!(got.iter().all(|&b| b == 0), "B is blanked, not ciphertext"); + assert!(!res.complete, "blanked units are not good bytes"); let (fx, set, mut src) = lazy_b(&[K1, K2]); let dest = tempfile::tempdir().unwrap(); @@ -1979,7 +1982,7 @@ fn session_resolves_a_key_set_through_its_reader() { } /// KU §2.5: a title scope — even every title (`-t all`) — covers only the files its titles -/// play. A file no title plays is a whole-disc concern (E7032 there), never a title refusal. +/// play. A file no title plays is a whole-disc concern (blanked there), never a title refusal. #[test] fn titles_scope_never_keys_a_file_no_title_plays() { let fx = fixture( @@ -1995,7 +1998,10 @@ fn titles_scope_never_keys_a_file_no_title_plays() { let keydb = [Spec::keydb(&[K1, K2], &calls)]; let set = resolve(&fx, KeyScope::Titles(vec![0, 1]), &keydb).unwrap(); assert_eq!(set.status().keyed, 2); - assert_eq!(code(resolve(&fx, KeyScope::WholeDisc, &keydb)), E7032); + // The file no title plays is a whole-disc concern: kept lazy there, blanked on read. + let whole = resolve(&fx, KeyScope::WholeDisc, &keydb).expect("damage, not E7032"); + assert_eq!(whole.status().keyed, 2); + assert_eq!(whole.lazy().len(), 1); } /// KS-25, KS-26 (evidence). A forensic segment whose index has no held key refuses (E7026): @@ -2744,6 +2750,8 @@ fn a_source_halted_mid_request_keeps_the_asked_step() { ); } +mod parity; +mod parity_live; mod stop_tests; /// LK21 (K-13), per spec — KS-5 [BD] §3.10.2: CPI "shall be set to 00₂ if the data is not @@ -2829,10 +2837,10 @@ fn whole_disc_reader_refuses_titles_without_a_stream_folder() { } /// KU §2.1 (6) "Refuse first": a whole-disc sweep reads every unit, so a Lazy piece -/// holding a readable encrypted probe no held key opens is refused E7032 by the reader, -/// before any output (engine `a_multi_cps_sweep_refuses_a_first_unit_no_key_opens_*`). +/// holding a readable encrypted probe no held key opens is blanked and counted by the +/// reader (E7013 option A policy), never refused E7032. #[test] -fn whole_disc_reader_refuses_a_lazy_piece_no_held_key_opens() { +fn whole_disc_reader_blanks_a_lazy_piece_no_held_key_opens() { let mut fx = fixture(&[stream(1, 10, Some(K1)), stream(2, 10, None)], 2, &[&[0]]); let at = fx.unit(1, 0) as usize * 2048; // An intact unit: TS sync at byte 4 of each packet, so its seed is not damage (KS-4). @@ -2848,18 +2856,18 @@ fn whole_disc_reader_refuses_a_lazy_piece_no_held_key_opens() { &[(b, b + n)], "one unopened unit: Lazy (step 9.2)" ); - assert_eq!( - code( - set.whole_disc_reader(&fx.disc, fx.source(), None) - .map(|_| ()) - ), - E7032 - ); + let mut w = set + .whole_disc_reader(&fx.disc, fx.source(), None) + .expect("a piece no key opens is blanked, not refused"); + let mut buf = vec![0xAAu8; ALIGNED_UNIT_LEN]; + w.read_sectors(fx.unit(1, 0), 3, &mut buf, true).unwrap(); + assert!(buf.iter().all(|&b| b == 0)); + assert_eq!(w.blanked_units(), 1); // A title rip that never reads B is unaffected. assert!(set.title_reader(&fx.disc, 0, fx.source()).is_ok()); } -/// E7013 option A with the E7032 refusal above: a probe whose seed is damaged (no TS sync, +/// E7013 option A, like the blanking above: a probe whose seed is damaged (no TS sync, /// KS-4 [BD] §3.10.1 "The first 16 bytes of each Aligned Unit is used as the seed") opens /// under no key and is blanked on read, never a stop, so it refuses no sweep up front. #[test] diff --git a/src/keys/tests/parity.rs b/src/keys/tests/parity.rs new file mode 100644 index 00000000..67aa40a1 --- /dev/null +++ b/src/keys/tests/parity.rs @@ -0,0 +1,447 @@ +//! Parity goldens for the image and live-reader paths over synthetic muxable AACS discs: +//! the whole-disc ISO copy, the decrypted folder, the title reader and every mux input +//! (`Iso`, `Live`) × sink, with loss counters and key-gate verdicts. Cells are `parity_*`; +//! see [`Golden`](crate::test_util::Golden) for the re-bless command. + +use super::*; +use crate::mux::driver::{MuxOptions, MuxSource, NoopEvents, mux_with_keys}; +use crate::mux::parity_tests::{golden, record_run, record_tree}; +use crate::mux::select::{PidFilter, StreamSelection}; +use crate::test_util::{CLIP_AUDIO_PIDS, Golden, synthetic_bd_clip}; + +fn refused(g: &mut Golden, key: &str, r: &Result) -> bool { + match r { + Ok(_) => false, + Err(e) => { + g.kv(key, format_args!("refused E{}", e.code())); + true + } + } +} + +// A disc whose title 0 plays one clip file per entry of `keys` (`None`: a clear file), +// declaring that many CPS units, with the clip's streams on the title. +fn clip_fx(keys: &[Option<[u8; 16]>]) -> Fx { + let clip = synthetic_bd_clip(6); + let files: Vec = keys + .iter() + .enumerate() + .map(|(i, k)| { + BdFile::new(format!("BDMV/STREAM/{:05}.m2ts", i + 1), 0, *k).with_clip(clip.clone()) + }) + .collect(); + let all: Vec = (0..keys.len()).collect(); + let mut fx = fixture(&files, keys.len(), &[&all]); + fx.disc.titles[0].streams = crate::mux::ts::scan_streams(&clip).unwrap(); + fx +} + +fn write_iso(fx: &Fx, dir: &std::path::Path) -> std::path::PathBuf { + let p = dir.join("disc.iso"); + std::fs::write(&p, &fx.img.image).unwrap(); + p +} + +// One whole-disc ISO copy through `reader`: the image bytes (or the failure) and the +// reader's blanked-unit count. +fn copy_image(g: &mut Golden, tag: &str, fx: &Fx, reader: &mut dyn SectorSource) { + let dir = tempfile::tempdir().unwrap(); + let dest = dir.path().join("out.iso"); + let cap = fx.disc.capacity_sectors; + let r = crate::io::image_writer::write_image(reader, &dest, cap, &Halt::new(), |_| {}); + match r { + Ok(n) => { + g.kv(&format!("{tag} written"), n); + g.bytes(&format!("{tag} image"), &std::fs::read(&dest).unwrap()); + } + Err(e) => { + g.kv(&format!("{tag} failed"), format_args!("E{}", e.code())); + } + } +} + +fn extract(g: &mut Golden, tag: &str, fx: &Fx, keys: Option<&ResolvedKeySet>, src: Faulty) { + let dest = tempfile::tempdir().unwrap(); + let opts = crate::disc::ExtractOptions { + keys, + ..Default::default() + }; + let mut src = src; + match fx.disc.extract_tree(&mut src, dest.path(), &opts) { + Ok(r) => { + g.kv( + &format!("{tag} result"), + format_args!( + "good={} unreadable={} complete={} halted={} files={:?}", + r.bytes_good, + r.bytes_unreadable, + r.complete, + r.halted, + r.files + .iter() + .map(|f| ( + f.path.display().to_string(), + f.bytes_good, + f.bytes_unreadable, + f.complete + )) + .collect::>() + ), + ); + record_tree(g, tag, dest.path()); + } + Err(e) => { + g.kv(&format!("{tag} failed"), format_args!("E{}", e.code())); + } + } +} + +fn mux_cells(g: &mut Golden, fx: &Fx, set: Option<&ResolvedKeySet>, dead: Option<(u32, u32)>) { + let _serial = crate::sector::prefetched::holder_test_lock(); + let dir = tempfile::tempdir().unwrap(); + let iso = write_iso(fx, dir.path()); + let title = fx.disc.titles[0].clone(); + let format = fx.disc.content_format; + let base = MuxOptions { + batch_sectors: 30, + ..Default::default() + }; + let selected = MuxOptions { + batch_sectors: 30, + selection: StreamSelection { + audio: PidFilter::Only(vec![CLIP_AUDIO_PIDS[1]]), + subtitle: PidFilter::Only(vec![]), + }, + ..Default::default() + }; + let raw = MuxOptions { + raw: true, + batch_sectors: 30, + ..Default::default() + }; + let skip = MuxOptions { + skip_errors: true, + batch_sectors: 30, + ..Default::default() + }; + let run = |g: &mut Golden, tag: &str, sink: &str, live: bool, opts: &MuxOptions| { + let out = dir.path().join(format!("out-{tag}")); + std::fs::create_dir_all(&out).unwrap(); + let dest = match sink { + "null" => "null://".to_string(), + s => format!("{s}://{}", out.join(format!("o.{s}")).display()), + }; + let source = if live { + let src = fx.source(); + if let Some((a, b)) = dead { + src.kill(a, b); + } + MuxSource::Live { + reader: Box::new(src), + title: title.clone(), + format, + } + } else { + MuxSource::Iso { + path: &iso, + title: title.clone(), + format, + } + }; + let r = mux_with_keys(source, set, &dest, opts, &Halt::new(), Arc::new(NoopEvents)); + record_run(g, tag, &r); + record_tree(g, tag, &out); + }; + for sink in ["mkv", "m2ts", "null"] { + run(g, &format!("iso-{sink}"), sink, false, &base); + } + run(g, "iso-mkv-raw", "mkv", false, &raw); + run(g, "iso-mkv-selection", "mkv", false, &selected); + run(g, "live-mkv", "mkv", true, &base); + run(g, "live-mkv-raw", "mkv", true, &raw); + if dead.is_some() { + run(g, "live-mkv-skip", "mkv", true, &skip); + } +} + +// The full matrix for one disc: the whole-disc copy (decrypting and raw), the decrypted +// folder and the raw folder, the title reader, then every mux input and sink. +fn image_cell(name: &str, fx: &Fx, pool: &[[u8; 16]], dead: Option<(u32, u32)>) { + let mut g = golden(name); + let src = || { + let s = fx.source(); + if let Some((a, b)) = dead { + s.kill(a, b); + } + s + }; + let calls = Calls::default(); + let specs = [Spec::keydb(pool, &calls)]; + let whole = resolve_with( + fx, + &mut fx.source(), + KeyScope::WholeDisc, + &specs, + ResolveKeysOptions::default(), + &FakeClock::default(), + ); + if !refused(&mut g, "resolve-whole", &whole) { + let set = whole.as_ref().unwrap(); + g.kv("resolve-whole status", format_args!("{:?}", set.status())); + match set.whole_disc_reader(&fx.disc, src(), None) { + Ok(mut w) => { + copy_image(&mut g, "copy", fx, &mut w); + g.kv("copy blanked", w.blanked_units()); + } + Err(e) => { + g.kv("copy", format_args!("refused E{}", e.code())); + } + } + extract(&mut g, "extract", fx, Some(set), src()); + } + let mut raw = crate::whole_disc::raw_whole_disc_reader(src()); + copy_image(&mut g, "copy-raw", fx, &mut raw); + extract(&mut g, "extract-raw", fx, None, src()); + let titles = resolve_with( + fx, + &mut fx.source(), + KeyScope::Titles(vec![0]), + &specs, + ResolveKeysOptions::default(), + &FakeClock::default(), + ); + if !refused(&mut g, "resolve-title", &titles) { + let set = titles.as_ref().unwrap(); + g.kv("resolve-title status", format_args!("{:?}", set.status())); + match set.title_reader(&fx.disc, 0, src()) { + Ok(mut r) => { + let mut all = Vec::new(); + let mut failed = None; + for e in &fx.disc.titles[0].extents { + let mut at = e.start_lba; + let end = e.start_lba + e.sector_count; + r.set_unit_base(e.start_lba); + while at < end { + let n = (end - at).min(30); + let mut buf = vec![0u8; n as usize * 2048]; + match r.read_sectors(at, n as u16, &mut buf, true) { + Ok(_) => all.extend_from_slice(&buf), + Err(e) => { + failed = Some(e.code()); + break; + } + } + at += n; + } + } + g.bytes("title-read", &masked(&all)); + g.kv("title-read blanked", r.blanked_units()); + if let Some(c) = failed { + g.kv("title-read failed", format_args!("E{c}")); + } + } + Err(e) => { + g.kv("title-read", format_args!("refused E{}", e.code())); + } + } + mux_cells(&mut g, fx, Some(set), dead); + } else { + // No key set: the mux still runs keyless and refuses on the first AACS unit. + mux_cells(&mut g, fx, None, dead); + } + g.check(); +} + +#[test] +fn parity_image_bd_one_cps() { + image_cell( + "parity_image_bd_one_cps", + &clip_fx(&[Some(K1)]), + &[K1], + None, + ); +} + +#[test] +fn parity_image_bd_multi_cps() { + let fx = clip_fx(&[Some(K1), Some(K2)]); + image_cell("parity_image_bd_multi_cps", &fx, &[K1, K2], None); +} + +#[test] +fn parity_image_bd_clear_files() { + image_cell( + "parity_image_bd_clear_files", + &clip_fx(&[None, None]), + &[K1], + None, + ); +} + +// A held key set that opens only one of the two files refuses before any output. +#[test] +fn parity_image_bd_missing_key() { + let fx = clip_fx(&[Some(K1), Some(K2)]); + image_cell("parity_image_bd_missing_key", &fx, &[K1], None); +} + +// Seed-damaged units inside a keyed file: blanked and counted on every path (E7013 option A). +#[test] +fn parity_image_bd_damaged_units() { + let mut fx = clip_fx(&[Some(K1), Some(K2)]); + for u in [1, 2, 4] { + damage_seed(&mut fx, 1, u); + } + image_cell("parity_image_bd_damaged_units", &fx, &[K1, K2], None); +} + +// A read-failure map: a sector range that fails to read, over a keyed file. +#[test] +fn parity_image_bd_bad_sectors() { + let fx = clip_fx(&[Some(K1)]); + let (b, _) = fx.file(0); + image_cell( + "parity_image_bd_bad_sectors", + &fx, + &[K1], + Some((b + 9, b + 12)), + ); +} + +// A UHD disc whose AACS state says bus encryption: an image carries it removed or not at all. +#[test] +fn parity_image_uhd_bus_flag() { + let mut fx = clip_fx(&[Some(K1)]); + let a = fx.disc.aacs.take().unwrap(); + fx.disc.aacs = Some( + aacs_state() + .disc_hash(HASH) + .volume_id(VID) + .uk_ro(a.uk_ro.clone()) + .bus_encryption(true) + .build(), + ); + image_cell("parity_image_uhd_bus_flag", &fx, &[K1], None); +} + +// The FMTS alternate-phase disc: the forensic clip's even units open under the held phase, +// the alternate phase stays ciphertext (read through the title reader, never muxed). +#[test] +fn parity_image_fmts_alternate_phase() { + let mut g = golden("parity_image_fmts_alternate_phase"); + for held_phase in [0usize, 1] { + let mut fx = fmts_fixture(); + if held_phase == 1 { + for (a, b, key) in [(0, 16, F1), (20, 36, F2)] { + for u in a..b { + fx.reencrypt(1, u, if u % 2 == 1 { &key } else { &ALT }); + } + } + } + let calls = Calls::default(); + let mut source = fmts_online(&calls); + source.keys = vec![K1, K2]; + source.match_fmts_phase = true; + let set = resolve(&fx, KeyScope::Titles(vec![2]), &[source]).unwrap(); + g.kv( + &format!("phase{held_phase} status"), + format_args!("{:?}", set.status()), + ); + let mut r = set.title_reader(&fx.disc, 2, fx.source()).unwrap(); + let got = read(&mut r, &fx, 1, 0, 16).unwrap(); + g.bytes(&format!("phase{held_phase} forensic-clip"), &got); + let mut w = set.whole_disc_reader(&fx.disc, fx.source(), None); + match &mut w { + Ok(w) => copy_image(&mut g, &format!("phase{held_phase} copy"), &fx, w), + Err(e) => { + g.kv( + &format!("phase{held_phase} copy"), + format_args!("refused E{}", e.code()), + ); + } + } + } + g.check(); +} + +// An HD DVD whose single content file is clear: the whole-disc copy and folder are byte +// copies of the image. +#[test] +fn parity_image_hddvd_clear() { + let files = [ + BdFile::new("HVDVD_TS/FEATURE.EVO", 30, None), + BdFile::new("BDMV/index.bdmv", 1, None), + ]; + let uk_ro = unit_key_ro(AacsVersion::V10, &[[0xEE; 16]], &[1]); + let img = encrypted_bd_image(&files, &uk_ro); + let disc = disc_over(&img, &uk_ro, &[&[0]], DiscFormat::HdDvd); + let fx = Fx { img, disc }; + let mut g = golden("parity_image_hddvd_clear"); + let calls = Calls::default(); + let set = resolve(&fx, KeyScope::WholeDisc, &[Spec::keydb(&[K1], &calls)]); + if !refused(&mut g, "resolve-whole", &set) { + let set = set.unwrap(); + g.kv("resolve-whole status", format_args!("{:?}", set.status())); + let mut w = set.whole_disc_reader(&fx.disc, fx.source(), None).unwrap(); + copy_image(&mut g, "copy", &fx, &mut w); + extract(&mut g, "extract", &fx, Some(&set), fx.source()); + } + g.check(); +} + +// A DVD of two title sets, each VOB scrambled under its own title key: the folder re-cracks +// per VTS from the disc-wide key's start. +fn dvd_fx(live: bool) -> (Fx, Vec, Vec) { + use crate::mux::parity_tests::{clear_mpg, scramble_vob}; + let pad = |mut v: Vec| { + v.resize(v.len().div_ceil(ALIGNED_UNIT_LEN) * ALIGNED_UNIT_LEN, 0); + v + }; + let clear = pad(clear_mpg()); + let (k1, k2) = ( + [0x42, 0x13, 0x37, 0xBE, 0xEF], + [0x17, 0x71, 0x29, 0x93, 0x05], + ); + let files = [ + BdFile::new("VIDEO_TS/VTS_01_1.VOB", 0, None).with_clip(scramble_vob(&clear, &k1)), + BdFile::new("VIDEO_TS/VTS_02_1.VOB", 0, None).with_clip(scramble_vob(&clear, &k2)), + ]; + let uk_ro = unit_key_ro(AacsVersion::V10, &[[0xEE; 16]], &[1]); + let img = encrypted_bd_image(&files, &uk_ro); + let mut disc = disc_over(&img, &uk_ro, &[&[0, 1]], DiscFormat::Dvd); + disc.aacs = None; + disc.encrypted = true; + disc.content_format = ContentFormat::MpegPs; + // A live scan leaves `css` unset; a scanned image carries the first VTS's cracked key. + disc.css = (!live).then_some(crate::css::CssState { + title_key: k1, + crack_span: None, + }); + (Fx { img, disc }, clear, k1.to_vec()) +} + +#[test] +fn parity_image_dvd_css_multi_vts() { + let (fx, _, _) = dvd_fx(false); + let mut g = golden("parity_image_dvd_css_multi_vts"); + extract(&mut g, "extract", &fx, None, fx.source()); + g.check(); +} + +// BUG-1 (pipeline design §1.4): a live-scanned DVD has no `disc.css`, so the folder must +// still come out descrambled. Red today (the VOBs are written scrambled); ignored until the +// decrypt stage detects CSS by content (slice 3b). +#[test] +#[ignore = "BUG-1: live DVD -> dir never engages CSS (fixed in slice 3b)"] +fn parity_bug1_live_dvd_folder_is_descrambled() { + let (fx, clear, _) = dvd_fx(true); + let dest = tempfile::tempdir().unwrap(); + let mut src = fx.source(); + fx.disc + .extract_tree(&mut src, dest.path(), &Default::default()) + .unwrap(); + for vts in ["VTS_01_1.VOB", "VTS_02_1.VOB"] { + let got = std::fs::read(dest.path().join("VIDEO_TS").join(vts)).unwrap(); + assert!(got[..clear.len()] == clear[..], "{vts} is still scrambled"); + } +} diff --git a/src/keys/tests/parity_live.rs b/src/keys/tests/parity_live.rs new file mode 100644 index 00000000..e9013286 --- /dev/null +++ b/src/keys/tests/parity_live.rs @@ -0,0 +1,245 @@ +//! Parity goldens for the live path: a real `Drive` over the in-crate `FakeTransport` serving +//! a synthetic AACS disc, scanned, keyed (clean and lazy) and muxed through a `DiscSession`. +//! Bad sectors, a stop mid-read and the keyless refusal are pinned as counters and verdicts. + +use super::*; +use crate::dirimage::tests::{minimal_clpi, one_item_mpls}; +use crate::disc::ScanOptions; +use crate::drive::Drive; +use crate::mux::driver::{MuxOptions, MuxSource, NoopEvents, mux_with_keys}; +use crate::mux::parity_tests::{golden, record_run, record_tree}; +use crate::session::{DiscSession, KeySpec}; +use crate::test_util::{FakeMode, FakeTransport, Golden, synthetic_bd_clip}; + +// A scannable one-title BD whose only stream file is a K1 clip of `UNITS` units. +fn disc_image() -> (EncryptedBdImage, Vec) { + let clip = synthetic_bd_clip(6); + let uk_ro = unit_key_ro(AacsVersion::V10, &[[0xEE; 16]], &[1]); + let files = [ + BdFile::new("BDMV/index.bdmv", 1, None), + BdFile::new("BDMV/PLAYLIST/00000.mpls", 1, None), + BdFile::new("BDMV/CLIPINF/00000.clpi", 1, None), + BdFile::new("BDMV/STREAM/00000.m2ts", 0, Some(K1)).with_clip(clip.clone()), + ]; + let mut img = encrypted_bd_image(&files, &uk_ro); + let packets = (clip.len() / 192) as u32; + for (i, bytes) in [(1, one_item_mpls(b"00000")), (2, minimal_clpi(packets))] { + let at = img.files[i].0 as usize * 2048; + img.image[at..at + bytes.len()].copy_from_slice(&bytes); + } + (img, clip) +} + +fn is_read10(c: &[u8]) -> bool { + c[0] == crate::scsi::SCSI_READ_10 +} + +// Whether a READ(10) overlaps `[a, b)`. +fn reads(c: &[u8], a: u32, b: u32) -> bool { + if !is_read10(c) { + return false; + } + let lba = u32::from_be_bytes([c[2], c[3], c[4], c[5]]); + let n = u16::from_be_bytes([c[7], c[8]]) as u32; + lba < b && lba + n > a +} + +// A scanned session over a fake drive serving `img`, the clip's streams put on title 0. +fn session(t: FakeTransport, halt: &Halt, clip: &[u8]) -> DiscSession { + let drive = Drive::from_transport_with(Box::new(t), halt); + let mut s = DiscSession::bring_up(drive, KeySpec::default(), Some(halt.clone())).unwrap(); + s.scan(ScanOptions::default()).expect("the fake disc scans"); + s.disc_mut().unwrap().titles[0].streams = crate::mux::ts::scan_streams(clip).unwrap(); + // The UDF volume id carries the fixture's temp-dir name, which varies per run. + s.disc_mut().unwrap().volume_id = "PARITY".into(); + s.stage_drive_as_reader(); + s +} + +fn medium_error() -> FakeMode { + FakeMode::Sense { + sense: crate::scsi::ScsiSense { + sense_key: 3, + asc: 0x11, + ascq: 0x00, + }, + progress: None, + } +} + +// Mux title 0 of `s` to mkv under `set`, recording the outcome and output. +fn mux( + g: &mut Golden, + tag: &str, + s: &mut DiscSession, + set: Option<&ResolvedKeySet>, + opts: &MuxOptions, + halt: &Halt, +) { + let dir = tempfile::tempdir().unwrap(); + let dest = format!("mkv://{}", dir.path().join("o.mkv").display()); + let r = mux_with_keys( + MuxSource::Session { + session: s, + title_index: 0, + }, + set, + &dest, + opts, + halt, + Arc::new(NoopEvents), + ); + record_run(g, tag, &r); + record_tree(g, tag, dir.path()); +} + +fn opts(skip: bool) -> MuxOptions { + MuxOptions { + skip_errors: skip, + batch_sectors: 30, + ..Default::default() + } +} + +fn keys(s: &mut DiscSession, kdb: &[[u8; 16]]) -> crate::error::Result { + let calls = Calls::default(); + let f = factory(&[Spec::keydb(kdb, &calls)]); + s.resolve_key_set(KeyScope::Titles(vec![0]), &f, ResolveKeysOptions::default()) + .map(|r| r.keys) +} + +#[test] +fn parity_live_bd_session() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let (img, clip) = disc_image(); + let mut g = golden("parity_live_bd_session"); + let halt = Halt::new(); + let (t, _h) = FakeTransport::new(); + let mut s = session(t.with_image(img.image.clone()), &halt, &clip); + g.kv( + "scan", + format_args!("titles={}", s.disc().unwrap().titles.len()), + ); + let set = keys(&mut s, &[K1]).unwrap(); + g.kv("keys", format_args!("{:?}", set.status())); + mux(&mut g, "keyed", &mut s, Some(&set), &opts(false), &halt); + let (t, _h) = FakeTransport::new(); + let mut s2 = session(t.with_image(img.image.clone()), &halt, &clip); + let (t, _h) = FakeTransport::new(); + let mut s = session(t.with_image(img.image.clone()), &halt, &clip); + mux(&mut g, "keyless", &mut s, None, &opts(false), &halt); + mux( + &mut g, + "keyed-again", + &mut s2, + Some(&set), + &opts(false), + &halt, + ); + let (t, _h) = FakeTransport::new(); + let mut s = session(t.with_image(img.image.clone()), &halt, &clip); + let raw = MuxOptions { + raw: true, + ..opts(false) + }; + mux(&mut g, "raw", &mut s, None, &raw, &halt); + g.check(); +} + +// The key set resolved while the file's probes failed to read: the piece is lazy and is +// proven on arrival while the live mux reads it. +#[test] +fn parity_live_bd_lazy_piece() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let (img, clip) = disc_image(); + let mut g = golden("parity_live_bd_lazy_piece"); + let halt = Halt::new(); + let (t, _h) = FakeTransport::new(); + let mut s = session(t.with_image(img.image.clone()), &halt, &clip); + let (start, n) = img.files[3]; + let dead = Faulty::new(MemSource::new(img.image.clone())); + dead.kill(start, start + n); + let calls = Calls::default(); + let f = factory(&[Spec::keydb(&[K1], &calls)]); + let (t2, _h2) = FakeTransport::new(); + let disc = session(t2.with_image(img.image.clone()), &halt, &clip) + .take_disc() + .unwrap(); + let set = ResolvedKeySet::resolve( + &disc, + &mut dead.clone(), + KeyScope::Titles(vec![0]), + &f, + ResolveKeysOptions::default(), + ) + .unwrap() + .keys; + g.kv("keys", format_args!("{:?}", set.status())); + mux(&mut g, "lazy", &mut s, Some(&set), &opts(false), &halt); + g.check(); +} + +// Media errors inside the clip: strict ends the mux, skip zero-fills and counts. +#[test] +fn parity_live_bd_bad_sectors() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let (img, clip) = disc_image(); + let mut g = golden("parity_live_bd_bad_sectors"); + let (start, _) = img.files[3]; + for (tag, skip) in [("strict", false), ("skip", true)] { + let halt = Halt::new(); + let (t, _h) = FakeTransport::new(); + let bad = (start + 9, start + 12); + let t = t + .with_image(img.image.clone()) + .rule(move |c| reads(c, bad.0, bad.1), medium_error()); + let mut s = session(t, &halt, &clip); + let set = keys(&mut s, &[K1]).unwrap(); + mux(&mut g, tag, &mut s, Some(&set), &opts(skip), &halt); + } + g.check(); +} + +// A Stop raised by the first read of the clip: the mux ends not-completed, never an error. +#[test] +fn parity_live_bd_stop_mid_read() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let (img, clip) = disc_image(); + let mut g = golden("parity_live_bd_stop_mid_read"); + let (start, _) = img.files[3]; + let halt = Halt::new(); + let (t, _h) = FakeTransport::new(); + // Armed once the key set exists, so the Stop lands in the mux's own reads. + let armed = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let seen = std::sync::atomic::AtomicUsize::new(0); + let gate = armed.clone(); + let t = t.with_image(img.image.clone()).cancel_on( + move |c| { + gate.load(std::sync::atomic::Ordering::Relaxed) + && reads(c, start, start + 36) + && seen.fetch_add(1, std::sync::atomic::Ordering::Relaxed) == 0 + }, + &halt, + ); + let mut s = session(t, &halt, &clip); + let set = keys(&mut s, &[K1]).unwrap(); + armed.store(true, std::sync::atomic::Ordering::Relaxed); + let dir = tempfile::tempdir().unwrap(); + let r = mux_with_keys( + MuxSource::Session { + session: &mut s, + title_index: 0, + }, + Some(&set), + &format!("mkv://{}", dir.path().join("o.mkv").display()), + &MuxOptions { + batch_sectors: 2, + ..Default::default() + }, + &halt, + Arc::new(NoopEvents), + ); + let o = r.as_ref().map(|o| (o.completed, o.errors, o.lost_bytes)); + g.kv("stop", format_args!("{o:?}")); + g.check(); +} diff --git a/src/lib.rs b/src/lib.rs index 0a5d2a48..32bd83d1 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -230,7 +230,7 @@ pub use mux::WriteSeek; #[cfg(feature = "rip")] pub use mux::{FitReport, SkipReason, fit_report}; #[cfg(feature = "rip")] -pub use mux::{InputOptions, StreamUrl, input, output, parse_url}; +pub use mux::{InputOptions, StreamUrl, disc_root_of, input, output, parse_url}; #[cfg(feature = "rip")] pub use mux::{Medium, SourceInfo}; #[cfg(feature = "rip")] diff --git a/src/mpls.rs b/src/mpls.rs index c2461d05..1165e5bc 100644 --- a/src/mpls.rs +++ b/src/mpls.rs @@ -93,7 +93,8 @@ pub struct StreamEntry { /// Parse an MPLS file from raw bytes. /// /// `data` is the raw contents of a `BDMV/PLAYLIST/*.mpls` file. Returns -/// [`Error::MplsParse`] on malformed or truncated input. +/// [`Error::MplsParse`] on a malformed header, or when truncation leaves no play item; +/// a truncated item list keeps the items before the damage (warned). /// /// Note: [`Playlist::streams`] is extracted ONLY from the first play /// item's STN table. Multi-item playlists whose later items carry a @@ -124,14 +125,18 @@ pub fn parse(data: &[u8]) -> Result { let mut play_items = Vec::with_capacity(num_play_items.min(256)); let mut streams = Vec::new(); let mut pos = 10; + let mut truncated = false; for item_idx in 0..num_play_items { + // A truncated item ends the list; the items parsed so far are kept. if pos + 2 > pl.len() { - return Err(Error::MplsParse); + truncated = true; + break; } let item_length = u16::from_be_bytes([pl[pos], pl[pos + 1]]) as usize; if pos + 2 + item_length > pl.len() { - return Err(Error::MplsParse); + truncated = true; + break; } let item = &pl[pos + 2..pos + 2 + item_length]; @@ -283,6 +288,18 @@ pub fn parse(data: &[u8]) -> Result { pos += 2 + item_length; } + if truncated { + if play_items.is_empty() { + return Err(Error::MplsParse); + } + tracing::warn!( + target: "freemkv::mpls", + kept = play_items.len(), + declared = num_play_items, + "truncated playlist: keeping the play items parsed so far" + ); + } + // Parse PlayListMark section let mut marks = Vec::new(); // The first real read is num_marks at ms[4..6], so the section needs @@ -835,6 +852,26 @@ mod tests { assert!(parse(&[0u8; 39]).is_err()); } + #[test] + fn parse_truncated_play_item_keeps_the_items_before_it() { + let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None); + let data = build_mpls( + &[ + (b"00001", 1, 90000, 4500000), + (b"00002", 5, 4500000, 9000000), + ], + (1, 0, 0, 0, 0, 0, 0, 0), + &[video], + ); + let ps = u32::from_be_bytes([data[8], data[9], data[10], data[11]]) as usize; + let first = u16::from_be_bytes([data[ps + 10], data[ps + 11]]) as usize; + // Cut inside the second item: one whole item survives. + let cut = &data[..ps + 10 + 2 + first + 10]; + let playlist = parse(cut).expect("damage must not drop the parsed items"); + assert_eq!(playlist.play_items.len(), 1); + assert_eq!(playlist.play_items[0].clip_id, "00001"); + } + #[test] fn parse_multiple_play_items() { let video = build_stream_entry_video(0x1011, 0x1B, 6, 1, None); @@ -1819,14 +1856,14 @@ mod tests { } #[test] - fn truncated_play_item_is_an_error() { + fn truncated_play_item_errors_only_when_nothing_is_left() { let full = build_mpls_raw_items(&[play_item_20(b"00001", 1, 0, 9000)]); // Cut into the item body: declared item_length runs past EOF. assert!(parse(&full[..full.len() - 3]).is_err()); // num_play_items overshoots the items actually present. let mut over = full.clone(); over[46..48].copy_from_slice(&2u16.to_be_bytes()); - assert!(parse(&over).is_err()); + assert_eq!(parse(&over).expect("kept").play_items.len(), 1); } #[test] diff --git a/src/mux/codec/adts.rs b/src/mux/codec/adts.rs index 5515ebea..25459165 100644 --- a/src/mux/codec/adts.rs +++ b/src/mux/codec/adts.rs @@ -80,6 +80,72 @@ fn adts_frame_ns(d: &[u8]) -> Option { Some(1024 * (u64::from(d[6] & 3) + 1) * 1_000_000_000 / u64::from(rate)) } +/// The ADTS header (MPEG-4 ID, no CRC, one raw_data_block, frame_length 0) for the access +/// units an AudioSpecificConfig describes (14496-3 §1.6.2.1); `None` when ADTS cannot signal +/// it: no 2-bit profile, an explicit rate, an in-band PCE layout or 960-sample frames +/// (GASpecificConfig frameLengthFlag). SBR/PS signal the core. +pub(crate) fn adts_template(asc: &[u8]) -> Option<[u8; ADTS_HEADER_BYTES]> { + let (b0, b1) = (*asc.first()?, *asc.get(1)?); + let rate = ((b0 & 7) << 1) | (b1 >> 7); + let channels = (b1 >> 3) & 0x0F; + let (object, frame_960) = match b0 >> 3 { + // extensionSamplingFrequencyIndex (4 bits), then the core audioObjectType. + 5 | 29 => { + let b2 = *asc.get(2)?; + let ext = ((b1 & 7) << 1) | (b2 >> 7); + (ext != 0x0F).then_some(((b2 >> 2) & 0x1F, b2 & 0x02 != 0))? + } + o => (o, b1 & 0x04 != 0), + }; + if frame_960 || !(1..=4).contains(&object) || rate > 0x0B || !(1..=7).contains(&channels) { + return None; + } + Some([ + 0xFF, + 0xF1, + ((object - 1) << 6) | (rate << 2) | (channels >> 2), + (channels & 3) << 6, + 0, + 0x1F, + 0xFC, + ]) +} + +/// raw_data_block()s in an access unit lasting `duration_ns` at `template`'s rate: an +/// ADTS source frame keeps all of its 1..=4 blocks. 1 unless the duration is within a +/// sample of a whole number of 1024-sample blocks. +pub(crate) fn raw_data_blocks(template: [u8; ADTS_HEADER_BYTES], duration_ns: Option) -> u8 { + let rate = u64::from(ADTS_SAMPLE_RATE_VALID[usize::from((template[2] >> 2) & 15)]); + let Some(scaled) = duration_ns.and_then(|d| d.checked_mul(rate)) else { + return 1; + }; + let n = scaled.saturating_add(512_000_000_000) / 1_024_000_000_000; + if (1..=4).contains(&n) && scaled.abs_diff(n * 1_024_000_000_000) <= 1_000_000_000 { + n as u8 + } else { + 1 + } +} + +/// `raw` (`blocks` raw_data_block()s) as one ADTS frame under `template`; `None` past the +/// 13-bit frame_length. +pub(crate) fn adts_frame( + template: [u8; ADTS_HEADER_BYTES], + raw: &[u8], + blocks: u8, +) -> Option> { + let len = ADTS_HEADER_BYTES + raw.len(); + if len >= 1 << 13 || !(1..=4).contains(&blocks) { + return None; + } + let mut h = template; + h[6] |= blocks - 1; + h[3] |= (len >> 11) as u8; + h[4] = (len >> 3) as u8; + h[5] |= ((len & 7) << 5) as u8; + Some([&h[..], raw].concat()) +} + pub struct AdtsParser { frames: AudioFrames, config: Option>, @@ -359,6 +425,18 @@ mod tests { assert_eq!(p.config_changes(), 0); } + // The block count comes only from a duration of whole 1024-sample blocks. + #[test] + fn raw_data_blocks_trusts_only_whole_block_durations() { + let t = adts_template(&[0x11, 0x90]).unwrap(); + let ns = |samples: u64| Some(samples * 1_000_000_000 / 48_000); + assert_eq!(raw_data_blocks(t, ns(2048)), 2); + assert_eq!(raw_data_blocks(t, ns(4096)), 4); + for d in [None, ns(1536), ns(5120), Some(u64::MAX)] { + assert_eq!(raw_data_blocks(t, d), 1, "{d:?}"); + } + } + #[test] fn multi_block_frame_with_crc_is_refused() { let mut p = AdtsParser::new(); diff --git a/src/mux/codec/lpcm.rs b/src/mux/codec/lpcm.rs index cc1dd722..9bfc5c2a 100644 --- a/src/mux/codec/lpcm.rs +++ b/src/mux/codec/lpcm.rs @@ -4,7 +4,7 @@ //! 3 on DVD-PS (`PsDemuxer` strips only the sub_id/frames/pointer bytes). Output is //! interleaved big-endian PCM in WAVE_FORMAT_EXTENSIBLE channel order at the source //! depth: 16-bit stays 16-bit, 20- and 24-bit sources output 24-bit. The depth rides -//! in `codec_private` ([`output_depth`]) so "A_PCM/INT/BIG" BitDepth matches. Layouts follow ffmpeg pcm-bluray.c (pad channel, LFE/surround +//! in `codec_private` (`output_depth`) so "A_PCM/INT/BIG" BitDepth matches. Layouts follow ffmpeg pcm-bluray.c (pad channel, LFE/surround //! remap) and pcm-dvd.c (20/24-bit sample groups and blocks). use super::{CodecParser, Frame, PesPacket, pts_to_ns}; diff --git a/src/mux/fit.rs b/src/mux/fit.rs index 90552b69..64ef23e9 100644 --- a/src/mux/fit.rs +++ b/src/mux/fit.rs @@ -96,6 +96,12 @@ pub fn fit_report(dest: &StreamUrl, title: &DiscTitle) -> FitReport { continue; } let refused = match (dest, s) { + (StreamUrl::M2ts { .. }, Stream::Audio(a)) if a.codec == Codec::Aac => { + let cp = title.codec_privates.get(i).and_then(|c| c.as_deref()); + super::m2ts::aac_adts_template(cp) + .is_none() + .then_some(SkipReason::UnmappableAudio) + } (StreamUrl::M2ts { .. }, Stream::Audio(a)) if a.codec == Codec::Lpcm => { let cp = title.codec_privates.get(i).and_then(|c| c.as_deref()); super::m2ts::lpcm_bd_header(a, cp) @@ -268,6 +274,30 @@ mod tests { assert_eq!(r.skipped, vec![(1, SkipReason::UnmappableAudio)]); } + // AAC needs an ADTS-signallable ASC to be re-framed for the TS (HE-AAC signals its core). + #[test] + fn m2ts_plans_out_aac_adts_cannot_signal() { + let mut t = title(vec![ + video(Codec::H264), + audio(0x1100, Codec::Aac, SampleRate::S48, ""), + audio(0x1101, Codec::Aac, SampleRate::S48, ""), + audio(0x1102, Codec::Aac, SampleRate::S48, ""), + audio(0x1103, Codec::Aac, SampleRate::S48, ""), + ]); + // LC, HE-AAC, no ASC, and LC with 960-sample frames (frameLengthFlag). + t.codec_privates = vec![ + None, + Some(vec![0x11, 0x90]), + Some(vec![0x2B, 0x92, 0x08]), + None, + Some(vec![0x11, 0x94]), + ]; + let r = fit_report(&parse_url("m2ts:///o/x.m2ts"), &t); + assert_eq!(r.included, vec![0, 1, 2]); + let skip = SkipReason::UnmappableAudio; + assert_eq!(r.skipped, vec![(3, skip), (4, skip)]); + } + // A class sink writes only its class by the user's choice: the rest is not a loss, // so it is neither included nor skipped. #[test] diff --git a/src/mux/m2ts.rs b/src/mux/m2ts.rs index e93ba3a5..90abd289 100644 --- a/src/mux/m2ts.rs +++ b/src/mux/m2ts.rs @@ -20,6 +20,58 @@ pub(crate) fn lpcm_bd_header(a: &crate::disc::AudioStream, cp: Option<&[u8]>) -> super::codec::lpcm::bd_header(a.channels.count(), a.sample_rate.hz() as u32, src, depth) } +/// The ADTS header an `m2ts://` sink re-frames an AAC track with, or `None` when its +/// AudioSpecificConfig is missing or not signallable in ADTS and `create` leaves it out. +pub(crate) fn aac_adts_template(cp: Option<&[u8]>) -> Option<[u8; 7]> { + cp.and_then(super::codec::adts::adts_template) +} + +// PMT stream_type (13818-1 Table 2-34; BD-ROM HDMV types under the HDMV registration); +// codecs with no TS mapping are PES private data (0x06). +fn stream_type(s: &DiscStream) -> u8 { + use crate::disc::Codec as C; + match s { + DiscStream::Video(v) => match v.codec { + C::H264 if v.is_mvc_dependent() => 0x20, + C::H264 => 0x1B, + C::Hevc => 0x24, + C::Vc1 => 0xEA, + C::Mpeg2 => 0x02, + C::Mpeg1 => 0x01, + _ => 0x06, + }, + DiscStream::Audio(a) => match a.codec { + C::Lpcm => 0x80, + C::Ac3 => 0x81, + C::Dts => 0x82, + C::TrueHd => 0x83, + C::Ac3Plus if a.secondary => 0xA1, + C::Ac3Plus => 0x84, + C::DtsHdHr if a.secondary => 0xA2, + C::DtsHdHr => 0x85, + C::DtsHdMa => 0x86, + C::Aac => 0x0F, + // As FFmpeg declares MPEG audio; its readers take 0x03 for either part. + C::Mp2 | C::Mp3 => 0x03, + _ => 0x06, + }, + DiscStream::Subtitle(t) => match t.codec { + C::Pgs => 0x90, + _ => 0x06, + }, + } +} + +/// How the sink re-frames a track's IR frames for BD-TS. +#[derive(Clone, Copy)] +enum Repack { + Verbatim, + /// IR PCM → BD LPCM under this header. + Lpcm([u8; 2]), + /// Raw AAC access units → ADTS under this header. + Adts([u8; 7]), +} + fn stream_pid(s: &DiscStream) -> u16 { match s { DiscStream::Video(v) => v.pid, @@ -32,7 +84,10 @@ fn stream_pid(s: &DiscStream) -> u16 { // nothing else may; DVD ids (0xE0, 0xBD80) and the null PID are not carriable at all. fn pid_fits(s: &DiscStream, pid: u16) -> bool { let video = matches!(s, DiscStream::Video(_)); - video == super::tsmux::is_video_pid(pid) && (0x0010..=0x1FFE).contains(&pid) + video == super::tsmux::is_video_pid(pid) + && (0x0010..=0x1FFE).contains(&pid) + && pid != super::tsmux::PMT_PID + && pid != super::tsmux::PCR_PID } // Lowest free carriable PID for `s`, searched from its BD base. @@ -52,10 +107,9 @@ fn free_pid(s: &DiscStream, used: &[bool]) -> Option { pub struct M2tsStream { disc_title: DiscTitle, muxer: super::tsmux::TsMuxer>, - /// Per input track: output track index (None = dropped at create) and, for LPCM, - /// the BD LPCM header to re-pack parser PCM with. - route: Vec)>>, - /// MPEG-2 multichannel extension tracks: no descriptor carrier (tsmux writes no PMT). + /// Per input track: output track index (None = dropped at create) and its re-framing. + route: Vec>, + /// MPEG-2 multichannel extension tracks: no PMT descriptor binds them to their base. excluded: super::ps::UnstoredExtensions, } @@ -94,7 +148,19 @@ impl M2tsStream { route.push(None); continue; } - let lpcm = match s { + let repack = match s { + DiscStream::Audio(a) if a.codec == crate::disc::Codec::Aac => { + let Some(h) = aac_adts_template(cp.as_deref()) else { + tracing::warn!( + target: "mux", + track = i, + "AAC config not representable as ADTS; track omitted from M2TS" + ); + route.push(None); + continue; + }; + Repack::Adts(h) + } DiscStream::Audio(a) if a.codec == crate::disc::Codec::Lpcm => { let h = lpcm_bd_header(a, cp.as_deref()); if h.is_none() { @@ -109,9 +175,9 @@ impl M2tsStream { // Advertise the layout actually packed, not a rejected source byte. let depth = super::codec::lpcm::output_depth(cp.as_deref()); cp = h.map(|h| super::codec::lpcm::tagged_layout(h[0], depth)); - h + h.map_or(Repack::Verbatim, Repack::Lpcm) } - _ => None, + _ => Repack::Verbatim, }; let mut s = s.clone(); if !keep[i] { @@ -127,7 +193,7 @@ impl M2tsStream { DiscStream::Subtitle(t) => t.pid = pid, } } - route.push(Some((out.streams.len(), lpcm))); + route.push(Some((out.streams.len(), repack))); out.streams.push(s); out.codec_privates.push(cp); } @@ -139,6 +205,7 @@ impl M2tsStream { let pids: Vec = out.streams.iter().map(stream_pid).collect(); let boxed: Box = Box::new(writer); let mut muxer = super::tsmux::TsMuxer::new(boxed, &pids); + muxer.set_program(out.streams.iter().map(stream_type).collect())?; // Declaring the codec decides both ES framing (HEVC/H.264 are length- // prefixed → need Annex-B conversion; MPEG-2/VC-1 are already start-code // ES) and which param-set parser applies (avcC vs hvcC) — kept as one fact. @@ -189,7 +256,7 @@ impl crate::pes::Stream for M2tsStream { // Dropped at create (already warned): nothing to write. Some(None) => Ok(()), // Parser output is plain PCM (16 or 24-bit); BD-TS needs the BD LPCM framing back. - Some(Some((track, Some(header)))) => { + Some(Some((track, Repack::Lpcm(header)))) => { for (offset_ns, payload) in super::codec::lpcm::bd_payloads(&frame.data, header) { self.muxer.write_frame( track, @@ -200,7 +267,16 @@ impl crate::pes::Stream for M2tsStream { } Ok(()) } - Some(Some((track, None))) => { + Some(Some((track, Repack::Adts(header)))) => { + let blocks = super::codec::adts::raw_data_blocks(header, frame.duration_ns); + let Some(adts) = super::codec::adts::adts_frame(header, &frame.data, blocks) else { + tracing::warn!(target: "mux", track, len = frame.data.len(), "AAC access unit too long for ADTS; dropped"); + return Ok(()); + }; + self.muxer + .write_frame(track, frame.pts, frame.keyframe, &adts) + } + Some(Some((track, Repack::Verbatim))) => { self.muxer .write_frame(track, frame.pts, frame.keyframe, &frame.data) } @@ -221,7 +297,7 @@ impl crate::pes::Stream for M2tsStream { } fn undelivered_streams(&self) -> Vec { - // LPCM BD LPCM can't carry (known from create), and MPEG-2 extension tracks whose + // LPCM/AAC BD-TS can't carry (known from create), and MPEG-2 extension tracks whose // packets arrived. let mut out: Vec = (0..self.route.len()) .filter(|&i| self.route[i].is_none() && !self.excluded.contains(i)) @@ -1005,6 +1081,105 @@ mod tests { assert_eq!(got(0x1200), Some(sub)); } + fn audio(pid: u16, codec: Codec) -> DiscStream { + let DiscStream::Audio(a) = ac3_audio(pid) else { + unreachable!() + }; + DiscStream::Audio(crate::disc::AudioStream { codec, ..a }) + } + + // ffmpeg names streams only from a PMT: every track is declared under the program's + // HDMV registration with its stream_type (BD LPCM 0x80, AAC 0x0F, MPEG audio 0x03). + #[test] + fn every_track_is_declared_in_a_crc_valid_hdmv_pmt() { + let mut title = make_title(); + title.streams.push(audio(0x1100, Codec::Lpcm)); + title.streams.push(audio(0x1101, Codec::Aac)); + title.streams.push(audio(0x1102, Codec::Mp3)); + let cp0 = title.codec_privates[0].clone(); + title.codec_privates = vec![ + cp0, + Some(b"BDLP\x31\x10".to_vec()), + Some(vec![0x11, 0x90]), + None, + ]; + let mp3 = [0xFF, 0xFB, 0x90, 0x64, 0, 0, 0, 0].to_vec(); + let data = [ + fake_idr_pes_data(), + vec![0; 960 * 4], + vec![0x21, 0x10, 0x04], + mp3, + ]; + let (pids, ts) = mux_one_frame_each(&title, &data); + let streams = crate::mux::ts::scan_streams(&ts).expect("PAT/PMT present"); + let got: Vec<(u16, Codec)> = streams + .iter() + .map(|s| match s { + DiscStream::Video(v) => (v.pid, v.codec), + DiscStream::Audio(a) => (a.pid, a.codec), + DiscStream::Subtitle(t) => (t.pid, t.codec), + }) + .collect(); + let want = [Codec::Hevc, Codec::Lpcm, Codec::Aac, Codec::Mp3]; + assert_eq!(got, pids.iter().copied().zip(want).collect::>()); + assert!( + ts.windows(6) + .any(|w| w == [0x05, 0x04, b'H', b'D', b'M', b'V']), + "HDMV registration descriptor" + ); + } + + // ISO/IEC 13818-7 AAC in a TS is ADTS: the raw access units are re-framed from the ASC. + #[test] + fn aac_is_reframed_as_adts_from_its_audio_specific_config() { + let mut title = make_title(); + title.streams = vec![audio(0x1100, Codec::Aac)]; + // AAC-LC (2), 48 kHz (index 3), stereo. + title.codec_privates = vec![Some(vec![0x11, 0x90])]; + let raw = vec![0x21, 0x10, 0x04, 0x60, 0x8C]; + let (pids, ts) = mux_one_frame_each(&title, std::slice::from_ref(&raw)); + let mut demux = crate::mux::ts::TsDemuxer::new(&pids); + let mut pes = demux.feed(&ts); + pes.extend(demux.flush()); + let len = 7 + raw.len(); + let adts = [ + 0xFF, + 0xF1, + 0x4C, + 0x80, + (len >> 3) as u8, + ((len & 7) << 5) as u8 | 0x1F, + 0xFC, + ]; + assert_eq!(pes[0].data, [&adts[..], &raw].concat()); + } + + // An ADTS source frame of two raw_data_blocks (2048 samples) keeps both: the header + // says number_of_raw_data_blocks_in_frame 1 (13818-7 §8.1.1.2), so none is skipped. + #[test] + fn a_multi_block_aac_frame_declares_every_raw_data_block() { + let mut title = make_title(); + title.streams = vec![audio(0x1100, Codec::Aac)]; + title.codec_privates = vec![Some(vec![0x11, 0x90])]; + let shared = std::sync::Arc::new(std::sync::Mutex::new(Vec::::new())); + let mut stream = M2tsStream::create(SharedSink(shared.clone()), &title).unwrap(); + let mut two = frame(0, 0, true, vec![0x21, 0x10, 0x04, 0x60, 0x8C, 0x21, 0x10]); + two.duration_ns = Some(2048 * 1_000_000_000 / 48_000); + stream.write(&two).unwrap(); + stream.finish().unwrap(); + drop(stream); + let (meta, ts) = ts_after_header(&shared.lock().unwrap()); + let pid = match &meta.to_title().streams[0] { + DiscStream::Audio(a) => a.pid, + _ => unreachable!(), + }; + let mut demux = crate::mux::ts::TsDemuxer::new(&[pid]); + let mut pes = demux.feed(&ts); + pes.extend(demux.flush()); + assert_eq!(pes[0].data[6] & 3, 1, "two raw_data_blocks"); + assert_eq!(&pes[0].data[7..], &two.data[..]); + } + #[test] fn mvc_dependent_view_matches_the_base_dts() { mvc_dependent_dts(false); diff --git a/src/mux/mod.rs b/src/mux/mod.rs index f91caacf..744c8292 100644 --- a/src/mux/mod.rs +++ b/src/mux/mod.rs @@ -97,10 +97,10 @@ pub use videomap::{Medium, SourceInfo}; // and two of its inclusions can still be dropped at `finish()`. pub use mp4::{Mp4FitReport, Mp4Sink, Mp4SkipReason, fit_report as mp4_fit_report}; pub use mpg::MpgSink; -pub use network::NetworkStream; +pub use network::{NetworkStream, is_blocked_ip}; pub use null::NullStream; pub use pipelined_stream::PipelinedPesStream; -pub use resolve::{InputOptions, StreamUrl, input, output, parse_url}; +pub use resolve::{InputOptions, StreamUrl, disc_root_of, input, output, parse_url}; pub use stdio::StdioStream; use std::io::{Seek, Write}; @@ -120,6 +120,10 @@ impl WriteSeek for T {} mod fvi_pipeline_tests; #[cfg(test)] pub(crate) mod interop_tests; +#[cfg(test)] +pub(crate) mod parity_tests; +#[cfg(test)] +mod stage_tests; #[cfg(test)] mod tests { diff --git a/src/mux/mp4/audio.rs b/src/mux/mp4/audio.rs index 5c71da2e..9cd6c6c3 100644 --- a/src/mux/mp4/audio.rs +++ b/src/mux/mp4/audio.rs @@ -2,8 +2,9 @@ //! //! Covers codecs that map cleanly into MP4: **AC-3** (`ac-3` + `dac3`), //! **E-AC-3 / Dolby Digital Plus** (`ec-3` + `dec3`, incl. Atmos-in-DD+ JOC), -//! and **DTS / DTS-HD** (`dtsc`/`dtsh` + `ddts`, core described, whole access -//! units passed through so an HD decoder finds the extension). Config boxes +//! **DTS / DTS-HD** (`dtsc`/`dtsh` + `ddts`, core described, whole access +//! units passed through so an HD decoder finds the extension), and AAC / MPEG-1/2 +//! audio (`mp4a` + `esds`). Config boxes //! are derived from the first audio frame's bitstream (ISO/IEC 14496-12 //! amendments; ETSI TS 102 366 / 102 114). Codecs with no clean MP4 mapping //! (TrueHD, LPCM, bitmap subtitles) are excluded by the fit oracle. @@ -483,12 +484,80 @@ pub(super) fn dolby_sample_entry( } } +/// (channels, rate) of an AudioSpecificConfig the `esds` can carry: a table rate and +/// channel configuration, no escaped object type, short enough for one-byte sizes. +pub(super) fn aac_config(asc: &[u8]) -> Option<(u16, u32)> { + const RATES: [u32; 12] = [ + 96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, + ]; + let (b0, b1) = (*asc.first()?, *asc.get(1)?); + if b0 >> 3 == 31 || asc.len() > 100 { + return None; + } + let rate = *RATES.get(usize::from(((b0 & 7) << 1) | (b1 >> 7)))?; + let channels = match (b1 >> 3) & 0x0F { + c @ 1..=6 => u16::from(c), + 7 => 8, + _ => return None, + }; + Some((channels, rate)) +} + +// One MPEG-4 descriptor (14496-1 §7.2.2.1): tag, one-byte size (every body here is < 128). +fn descriptor(tag: u8, body: &[u8]) -> Vec { + [&[tag, body.len() as u8][..], body].concat() +} + +/// The `mp4a` + `esds` AudioSampleEntry (14496-14 §5.6) for AAC (from its ASC) or MPEG-1/2 +/// audio (from the first frame header), or `None` if neither describes the track. +pub(super) fn mpeg_sample_entry(codec: Codec, first_frame: &[u8], asc: &[u8]) -> Option> { + // (objectTypeIndication, channels, rate, DecoderSpecificInfo) + let (oti, channels, rate, dsi) = match codec { + Codec::Aac => { + let (channels, rate) = aac_config(asc)?; + (0x40, channels, rate, descriptor(0x05, asc)) + } + Codec::Mp2 | Codec::Mp3 => { + let h = first_frame.get(..4)?; + if h[0] != 0xFF || h[1] & 0xE0 != 0xE0 || h[1] & 0x06 == 0 { + return None; + } + let base = [44_100, 48_000, 32_000].get(usize::from((h[2] >> 2) & 3))?; + // version '11' MPEG-1 (11172-3), '10' MPEG-2 half rates, '00' MPEG-2.5 quarter. + let (oti, rate) = match (h[1] >> 3) & 3 { + 3 => (0x6B, *base), + 2 => (0x69, base / 2), + 0 => (0x69, base / 4), + _ => return None, + }; + (oti, if h[3] >> 6 == 3 { 1 } else { 2 }, rate, Vec::new()) + } + _ => return None, + }; + // streamType 5 (audio) << 2 | reserved 1; bufferSizeDB, max/avg bitrate unknown (0). + let mut config = vec![oti, 0x15]; + config.extend_from_slice(&[0; 11]); + config.extend_from_slice(&dsi); + let mut es = vec![0, 0, 0]; + es.extend(descriptor(0x04, &config)); + es.extend(descriptor(0x06, &[0x02])); + let esds = bx(b"esds", &[&[0u8; 4][..], &descriptor(0x03, &es)].concat()); + Some(audio_sample_entry(b"mp4a", channels, rate, &esds)) +} + /// Fit oracle for an audio codec: does `mp4://` currently carry it? -// TrueHD, LPCM, AAC are not yet mapped; skipped with a loud report, not silently dropped. +// TrueHD and LPCM are not mapped; skipped with a loud report, not silently dropped. pub(super) fn audio_fits(codec: Codec) -> bool { matches!( codec, - Codec::Ac3 | Codec::Ac3Plus | Codec::Dts | Codec::DtsHdMa | Codec::DtsHdHr + Codec::Ac3 + | Codec::Ac3Plus + | Codec::Dts + | Codec::DtsHdMa + | Codec::DtsHdHr + | Codec::Aac + | Codec::Mp2 + | Codec::Mp3 ) } @@ -650,6 +719,60 @@ mod tests { assert_eq!(ch, 6); } + // MPEG-4 audio and MPEG-1/2 audio are `mp4a` + `esds` (14496-14 §5.6, OTI 0x40/0x6B/0x69). + #[test] + fn aac_and_mpeg_audio_get_mp4a_esds_entries() { + for c in [Codec::Aac, Codec::Mp2, Codec::Mp3] { + assert!(audio_fits(c), "{c:?}"); + } + let esds = |e: &[u8]| { + e.windows(4) + .position(|w| w == b"esds") + .map(|i| e[i + 4..].to_vec()) + }; + // AAC-LC 48 kHz stereo: channels/rate from the ASC, which is the DecoderSpecificInfo. + let e = mpeg_sample_entry(Codec::Aac, &[0x21], &[0x11, 0x90]).unwrap(); + assert_eq!(&e[4..8], b"mp4a"); + assert_eq!(u16::from_be_bytes([e[24], e[25]]), 2); + assert_eq!( + u32::from_be_bytes([e[32], e[33], e[34], e[35]]) >> 16, + 48_000 + ); + let d = esds(&e).unwrap(); + assert_eq!( + d[4..9], + [0x03, 0x19, 0x00, 0x00, 0x00], + "ES_Descriptor, ES_ID 0" + ); + assert_eq!(d[9..12], [0x04, 0x11, 0x40], "DecoderConfig, OTI 0x40"); + assert_eq!(d[12], 0x15, "audio stream"); + assert_eq!( + d[24..28], + [0x05, 0x02, 0x11, 0x90], + "ASC as DecoderSpecificInfo" + ); + assert_eq!(d[28..], [0x06, 0x01, 0x02], "SLConfig predefined 2"); + // MPEG-1 Layer III 44.1 kHz mono → 0x6B; MPEG-2 Layer II 24 kHz stereo → 0x69. + let e = mpeg_sample_entry(Codec::Mp3, &[0xFF, 0xFB, 0x90, 0xC4], &[]).unwrap(); + assert_eq!(u16::from_be_bytes([e[24], e[25]]), 1); + assert_eq!( + u32::from_be_bytes([e[32], e[33], e[34], e[35]]) >> 16, + 44_100 + ); + assert_eq!(esds(&e).unwrap()[11], 0x6B); + let e = mpeg_sample_entry(Codec::Mp2, &[0xFF, 0xF5, 0x84, 0x04], &[]).unwrap(); + assert_eq!( + u32::from_be_bytes([e[32], e[33], e[34], e[35]]) >> 16, + 24_000 + ); + assert_eq!(esds(&e).unwrap()[11], 0x69); + assert!( + mpeg_sample_entry(Codec::Aac, &[0x21], &[]).is_none(), + "no ASC" + ); + assert!(mpeg_sample_entry(Codec::Mp3, &[0x0B, 0x77, 0, 0], &[]).is_none()); + } + #[test] fn fit_oracle_covers_dolby_and_dts() { assert!(audio_fits(Codec::Ac3)); diff --git a/src/mux/mp4/mod.rs b/src/mux/mp4/mod.rs index 37e26631..4f88a023 100644 --- a/src/mux/mp4/mod.rs +++ b/src/mux/mp4/mod.rs @@ -123,8 +123,8 @@ struct Track { /// in [`Mp4Sink::final_report`]. stream_idx: usize, codec: Codec, - /// Video: `hvcC`/`avcC`. Audio: unused (the sample entry is built from the - /// first frame's bitstream and cached in `audio_entry`). + /// Video: `hvcC`/`avcC`. Audio: AAC's ASC; other entries come from the first + /// frame's bitstream (cached in `audio_entry`). codec_private: Vec, width: u32, height: u32, @@ -190,7 +190,7 @@ pub struct Mp4FitReport { /// Compute the fit plan without opening a file. Video: the first primary /// HEVC/H.264 track. Audio: every track `audio::audio_fits` carries — the Dolby -/// family (AC-3 / E-AC-3) and DTS (core / DTS-HD HRA / DTS-HD MA). Everything +/// family (AC-3 / E-AC-3), DTS (core / DTS-HD HRA / DTS-HD MA), AAC and MPEG audio. Everything /// else is skipped with a reason, except a DVD MPEG-2 multichannel extension track, which /// only [`Mp4Sink::final_report`] lists, and only once its packets arrived. pub fn fit_report(title: &DiscTitle) -> Mp4FitReport { @@ -217,7 +217,9 @@ pub fn fit_report(title: &DiscTitle) -> Mp4FitReport { // (Mp2Extension, post-mux) once its packets actually arrive. DiscStream::Audio(a) if a.is_mp2_extension() => {} DiscStream::Audio(a) => { - if audio::audio_fits(a.codec) { + let cp = title.codec_privates.get(i).and_then(|c| c.as_deref()); + let described = a.codec != Codec::Aac || cp.and_then(audio::aac_config).is_some(); + if audio::audio_fits(a.codec) && described { included.push(i); } else { skipped.push((i, Mp4SkipReason::UnmappableAudio)); @@ -330,7 +332,13 @@ impl Mp4Sink { track_id, stream_idx: i, codec: a.codec, - codec_private: Vec::new(), + // AAC's AudioSpecificConfig; the other codecs describe in-band. + codec_private: title + .codec_privates + .get(i) + .cloned() + .flatten() + .unwrap_or_default(), width: 0, height: 0, colr: None, @@ -456,7 +464,16 @@ impl Stream for Mp4Sink { &frame.data, self.tracks[slot].audio_timescale, ) + .or_else(|| { + let t = &self.tracks[slot]; + audio::mpeg_sample_entry(t.codec, &frame.data, &t.codec_private) + }) { + // A rate the title could not name (e.g. 22.05 kHz MP3) comes from the entry. + if self.tracks[slot].audio_timescale == 0 { + self.tracks[slot].audio_timescale = + u32::from_be_bytes([entry[32], entry[33], entry[34], entry[35]]) >> 16; + } self.tracks[slot].audio_entry = Some(entry); } // Nothing decodes before the first video keyframe: it is not stored. @@ -1138,6 +1155,32 @@ mod tests { t } + // An AAC track the esds cannot describe (no ASC, PCE layout) is planned out, not + // promised and then dropped at finish(). + #[test] + fn fit_report_includes_aac_only_with_a_describable_config() { + let t = title( + vec![ + hevc_video(), + audio(Codec::Aac, "eng"), + audio(Codec::Aac, "eng"), + audio(Codec::Aac, "eng"), + audio(Codec::Mp3, "eng"), + ], + vec![ + Some(vec![1, 2, 3]), + Some(vec![0x11, 0x90]), + None, + Some(vec![0x11, 0x80]), + None, + ], + ); + let r = fit_report(&t); + assert_eq!(r.included, vec![0, 1, 4]); + let skip = Mp4SkipReason::UnmappableAudio; + assert_eq!(r.skipped, vec![(2, skip), (3, skip)]); + } + #[test] fn fit_report_includes_video_and_dolby_only() { let t = title( diff --git a/src/mux/mp4/read.rs b/src/mux/mp4/read.rs index 7768af6c..351ced20 100644 --- a/src/mux/mp4/read.rs +++ b/src/mux/mp4/read.rs @@ -15,9 +15,7 @@ use crate::disc::{ }; use crate::labels::LabelPurpose; use crate::pes::{PesFrame, Stream}; -use std::fs::File; use std::io::{self, Read, Seek, SeekFrom}; -use std::path::Path; const NS: i128 = 1_000_000_000; @@ -61,18 +59,6 @@ pub struct Mp4Reader { cursor: usize, } -impl Mp4Reader { - /// Open and index an MP4 file by path. - pub fn open(path: &Path) -> io::Result { - let name = path - .file_stem() - .and_then(|s| s.to_str()) - .unwrap_or("mp4") - .to_string(); - Self::from_reader(File::open(path)?, name) - } -} - impl Mp4Reader { /// Index an already-opened seekable MP4 reader. pub fn from_reader(mut file: R, name: String) -> io::Result { @@ -1605,6 +1591,8 @@ mod tests { sink.finish().unwrap(); } + let path = std::env::temp_dir().join(format!("fmkv-mp4-rt-{}.mp4", std::process::id())); + std::fs::write(&path, &buf).unwrap(); let mut rd = Mp4Reader::from_reader(Cursor::new(buf), "rt".into()).unwrap(); // Two streams: HEVC video + AC-3 audio, and the hvcC round-trips. assert_eq!(rd.info().streams.len(), 2); @@ -1633,6 +1621,17 @@ mod tests { assert_eq!(vids[1].1, 350, "second video sample size"); assert!(vids[0].2, "first video frame is a keyframe"); assert_eq!(auds[0].1, ac3.len()); + + // `mp4://` reads the same file through the decryption stage: a clear container + // (Opaque) is handed through untouched, seeks included. + let url = format!("mp4://{}", path.display()); + let mut staged = crate::mux::resolve::input(&url, &Default::default()).unwrap(); + let mut again = Vec::new(); + while let Some(f) = staged.read().unwrap() { + again.push((f.track, f.data.len(), f.keyframe, f.data)); + } + let _ = std::fs::remove_file(&path); + assert_eq!(again, got); } // `stts` run-length expansion (ISO/IEC 14496-12 §8.6.1.2) must expand `(sample_count, diff --git a/src/mux/mpg/tests.rs b/src/mux/mpg/tests.rs index 91401d08..5e06db45 100644 --- a/src/mux/mpg/tests.rs +++ b/src/mux/mpg/tests.rs @@ -1766,6 +1766,21 @@ fn stuffed_scrambled_packs_are_never_muxed_as_ciphertext() { } } +// Review: a VOB whose first 33 sectors are zero-filled (a bad-read start) is cracked from its +// first written pack; behind a head of non-pack bytes its scrambled packs are refused (E7023). +// Never muxed as ciphertext. +#[test] +fn a_blank_or_garbled_head_does_not_hide_scrambled_packs() { + let (mut body, es) = clear_ps(false, 0); + scramble_video_packs(&mut body); + let blank = [vec![0; 33 * pack::PACK_BYTES], body.clone()].concat(); + let got = read_video(&blank, "blank-head").map(|v| first_diff(&v, &es)); + assert_eq!(got, Ok(None), "descrambled"); + let garbled = [vec![0x30; 33 * pack::PACK_BYTES], body].concat(); + let got = read_video(&garbled, "garbled-head").map(|v| v.len()); + assert_eq!(got, Err(Some(crate::error::E_CSS_KEY_MISSING))); +} + // m1: scrambling first met past the crack's 50 000-sector budget is refused with E7023 when // it is read, never muxed as ciphertext. #[test] diff --git a/src/mux/network.rs b/src/mux/network.rs index 4e720512..5eb135bd 100644 --- a/src/mux/network.rs +++ b/src/mux/network.rs @@ -9,6 +9,7 @@ use super::meta; use crate::disc::DiscTitle; use crate::halt::{Halt, WAIT_SLICE}; +use crate::sector::stage::Stage; use rustix::event::{PollFd, PollFlags, Timespec}; use std::io::{self, BufReader, BufWriter, Read, Write}; use std::net::{IpAddr, SocketAddr, TcpListener, TcpStream, ToSocketAddrs}; @@ -119,68 +120,28 @@ impl Read for HaltRead { } } -// True if `ip` must never be a `network://` connect target (loopback, private, link-local, -// unspecified, multicast). Re-checked here at connect time to close a DNS-rebinding TOCTOU. -pub(crate) fn is_blocked_ip(ip: IpAddr) -> bool { +// True if `ip` can never be a `network://` peer (unspecified, multicast, broadcast, +// 0.0.0.0/8, Class E). Loopback, private, link-local and ULA are valid LAN targets. +pub fn is_blocked_ip(ip: IpAddr) -> bool { match ip { IpAddr::V4(v4) => { - let o = v4.octets(); - v4.is_loopback() - || v4.is_private() - || v4.is_link_local() - || v4.is_unspecified() + v4.is_unspecified() || v4.is_multicast() || v4.is_broadcast() - // carrier-grade NAT 100.64.0.0/10 - || (o[0] == 100 && (o[1] & 0xc0) == 0x40) - // "this network" 0.0.0.0/8 - || o[0] == 0 - // benchmarking 198.18.0.0/15 (RFC 2544) — 198.18.x and 198.19.x - // (the /15 second octet is 18 with the low bit free, i.e. 18|19). - || (o[0] == 198 && (o[1] & 0xfe) == 18) - // IETF protocol assignments 192.0.0.0/24 (RFC 6890), which - // includes 192.0.0.170/171 (NAT64/DNS64 discovery). - || (o[0] == 192 && o[1] == 0 && o[2] == 0) - // Class E reserved 240.0.0.0/4 - || o[0] >= 240 + || v4.octets()[0] == 0 + || v4.octets()[0] >= 240 } IpAddr::V6(v6) => { - let seg = v6.segments(); - // 6to4 (2002::/16) embeds an IPv4 in segments[1..3]; Teredo (2001:0000::/32) - // embeds the client IPv4 in the last two segments, each XOR 0xffff. Re-check - // both as their embedded IPv4 or an internal target slips through the tunnel. - let sixtofour = (seg[0] == 0x2002) - .then(|| std::net::Ipv4Addr::from(((seg[1] as u32) << 16) | (seg[2] as u32))); - let teredo = (seg[0] == 0x2001 && seg[1] == 0x0000).then(|| { - std::net::Ipv4Addr::from( - (((seg[6] ^ 0xffff) as u32) << 16) | ((seg[7] ^ 0xffff) as u32), - ) - }); - // NAT64 well-known prefix 64:ff9b::/96 (RFC 6052) embeds the IPv4 in - // the last 32 bits (segments[6..8]); re-check it too so an internal - // target does not slip through a NAT64 translator. - let nat64 = (seg[0] == 0x0064 && seg[1] == 0xff9b) - .then(|| std::net::Ipv4Addr::from(((seg[6] as u32) << 16) | (seg[7] as u32))); - v6.is_loopback() - || v6.is_unspecified() + v6.is_unspecified() || v6.is_multicast() - // unique-local fc00::/7 - || (seg[0] & 0xfe00) == 0xfc00 - // link-local fe80::/10 - || (seg[0] & 0xffc0) == 0xfe80 - // IPv4-mapped (::ffff:x.x.x.x) and IPv4-compatible (::x.x.x.x); - // to_ipv4() returns Some for both forms — re-check as IPv4 so an - // IPv4-mapped private/loopback address can't bypass the block above. - || v6.to_ipv4().map(|m| is_blocked_ip(IpAddr::V4(m))) == Some(true) - || sixtofour.is_some_and(|v4| is_blocked_ip(IpAddr::V4(v4))) - || teredo.is_some_and(|v4| is_blocked_ip(IpAddr::V4(v4))) - || nat64.is_some_and(|v4| is_blocked_ip(IpAddr::V4(v4))) + // IPv4-mapped (::ffff:x.x.x.x) is judged as its IPv4 address. + || v6.to_ipv4_mapped().is_some_and(|m| is_blocked_ip(IpAddr::V4(m))) } } } -// Every resolved address of `addr` not blocked by `is_blocked_ip`, in order. Vetted IP -// literals (no second DNS lookup). Zero resolved or all blocked: NetworkAddrBlocked. +// Every resolved address of `addr` not refused by `is_blocked_ip`, in order. Zero +// resolved or all refused: NetworkAddrBlocked. fn allowed_addrs( addr: &str, resolved: impl Iterator, @@ -222,7 +183,7 @@ enum Mode { ended: bool, }, Read { - reader: BufReader, + reader: Box>>, meta: meta::M2tsMeta, }, } @@ -240,13 +201,9 @@ impl NetworkStream { Self::connect_vetted(addr, true) } - // `connect` with an explicit SSRF-vetting toggle: vet=true (the public - // path) rejects loopback/private/link-local/multicast; vet=false is - // for in-crate tests connecting to 127.0.0.1 ephemeral listeners. + // `connect` with a toggle: vet=true (the public path) refuses unspecified/ + // multicast/broadcast targets; vet=false skips that for in-crate tests. fn connect_vetted(addr: &str, vet: bool) -> io::Result { - // Connect to the vetted IP literal (not the raw name) so a DNS rebind between - // settings-save validation and now can't redirect us to a loopback/private/ - // link-local host (SSRF). let stream = if vet { let vetted = allowed_addrs(addr, addr.to_socket_addrs()?)?; connect_first(&vetted)? @@ -296,7 +253,12 @@ impl NetworkStream { /// [`listen`](Self::listen) that a [`Halt`] can interrupt, while waiting for /// the sender to connect and while blocked on a stalled sender. pub fn listen_with_halt(addr: &str, halt: Option) -> io::Result { - Self::accept_from_with_halt(TcpListener::bind(addr)?, halt) + Self::listen_staged(addr, halt, false) + } + + // `listen_with_halt` whose decryption stage passes ciphertext when `raw`. + pub(crate) fn listen_staged(addr: &str, halt: Option, raw: bool) -> io::Result { + Self::accept_staged(TcpListener::bind(addr)?, halt, raw) } /// Accept one connection from an already-bound listener and read from it. @@ -309,11 +271,15 @@ impl NetworkStream { /// [`accept_from`](Self::accept_from) that a [`Halt`] can interrupt. pub fn accept_from_with_halt(listener: TcpListener, halt: Option) -> io::Result { + Self::accept_staged(listener, halt, false) + } + + fn accept_staged(listener: TcpListener, halt: Option, raw: bool) -> io::Result { let Some(h) = halt else { let (stream, _peer) = listener.accept()?; stream.set_nodelay(true)?; arm_keepalive(&stream); - return Self::read_from(stream, None); + return Self::read_from(stream, None, raw); }; listener.set_nonblocking(true)?; let halt = Some(h); @@ -334,21 +300,25 @@ impl NetworkStream { stream.set_nonblocking(false)?; stream.set_nodelay(true)?; arm_keepalive(&stream); - Self::read_from(stream, halt) + Self::read_from(stream, halt, raw) } - // Wrap an accepted connection and read its FMKV header. - fn read_from(stream: TcpStream, halt: Option) -> io::Result { - let mut reader = BufReader::with_capacity(NET_BUF_SIZE, HaltRead::new(stream, halt)); + // Wrap an accepted connection in the decryption stage and read its FMKV header. + fn read_from(stream: TcpStream, halt: Option, raw: bool) -> io::Result { + let staged = Stage::lazy(HaltRead::new(stream, halt), raw); + let mut reader = BufReader::with_capacity(NET_BUF_SIZE, staged); // Read FMKV metadata header let meta = meta::read_header(&mut reader) - .map_err(|e| reader.get_ref().halted_or(e))? + .map_err(|e| reader.get_ref().get_ref().halted_or(e))? .ok_or_else(|| -> io::Error { crate::error::Error::NoMetadata.into() })?; Ok(Self { disc_title: meta.to_title(), - mode: Mode::Read { reader, meta }, + mode: Mode::Read { + reader: Box::new(reader), + meta, + }, }) } } @@ -377,7 +347,7 @@ impl crate::pes::Stream for NetworkStream { match &mut self.mode { Mode::Read { reader, meta } => { crate::pes::PesFrame::deserialize_ext(reader, meta.frame_padding) - .map_err(|e| reader.get_ref().halted_or(e)) + .map_err(|e| reader.get_ref().get_ref().halted_or(e)) } _ => Err(crate::error::Error::StreamWriteOnly.into()), } @@ -502,191 +472,72 @@ mod tests { }; use std::net::TcpListener; - // SSRF guard: every loopback/private/link-local/multicast/unspecified - // address (v4+v6) must be rejected, ordinary public addresses allowed — - // this is what closes the DNS-rebinding window in `connect`. + // LAN targets are valid for `network://`: loopback, private, link-local, + // ULA and CGNAT connect; only addresses that can never be a peer are refused. #[test] - fn is_blocked_ip_rejects_internal_targets() { + fn is_blocked_ip_allows_lan_and_refuses_invalid() { use std::net::{Ipv4Addr, Ipv6Addr}; - // Built from octets (not string literals) so the repo's internal-infra - // secret scanner doesn't flag the RFC1918 addresses. + // Built from octets so the internal-infra secret scanner doesn't flag RFC1918. let v4 = |a, b, c, d| IpAddr::V4(Ipv4Addr::new(a, b, c, d)); - let blocked: &[(IpAddr, &str)] = &[ - (v4(127, 0, 0, 1), "loopback"), - (v4(127, 10, 20, 30), "loopback /8"), - (v4(10, 0, 0, 1), "private 10/8"), - (v4(172, 16, 5, 5), "private 172.16/12"), - (v4(192, 168, 1, 1), "private 192.168/16"), - (v4(169, 254, 10, 10), "link-local"), - (v4(0, 0, 0, 0), "unspecified"), - (v4(224, 0, 0, 1), "multicast"), - (v4(255, 255, 255, 255), "broadcast"), - (IpAddr::V6(Ipv6Addr::LOCALHOST), "loopback v6"), - (IpAddr::V6(Ipv6Addr::UNSPECIFIED), "unspecified v6"), - ( - IpAddr::V6(Ipv6Addr::new(0xfc00, 0, 0, 0, 0, 0, 0, 1)), - "ULA", - ), - ( - IpAddr::V6(Ipv6Addr::new(0xfd12, 0x3456, 0, 0, 0, 0, 0, 1)), - "ULA", - ), - ( - IpAddr::V6(Ipv6Addr::new(0xfe80, 0, 0, 0, 0, 0, 0, 1)), - "link-local v6", - ), - ( - IpAddr::V6(Ipv6Addr::new(0xff02, 0, 0, 0, 0, 0, 0, 1)), - "multicast v6", - ), - // CGNAT / 0.0.0.0/8 / Class E (finding 8). - (v4(100, 64, 0, 1), "carrier-grade NAT"), - (v4(100, 127, 255, 254), "carrier-grade NAT edge"), - (v4(0, 1, 2, 3), "0.0.0.0/8"), - (v4(240, 0, 0, 1), "Class E"), - (v4(255, 0, 0, 1), "Class E high"), - // IPv4-mapped / -compatible IPv6 bypass (finding 7). - ( - IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0xffff, 0x0a00, 0x0001)), - "IPv4-mapped RFC1918 (::ffff:0a00:0001)", - ), - ( - IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0xffff, 0x7f00, 0x0001)), - "::ffff:127.0.0.1 mapped", - ), - ( - IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0, 0x7f00, 0x0001)), - "::127.0.0.1 compatible", - ), + let v6 = |s: [u16; 8]| IpAddr::V6(Ipv6Addr::from(s)); + let invalid = [ + v4(0, 0, 0, 0), + v4(0, 1, 2, 3), + v4(224, 0, 0, 1), + v4(255, 255, 255, 255), + v4(240, 0, 0, 1), + IpAddr::V6(Ipv6Addr::UNSPECIFIED), + v6([0xff02, 0, 0, 0, 0, 0, 0, 1]), + v6([0, 0, 0, 0, 0, 0xffff, 0xe000, 1]), + v6([0, 0, 0, 0, 0, 0xffff, 0, 0]), ]; - for (ip, label) in blocked { - assert!(is_blocked_ip(*ip), "{label} ({ip}) must be blocked"); + for ip in invalid { + assert!(is_blocked_ip(ip), "{ip} must be refused"); } - - let allowed: &[(IpAddr, &str)] = &[ - (v4(8, 8, 8, 8), "public dns"), - (v4(1, 1, 1, 1), "public dns"), - (v4(93, 184, 216, 34), "example.com"), - ( - IpAddr::V6(Ipv6Addr::new(0x2606, 0x2800, 0x220, 1, 0, 0, 0, 1)), - "public v6", - ), - ( - IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0xffff, 0x0808, 0x0808)), - "::ffff:8.8.8.8 public mapped", - ), + let allowed = [ + v4(127, 0, 0, 1), + v4(10, 0, 0, 1), + v4(172, 16, 5, 5), + v4(192, 168, 1, 1), + v4(169, 254, 10, 10), + v4(100, 64, 0, 1), + v4(8, 8, 8, 8), + IpAddr::V6(Ipv6Addr::LOCALHOST), + v6([0xfd12, 0x3456, 0, 0, 0, 0, 0, 1]), + v6([0xfe80, 0, 0, 0, 0, 0, 0, 1]), + v6([0, 0, 0, 0, 0, 0xffff, 0x0a00, 1]), + v6([0, 0, 0, 0, 0, 0xffff, 0x7f00, 1]), + v6([0x2606, 0x2800, 0x220, 1, 0, 0, 0, 1]), ]; - for (ip, label) in allowed { - assert!(!is_blocked_ip(*ip), "{label} ({ip}) must be allowed"); + for ip in allowed { + assert!(!is_blocked_ip(ip), "{ip} must be allowed"); } } - // Benchmarking 198.18.0.0/15 (RFC 2544) and IETF protocol assignments - // 192.0.0.0/24 (RFC 6890, incl. the 192.0.0.170/171 NAT64/DNS64 anycast) - // are non-public and must be blocked outbound — parity with keysources. + // The public `connect` reaches a loopback listener (the LAN-send path). #[test] - fn ssrf_guard_blocks_benchmarking_and_protocol_assignment_ranges() { - use std::net::Ipv4Addr; - let v4 = |a, b, c, d| IpAddr::V4(Ipv4Addr::new(a, b, c, d)); - // 198.18.0.0/15 spans 198.18.x AND 198.19.x — both octets blocked. - assert!(is_blocked_ip(v4(198, 18, 0, 1))); - assert!(is_blocked_ip(v4(198, 18, 255, 255))); - assert!(is_blocked_ip(v4(198, 19, 0, 1))); - assert!(is_blocked_ip(v4(198, 19, 200, 5))); - // 198.17.x and 198.20.x are OUTSIDE the /15 — must stay allowed. - assert!(!is_blocked_ip(v4(198, 17, 0, 1))); - assert!(!is_blocked_ip(v4(198, 20, 0, 1))); - // 192.0.0.0/24, including 192.0.0.170 / 192.0.0.171. - assert!(is_blocked_ip(v4(192, 0, 0, 0))); - assert!(is_blocked_ip(v4(192, 0, 0, 170))); - assert!(is_blocked_ip(v4(192, 0, 0, 171))); - assert!(is_blocked_ip(v4(192, 0, 0, 255))); - // The adjacent 192.0.1.0 is a different block — not covered here. - assert!(!is_blocked_ip(v4(192, 0, 1, 1))); - } - - // 6to4 (2002::/16) and Teredo (2001:0000::/32) tunnel an IPv4 inside an - // IPv6 address; the guard must decode and re-check that embedded IPv4 or an - // internal target slips through the tunnel — parity with keysources. - #[test] - fn ssrf_guard_blocks_embedded_ipv4_via_6to4_and_teredo() { - use std::net::Ipv6Addr; - // 6to4 for 127.0.0.1: 2002:7f00:0001:: (embedded in segments[1..3]). - assert!(is_blocked_ip(IpAddr::V6(Ipv6Addr::new( - 0x2002, 0x7f00, 0x0001, 0, 0, 0, 0, 0 - )))); - // 6to4 for 169.254.169.254 (cloud metadata): 2002:a9fe:a9fe::. - assert!(is_blocked_ip(IpAddr::V6(Ipv6Addr::new( - 0x2002, 0xa9fe, 0xa9fe, 0, 0, 0, 0, 0 - )))); - // Teredo for 127.0.0.1: client IPv4 lives in the last two segments XOR - // 0xffff, so 0x7f00^0xffff=0x80ff and 0x0001^0xffff=0xfffe. - assert!(is_blocked_ip(IpAddr::V6(Ipv6Addr::new( - 0x2001, 0x0000, 0, 0, 0, 0, 0x80ff, 0xfffe - )))); - // A 6to4 wrapping a PUBLIC IPv4 (8.8.8.8 → 2002:0808:0808::) is allowed. - assert!(!is_blocked_ip(IpAddr::V6(Ipv6Addr::new( - 0x2002, 0x0808, 0x0808, 0, 0, 0, 0, 0 - )))); - } - - /// The NAT64 well-known prefix 64:ff9b::/96 (RFC 6052) embeds an IPv4 in its - /// last 32 bits; the guard must decode and re-check that embedded IPv4 so an - /// internal target does not slip through a NAT64 translator — parity with - /// keysources. A NAT64 address embedding a private/special IPv4 must classify - /// exactly as that IPv4 does directly. - #[test] - fn ssrf_guard_blocks_embedded_ipv4_via_nat64() { - use std::net::{Ipv4Addr, Ipv6Addr}; - // NAT64 for 169.254.169.254 (cloud metadata): 64:ff9b::a9fe:a9fe — the - // embedded IPv4 is link-local, so both forms must be blocked identically. - assert!(is_blocked_ip(IpAddr::V4(Ipv4Addr::new(169, 254, 169, 254)))); - assert!(is_blocked_ip(IpAddr::V6(Ipv6Addr::new( - 0x0064, 0xff9b, 0, 0, 0, 0, 0xa9fe, 0xa9fe - )))); - // NAT64 for 127.0.0.1: 64:ff9b::7f00:0001 — embedded loopback, blocked. - assert!(is_blocked_ip(IpAddr::V6(Ipv6Addr::new( - 0x0064, 0xff9b, 0, 0, 0, 0, 0x7f00, 0x0001 - )))); - // NAT64 wrapping a PUBLIC IPv4 (8.8.8.8 → 64:ff9b::0808:0808) is allowed, - // exactly as 8.8.8.8 itself is. - assert!(!is_blocked_ip(IpAddr::V4(Ipv4Addr::new(8, 8, 8, 8)))); - assert!(!is_blocked_ip(IpAddr::V6(Ipv6Addr::new( - 0x0064, 0xff9b, 0, 0, 0, 0, 0x0808, 0x0808 - )))); - } - - /// The public `connect` must refuse a loopback target with the typed - /// `NetworkAddrBlocked` error (PermissionDenied) rather than attempting - /// the TCP connect — this is the rebinding TOCTOU close at the connect. - #[test] - fn connect_refuses_blocked_loopback_target() { - // Bind a real loopback listener so a non-vetting connect WOULD - // succeed; the vetting connect must still refuse it. + fn connect_reaches_loopback_listener() { let listener = TcpListener::bind("127.0.0.1:0").unwrap(); let addr = listener.local_addr().unwrap(); - let err = match NetworkStream::connect(&addr.to_string()) { - Ok(_) => panic!("loopback target must be refused by the SSRF guard"), - Err(e) => e, - }; - assert_eq!(err.kind(), io::ErrorKind::PermissionDenied); + NetworkStream::connect(&addr.to_string()).expect("loopback send must connect"); } - // The SSRF filter keeps every allowed address in order (not just the first) and - // refuses an all-blocked or empty resolution. + // The filter keeps every allowed address in order and refuses a resolution + // that is empty or all invalid. #[test] - fn allowed_addrs_keeps_every_public_address_and_refuses_none() { + fn allowed_addrs_keeps_every_valid_address_and_refuses_none() { let a = |s: &str| s.parse::().unwrap(); let mixed = [ a("127.0.0.1:9"), - a("[2001:db8::1]:9"), a("224.0.0.1:9"), + a("[::1]:9"), + a("0.0.0.0:9"), a("8.8.8.8:9"), ]; let got = allowed_addrs("h:9", mixed.into_iter()).unwrap(); - assert_eq!(got, vec![a("[2001:db8::1]:9"), a("8.8.8.8:9")]); - for list in [vec![a("127.0.0.1:9"), a("[::1]:9")], vec![]] { - let err = allowed_addrs("h:9", list.into_iter()).expect_err("no safe address"); + assert_eq!(got, vec![a("127.0.0.1:9"), a("[::1]:9"), a("8.8.8.8:9")]); + for list in [vec![a("0.0.0.0:9"), a("[ff02::1]:9")], vec![]] { + let err = allowed_addrs("h:9", list.into_iter()).expect_err("no valid address"); assert_eq!( crate::error::error_code(&err), Some(crate::error::E_NETWORK_ADDR_BLOCKED) @@ -1210,9 +1061,11 @@ mod tests { let handle = std::thread::spawn(move || { let ns = NetworkStream::accept_from(listener).unwrap(); match &ns.mode { - Mode::Read { reader, .. } => socket2::SockRef::from(&reader.get_ref().stream) - .keepalive() - .unwrap(), + Mode::Read { reader, .. } => { + socket2::SockRef::from(&reader.get_ref().get_ref().stream) + .keepalive() + .unwrap() + } Mode::Write { .. } => false, } }); @@ -1242,7 +1095,9 @@ mod tests { let halt = crate::halt::Halt::new(); let ns = NetworkStream::accept_from_with_halt(listener, Some(halt)).unwrap(); match &ns.mode { - Mode::Read { reader, .. } => reader.get_ref().stream.read_timeout().unwrap(), + Mode::Read { reader, .. } => { + reader.get_ref().get_ref().stream.read_timeout().unwrap() + } Mode::Write { .. } => None, } }); @@ -1327,9 +1182,11 @@ mod tests { let halt = crate::halt::Halt::new(); let mut ns = NetworkStream::accept_from_with_halt(listener, Some(halt)).unwrap(); let keepalive = match &ns.mode { - Mode::Read { reader, .. } => socket2::SockRef::from(&reader.get_ref().stream) - .keepalive() - .unwrap(), + Mode::Read { reader, .. } => { + socket2::SockRef::from(&reader.get_ref().get_ref().stream) + .keepalive() + .unwrap() + } Mode::Write { .. } => false, }; ( @@ -1462,4 +1319,75 @@ mod tests { assert_eq!(back.track_timing(0), timing); assert_eq!(back.read().unwrap().unwrap().discard_padding_ns, -2_500_000); } + // Parity golden: the FMKV wire bytes a sender emits for the synthetic clip, and the + // frames a receiver reads back (public `connect` refuses loopback, so this seam). + #[test] + fn parity_network_fmkv_wire() { + use crate::pes::Stream as _; + use std::io::Read as _; + let clip = crate::test_util::synthetic_bd_clip(6); + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("c.m2ts"); + std::fs::write(&path, &clip).unwrap(); + let mut src = + crate::mux::resolve::input(&format!("m2ts://{}", path.display()), &Default::default()) + .unwrap(); + let title = src.info().clone(); + let mut frames = Vec::new(); + while let Some(f) = src.read().unwrap() { + frames.push(f); + } + let mut g = + crate::test_util::Golden::new(env!("CARGO_MANIFEST_DIR"), "parity_network_fmkv_wire"); + + // Sender -> a raw socket: the wire bytes. + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let raw = std::thread::spawn(move || { + let mut bytes = Vec::new(); + listener + .accept() + .unwrap() + .0 + .read_to_end(&mut bytes) + .unwrap(); + bytes + }); + let mut w = NetworkStream::connect_vetted(&addr.to_string(), false) + .unwrap() + .meta(&title); + for f in &frames { + w.write(f).unwrap(); + } + w.finish().unwrap(); + drop(w); + g.bytes("wire", &raw.join().unwrap()); + + // Sender -> NetworkStream receiver: the frames round-trip. + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let rx = std::thread::spawn(move || { + let mut ns = NetworkStream::accept_from(listener).unwrap(); + let mut got = Vec::new(); + while let Some(f) = ns.read().unwrap() { + got.push((f.track, f.pts, f.keyframe, f.data)); + } + got + }); + let mut w = NetworkStream::connect_vetted(&addr.to_string(), false) + .unwrap() + .meta(&title); + for f in &frames { + w.write(f).unwrap(); + } + w.finish().unwrap(); + let got = rx.join().unwrap(); + let sent: Vec<_> = frames + .iter() + .map(|f| (f.track, f.pts, f.keyframe, f.data.clone())) + .collect(); + g.kv("frames", got.len()); + g.kv("round-trip identical", got == sent); + g.check(); + } } diff --git a/src/mux/parity_tests.rs b/src/mux/parity_tests.rs new file mode 100644 index 00000000..d8d4573f --- /dev/null +++ b/src/mux/parity_tests.rs @@ -0,0 +1,450 @@ +//! Parity goldens for the loose-file paths: every file source read through `input()` and +//! every sink driven through `mux_with_keys`, over the synthetic BD clip. Pinned as output +//! hashes, loss counters and refusal codes (see [`Golden`]); cells are `parity_*`. + +use super::driver::{MuxOptions, MuxOutcome, MuxSource, NoopEvents, mux_with_keys}; +use super::resolve::{InputOptions, input}; +use super::select::{PidFilter, StreamSelection}; +use crate::aacs::content::{ALIGNED_UNIT_LEN, encrypt_unit}; +use crate::consts::BD_SOURCE_PACKET_BYTES as PKT; +use crate::halt::Halt; +use crate::keys::ResolvedKeySet; +use crate::test_util::{CLIP_AUDIO_PIDS, Golden, synthetic_bd_clip}; +use std::path::Path; +use std::sync::Arc; + +// A synthetic test key, not key material from any disc. +const KEY: [u8; 16] = [0x5A; 16]; +const UNITS: usize = 12; + +pub(crate) fn golden(cell: &str) -> Golden { + Golden::new(env!("CARGO_MANIFEST_DIR"), cell) +} + +fn replace(hay: &[u8], from: &[u8], to: &[u8]) -> Vec { + let mut out = Vec::with_capacity(hay.len()); + let mut i = 0; + while i < hay.len() { + if hay[i..].starts_with(from) { + out.extend_from_slice(to); + i += from.len(); + } else { + out.push(hay[i]); + i += 1; + } + } + out +} + +// Every file under `dir`, sorted by relative path, as ` out = len sha256`. +// The temp root appears in provenance headers (`fvi://`) and differs per run: masked. +pub(crate) fn record_tree(g: &mut Golden, prefix: &str, dir: &Path) { + let root = dir.parent().unwrap().display().to_string().into_bytes(); + let mut files = Vec::new(); + let mut stack = vec![dir.to_path_buf()]; + while let Some(d) = stack.pop() { + for e in std::fs::read_dir(&d).unwrap().flatten() { + let p = e.path(); + if p.is_dir() { + stack.push(p); + } else { + files.push(p); + } + } + } + files.sort(); + for f in files { + let rel = f.strip_prefix(dir).unwrap().display().to_string(); + let mut bytes = replace(&std::fs::read(&f).unwrap(), &root, b""); + // The build stamp (version and commit) changes with every commit: same-length mask. + for stamp in [crate::MUX_APP, super::videomap::FVI_GENERATOR] { + bytes = replace(&bytes, stamp.as_bytes(), &vec![b'#'; stamp.len()]); + } + g.bytes(&format!("{prefix} out {rel}"), &bytes); + } +} + +// The outcome (or refusal code) of one mux run. +pub(crate) fn record_run(g: &mut Golden, prefix: &str, r: &std::io::Result) { + match r { + Ok(o) => g.kv( + &format!("{prefix} outcome"), + format_args!( + "completed={} opened={} bytes={} errors={} lost={} streams={} undelivered={:?}", + o.completed, + o.output_opened, + o.bytes_written, + o.errors, + o.lost_bytes, + o.streams, + o.undelivered_streams + ), + ), + Err(e) => g.kv( + &format!("{prefix} refused"), + format_args!("E{}", crate::error::error_code(e).unwrap_or(0)), + ), + }; +} + +fn dest_name(scheme: &str) -> &'static str { + match scheme { + "mkv" => "o.mkv", + "mp4" => "o.mp4", + "mpg" => "o.mpg", + "m2ts" => "o.m2ts", + "fvi" => "o.fvi", + "chapters" => "o.xml", + "json" => "o.json", + _ => "o", + } +} + +// Mux the loose file `bytes` (read as `src_scheme://`) to each `sinks` entry, recording the +// outcome and every output file under the sink's name. +fn sinks( + g: &mut Golden, + src_scheme: &str, + bytes: &[u8], + sinks: &[&str], + mopts: &MuxOptions, + iopts: &InputOptions, +) { + let dir = tempfile::tempdir().unwrap(); + let input = dir.path().join(format!("in.{src_scheme}")); + std::fs::write(&input, bytes).unwrap(); + let url = format!("{src_scheme}://{}", input.display()); + for sink in sinks { + let out = dir.path().join(format!("out-{sink}")); + std::fs::create_dir_all(&out).unwrap(); + let dest = match *sink { + "null" => "null://".to_string(), + s => format!("{s}://{}", out.join(dest_name(s)).display()), + }; + let r = mux_with_keys( + MuxSource::Url { + url: &url, + opts: iopts.clone(), + }, + None, + &dest, + mopts, + &Halt::new(), + Arc::new(NoopEvents), + ); + record_run(g, sink, &r); + record_tree(g, sink, &out); + } +} + +const ALL_SINKS: &[&str] = &[ + "mkv", "mp4", "mpg", "m2ts", "fvi", "json", "chapters", "demux", "video", "audio", "sub", + "null", +]; + +#[test] +fn parity_sinks_clear_m2ts_default() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let mut g = golden("parity_sinks_clear_m2ts_default"); + sinks( + &mut g, + "m2ts", + &synthetic_bd_clip(UNITS), + ALL_SINKS, + &MuxOptions::default(), + &Default::default(), + ); + g.check(); +} + +// `raw` is ciphertext passthrough: on a clear clip it changes no output byte. +#[test] +fn parity_sinks_clear_m2ts_raw() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let mut g = golden("parity_sinks_clear_m2ts_raw"); + let opts = MuxOptions { + raw: true, + ..Default::default() + }; + let iopts = InputOptions { + raw: true, + ..Default::default() + }; + sinks( + &mut g, + "m2ts", + &synthetic_bd_clip(UNITS), + &["mkv", "m2ts", "demux"], + &opts, + &iopts, + ); + g.check(); +} + +// Selection through a URL source goes in `InputOptions` (the second audio only, no subs). +#[test] +fn parity_sinks_clear_m2ts_selection() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let mut g = golden("parity_sinks_clear_m2ts_selection"); + let iopts = InputOptions { + selection: StreamSelection { + audio: PidFilter::Only(vec![CLIP_AUDIO_PIDS[1]]), + subtitle: PidFilter::Only(vec![]), + }, + ..Default::default() + }; + sinks( + &mut g, + "m2ts", + &synthetic_bd_clip(UNITS), + &["mkv", "m2ts", "demux", "json"], + &MuxOptions::default(), + &iopts, + ); + g.check(); +} + +// Every container this crate writes is also a source: re-mux its output to mkv. +#[test] +fn parity_sinks_from_mkv_mpg_and_fmkv_sources() { + let mut g = golden("parity_sinks_from_mkv_mpg_and_fmkv_sources"); + let _serial = crate::sector::prefetched::holder_test_lock(); + let clip = synthetic_bd_clip(UNITS); + let dir = tempfile::tempdir().unwrap(); + let src = dir.path().join("clip.m2ts"); + std::fs::write(&src, &clip).unwrap(); + for (scheme, name) in [("mkv", "a.mkv"), ("mpg", "a.mpg"), ("m2ts", "a.fmkv.m2ts")] { + let p = dir.path().join(name); + mux_with_keys( + MuxSource::Url { + url: &format!("m2ts://{}", src.display()), + opts: Default::default(), + }, + None, + &format!("{scheme}://{}", p.display()), + &MuxOptions::default(), + &Halt::new(), + Arc::new(NoopEvents), + ) + .unwrap(); + let bytes = std::fs::read(&p).unwrap(); + sinks( + &mut g, + scheme, + &bytes, + &["mkv"], + &MuxOptions::default(), + &Default::default(), + ); + let tag = format!("{scheme}-src"); + let mut s = input(&format!("{scheme}://{}", p.display()), &Default::default()).unwrap(); + record_frames(&mut g, &tag, s.as_mut()); + } + g.check(); +} + +// The frames a source yields (count, a digest over track/pts/keyframe/data), its stream +// list, and the stream's loss counters; or the code that refused it first. +fn record_frames(g: &mut Golden, tag: &str, s: &mut dyn crate::pes::Stream) { + let streams: Vec = s + .info() + .streams + .iter() + .map(|st| match st { + crate::disc::Stream::Video(v) => format!("v:{:?}:{:#x}", v.codec, v.pid), + crate::disc::Stream::Audio(a) => format!("a:{:?}:{:#x}", a.codec, a.pid), + crate::disc::Stream::Subtitle(x) => format!("s:{:?}:{:#x}", x.codec, x.pid), + }) + .collect(); + let mut digest = Vec::new(); + let mut n = 0u64; + while let Some(f) = s.read().unwrap() { + n += 1; + digest.extend_from_slice(&(f.track as u64).to_le_bytes()); + digest.extend_from_slice(&f.pts.to_le_bytes()); + digest.push(f.keyframe as u8); + digest.extend_from_slice(&f.data); + } + g.kv(&format!("{tag} streams"), streams.join(",")); + g.kv(&format!("{tag} frames"), n); + g.bytes(&format!("{tag} frame-digest"), &digest); + g.kv( + &format!("{tag} errors"), + format_args!("{} lost={}", s.errors(), s.lost_bytes()), + ); +} + +// ── Loose AACS and CSS file sources (input()) ────────────────────────────── + +fn flagged(clip: &[u8], key: Option<&[u8; 16]>) -> Vec { + let mut out = clip.to_vec(); + for p in out.chunks_mut(PKT) { + p[0] |= 0xC0; + } + if let Some(key) = key { + for u in out.as_chunks_mut::().0 { + assert!(encrypt_unit(u, key)); + } + } + out +} + +fn record_source(g: &mut Golden, tag: &str, scheme: &str, bytes: &[u8], opts: &InputOptions) { + // Keep the temp file until the stream is drained (the sector pipeline reads lazily). + let p = std::env::temp_dir().join(format!("fmkv-parity-{tag}-{}", std::process::id())); + std::fs::write(&p, bytes).unwrap(); + let r = input(&format!("{scheme}://{}", p.display()), opts); + match r { + Ok(mut s) => { + let mut digest = Vec::new(); + let mut n = 0u64; + let failed = loop { + match s.read() { + Ok(Some(f)) => { + n += 1; + digest.extend_from_slice(&(f.track as u64).to_le_bytes()); + digest.extend_from_slice(&f.pts.to_le_bytes()); + digest.extend_from_slice(&f.data); + } + Ok(None) => break None, + Err(e) => break Some(crate::error::error_code(&e).unwrap_or(0)), + } + }; + g.kv(&format!("{tag} frames"), n); + g.bytes(&format!("{tag} frame-digest"), &digest); + g.kv(&format!("{tag} blanked"), s.errors()); + if let Some(code) = failed { + g.kv(&format!("{tag} read-failed"), format_args!("E{code}")); + } + } + Err(e) => { + g.kv( + &format!("{tag} refused"), + format_args!("E{}", crate::error::error_code(&e).unwrap_or(0)), + ); + } + } + let _ = std::fs::remove_file(&p); +} + +fn keyed(keys: &[[u8; 16]]) -> InputOptions { + InputOptions { + keys: Some(ResolvedKeySet::held_for_test(keys)), + ..Default::default() + } +} + +#[test] +fn parity_sources_loose_m2ts_aacs() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let mut g = golden("parity_sources_loose_m2ts_aacs"); + let clip = synthetic_bd_clip(UNITS); + let enc = flagged(&clip, Some(&KEY)); + record_source(&mut g, "clear", "m2ts", &clip, &Default::default()); + record_source(&mut g, "clear-keyed", "m2ts", &clip, &keyed(&[KEY])); + record_source( + &mut g, + "stale-cpi", + "m2ts", + &flagged(&clip, None), + &Default::default(), + ); + record_source( + &mut g, + "aacs-held-key", + "m2ts", + &enc, + &keyed(&[[0x11; 16], KEY]), + ); + record_source(&mut g, "aacs-no-keys", "m2ts", &enc, &Default::default()); + record_source( + &mut g, + "aacs-wrong-key", + "m2ts", + &enc, + &keyed(&[[0x22; 16]]), + ); + let raw = InputOptions { + raw: true, + ..Default::default() + }; + record_source(&mut g, "aacs-raw", "m2ts", &enc, &raw); + let mut trunc = enc.clone(); + trunc.truncate(enc.len() - ALIGNED_UNIT_LEN + 20 * PKT); + record_source(&mut g, "aacs-truncated", "m2ts", &trunc, &keyed(&[KEY])); + let mut damaged = enc.clone(); + crate::test_util::damage_unit_seed(&mut damaged[4 * ALIGNED_UNIT_LEN..5 * ALIGNED_UNIT_LEN]); + record_source( + &mut g, + "aacs-damaged-unit", + "m2ts", + &damaged, + &keyed(&[KEY]), + ); + g.check(); +} + +// The sink's own `.mpg` of the synthetic clip (clear), as a DVD-like program stream. +pub(crate) fn clear_mpg() -> Vec { + let clip = synthetic_bd_clip(UNITS * 2); + let dir = tempfile::tempdir().unwrap(); + let src = dir.path().join("c.m2ts"); + std::fs::write(&src, &clip).unwrap(); + let dst = dir.path().join("c.mpg"); + mux_with_keys( + MuxSource::Url { + url: &format!("m2ts://{}", src.display()), + opts: Default::default(), + }, + None, + &format!("mpg://{}", dst.display()), + &MuxOptions::default(), + &Halt::new(), + Arc::new(NoopEvents), + ) + .unwrap(); + std::fs::read(&dst).unwrap() +} + +// `clear` with every whole-sector video pack CSS-scrambled under `key`. +pub(crate) fn scramble_vob(clear: &[u8], key: &[u8; 5]) -> Vec { + let mut vob = clear.to_vec(); + for pk in vob.as_chunks_mut::<2048>().0 { + if pk[13] & 7 == 0 && pk[17] == 0xE0 && pk[0x14] & 0x30 == 0 { + pk[0x14] |= 0x10; + crate::css::lfsr::scramble_sector(key, pk); + } + } + vob +} + +fn css_vob() -> (Vec, Vec) { + let clear = clear_mpg(); + let vob = scramble_vob(&clear, &[0x42, 0x13, 0x37, 0xBE, 0xEF]); + (clear, vob) +} + +#[test] +fn parity_sources_loose_mpg_css() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let mut g = golden("parity_sources_loose_mpg_css"); + let (clear, vob) = css_vob(); + record_source(&mut g, "clear", "mpg", &clear, &Default::default()); + record_source(&mut g, "scrambled", "mpg", &vob, &Default::default()); + let raw = InputOptions { + raw: true, + ..Default::default() + }; + record_source(&mut g, "scrambled-raw", "mpg", &vob, &raw); + // A scrambled stream under a clear container's scheme is refused by content, not name. + record_source(&mut g, "scrambled-as-mkv", "mkv", &vob, &Default::default()); + sinks( + &mut g, + "mpg", + &vob, + &["mkv", "m2ts"], + &MuxOptions::default(), + &Default::default(), + ); + g.check(); +} diff --git a/src/mux/resolve.rs b/src/mux/resolve.rs index 38f66f46..79ce778b 100644 --- a/src/mux/resolve.rs +++ b/src/mux/resolve.rs @@ -16,6 +16,7 @@ use super::stdio::StdioStream; use super::{M2tsStream, MkvStream}; use crate::disc::{ContentFormat, DiscTitle}; use crate::sector::SectorSource; +use crate::sector::stage::Stage; use std::io; use std::path::{Path, PathBuf}; @@ -327,6 +328,28 @@ fn validate_network_addr(addr: &str) -> io::Result<()> { Ok(()) } +/// The AACS disc folder a loose Blu-ray clip sits in, found only by walking up from `file`: +/// `/BDMV/STREAM/x.m2ts` (or `STREAM/SSIF/x.ssif`) with `/AACS`. Open `` as +/// `dir://`, resolve its keys, and pass them in [`InputOptions::keys`]. `None`: no disc +/// structure, so an encrypted clip refuses (E7022). A `.vob` needs none: CSS self-cracks. +pub fn disc_root_of(file: &Path) -> Option { + let file = std::fs::canonicalize(file).ok()?; + let up = |p: &Path, name: &str| { + let parent = p.parent()?; + let dir = parent.file_name()?.to_str()?; + dir.eq_ignore_ascii_case(name).then(|| parent.to_path_buf()) + }; + let stream = up(&file, "STREAM").or_else(|| up(&up(&file, "SSIF")?, "STREAM"))?; + let root = up(&stream, "BDMV")?.parent()?.to_path_buf(); + let aacs = std::fs::read_dir(&root).ok()?.flatten().any(|e| { + e.file_name() + .to_str() + .is_some_and(|n| n.eq_ignore_ascii_case("AACS")) + && e.path().is_dir() + }); + aacs.then_some(root) +} + /// Options for opening an input stream. #[derive(Clone, Default)] pub struct InputOptions { @@ -415,29 +438,43 @@ pub(crate) fn input_with_halt( } StreamUrl::M2ts { ref path } => { validate_file_path(path, "m2ts")?; - let file = std::fs::File::open(path)?; - let reader = std::io::BufReader::with_capacity(IO_BUF_SIZE, file); - let stream = build_m2ts_pipeline(reader)?; - Ok(Box::new(stream)) + let len = std::fs::metadata(path)?.len(); + let stage = crate::sector::DecryptingSectorSource::detecting( + Box::new(crate::io::file_sector_source::FileSectorSource::open_padded(path)?) + as Box, + stage_options(opts, halt, opts.raw), + ); + let blanked = stage.blanked_counter(); + let reader = crate::sector::stage::SectorBytes::new(stage, len); + Ok(Box::new(build_m2ts_pipeline(reader)?.with_blanked(blanked))) } StreamUrl::Mkv { ref path } => { validate_file_path(path, "mkv")?; - let file = std::fs::File::open(path)?; - let reader = std::io::BufReader::with_capacity(IO_BUF_SIZE, file); + let staged = Stage::eager(std::fs::File::open(path)?, opts.raw)?; + let reader = std::io::BufReader::with_capacity(IO_BUF_SIZE, staged); Ok(Box::new(MkvStream::open(reader)?)) } StreamUrl::Network { ref addr } => { validate_network_addr(addr)?; - Ok(Box::new(NetworkStream::listen_with_halt( + Ok(Box::new(NetworkStream::listen_staged( addr, halt.cloned(), + opts.raw, )?)) } - StreamUrl::Stdio => Ok(Box::new(StdioStream::input())), + StreamUrl::Stdio => Ok(Box::new(StdioStream::input_staged(opts.raw))), StreamUrl::Null => Err(crate::error::Error::StreamWriteOnly.into()), // `mp4://` as a source: demux a progressive MP4 back into PES frames, so // `mp4://` flows to every sink (mkv://, audio://, json://, …). - StreamUrl::Mp4 { ref path } => Ok(Box::new(super::mp4::Mp4Reader::open(path)?)), + StreamUrl::Mp4 { ref path } => { + let name = path + .file_stem() + .and_then(|s| s.to_str()) + .unwrap_or("mp4") + .to_string(); + let staged = Stage::eager(std::fs::File::open(path)?, opts.raw)?; + Ok(Box::new(super::mp4::Mp4Reader::from_reader(staged, name)?)) + } StreamUrl::Mpg { ref path } => { validate_file_path(path, "mpg")?; Ok(Box::new(build_ps_pipeline(path, opts, halt)?)) @@ -713,9 +750,8 @@ pub(crate) fn output_with( Ok(Box::new(M2tsStream::create(writer, title)?)) } StreamUrl::Network { ref addr } => { - // `NetworkStream::connect` re-resolves the host and refuses - // loopback/private/link-local/multicast — the SSRF/rebinding - // guard, not in `validate_network_addr` (shared with listen). + // `NetworkStream::connect` refuses only unspecified/multicast/ + // broadcast targets; LAN and loopback hosts are allowed. validate_network_addr(addr)?; Ok(Box::new(NetworkStream::connect(addr)?.meta(title))) } @@ -998,9 +1034,40 @@ fn iso_pipeline_tail( ) } -// An `mpg://` source (design §4 step 2, J13): the sector path, one extent over the zero-padded -// file. Crack first (a scrambled `.vob` descrambles, a clear one passes), scan the decrypted -// head, then build the DVD pipeline with the RESOLVED keys, so its own crack is a no-op. +// The decryption stage's options for a content-detected input. +fn stage_options( + opts: &InputOptions, + halt: Option<&crate::halt::Halt>, + raw: bool, +) -> crate::sector::decrypting::StageOptions { + crate::sector::decrypting::StageOptions { + raw, + keys: opts.keys.clone(), + halt: halt.cloned(), + } +} + +// The first `sectors` of `src`, read in `batch`-sector reads. +fn read_head(src: &mut dyn SectorSource, sectors: u32, batch: u16) -> io::Result> { + let mut head = vec![0u8; sectors as usize * 2048]; + let mut at = 0u32; + while at < sectors { + let count = (sectors - at).min(u32::from(batch)) as u16; + let off = at as usize * 2048; + src.read_sectors( + at, + count, + &mut head[off..off + count as usize * 2048], + false, + ) + .map_err(|e| -> io::Error { e.into() })?; + at += u32::from(count); + } + Ok(head) +} + +// An `mpg://` source (design §4 step 2, J13): the file → the decryption stage → the PS +// pipeline. The stage cracks once (D3) on its first read; the head scan reads through it. fn build_ps_pipeline( path: &Path, opts: &InputOptions, @@ -1008,8 +1075,15 @@ fn build_ps_pipeline( ) -> io::Result { const PS_MUX_BATCH_SECTORS: u16 = 8192; const HEAD_SECTORS: u32 = 2048; // 4 MiB - let mut reader = crate::io::file_sector_source::FileSectorSource::open_padded(path)?; - let capacity = reader.capacity_sectors(); + let open = |raw: bool| -> io::Result<_> { + let file = crate::io::file_sector_source::FileSectorSource::open_padded(path)?; + Ok(crate::sector::DecryptingSectorSource::detecting( + Box::new(file) as Box, + stage_options(opts, halt, raw), + )) + }; + let mut stage = open(opts.raw)?; + let capacity = stage.capacity_sectors(); if capacity == 0 { return Err(crate::error::Error::NoStreams.into()); } @@ -1017,62 +1091,23 @@ fn build_ps_pipeline( start_lba: 0, sector_count: capacity, }; - // The crack's verdict (`None` = clear) is final: the pipeline never scans again (D3). - let mut keys = crate::decrypt::DecryptKeys::None; - // B2: CSS scrambles DVD-Video's 13818-1 packs only; an 11172-1 system stream (the - // first pack's '0010') cannot be CSS, so it is never cracked. - let probe = capacity.min(32); - let mut first = vec![0u8; probe as usize * 2048]; - reader - .read_sectors(0, probe as u16, &mut first, false) - .map_err(|e| -> io::Error { e.into() })?; - let mpeg2 = first - .windows(5) - .find(|w| w[..4] == crate::css::PACK_START) - .is_some_and(|w| w[4] >> 6 == 0b01); - if mpeg2 { - // Design §4 step 2.1: "css::resolve_dvd_title_key(… raw = false, halt)". Even a - // raw read cracks, so the head scan sees the streams; the mux itself stays raw. - let cracked = crate::css::resolve_ps_file_title_key( - &mut reader, - &[extent], - PS_MUX_BATCH_SECTORS, - halt, - ); - match cracked { - Ok(k) => keys = k, - // `--raw` never hard-fails on scrambled-uncrackable: scan the ciphertext. - Err(e) - if opts.raw - && crate::error::error_code(&e) == Some(crate::error::E_CSS_KEY_MISSING) => {} - Err(e) => return Err(e), - } - } - let title_key = match keys { - crate::decrypt::DecryptKeys::Css { title_key } => Some(title_key), - _ => None, - }; - let mut head_reader = PsFileCss { - inner: crate::io::file_sector_source::FileSectorSource::open_padded(path)?, - title_key, - refuse: false, - }; let n = capacity.min(HEAD_SECTORS); - let mut head = vec![0u8; n as usize * 2048]; - let mut at = 0u32; - while at < n { - let count = (n - at).min(u32::from(PS_MUX_BATCH_SECTORS)) as u16; - let off = at as usize * 2048; - head_reader - .read_sectors( - at, - count, - &mut head[off..off + count as usize * 2048], - false, - ) - .map_err(|e| -> io::Error { e.into() })?; - at += u32::from(count); - } + // `--raw` still scans a descrambled head where a crack reaches one; the mux stays raw. + let head = if opts.raw { + match read_head(&mut open(false)?, n, PS_MUX_BATCH_SECTORS) { + Err(e) + if matches!( + crate::error::error_code(&e), + Some(crate::error::E_CSS_KEY_MISSING | crate::error::E_NO_DISC_KEY) + ) => + { + read_head(&mut stage, n, PS_MUX_BATCH_SECTORS)? + } + head => head?, + } + } else { + read_head(&mut stage, n, PS_MUX_BATCH_SECTORS)? + }; let scan = super::mpg::scan::scan(&head) .ok_or_else(|| -> io::Error { crate::error::Error::NoStreams.into() })?; let streams = scan.streams; @@ -1091,76 +1126,15 @@ fn build_ps_pipeline( opts.selection .apply(&mut title) .map_err(|e| -> io::Error { e.into() })?; - // CSS is PsFileCss's (m1/m2): descrambled where each pack's own stuffing puts its - // flags, and a scrambled pack with no key refused rather than muxed as ciphertext. - let reader = PsFileCss { - inner: reader, - title_key: title_key.filter(|_| !opts.raw), - refuse: !opts.raw, + let plan = IsoPlan { + extents: vec![extent], + full_extents: vec![extent], + batch_sectors: PS_MUX_BATCH_SECTORS, + unit_align: 1, + format: ContentFormat::MpegPs, }; - build_iso_pipeline( - reader, - title, - crate::decrypt::DecryptKeys::None, - PS_MUX_BATCH_SECTORS, - ContentFormat::MpegPs, - // `true` skips build_iso_pipeline's own crack: the verdict above is final (D3). - true, - halt.cloned(), - None, - ) - .map(|p| p.with_video_stream_id(scan.video_id)) -} - -/// An `mpg://` file's CSS layer (design §4 step 2, m1/m2): with the title key, each scrambled -/// pack is descrambled where its own pack stuffing puts the PES flags; without one, a -/// scrambled pack is `CssKeyMissing` (E7023) unless `refuse` is off (`--raw`). -struct PsFileCss { - inner: S, - title_key: Option<[u8; 5]>, - refuse: bool, -} - -impl SectorSource for PsFileCss { - fn capacity_sectors(&self) -> u32 { - self.inner.capacity_sectors() - } - - fn set_speed(&mut self, kbs: u16) { - self.inner.set_speed(kbs); - } - - fn set_unit_base(&mut self, lba: u32) { - self.inner.set_unit_base(lba); - } - - fn unmapped_stream_files(&self) -> &[crate::sector::bus_removal::UnmappedStreamFile] { - self.inner.unmapped_stream_files() - } - - fn random_access(&self) -> bool { - self.inner.random_access() - } - - fn read_sectors( - &mut self, - lba: u32, - count: u16, - buf: &mut [u8], - recovery: bool, - ) -> crate::error::Result { - let n = self.inner.read_sectors(lba, count, buf, recovery)?; - let len = n.min(buf.len()); - let got = &mut buf[..len]; - match &mut self.title_key { - Some(key) => crate::css::descramble_ps_region(got, key), - None if self.refuse && got.chunks(2048).any(crate::css::is_scrambled_ps_pack) => { - return Err(crate::error::Error::CssKeyMissing); - } - None => {} - } - Ok(n) - } + iso_pipeline_tail(stage, plan, title, halt.cloned(), None) + .map(|p| p.with_video_stream_id(scan.video_id)) } // Assemble the M2TS file mux pipeline (read -> demux -> parse). Scans the head @@ -1936,6 +1910,7 @@ mod tests { sec[4] = 0x44; // '01': a 13818-1 pack sec[0x0D] = 0xF8; // pack_stuffing_length 0 sec[0x14] = 0x10; // scramble flag + crate::css::dvd_pack_header(&mut sec, 0xE0); for (i, b) in sec.iter_mut().enumerate().skip(0x80) { *b = (i as u8) ^ 0x3C; } diff --git a/src/mux/stage_tests.rs b/src/mux/stage_tests.rs new file mode 100644 index 00000000..abddce69 --- /dev/null +++ b/src/mux/stage_tests.rs @@ -0,0 +1,347 @@ +//! The decryption stage as every input sees it: `input()` of a loose `.m2ts` (AACS), and +//! the content-detected pass-through of files that hold nothing encrypted. + +use crate::aacs::content::{ALIGNED_UNIT_LEN, encrypt_unit}; +use crate::consts::BD_SOURCE_PACKET_BYTES as PKT; +use crate::error::{E_CSS_KEY_MISSING, E_MP4_INVALID, E_NO_DISC_KEY, E_NO_STREAMS, error_code}; +use crate::keys::ResolvedKeySet; +use crate::mux::resolve::{InputOptions, input}; +use crate::pes::PesFrame; + +// A synthetic test key, not key material from any disc. +const KEY: [u8; 16] = [0x5A; 16]; +const AUDIO_PID: u16 = 0x1100; +const PMT_PID: u16 = 0x0100; +const UNITS: usize = 6; + +// One BD source packet: 4-byte TP_extra_header (CPI 00) + a TS packet on `pid`. +fn packet(pid: u16, pusi: bool, cc: u8, payload: &[u8]) -> Vec { + let mut p = vec![0xFFu8; PKT]; + p[..4].copy_from_slice(&[0, 0, 0, 0]); + p[4] = 0x47; + p[5] = ((pid >> 8) as u8 & 0x1F) | if pusi { 0x40 } else { 0 }; + p[6] = pid as u8; + p[7] = 0x10 | (cc & 0x0F); + let n = payload.len().min(184); + p[8..8 + n].copy_from_slice(&payload[..n]); + p +} + +fn pat() -> Vec { + let s = [ + 0x00, 0x00, 0xB0, 0x0D, 0x00, 0x01, 0xC1, 0x00, 0x00, 0x00, 0x01, + ]; + let mut body = s.to_vec(); + body.extend_from_slice(&[0xE0 | (PMT_PID >> 8) as u8, PMT_PID as u8, 0, 0, 0, 0]); + packet(0, true, 0, &body) +} + +fn pmt() -> Vec { + let mut b = vec![0x00, 0x02, 0xB0, 9 + 5 + 4, 0x00, 0x01, 0xC1, 0x00, 0x00]; + b.extend_from_slice(&[0xE0, 0x00, 0xF0, 0x00]); + b.extend_from_slice(&[ + 0x0F, + 0xE0 | (AUDIO_PID >> 8) as u8, + AUDIO_PID as u8, + 0xF0, + 0x00, + ]); + b.extend_from_slice(&[0, 0, 0, 0]); + packet(PMT_PID, true, 0, &b) +} + +// A clear BD-style clip of `UNITS` whole aligned units: PAT, PMT, then one audio PES per +// packet with a payload unique to its index. +fn clear_clip() -> Vec { + let mut clip = [pat(), pmt()].concat(); + let mut i = 0u32; + while clip.len() < UNITS * ALIGNED_UNIT_LEN { + let mut pes = vec![0, 0, 1, 0xC0, 0, 0, 0x80, 0x00, 0x00]; + pes.extend((0..175u32).map(|k| (i.wrapping_mul(31).wrapping_add(k) % 253) as u8)); + clip.extend(packet(AUDIO_PID, true, i as u8, &pes)); + i += 1; + } + clip +} + +// `clip` with every packet's CPI set to 11₂ (KS-5) and, when `key` is given, every whole +// aligned unit encrypted under it (KS-3, KS-4). +fn flagged(clip: &[u8], key: Option<&[u8; 16]>) -> Vec { + let mut out = clip.to_vec(); + for p in out.chunks_mut(PKT) { + p[0] |= 0xC0; + } + if let Some(key) = key { + for u in out.as_chunks_mut::().0 { + assert!(encrypt_unit(u, key)); + } + } + out +} + +fn path(tag: &str) -> std::path::PathBuf { + static N: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + let n = N.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + std::env::temp_dir().join(format!("fmkv-stage-{tag}-{}-{n}.m2ts", std::process::id())) +} + +// Every frame of `bytes` read as `scheme://`, and the stream's blanked-unit count. +fn read(scheme: &str, bytes: &[u8], opts: &InputOptions) -> std::io::Result<(Vec, u64)> { + let p = path(scheme); + std::fs::write(&p, bytes).unwrap(); + let got = (|| { + let mut s = input(&format!("{scheme}://{}", p.display()), opts)?; + let mut frames = Vec::new(); + while let Some(f) = s.read()? { + frames.push(f); + } + Ok((frames, s.errors())) + })(); + let _ = std::fs::remove_file(&p); + got +} + +fn keyed() -> InputOptions { + InputOptions { + keys: Some(ResolvedKeySet::held_for_test(&[[0x11; 16], KEY])), + ..Default::default() + } +} + +fn data(frames: &[PesFrame]) -> Vec> { + frames.iter().map(|f| f.data.clone()).collect() +} + +#[test] +fn a_clear_bd_m2ts_reads_its_frames() { + let (frames, blanked) = read("m2ts", &clear_clip(), &Default::default()).unwrap(); + assert!(frames.len() > 100, "{} frames", frames.len()); + assert_eq!(blanked, 0); +} + +// KS-1: every aligned unit of the clip file is encrypted; the held key opens them, the +// frames match the clear clip's, and the stage is the only decrypt on the way. +#[test] +fn an_aacs_m2ts_is_decrypted_with_the_held_keys() { + let clear = data(&read("m2ts", &clear_clip(), &Default::default()).unwrap().0); + let enc = flagged(&clear_clip(), Some(&KEY)); + let (frames, blanked) = read("m2ts", &enc, &keyed()).expect("decrypts"); + assert_eq!(data(&frames), clear); + assert_eq!(blanked, 0); +} + +// The refusal comes from `input()` itself: the stream never opens, so no frame exists. +#[test] +fn an_aacs_m2ts_with_no_keys_is_refused_before_any_frame() { + let p = path("nokeys"); + std::fs::write(&p, flagged(&clear_clip(), Some(&KEY))).unwrap(); + let opened = input(&format!("m2ts://{}", p.display()), &Default::default()); + let _ = std::fs::remove_file(&p); + assert_eq!( + opened.err().and_then(|e| error_code(&e)), + Some(E_NO_DISC_KEY) + ); +} + +#[test] +fn an_aacs_m2ts_no_held_key_opens_is_refused() { + let enc = flagged(&clear_clip(), Some(&KEY)); + let opts = InputOptions { + keys: Some(ResolvedKeySet::held_for_test(&[[0x22; 16]])), + ..Default::default() + }; + let e = read("m2ts", &enc, &opts).unwrap_err(); + assert_eq!(error_code(&e), Some(E_NO_DISC_KEY)); +} + +// `--raw` never refuses on a missing key: the ciphertext reaches the demuxer, whose PMT scan +// finds nothing it can read. +#[test] +fn an_aacs_m2ts_under_raw_is_not_a_key_refusal() { + let enc = flagged(&clear_clip(), Some(&KEY)); + let opts = InputOptions { + raw: true, + ..Default::default() + }; + let e = read("m2ts", &enc, &opts).unwrap_err(); + assert_eq!(error_code(&e), Some(E_NO_STREAMS)); +} + +// A clear clip whose decrypter left CPI 11₂ (KS-5 says 00₂) is clear by structure: no key. +#[test] +fn a_clear_m2ts_with_stale_cpi_needs_no_key() { + let clear = data(&read("m2ts", &clear_clip(), &Default::default()).unwrap().0); + let (frames, blanked) = read("m2ts", &flagged(&clear_clip(), None), &Default::default()) + .expect("clear content opens without keys"); + assert_eq!(data(&frames), clear); + assert_eq!(blanked, 0); +} + +// Review: a damaged unit (10 of 31 packets off sync) in a stale-CPI clear clip is not +// mistaken for ciphertext: the clip opens with or without keys. +#[test] +fn a_damaged_unit_in_a_stale_cpi_clip_is_not_a_key_refusal() { + let mut clip = flagged(&clear_clip(), None); + let unit = &mut clip[3 * ALIGNED_UNIT_LEN..4 * ALIGNED_UNIT_LEN]; + for p in unit.chunks_mut(PKT).skip(1).take(10) { + p[4] = 0x9D; + } + for opts in [Default::default(), keyed()] { + let (frames, blanked) = read("m2ts", &clip, &opts).expect("opens"); + assert!( + frames.len() > 100 && blanked == 0, + "{} {blanked}", + frames.len() + ); + } +} + +// Review: an encrypted clip behind a zero-filled start (11 blank units, past the 32-sector +// head) is judged by its first written unit: decrypted with the held keys, refused without. +#[test] +fn a_blank_head_does_not_hide_an_aacs_clip() { + let clear = data(&read("m2ts", &clear_clip(), &Default::default()).unwrap().0); + let enc = [ + vec![0; 11 * ALIGNED_UNIT_LEN], + flagged(&clear_clip(), Some(&KEY)), + ] + .concat(); + let (frames, _) = read("m2ts", &enc, &keyed()).expect("decrypts"); + assert_eq!(data(&frames), clear); + let e = read("m2ts", &enc, &Default::default()).unwrap_err(); + assert_eq!(error_code(&e), Some(E_NO_DISC_KEY)); +} + +// Review: a blank run inside the 32-sector head (28 sectors) does not hide the clip either. +#[test] +fn a_short_blank_run_does_not_hide_an_aacs_clip() { + let mut enc = flagged(&clear_clip().repeat(3), Some(&KEY)); + enc[..28 * 2048].fill(0); + let e = read("m2ts", &enc, &Default::default()).unwrap_err(); + assert_eq!(error_code(&e), Some(E_NO_DISC_KEY)); +} + +// A truncated copy: the encrypted partial unit at the end cannot be opened as a unit, so it +// is blanked and counted (E7013 option A), never muxed as ciphertext. +#[test] +fn a_truncated_aacs_m2ts_blanks_its_partial_unit() { + let mut clip = clear_clip(); + let whole = data(&read("m2ts", &clip, &Default::default()).unwrap().0); + let mut extra = clip[2 * PKT..2 * PKT + ALIGNED_UNIT_LEN].to_vec(); + for (j, p) in extra.chunks_mut(PKT).enumerate() { + p[7] = 0x10 | (j as u8 & 0x0F); + } + clip.extend(extra); + let mut enc = flagged(&clip, Some(&KEY)); + enc.truncate(UNITS * ALIGNED_UNIT_LEN + 20 * PKT); + let (frames, blanked) = read("m2ts", &enc, &keyed()).expect("decrypts"); + assert_eq!(blanked, 1, "the partial unit is blanked and counted"); + assert_eq!(data(&frames)[..whole.len()], whole[..]); +} + +// freemkv's own m2ts (an FMKV header, then TS off the unit grid) is always clear. +#[test] +fn an_fmkv_m2ts_still_reads() { + let title = crate::mux::meta::M2tsMeta::from_title(&{ + let mut t = crate::disc::DiscTitle::empty(); + t.streams = crate::mux::ts::scan_streams(&clear_clip()).unwrap(); + t + }); + let mut file = Vec::new(); + crate::mux::meta::write_header(&mut file, &title).unwrap(); + file.extend(clear_clip()); + let clear = data(&read("m2ts", &clear_clip(), &Default::default()).unwrap().0); + let (frames, _) = read("m2ts", &file, &keyed()).expect("reads"); + assert_eq!(data(&frames), clear); +} + +// A CSS-scrambled pack stream. +fn scrambled_ps() -> Vec { + let mut out = Vec::new(); + for scramble in [0u8, 0x10] { + let mut p = vec![0x5Au8; 2048]; + p[..4].copy_from_slice(&crate::css::PACK_START); + p[4] = 0x44; + p[0x0D] = 0xF8; + p[0x0E..0x12].copy_from_slice(&[0, 0, 1, 0xE0]); + p[0x12..0x14].copy_from_slice(&0x07ECu16.to_be_bytes()); + p[0x14] = 0x80 | scramble; + out.extend(p); + } + out +} + +// Every file arm passes the stage: scrambled content under a clear container's scheme is +// judged by its bytes and refused. MP4's box walk seeks past what the stage has judged and +// fails as not-MP4 (E9049) first; either way no ciphertext reaches a sink. +#[test] +fn every_file_scheme_passes_the_stage() { + let enc = flagged(&clear_clip(), Some(&KEY)); + for (bytes, code) in [(scrambled_ps(), E_CSS_KEY_MISSING), (enc, E_NO_DISC_KEY)] { + let e = read("mkv", &bytes, &Default::default()).unwrap_err(); + assert_eq!(error_code(&e), Some(code), "mkv"); + let e = read("mp4", &bytes, &Default::default()).unwrap_err(); + assert!(matches!(error_code(&e), Some(c) if c == code || c == E_MP4_INVALID)); + } +} + +// §8.5: every file arm builds the stage around its source, whatever the bytes turn out to be. +#[test] +fn every_file_arm_builds_the_stage() { + for scheme in ["mpg", "m2ts", "mkv", "mp4"] { + let before = crate::sector::stage::STAGES.with(|n| n.get()); + let _ = read( + scheme, + &vec![0x5A; 4 * ALIGNED_UNIT_LEN], + &Default::default(), + ); + let built = crate::sector::stage::STAGES.with(|n| n.get()) - before; + assert!(built >= 1, "{scheme}"); + } +} + +// S7: `--raw` mpg:// reads its head through a non-raw stage only where that stage can; an +// AACS clip under the mpg scheme refuses there (E7022) and the raw head is used instead. +#[test] +fn a_raw_mpg_head_scan_falls_back_on_a_key_refusal() { + let opts = InputOptions { + raw: true, + ..Default::default() + }; + let e = read("mpg", &flagged(&clear_clip(), Some(&KEY)), &opts).err(); + assert_eq!(e.and_then(|e| error_code(&e)), Some(E_NO_STREAMS)); +} + +// A loose file's keys are found only by walking up to its disc folder (never a sidecar). +#[test] +fn a_loose_clip_finds_its_disc_folder_by_walking_up() { + use crate::mux::resolve::disc_root_of; + let root = std::env::temp_dir().join(format!("fmkv-root-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + let stream = root.join("BDMV/STREAM"); + std::fs::create_dir_all(&stream).unwrap(); + let clip = stream.join("00001.m2ts"); + std::fs::write(&clip, b"").unwrap(); + std::fs::write(root.join("00001.m2ts"), b"").unwrap(); + let root = std::fs::canonicalize(&root).unwrap(); + assert_eq!( + disc_root_of(&clip), + None, + "no AACS folder: no disc structure" + ); + std::fs::create_dir_all(root.join("AACS")).unwrap(); + assert_eq!(disc_root_of(&clip), Some(root.clone())); + assert_eq!( + disc_root_of(&root.join("00001.m2ts")), + None, + "not under BDMV/STREAM" + ); + // N11: a 3D clip's interleaved file sits one level deeper. + std::fs::create_dir_all(stream.join("SSIF")).unwrap(); + std::fs::write(stream.join("SSIF/00001.ssif"), b"").unwrap(); + assert_eq!( + disc_root_of(&stream.join("SSIF/00001.ssif")), + Some(root.clone()) + ); + let _ = std::fs::remove_dir_all(&root); +} diff --git a/src/mux/stdio.rs b/src/mux/stdio.rs index 96d6b860..a0620a6f 100644 --- a/src/mux/stdio.rs +++ b/src/mux/stdio.rs @@ -32,7 +32,15 @@ pub struct StdioStream { impl StdioStream { /// Create a stdio stream for reading (stdin). pub fn input() -> Self { - Self::from_reader(Box::new(io::stdin())) + Self::input_staged(false) + } + + // `input` whose decryption stage passes ciphertext when `raw`. + pub(crate) fn input_staged(raw: bool) -> Self { + Self::from_reader(Box::new(crate::sector::stage::Stage::lazy( + io::stdin(), + raw, + ))) } fn from_reader(reader: Box) -> Self { diff --git a/src/mux/ts.rs b/src/mux/ts.rs index c98cad42..3444c678 100644 --- a/src/mux/ts.rs +++ b/src/mux/ts.rs @@ -706,10 +706,17 @@ fn ts_payload_base(pkt: &[u8]) -> Option { } } -// Per PID, the MPEG audio layer (1-3) from the frame header at the start of its first PES -// in `data` (`None` when that PES has no such header), found in one walk over `data`. -fn mpeg_audio_layers(data: &[u8]) -> std::collections::HashMap> { - let mut layers = std::collections::HashMap::new(); +// The audio sync at the head of a PES's ES: ADTS (13818-7 layer '00') or MPEG audio Layer 1-3. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum AudioSync { + Adts, + Layer(u8), +} + +// Per PID, the stream_id and audio sync opening its first PES in `data` (`None` when that +// PES has no PES/sync header), found in one walk over `data`. +fn first_pes_heads(data: &[u8]) -> std::collections::HashMap)>> { + let mut heads = std::collections::HashMap::new(); let mut offset = 0; while offset + BD_SOURCE_PACKET_BYTES <= data.len() { if !is_resync_point(data, offset) { @@ -719,30 +726,33 @@ fn mpeg_audio_layers(data: &[u8]) -> std::collections::HashMap> let pkt = &data[offset..offset + BD_SOURCE_PACKET_BYTES]; let pkt_pid = (((pkt[5] & 0x1F) as u16) << 8) | pkt[6] as u16; if pkt[5] & 0x40 != 0 { - layers - .entry(pkt_pid) - .or_insert_with(|| first_pes_layer(pkt)); + heads.entry(pkt_pid).or_insert_with(|| first_pes_head(pkt)); } offset += BD_SOURCE_PACKET_BYTES; } - layers + heads } -// MPEG audio layer of the frame header opening the PES that starts in `pkt`. -fn first_pes_layer(pkt: &[u8]) -> Option { +// stream_id and audio sync of the PES that starts in `pkt`. +fn first_pes_head(pkt: &[u8]) -> Option<(u8, Option)> { let pes = &pkt[ts_payload_base(pkt)?..]; if pes.get(..3)? != [0, 0, 1] { return None; } - let es = pes.get(9 + *pes.get(8)? as usize..)?; - let (b0, b1) = (*es.first()?, *es.get(1)?); - if b0 != 0xFF || b1 & 0xE0 != 0xE0 { - return None; - } - match (b1 >> 1) & 0x03 { - 0 => None, - l => Some(4 - l), - } + let id = *pes.get(3)?; + let es = pes.get(9 + *pes.get(8)? as usize..); + let sync = es.and_then(|es| { + let (b0, b1) = (*es.first()?, *es.get(1)?); + if b0 != 0xFF || b1 & 0xE0 != 0xE0 { + return None; + } + match (b1 >> 1) & 0x03 { + 0 if b1 & 0xF0 == 0xF0 => Some(AudioSync::Adts), + 0 => None, + l => Some(AudioSync::Layer(4 - l)), + } + }); + Some((id, sync)) } // Reassemble a single PSI section (PAT/PMT) for `target_pid`/`table_id` across TS-packet @@ -909,7 +919,7 @@ pub fn scan_streams(data: &[u8]) -> Option> { let prog_info_len = ((((pmt[10] & 0x0F) as usize) << 8) | pmt[11] as usize).min(end.saturating_sub(12)); let mut pos = 12 + prog_info_len; - let mut layers = None; + let mut heads = None; while pos + 5 <= end { let stream_type = pmt[pos]; @@ -918,17 +928,25 @@ pub fn scan_streams(data: &[u8]) -> Option> { // PMTs may also carry ISO/IEC 13818-1 audio stream types that are // absent from Blu-ray's STN table. Keep that distinction local to TS. + let mut head = || { + heads + .get_or_insert_with(|| first_pes_heads(data)) + .get(&es_pid) + .copied() + .flatten() + }; let codec = match stream_type { // MPEG-1/2 audio covers Layers I-III; only the ES says which. - 0x03 | 0x04 - if layers - .get_or_insert_with(|| mpeg_audio_layers(data)) - .get(&es_pid) - == Some(&Some(3)) => - { - Codec::Mp3 - } + 0x03 | 0x04 if head().and_then(|h| h.1) == Some(AudioSync::Layer(3)) => Codec::Mp3, 0x03 | 0x04 => Codec::Mp2, + // PES private data (as other m2ts muxers write it): an MPEG-audio stream_id plus the ES + // sync names AAC/MP2/MP3 (13818-1 Table 2-22); anything else stays unknown. + 0x06 => match head() { + Some((0xC0..=0xDF, Some(AudioSync::Adts))) => Codec::Aac, + Some((0xC0..=0xDF, Some(AudioSync::Layer(3)))) => Codec::Mp3, + Some((0xC0..=0xDF, Some(AudioSync::Layer(_)))) => Codec::Mp2, + _ => Codec::Unknown(stream_type), + }, 0x0f => Codec::Aac, 0x87 => Codec::Ac3Plus, _ => Codec::from_coding_type(stream_type), @@ -1786,6 +1804,32 @@ mod tests { } } + // Other m2ts muxers declare AAC/MP2/MP3 as PES private data (0x06); the + // MPEG-audio stream_id and the ES sync name the codec. + #[test] + fn scan_streams_types_private_data_audio_from_its_es() { + use crate::disc::{Codec, Stream}; + for (id, header, expected) in [ + (0xC0, [0xFF, 0xF1], Some(Codec::Aac)), + (0xC1, [0xFF, 0xFB], Some(Codec::Mp3)), + (0xC2, [0xFF, 0xFD], Some(Codec::Mp2)), + (0xBD, [0xFF, 0xFD], None), + ] { + let mut pes = vec![0x00, 0x00, 0x01, id, 0x00, 0x00, 0x80, 0x00, 0x00]; + pes.extend_from_slice(&header); + pes.extend_from_slice(&[0x50, 0x80]); + let mut data = pat_packet(0x100); + data.extend(pmt_packet(0x100, &[(0x1b, 0x1011), (0x06, 0x1100)])); + data.extend(es_packet_exact(0x1100, true, &pes)); + let streams = scan_streams(&data).unwrap(); + let got = streams.iter().find_map(|s| match s { + Stream::Audio(a) => Some(a.codec), + _ => None, + }); + assert_eq!(got, expected, "stream_id {id:#x}"); + } + } + #[test] fn scan_streams_maps_lpcm_via_from_coding_type() { use crate::disc::{Codec, Stream}; diff --git a/src/mux/tsmux.rs b/src/mux/tsmux.rs index 9c682a5b..98b70706 100644 --- a/src/mux/tsmux.rs +++ b/src/mux/tsmux.rs @@ -41,6 +41,27 @@ const ORIGIN_HEADROOM_TICKS: i64 = 90_000; const HOLD_CAP_TICKS: u64 = 90_000; const HOLD_CAP_BYTES: usize = 64 * 1024 * 1024; +/// PID of the program_map_section (BD-ROM convention); no stream may use it. +pub(crate) const PMT_PID: u16 = 0x0100; +/// PID of the adaptation-field-only PCR packets (BD-ROM convention); no stream may use it. +pub(crate) const PCR_PID: u16 = 0x1001; +// System clock (27 MHz) of a declared program: a frame's packets arrive this far before its +// DTS, the T-STD delay cap (13818-1 §2.4.2.6), at most at 128 Mb/s (the UHD BD TS rate). +const STC_LEAD: u64 = 27_000_000; +const PACKET_STC: u64 = 188 * 8 * 27_000_000 / 128_000_000; +// PCR at most 100 ms apart (BD-ROM, 13818-1 §2.7.2); a timeline jump past 10 s restarts it. +const PCR_INTERVAL: u64 = 2_700_000; +const STC_RESET: u64 = 10 * 27_000_000; +// PAT/PMT repeat every 100 ms of PTS progress (the ETSI TR 101 290 PSI interval); a PTS +// this far below the highest seen is a discontinuity and restarts the clock. +const PSI_INTERVAL_TICKS: u64 = 9_000; +const PSI_RESET_TICKS: u64 = 10 * 90_000; +// HDMV registration_descriptor (13818-1 §2.6.8): the program's 0x80-0xFF stream_types are +// BD-ROM's, e.g. 0x80 = BD LPCM. +const HDMV_REGISTRATION: [u8; 6] = [0x05, 0x04, b'H', b'D', b'M', b'V']; +// ES_info loop entries fit one 1024-byte section: 1021 - 13 fixed - 6 descriptor bytes. +const MAX_PMT_ENTRIES: usize = (1021 - 13 - HDMV_REGISTRATION.len()) / 5; + /// A frame held, in arrival order, while a video track's start-up window is open. struct Held { track: usize, @@ -100,6 +121,21 @@ pub struct TsMuxer { hold_span: Option<(u64, u64)>, /// Warn once at `finish` when a DTS anomaly was counted. warned: bool, + /// Per-track PMT stream_type; `None` writes no PAT/PMT. + stream_types: Option>, + /// Continuity counters of the PAT and PMT PIDs. + psi_cc: [u8; 2], + /// (PTS of the last PAT/PMT, highest PTS since): decode-order PTS jitter and + /// interleaved tracks do not re-send them. + last_psi: Option<(u64, u64)>, + /// Arrival time of the next packet on the 27 MHz system clock; `None` before a + /// declared program's first frame. + stc: Option, + /// The last PCR written, and whether the next one flags a discontinuity. + last_pcr: Option, + pcr_discontinuity: bool, + /// Frames whose packets arrived after their DTS (input interleave beyond `STC_LEAD`). + late_frames: u64, } impl TsMuxer { @@ -122,9 +158,162 @@ impl TsMuxer { hold_bytes: 0, hold_span: None, warned: false, + stream_types: None, + psi_cc: [0; 2], + last_psi: None, + stc: None, + last_pcr: None, + pcr_discontinuity: false, + late_frames: 0, } } + /// Declare each track's PMT stream_type: the output then carries a PAT and an HDMV + /// PMT (PID [`PMT_PID`]) before the first PES and every 100 ms of PTS after it. + pub(crate) fn set_program(&mut self, stream_types: Vec) -> io::Result<()> { + if stream_types.len() != self.pids.len() { + return Err(crate::error::Error::MuxTrackRange { + track: stream_types.len(), + tracks: self.pids.len(), + } + .into()); + } + self.stream_types = Some(stream_types); + Ok(()) + } + + // PES stream_id: video, MPEG audio for 11172-3/13818-3/13818-7 stream_types + // (13818-1 Table 2-22), else private_stream_1 as BD-ROM uses. + fn stream_id(&self, track: usize) -> u8 { + use crate::consts::pes_stream_id; + if is_video_pid(self.pids[track]) { + return pes_stream_id::VIDEO; + } + match self.stream_types.as_ref().map(|t| t[track]) { + Some(0x03 | 0x04 | 0x0F) => pes_stream_id::MPEG_AUDIO, + _ => pes_stream_id::PRIVATE_STREAM_1, + } + } + + // PAT and PMT, when declared and due at `pts`. + fn write_psi_if_due(&mut self, pts: u64) -> io::Result<()> { + let Some(types) = &self.stream_types else { + return Ok(()); + }; + if let Some((sent, hi)) = &mut self.last_psi { + *hi = (*hi).max(pts); + if pts < *sent + PSI_INTERVAL_TICKS && pts + PSI_RESET_TICKS >= *hi { + return Ok(()); + } + } + if types.len() > MAX_PMT_ENTRIES && self.last_psi.is_none() { + tracing::warn!(target: "mux", tracks = types.len(), "bd-ts: PMT lists the first {MAX_PMT_ENTRIES} tracks only"); + } + self.last_psi = Some((pts, pts)); + let mut pat = vec![0x00, 0x01, 0xC1, 0x00, 0x00, 0x00, 0x01]; + pat.extend_from_slice(&(0xE000 | PMT_PID).to_be_bytes()); + let pat = psi_section(0x00, &pat); + let mut pmt = vec![0x00, 0x01, 0xC1, 0x00, 0x00]; + pmt.extend_from_slice(&(0xE000 | PCR_PID).to_be_bytes()); + pmt.extend_from_slice(&(0xF000 | HDMV_REGISTRATION.len() as u16).to_be_bytes()); + pmt.extend_from_slice(&HDMV_REGISTRATION); + for (&t, &pid) in types.iter().zip(&self.pids).take(MAX_PMT_ENTRIES) { + pmt.push(t); + pmt.extend_from_slice(&(0xE000 | pid).to_be_bytes()); + pmt.extend_from_slice(&[0xF0, 0x00]); + } + let pmt = psi_section(0x02, &pmt); + self.write_psi(0, 0, &pat)?; + self.write_psi(1, PMT_PID, &pmt) + } + + // One PSI section as TS packets: pointer_field 0, then the section, 0xFF-stuffed. + fn write_psi(&mut self, slot: usize, pid: u16, section: &[u8]) -> io::Result<()> { + let payload = [&[0u8][..], section].concat(); + for (i, chunk) in payload.chunks(TS_PAYLOAD_BYTES).enumerate() { + let cc = self.psi_cc[slot]; + self.psi_cc[slot] = (cc + 1) & 0x0F; + let pusi = if i == 0 { 0x40 } else { 0 }; + let mut pkt = [0xFFu8; 192]; + pkt[..4].copy_from_slice(&self.arrival()?); + pkt[4..8].copy_from_slice(&[SYNC_BYTE, pusi | (pid >> 8) as u8, pid as u8, 0x10 | cc]); + pkt[8..8 + chunk.len()].copy_from_slice(chunk); + self.writer.write_all(&pkt)?; + } + Ok(()) + } + + // Set the system clock for a frame decoded at `dts` (90 kHz): STC_LEAD before it, never + // backwards; a jump past STC_RESET either way restarts it. + fn clock_frame(&mut self, dts: u64) { + if self.stream_types.is_none() { + return; + } + let (dts, target) = (dts * 300, (dts * 300).saturating_sub(STC_LEAD)); + let stc = match self.stc { + Some(stc) if target <= stc + STC_RESET && stc <= target + STC_RESET => { + self.late_frames += u64::from(dts < stc); + stc.max(target) + } + Some(_) => { + self.pcr_discontinuity = true; + self.last_pcr = None; + target + } + None => target, + }; + self.stc = Some(stc); + } + + // TP_extra_header of the next packet: its arrival_time_stamp (low 30 bits of the system + // clock), after any PCR packet due by then. All zero without a declared program. + fn arrival(&mut self) -> io::Result<[u8; 4]> { + let Some(mut t) = self.stc else { + return Ok([0; 4]); + }; + loop { + let at = match self.last_pcr { + Some(p) if t < p + PCR_INTERVAL => break, + Some(p) => t.min(p + PCR_INTERVAL), + None => t, + }; + self.write_pcr(at)?; + if at == t { + t += PACKET_STC; + } + } + self.stc = Some(t + PACKET_STC); + Ok(((t & 0x3FFF_FFFF) as u32).to_be_bytes()) + } + + // An adaptation-field-only packet on PCR_PID carrying PCR `at` (13818-1 §2.4.3.4); its + // continuity_counter does not advance (no payload). + fn write_pcr(&mut self, at: u64) -> io::Result<()> { + let base = (at / 300) & 0x1_FFFF_FFFF; + let ext = at % 300; + let flags = 0x10 | if self.pcr_discontinuity { 0x80 } else { 0 }; + let mut pkt = [0xFFu8; 192]; + pkt[..4].copy_from_slice(&((at & 0x3FFF_FFFF) as u32).to_be_bytes()); + pkt[4..14].copy_from_slice(&[ + SYNC_BYTE, + (PCR_PID >> 8) as u8, + PCR_PID as u8, + 0x20, + 183, + flags, + (base >> 25) as u8, + (base >> 17) as u8, + (base >> 9) as u8, + (base >> 1) as u8, + ]); + pkt[14] = ((base & 1) as u8) << 7 | 0x7E | (ext >> 8) as u8; + pkt[15] = ext as u8; + self.writer.write_all(&pkt)?; + self.last_pcr = Some(at); + self.pcr_discontinuity = false; + Ok(()) + } + /// Mark `track` as the MVC dependent view of `base` (design §2.3). Its own deriver /// takes the base's R and frame period; the PTS sequences match, so the DTS match /// whichever view arrives first (SSIF reads deliver the dependent first). Returns @@ -312,6 +501,8 @@ impl TsMuxer { let pid = self.pids[track]; let is_video = is_video_pid(pid); let dts_90k = if is_video { self.next_dts(track) } else { None }; + self.clock_frame(dts_90k.unwrap_or(pts_90k)); + self.write_psi_if_due(pts_90k)?; // NAL video (HEVC/H.264): convert length-prefixed NALUs to Annex B and prepend // codec_private params on the first keyframe only (arm `params_written` even if @@ -413,7 +604,7 @@ impl TsMuxer { es_data: &[u8], ) -> io::Result<()> { let pid = self.pids[track]; - let pes_header = build_pes_header(pid, times, es_data.len()); + let pes_header = build_pes_header(self.stream_id(track), times, es_data.len()); // Logical PES packet = header bytes followed by es_data. It is // indexed (and written) in place, without materializing the @@ -443,8 +634,7 @@ impl TsMuxer { (TS_PAYLOAD_BYTES - remaining, remaining) }; - // TP_extra_header (4 bytes — arrival time, set to 0) - let tp_extra = [0u8; 4]; + let tp_extra = self.arrival()?; // TS header (4 bytes) let cc = self.continuity[track]; @@ -549,6 +739,13 @@ impl TsMuxer { "bd-ts: video DTS needed correction (PTS-only or guarded AUs)" ); } + if self.late_frames > 0 { + tracing::warn!( + late_frames = self.late_frames, + "bd-ts: frames arrive after their DTS (source interleave over 1 s)" + ); + self.late_frames = 0; + } if self.frame_count == 0 { return Err(crate::error::Error::MuxEmpty.into()); } @@ -584,15 +781,19 @@ fn push_timestamp(header: &mut Vec, prefix: u8, ts: u64) { header.push(0x01 | (((ts << 1) & 0xFE) as u8)); } +// A long-form PSI section: table_id, section_length, `body` (from the 16-bit id on), CRC_32. +fn psi_section(table_id: u8, body: &[u8]) -> Vec { + let len = (body.len() + 4) as u16; + let mut s = vec![table_id, 0xB0 | (len >> 8) as u8, len as u8]; + s.extend_from_slice(body); + let crc = super::mpg::pack::crc32(&s); + s.extend_from_slice(&crc.to_be_bytes()); + s +} + // Build a PES packet header for a BD stream. -fn build_pes_header(pid: u16, times: PesTimes, data_len: usize) -> Vec { +fn build_pes_header(stream_id: u8, times: PesTimes, data_len: usize) -> Vec { use crate::consts::pes_stream_id; - // Determine stream_id from PID range - let stream_id: u8 = if is_video_pid(pid) { - pes_stream_id::VIDEO - } else { - pes_stream_id::PRIVATE_STREAM_1 // audio, PGS subtitle, or default - }; // PES_header_data_length: 5 PTS bytes, plus 5 DTS bytes when present. let header_data_len: usize = match times { PesTimes::None => 0, @@ -657,6 +858,8 @@ mod tests { /// Parsed BD-TS packet (192 bytes total: 4 TP_extra + 4 TS header + 184 body). struct TsPacket { + /// TP_extra_header arrival_time_stamp (low 30 bits). + ats: u32, pid: u16, pusi: bool, #[allow(dead_code)] @@ -699,6 +902,7 @@ mod tests { }; out.push(TsPacket { + ats: u32::from_be_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]) & 0x3FFF_FFFF, pid, pusi, cc, @@ -1208,6 +1412,142 @@ mod tests { } } + // PSI repeats per 100 ms of PTS progress, not per frame whose PTS strays from the last + // PSI (interleaved tracks, decode-order jitter); a large backward jump re-sends at once. + #[test] + fn pat_repeats_per_100ms_of_pts_progress() { + let mut sink: Vec = Vec::new(); + { + let mut mux = TsMuxer::new(&mut sink, &[AUDIO_PID, AUDIO_PID + 1]); + mux.set_program(vec![0x81, 0x81]).unwrap(); + // Two tracks 500 ms apart, 10 ms frames: PSI at 0 and 0.5..=1.4 s, not ~200. + for n in 0..100i64 { + mux.write_frame(0, n * 10_000_000, true, &[0x0B, 0x77]) + .unwrap(); + mux.write_frame(1, 500_000_000 + n * 10_000_000, true, &[0x0B, 0x77]) + .unwrap(); + } + // 20 s is progress; back to 1.5 s is a discontinuity (> 10 s below the high). + for pts in [20_000_000_000, 1_500_000_000] { + mux.write_frame(0, pts, true, &[0x0B, 0x77]).unwrap(); + } + mux.finish().unwrap(); + } + let pats = parse_bd_ts(&sink).iter().filter(|p| p.pid == 0).count(); + assert_eq!(pats, 13); + } + + // A PES header's 33-bit PTS and DTS (DTS = PTS when absent), 90 kHz. + fn pes_pts_dts(payload: &[u8]) -> (u64, u64) { + let ts = |b: &[u8]| { + (u64::from(b[0] >> 1 & 7) << 30) + | (u64::from(b[1]) << 22) + | (u64::from(b[2] >> 1) << 15) + | (u64::from(b[3]) << 7) + | u64::from(b[4] >> 1) + }; + let pts = ts(&payload[9..14]); + let dts = if payload[7] & 0x40 != 0 { + ts(&payload[14..19]) + } else { + pts + }; + (pts, dts) + } + + // The PCR of an adaptation field (13818-1 §2.4.3.5): base * 300 + extension, 27 MHz. + fn af_pcr(af: &[u8]) -> Option { + (af.len() >= 7 && af[0] & 0x10 != 0).then(|| { + let base = (u64::from(u32::from_be_bytes([af[1], af[2], af[3], af[4]])) << 1) + | u64::from(af[5] >> 7); + base * 300 + (u64::from(af[5] & 1) << 8 | u64::from(af[6])) + }) + } + + // 13818-1 §2.4.3.5 / BD-ROM: a PCR on the PMT's PCR_PID at most 100 ms apart, before + // the first PES; arrival stamps on the same 27 MHz clock, never past a PES's DTS nor + // more than the 1 s T-STD delay (§2.4.2.6) ahead of it. + #[test] + fn pcr_paces_arrival_at_most_100ms_apart_and_never_past_a_dts() { + let mut sink: Vec = Vec::new(); + { + let mut mux = TsMuxer::new(&mut sink, &[VIDEO_PID, AUDIO_PID]); + mux.set_program(vec![0x24, 0x81]).unwrap(); + // 25 fps video + audio; frame 10 is a 2 MB keyframe (> 100 ms of packets), + // then a 400 ms gap from frame 30. + for i in 0..40i64 { + let pts = i * 40_000_000 + if i >= 30 { 400_000_000 } else { 0 }; + let size = if i == 10 { 2_000_000 } else { 3_000 }; + let nal = fake_hevc_nal(if i % 10 == 0 { 19 } else { 1 }, size); + mux.write_frame(0, pts, i % 10 == 0, &nal).unwrap(); + mux.write_frame(1, pts + 5_000_000, true, &[0x0B, 0x77, 0, 0]) + .unwrap(); + } + mux.finish().unwrap(); + } + let packets = parse_bd_ts(&sink); + let pmt = packets.iter().find(|p| p.pid == PMT_PID).unwrap(); + assert_eq!( + u16::from_be_bytes([pmt.payload[9], pmt.payload[10]]) & 0x1FFF, + PCR_PID + ); + let (mut last_ats, mut last_pcr, mut pes) = (0, None::, 0); + for p in &packets { + assert!(p.ats >= last_ats, "arrival stamps are monotonic"); + last_ats = p.ats; + if p.pid == PCR_PID { + let pcr = af_pcr(p.af.as_deref().unwrap()).unwrap(); + assert_eq!(pcr as u32 & 0x3FFF_FFFF, p.ats, "ATS on the PCR clock"); + if let Some(last) = last_pcr { + assert!(pcr > last && pcr - last <= 2_700_000, "PCR {last} -> {pcr}"); + } + last_pcr = Some(pcr); + } else if p.pusi && p.pid != PMT_PID && p.pid != 0 { + assert!(last_pcr.is_some(), "a PCR precedes the first PES"); + let dts = pes_pts_dts(&p.payload).1 * 300; + let ats = u64::from(p.ats); + assert!( + ats <= dts && dts - ats <= 27_000_000, + "ATS {ats} vs DTS {dts}" + ); + pes += 1; + } + } + assert_eq!(pes, 80); + } + + // A timeline jump back past the reset window restarts the clock, flagged by the + // discontinuity_indicator (13818-1 §2.4.3.5), instead of stamping every later PES late. + #[test] + fn a_backward_timeline_jump_restarts_the_pcr_with_a_discontinuity() { + let mut sink: Vec = Vec::new(); + { + let mut mux = TsMuxer::new(&mut sink, &[AUDIO_PID]); + mux.set_program(vec![0x81]).unwrap(); + for pts in [ + 0, + 5_000_000_000, + 10_000_000_000, + 15_000_000_000, + 1_000_000_000, + ] { + mux.write_frame(0, pts, true, &[0x0B, 0x77]).unwrap(); + } + mux.finish().unwrap(); + } + let pcrs: Vec<(u64, bool)> = parse_bd_ts(&sink) + .iter() + .filter(|p| p.pid == PCR_PID) + .map(|p| { + let af = p.af.as_deref().unwrap(); + (af_pcr(af).unwrap(), af[0] & 0x80 != 0) + }) + .collect(); + let (last, flagged) = pcrs[pcrs.len() - 1]; + assert!(flagged && last < pcrs[pcrs.len() - 2].0, "{pcrs:?}"); + assert!(pcrs[..pcrs.len() - 1].iter().all(|&(_, d)| !d)); + } + #[test] fn audio_es_round_trips_byte_for_byte_through_demuxer() { // The mux→demux round trip must preserve every audio ES byte. A diff --git a/src/sector/decrypting.rs b/src/sector/decrypting.rs index 7ba95e34..654b0d16 100644 --- a/src/sector/decrypting.rs +++ b/src/sector/decrypting.rs @@ -9,7 +9,7 @@ //! plaintext; `DecryptKeys::None` discs pass through unconditionally. use crate::decrypt::{DecryptKeys, decrypt_sectors, decrypt_sectors_in_content}; -use crate::error::Result; +use crate::error::{E_CSS_KEY_MISSING, Result}; use std::sync::Arc; use super::SectorSource; @@ -51,8 +51,29 @@ pub struct DecryptingSectorSource { arrival: Option>, /// Damaged AACS units blanked so far (see [`blanked_units`](Self::blanked_units)). blanked: BlankTally, + /// Content-detected stage: `Some` until the first read reaches the verdict. + detect: Option>, + /// Loose BD-TS: a CPI-flagged unit that is clean TS is clear (its flag is stale). + stale_cpi: bool, + /// A PS the crack left keyless: a scrambled pack read later is refused (E7023). + watch_css: bool, + /// The crack's verdict was scrambled-but-uncrackable: every read refuses (E7023). + refused: bool, } +/// What the content-detected stage may do with what it finds (from `InputOptions`). +#[derive(Clone, Default)] +pub(crate) struct StageOptions { + /// Pass ciphertext through: never crack, decrypt or refuse. + pub(crate) raw: bool, + /// Held AACS keys for a loose BD-TS file. + pub(crate) keys: Option, + pub(crate) halt: Option, +} + +// Sectors per CSS crack-scan read (the mpg:// batch). +const CRACK_BATCH: u16 = 8192; + // The blanked-unit count, shared with the stream that reports it as loss; logged once when // the reader is dropped, so a damaged rip never reads as clean. struct BlankTally(Arc); @@ -88,9 +109,71 @@ impl DecryptingSectorSource { key_map: None, arrival: None, blanked: BlankTally(Arc::default()), + detect: None, + stale_cpi: false, + watch_css: false, + refused: false, } } + /// The content-detected stage (every input but a disc, image or folder): the first read + /// classifies the head and resolves once (D3). A PS is cracked, a BD-TS read through + /// [`ResolvedKeySet::loose_file`](crate::keys::ResolvedKeySet::loose_file), else clear. + pub(crate) fn detecting(inner: S, opts: StageOptions) -> Self { + #[cfg(test)] + super::stage::STAGES.with(|n| n.set(n.get() + 1)); + let mut s = Self::new(inner, DecryptKeys::None); + s.detect = Some(Box::new(opts)); + s + } + + // Reach the verdict and install what it needs. On `Err` the stage stays undecided. + fn resolve_detect(&mut self) -> Result<()> { + use super::stage::{Kind, classify_sectors}; + let Some(opts) = self.detect.as_deref() else { + return Ok(()); + }; + let cap = self.inner.capacity_sectors(); + let head = detect_head(&mut self.inner, cap)?; + match classify_sectors(&head) { + Kind::Ps { mpeg2 } if !opts.raw => { + // B2: an 11172-1 stream cannot be CSS; it is never cracked. + if mpeg2 { + let whole = [crate::disc::Extent { + start_lba: 0, + sector_count: cap, + }]; + let halt = opts.halt.as_ref(); + let cracked = + crate::css::crack_title_key(&mut self.inner, &whole, CRACK_BATCH, halt); + self.keys = match cracked { + Ok(keys) => keys, + // A verdict, not a fault: kept, so a retry does not scan again (D3). + Err(e) if crate::error::error_code(&e) == Some(E_CSS_KEY_MISSING) => { + self.detect = None; + self.refused = true; + return Err(crate::error::Error::CssKeyMissing); + } + Err(e) => return Err(e.into()), + }; + } + self.watch_css = !self.keys.is_encrypted(); + } + Kind::BdTs if !opts.raw => { + let set = crate::keys::ResolvedKeySet::loose_file(opts.keys.as_ref(), cap); + self.keys = set.decrypt_keys(); + self.key_map = Some(set.key_map()); + self.arrival = set.arrival(set.title_stop()).map(Box::new); + self.unit_base = Some(0); + self.stale_cpi = true; + } + // Nothing seen that a key opens, yet a scrambled pack later is still refused. + _ => self.watch_css = !opts.raw, + } + self.detect = None; + Ok(()) + } + /// Install the key set's on-arrival proof (KU §2.4): an encrypted unit of a piece /// `resolve` left unproven is decrypted with a held key proven on it here. pub(crate) fn with_arrival(mut self, arrival: crate::keys::Arrival) -> Self { @@ -225,6 +308,12 @@ impl SectorSource for DecryptingSectorSource { recovery: bool, fua: bool, ) -> Result { + if self.refused { + return Err(crate::error::Error::CssKeyMissing); + } + if self.detect.is_some() { + self.resolve_detect()?; + } // Content-extent map (whole-disc readers): units outside the encrypted // extents are clear filesystem / nav and pass through untouched. Cheap Arc // bump; frees the &self borrow so we can decrypt against `&mut buf`. @@ -246,6 +335,7 @@ impl SectorSource for DecryptingSectorSource { let n = self .inner .read_sectors_fua(lba, count, buf, recovery, fua)?; + self.judge_detected(&mut buf[..n])?; // Read damage before any key sees the units: a damaged unit is blanked and counted, // never a key verdict (E7013 here, E7022 on arrival). "We rip bad discs." @@ -290,7 +380,72 @@ impl SectorSource for DecryptingSectorSource { } } +// The verdict's head: `HEAD_SECTORS` from the first written sector on the AACS unit grid, so +// a zero-filled bad-read start hides nothing. Blank sectors are skipped up to the crack budget. +fn detect_head(inner: &mut impl SectorSource, cap: u32) -> Result> { + use super::stage::HEAD_SECTORS; + const BLANK_BUDGET: u32 = 50_000; + const UNIT: u32 = crate::aacs::content::ALIGNED_UNIT_SECTORS; + let mut at = 0; + loop { + let head = read_head_at(inner, at, cap)?; + let blank = head + .chunks(crate::consts::SECTOR_BYTES) + .take_while(|s| s.iter().all(|&b| b == 0)) + .count() as u32; + let full = HEAD_SECTORS as usize * crate::consts::SECTOR_BYTES; + if (blank as usize) * crate::consts::SECTOR_BYTES < head.len() { + let start = (at + blank) / UNIT * UNIT; + return match start == at { + true => Ok(head), + false => read_head_at(inner, start, cap), + }; + } + if head.len() < full || at + HEAD_SECTORS >= BLANK_BUDGET { + return Ok(head); + } + at += HEAD_SECTORS; + } +} + +// Up to `HEAD_SECTORS` sectors at `lba`, fewer at the end of the source. +fn read_head_at(inner: &mut impl SectorSource, lba: u32, cap: u32) -> Result> { + let n = cap.saturating_sub(lba).min(super::stage::HEAD_SECTORS); + let mut head = vec![0u8; n as usize * crate::consts::SECTOR_BYTES]; + let got = match n { + 0 => 0, + n => inner.read_sectors(lba, n as u16, &mut head, false)?, + }; + head.truncate(got); + Ok(head) +} + impl DecryptingSectorSource { + // The content-detected stage's per-read rules, before any key sees the bytes: a stale CPI + // flag on clear TS is cleared (KS-5: "00₂ if the data is not encrypted"), and a + // keyless PS refuses a scrambled pack rather than pass ciphertext as clear. + fn judge_detected(&self, buf: &mut [u8]) -> Result<()> { + use crate::aacs::content::{ALIGNED_UNIT_LEN, aacs_unit_seed_encrypted}; + if self.stale_cpi { + let ts = crate::disc::ContentFormat::BdTs; + for unit in buf.chunks_mut(ALIGNED_UNIT_LEN) { + if aacs_unit_seed_encrypted(unit, ts) && super::stage::clear_ts(unit) { + for p in unit.chunks_mut(crate::consts::BD_SOURCE_PACKET_BYTES) { + p[0] &= 0x3F; + } + } + } + } + if self.watch_css + && buf + .chunks(crate::consts::SECTOR_BYTES) + .any(|c| crate::css::scrambled_at(c).is_some()) + { + return Err(crate::error::Error::CssKeyMissing); + } + Ok(()) + } + // Blank and count the damaged AACS units of a read at `lba` (see // `decrypt::blank_damaged_units`): judged are content units the map keys or arrival covers. fn blank_damage(&mut self, lba: u32, buf: &mut [u8], content: Option<&[(u32, u32)]>) { @@ -303,7 +458,10 @@ impl DecryptingSectorSource { && (map.is_some_and(|m| m.entry_for(at).is_some()) || arrival.is_some_and(|a| a.covers(at))) }; - let n = crate::decrypt::blank_damaged_units(buf, lba, format, &covered); + let cap = self.inner.capacity_sectors(); + let end = lba as u64 + buf.len().div_ceil(crate::consts::SECTOR_BYTES) as u64; + let at_end = cap != 0 && end >= cap as u64; + let n = crate::decrypt::blank_damaged_units(buf, lba, format, &covered, at_end); self.count_blanked(n); } @@ -353,6 +511,63 @@ mod tests { } } + // N8: an uncrackable scrambled stream's E7023 is a verdict: a retry refuses again without + // a second crack scan (D3). + #[test] + fn a_css_refusal_is_kept_for_the_retry() { + let mut pack: Vec = (0..2048u32) + .map(|i| (i as u8).wrapping_mul(7) ^ 0x3C) + .collect(); + pack[0x14] = 0x10; + crate::css::dvd_pack_header(&mut pack, 0xE0); + crate::css::lfsr::scramble_sector(&[0x11, 0x22, 0x33, 0x44, 0x55], &mut pack); + let src = crate::test_util::MemSource::new(pack.repeat(4)); + let opts = StageOptions { + raw: false, + keys: None, + halt: None, + }; + let mut stage = DecryptingSectorSource::detecting(src, opts); + let scans = crate::css::CRACK_SCANS.with(|n| n.get()); + let mut buf = vec![0u8; 2048]; + for _ in 0..2 { + let e = stage.read_sectors(0, 1, &mut buf, false).unwrap_err(); + assert_eq!(e.code(), crate::error::E_CSS_KEY_MISSING); + } + assert_eq!(crate::css::CRACK_SCANS.with(|n| n.get()), scans + 1); + } + + // Stale CPI is cleared only on clear TS: ciphertext that keeps a few syncs (the `is_clean` + // proof floor) stays flagged, while damaged packets do not hide a clear unit. + #[test] + fn stale_cpi_needs_half_the_packets_synced() { + use crate::aacs::content::ALIGNED_UNIT_LEN; + use crate::sector::stage::clear_ts; + let pkt = crate::consts::BD_SOURCE_PACKET_BYTES; + let mut unit: Vec = (0..ALIGNED_UNIT_LEN) + .map(|i| (i * 13 + 5) as u8 | 1) + .collect(); + for p in unit.chunks_mut(pkt) { + p[0] |= 0xC0; + p[4] = 0x47; + } + assert!(clear_ts(&unit)); + assert!(clear_ts(&unit[..20 * pkt]), "a clear partial tail"); + // Review #2: ten damaged packets of 31 leave a clear unit clear. + for p in unit.chunks_mut(pkt).skip(1).take(10) { + p[4] = 0x9D; + } + assert!(clear_ts(&unit), "ten damaged packets"); + for (i, p) in unit.chunks_mut(pkt).enumerate().skip(1) { + p[4] = if i < 5 { 0x47 } else { 0x9D }; + } + assert!(crate::aacs::content::is_clean( + &unit, + crate::disc::ContentFormat::BdTs + )); + assert!(!clear_ts(&unit), "four synced packets are not clear TS"); + } + // A DecryptingSectorSource must relay its inner source's unmapped list. #[test] fn decrypting_source_forwards_unmapped_stream_files() { @@ -422,11 +637,9 @@ mod tests { for (i, b) in buf.iter_mut().enumerate() { *b = (i as u8).wrapping_mul(29).wrapping_add(3); } - // A real MPEG-2 pack header, or `is_scrambled_pack` skips the sector. - buf[..4].copy_from_slice(&[0, 0, 1, 0xBA]); - buf[4] = 0x44; - buf[0x11] = 0xE0; + // A real DVD-Video pack header, or `is_scrambled_pack` skips the sector. buf[0x14] = 0x30; // scramble-control bits set → flags == 0x03 + crate::css::dvd_pack_header(buf, 0xE0); } } impl SectorSource for ShortReportSource { @@ -728,6 +941,7 @@ mod tests { template[4] = 0x44; // '01': a 13818-1 pack template[0x0D] = 0xF8; // pack_stuffing_length 0 template[0x14] = 0x30; // scramble bits (4-5) set → flags == 0x03 + crate::css::dvd_pack_header(&mut template, 0xE0); let pristine = template; // Start with None → pass-through (no descramble, flags stay set). diff --git a/src/sector/mod.rs b/src/sector/mod.rs index 40a33151..0da7a05a 100644 --- a/src/sector/mod.rs +++ b/src/sector/mod.rs @@ -11,6 +11,7 @@ pub mod bus_removal; pub mod decrypting; pub mod prefetched; +pub(crate) mod stage; use crate::error::Result; diff --git a/src/sector/stage.rs b/src/sector/stage.rs new file mode 100644 index 00000000..4b2c0ce1 --- /dev/null +++ b/src/sector/stage.rs @@ -0,0 +1,667 @@ +//! The decryption stage's content verdict, and its byte-stream forms (crate-private and +//! transitional: the pipeline's chunk stream replaces the sector/byte split). +//! +//! Every `input()` source passes the stage. A sector source (`mpg://`, `m2ts://`) is wrapped +//! by [`DecryptingSectorSource::detecting`](super::DecryptingSectorSource::detecting); a byte +//! reader (`mkv://`, `mp4://`, `network://`, `stdio://`) by [`Stage`], which hands a clear +//! container through untouched and refuses encrypted content it cannot decrypt. The verdict +//! comes from the bytes alone ([`classify`]), never from the URL scheme. + +use crate::aacs::content::{ALIGNED_UNIT_LEN, aacs_unit_seed_encrypted, is_clean}; +use crate::consts::{BD_SOURCE_PACKET_BYTES, SECTOR_BYTES}; +use crate::css::{PACK_START, scrambled_at}; +use crate::disc::ContentFormat; +use crate::error::Error; +use crate::sector::SectorSource; +use std::io::{self, Read, Seek, SeekFrom}; + +#[cfg(test)] +thread_local! { + /// Stages built on this thread (test-only: every `input()` arm must build one). + pub(crate) static STAGES: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + +/// Sectors a sector source's verdict is read from. +pub(crate) const HEAD_SECTORS: u32 = 32; +// Bytes a file's verdict is read from, and a stream's (one aligned unit: its latency bound). +const FILE_HEAD: usize = HEAD_SECTORS as usize * SECTOR_BYTES; +const STREAM_HEAD: usize = ALIGNED_UNIT_LEN; +// Aligned units a BD-TS verdict checks at most. +const TS_UNITS: usize = 16; + +/// What a stream's head says it holds. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum Kind { + /// Nothing CSS or AACS can scramble (MKV, MP4, FMKV, arbitrary bytes): pass-through. + Opaque, + /// An MPEG program stream on the 2048-byte pack grid; `mpeg2` is the 13818-1 marker. + Ps { mpeg2: bool }, + /// BD source packets (4-byte TP_extra_header + TS packet) on the 6144-byte unit grid. + BdTs, +} + +/// The stream kind `head` (the stream's first bytes) shows. A PS opens with a pack, as every +/// DVD-Video VOB does; BD-TS needs the seed sync of every head unit plus clean TS or a CPI flag. +pub(crate) fn classify(head: &[u8]) -> Kind { + // freemkv's own m2ts/network/stdio output: clear, and its header is off the unit grid. + if head.starts_with(b"FMKV") { + return Kind::Opaque; + } + if let Some(ps) = pack_at(head) { + return ps; + } + if is_bd_ts(head) { + Kind::BdTs + } else { + Kind::Opaque + } +} + +/// [`classify`] for a sector source (a file): a VOB whose first sectors are damaged or blank +/// is still a PS by its first sector-aligned pack. +pub(crate) fn classify_sectors(head: &[u8]) -> Kind { + match classify(head) { + Kind::Opaque if !head.starts_with(b"FMKV") => head + .chunks(SECTOR_BYTES) + .find_map(pack_at) + .unwrap_or(Kind::Opaque), + kind => kind, + } +} + +// A pack header opening `s`: 13818-1 ('01', a start code past its stuffing) or 11172-1 +// ('0010', a start code at byte 12). An MP4 box 0x1BA bytes long is neither. +fn pack_at(s: &[u8]) -> Option { + if s.get(..4)? != PACK_START { + return None; + } + let mpeg2 = s.get(4)? >> 6 == 0b01; + let at = match mpeg2 { + true => 0x0E + usize::from(s.get(0x0D)? & 0x07), + false if s[4] >> 4 == 0b0010 => 12, + false => return None, + }; + (s.get(at..at + 3)? == [0, 0, 1]).then_some(Kind::Ps { mpeg2 }) +} + +// KS-2: a source packet is "the TP_extra_header (4 bytes) and an MPEG Transport packet"; KS-4: +// a unit's clear seed keeps its sync. A damaged head unit must not hide the rest: half the +// non-blank head units, and two when there are two, must show that shape. +fn is_bd_ts(head: &[u8]) -> bool { + const PKT: usize = BD_SOURCE_PACKET_BYTES; + let units = head.as_chunks::().0; + let units = &units[..units.len().min(TS_UNITS)]; + if units.is_empty() { + let pkts = head.as_chunks::().0; + return !pkts.is_empty() && pkts.iter().all(|p| p[4] == 0x47); + } + let live = units.iter().filter(|u| u.iter().any(|&b| b != 0)).count(); + let shaped = units + .iter() + .filter(|u| u[4] == 0x47 && (is_clean(*u, ContentFormat::BdTs) || u[0] & 0xC0 != 0)) + .count(); + shaped >= live.clamp(1, 2) && shaped * 2 >= live +} + +// Clear TS: the seed's packet synced and half the other non-padding packets (16 of a whole +// unit's 31), not the 4-packet key-proof floor of `is_clean` (ciphertext passes ~7e-6). Damage +// to a few packets must not make a clear unit look encrypted; ciphertext passes this ~1e-30. +pub(crate) fn clear_ts(chunk: &[u8]) -> bool { + let pkts = chunk.as_chunks::().0; + let Some((seed, rest)) = pkts.split_first() else { + return false; + }; + let content = rest.iter().filter(|p| p[4..].iter().any(|&b| b != 0)); + let (n, synced) = content.fold((0, 0), |(n, s), p| (n + 1, s + usize::from(p[4] == 0x47))); + seed[4] == 0x47 && synced * 2 >= n +} + +// A BD-TS unit only a key opens: CPI-flagged and not clear TS. +fn needs_key(unit: &[u8]) -> bool { + aacs_unit_seed_encrypted(unit, ContentFormat::BdTs) && !clear_ts(unit) +} + +// `head` filled up to `max` bytes from `r`, fewer only at EOF; bytes read before an error stay. +fn fill_head(r: &mut impl Read, head: &mut Vec, max: usize) -> io::Result<()> { + let want = max.saturating_sub(head.len()) as u64; + r.by_ref().take(want).read_to_end(head).map(|_| ()) +} + +// Up to `max` bytes from the start of `r`, fewer only at EOF. +fn read_head(r: &mut impl Read, max: usize) -> io::Result> { + let mut head = Vec::with_capacity(max); + fill_head(r, &mut head, max)?; + Ok(head) +} + +/// The stage over a byte reader: a clear container passes untouched; PS or BD-TS content is +/// watched, and an encrypted pack or unit is refused (E7023 / E7022) unless `raw`. A byte +/// reader cannot crack or prove a key (no side reads, X-2), so it never decrypts. +pub(crate) struct Stage { + inner: R, + raw: bool, + kind: Option, + // A lazily read head, served before `inner`. + head: Vec, + served: usize, + // The watched block being assembled, and bytes to skip to the next block boundary. + carry: Vec, + skip: usize, +} + +impl Stage { + /// A stream (`network://`, `stdio://`): the verdict is read on the first read. + pub(crate) fn lazy(inner: R, raw: bool) -> Self { + #[cfg(test)] + STAGES.with(|n| n.set(n.get() + 1)); + Self { + inner, + raw, + kind: None, + head: Vec::new(), + served: 0, + carry: Vec::new(), + skip: 0, + } + } + + pub(crate) fn get_ref(&self) -> &R { + &self.inner + } + + /// The verdict, once reached. + #[cfg(test)] + pub(crate) fn kind(&self) -> Option { + self.kind + } + + fn block(&self) -> Option { + match self.kind? { + _ if self.raw => None, + Kind::Opaque => None, + Kind::Ps { .. } => Some(SECTOR_BYTES), + Kind::BdTs => Some(ALIGNED_UNIT_LEN), + } + } + + // Judge the bytes just read as whole aligned blocks complete. + fn judge(&mut self, mut bytes: &[u8]) -> io::Result<()> { + let Some(block) = self.block() else { + return Ok(()); + }; + let skip = self.skip.min(bytes.len()); + self.skip -= skip; + bytes = &bytes[skip..]; + while !bytes.is_empty() { + let take = (block - self.carry.len()).min(bytes.len()); + self.carry.extend_from_slice(&bytes[..take]); + bytes = &bytes[take..]; + if self.carry.len() == block { + let refuse = match self.kind { + Some(Kind::BdTs) => needs_key(&self.carry).then(|| Error::NoDiscKey { + disc_hash: String::new(), + }), + _ => scrambled_at(&self.carry).map(|_| Error::CssKeyMissing), + }; + self.carry.clear(); + if let Some(e) = refuse { + return Err(e.into()); + } + } + } + Ok(()) + } +} + +impl Stage { + /// A seekable file (`mkv://`, `mp4://`): the verdict is read now, then `inner` is rewound. + pub(crate) fn eager(mut inner: R, raw: bool) -> io::Result { + let head = read_head(&mut inner, FILE_HEAD)?; + inner.seek(SeekFrom::Start(0))?; + let mut s = Self::lazy(inner, raw); + s.kind = Some(classify(&head)); + Ok(s) + } +} + +impl Read for Stage { + fn read(&mut self, buf: &mut [u8]) -> io::Result { + if self.kind.is_none() { + // Five bytes rule a stream out (no pack start, no TS sync at byte 4), so an FMKV + // header is parsed as soon as it arrives; only a candidate waits for a whole unit. + fill_head(&mut self.inner, &mut self.head, 5)?; + if self.head.len() >= 5 && (self.head[..4] == PACK_START || self.head[4] == 0x47) { + fill_head(&mut self.inner, &mut self.head, STREAM_HEAD)?; + } + self.kind = Some(classify(&self.head)); + } + let n = if self.served < self.head.len() { + let n = (self.head.len() - self.served).min(buf.len()); + buf[..n].copy_from_slice(&self.head[self.served..self.served + n]); + self.served += n; + n + } else { + self.inner.read(buf)? + }; + self.judge(&buf[..n])?; + Ok(n) + } +} + +impl Seek for Stage { + fn seek(&mut self, to: SeekFrom) -> io::Result { + if self.served < self.head.len() { + return Err(io::ErrorKind::Unsupported.into()); + } + let at = self.inner.seek(to)?; + self.carry.clear(); + self.skip = self + .block() + .map_or(0, |b| (b - (at % b as u64) as usize) % b); + Ok(at) + } +} + +/// The byte view of a sector stage (`m2ts://`): its output as `Read`, clipped to `len` (the +/// file's real length, past its zero-padded last sector), read on the 6144-byte AACS grid. +pub(crate) struct SectorBytes { + src: S, + len: u64, + pos: u64, + buf: Vec, + buf_start: u64, +} + +// Sectors per refill: whole aligned units (the AACS read gate), ~1 MiB. +const REFILL_SECTORS: u16 = 510; + +impl SectorBytes { + pub(crate) fn new(src: S, len: u64) -> Self { + Self { + src, + len, + pos: 0, + buf: Vec::new(), + buf_start: 0, + } + } +} + +impl Read for SectorBytes { + fn read(&mut self, out: &mut [u8]) -> io::Result { + if self.pos >= self.len || out.is_empty() { + return Ok(0); + } + let buf_end = self.buf_start + self.buf.len() as u64; + if self.pos < self.buf_start || self.pos >= buf_end { + let unit = crate::aacs::content::ALIGNED_UNIT_SECTORS; + let lba = (self.pos / SECTOR_BYTES as u64) as u32 / unit * unit; + let left = self.src.capacity_sectors().saturating_sub(lba); + let count = left.min(u32::from(REFILL_SECTORS)) as u16; + if count == 0 { + return Err(io::ErrorKind::UnexpectedEof.into()); + } + self.buf.resize(count as usize * SECTOR_BYTES, 0); + let n = match self.src.read_sectors(lba, count, &mut self.buf, true) { + Ok(n) => n, + // A refused read must not leave its raw bytes to serve a retry. + Err(e) => { + self.buf.clear(); + return Err(e.into()); + } + }; + self.buf.truncate(n.min(self.buf.len())); + self.buf_start = lba as u64 * SECTOR_BYTES as u64; + if self.pos >= self.buf_start + self.buf.len() as u64 { + return Err(io::ErrorKind::UnexpectedEof.into()); + } + } + let at = (self.pos - self.buf_start) as usize; + let avail = (self.buf.len() - at).min((self.len - self.pos) as usize); + let n = avail.min(out.len()); + out[..n].copy_from_slice(&self.buf[at..at + n]); + self.pos += n as u64; + Ok(n) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::error::{E_CSS_KEY_MISSING, E_NO_DISC_KEY, error_code}; + use std::io::Cursor; + + // A DVD-Video pack: 14-byte pack header (stuffing 0), a PES of `id` at 0x0E whose + // MPEG-2 flags byte (0x14) carries `scramble`, and a varied body. + fn dvd_pack(id: u8, scramble: u8) -> Vec { + let mut p: Vec = (0..SECTOR_BYTES).map(|i| (i * 37 + 11) as u8).collect(); + p[..4].copy_from_slice(&PACK_START); + p[4] = 0x44; + p[0x0D] = 0xF8; + p[0x0E..0x11].copy_from_slice(&[0, 0, 1]); + p[0x11] = id; + p[0x12..0x14].copy_from_slice(&0x07ECu16.to_be_bytes()); + p[0x14] = 0x80 | (scramble << 4); + p + } + + fn ts_unit(cpi: bool) -> Vec { + let mut u: Vec = (0..ALIGNED_UNIT_LEN) + .map(|i| (i * 7 + 3) as u8 | 1) + .collect(); + for p in u.chunks_mut(BD_SOURCE_PACKET_BYTES) { + p[0] = if cpi { p[0] | 0xC0 } else { p[0] & 0x3F }; + p[4] = 0x47; + } + u + } + + fn noise(n: usize) -> Vec { + (0..n).map(|i| ((i * 2_654_435_761) >> 13) as u8).collect() + } + + #[test] + fn containers_and_arbitrary_bytes_are_opaque() { + let mut mkv = vec![0x1A, 0x45, 0xDF, 0xA3, 0x47]; + mkv.extend(noise(FILE_HEAD)); + let mut mp4 = vec![0, 0, 0, 0x20]; + mp4.extend(b"ftypisom"); + mp4.extend(noise(FILE_HEAD)); + let mut fmkv = b"FMKV\0\x01\0\0".to_vec(); + fmkv.extend(noise(FILE_HEAD)); + let mut ifo = b"DVDVIDEO-VTS".to_vec(); + ifo.extend(noise(FILE_HEAD)); + // A 188-byte TS (sync at byte 0, not 4) carries no AACS unit. + let mut ts188 = Vec::new(); + for _ in 0..200 { + ts188.push(0x47); + ts188.extend(noise(187)); + } + for (name, head) in [ + ("mkv", mkv), + ("mp4", mp4), + ("fmkv", fmkv), + ("ifo", ifo), + ("udf zeros", vec![0u8; FILE_HEAD]), + ("noise", noise(FILE_HEAD)), + ("ts188", ts188), + ("empty", Vec::new()), + ] { + assert_eq!(classify(&head), Kind::Opaque, "{name}"); + } + } + + #[test] + fn pack_streams_and_source_packets_are_recognised() { + let mut ps = dvd_pack(0xE0, 0); + ps.extend(noise(SECTOR_BYTES)); + assert_eq!(classify(&ps), Kind::Ps { mpeg2: true }); + let mut later = noise(SECTOR_BYTES); + later.extend(dvd_pack(0xE0, 1)); + assert_eq!( + classify(&later), + Kind::Opaque, + "a pack past the head of another stream" + ); + let mut mpeg1 = dvd_pack(0xE0, 0); + mpeg1[4] = 0x21; + mpeg1[12..15].copy_from_slice(&[0, 0, 1]); + assert_eq!(classify(&mpeg1), Kind::Ps { mpeg2: false }); + // Review: an MP4 whose first box is 0x1BA bytes opens `00 00 01 BA 'f'`. + let mut mp4 = vec![0, 0, 1, 0xBA]; + mp4.extend(b"ftypisom"); + mp4.extend(noise(FILE_HEAD)); + assert_eq!(classify(&mp4), Kind::Opaque); + let clear: Vec = (0..4).flat_map(|_| ts_unit(false)).collect(); + assert_eq!(classify(&clear), Kind::BdTs); + let mut enc = ts_unit(true); + enc[BD_SOURCE_PACKET_BYTES + 4] = 0x00; // an encrypted rest loses its syncs + for p in enc[BD_SOURCE_PACKET_BYTES..].chunks_mut(BD_SOURCE_PACKET_BYTES) { + p[4] = 0x5C; + } + assert_eq!(classify(&enc), Kind::BdTs); + let mut noisy = noise(ALIGNED_UNIT_LEN); + noisy[0] |= 0xC0; + noisy[4] = 0x47; + noisy.extend(noise(ALIGNED_UNIT_LEN)); + assert_eq!( + classify(&noisy), + Kind::Opaque, + "a second unit with no seed sync" + ); + } + + // An Opaque stream is handed through byte-identical, even if a later sector looks like a + // scrambled pack (a VOB muxed into an MKV attachment stays untouched). + #[test] + fn an_opaque_stream_passes_untouched() { + let mut bytes = b"FMKV\0\x01\0\0".to_vec(); + bytes.resize(SECTOR_BYTES, 0); + bytes.extend(dvd_pack(0xE0, 1)); + bytes.extend(ts_unit(true)); + let mut stage = Stage::lazy(Cursor::new(bytes.clone()), false); + let mut out = Vec::new(); + stage.read_to_end(&mut out).unwrap(); + assert_eq!(out, bytes); + assert_eq!(stage.kind(), Some(Kind::Opaque)); + } + + #[test] + fn a_byte_stream_refuses_encrypted_content_unless_raw() { + let mut ps = dvd_pack(0xE0, 0); + ps.extend(dvd_pack(0xE0, 1)); + let mut ts = ts_unit(false); + let mut enc = ts_unit(true); + for p in enc[BD_SOURCE_PACKET_BYTES..].chunks_mut(BD_SOURCE_PACKET_BYTES) { + p[4] = 0x5C; + } + ts.extend(enc); + for (bytes, code) in [(ps, E_CSS_KEY_MISSING), (ts, E_NO_DISC_KEY)] { + let mut out = Vec::new(); + let e = Stage::lazy(Cursor::new(bytes.clone()), false) + .read_to_end(&mut out) + .unwrap_err(); + assert_eq!(error_code(&e), Some(code)); + let mut raw = Vec::new(); + Stage::lazy(Cursor::new(bytes.clone()), true) + .read_to_end(&mut raw) + .unwrap(); + assert_eq!(raw, bytes, "raw passes ciphertext"); + } + } + + // A block straddling reads is judged once complete; a seek re-syncs to the next boundary. + #[test] + fn watched_blocks_are_judged_across_reads_and_seeks() { + let mut ps = dvd_pack(0xE0, 0); + ps.extend(dvd_pack(0xE0, 0)); + ps.extend(dvd_pack(0xE0, 2)); + let mut stage = Stage::eager(Cursor::new(ps.clone()), false).unwrap(); + let mut small = [0u8; 700]; + let mut e = None; + for _ in 0..10 { + if let Err(x) = stage.read(&mut small) { + e = Some(x); + break; + } + } + assert_eq!(error_code(&e.unwrap()), Some(E_CSS_KEY_MISSING)); + let mut stage = Stage::eager(Cursor::new(ps), false).unwrap(); + stage.seek(SeekFrom::Start(100)).unwrap(); + let mut two = vec![0u8; 2 * SECTOR_BYTES - 100]; + stage.read_exact(&mut two).unwrap(); + let mut rest = Vec::new(); + assert!( + stage.read_to_end(&mut rest).is_err(), + "the scrambled pack after the seek" + ); + } + + // X-3: ONE pack test for disc and file: every DVD-Video stream id and scramble value; an + // IFO/UDF sector (the 38→10-titles guard) or a pack whose 0x14 only looks flagged (stuffed, + // map-first, MPEG-1 PES) is never scrambled, so the disc path leaves it untouched too. + #[test] + fn one_pack_test_serves_disc_and_file() { + for id in [0xBDu8, 0xBE, 0xBF, 0xC0, 0xC7, 0xE0] { + for scramble in 0..4u8 { + let want = (scramble != 0 && !matches!(id, 0xBE | 0xBF)).then_some(0x14); + let got = scrambled_at(&dvd_pack(id, scramble)); + assert_eq!(got, want, "id {id:#x} scramble {scramble}"); + } + } + let mut ifo = b"DVDVIDEO-VTS".to_vec(); + ifo.resize(SECTOR_BYTES, 0x30); + let mut udf = vec![0u8; SECTOR_BYTES]; + udf[..6].copy_from_slice(&[0x02, 0x00, 0x02, 0x00, 0x30, 0x30]); + let mut stuffed = dvd_pack(0xE0, 0); + stuffed[0x0D] = 0xF8 | 1; + stuffed[0x14] = 0x30; + let mut map_first = dvd_pack(0xBC, 0); + map_first[0x14] = 0xB0; + let mut mpeg1_pes = dvd_pack(0xE0, 0); + mpeg1_pes[0x14] = 0xFF; + for s in [ + ifo, + udf, + vec![0x30; SECTOR_BYTES], + stuffed, + map_first, + mpeg1_pes, + ] { + assert_eq!(scrambled_at(&s), None); + let mut region = s.clone(); + crate::css::descramble_region(&mut region, &mut [0x42; 5]).unwrap(); + assert_eq!(region, s, "the disc path's descramble leaves it untouched"); + } + } + + // A VOB with a blank or damaged first sector is still a PS to a file stage, and one + // damaged unit at a clip's start does not hide its source packets. + #[test] + fn damage_at_the_head_does_not_hide_the_stream() { + let mut vob = vec![0u8; SECTOR_BYTES]; + vob.extend(dvd_pack(0xE0, 1)); + assert_eq!( + classify(&vob), + Kind::Opaque, + "a byte stream needs a pack at 0" + ); + assert_eq!(classify_sectors(&vob), Kind::Ps { mpeg2: true }); + let mut clip = vec![0u8; ALIGNED_UNIT_LEN]; + clip.extend((0..3).flat_map(|_| ts_unit(true))); + assert_eq!(classify(&clip), Kind::BdTs, "a zeroed first unit"); + let mut garbled = noise(ALIGNED_UNIT_LEN); + garbled.extend((0..3).flat_map(|_| ts_unit(false))); + assert_eq!(classify(&garbled), Kind::BdTs, "a garbled first unit"); + } + + // An FMKV stream is decided from five bytes: its header is parsed with no more data. + #[test] + fn a_stream_header_is_not_held_for_a_whole_unit() { + struct Trickle(Vec, usize); + impl Read for Trickle { + fn read(&mut self, buf: &mut [u8]) -> io::Result { + assert!(self.1 < self.0.len(), "the stage read past the sent header"); + let n = buf.len().min(self.0.len() - self.1); + buf[..n].copy_from_slice(&self.0[self.1..self.1 + n]); + self.1 += n; + Ok(n) + } + } + let header = b"FMKV\0\x01\0\0header".to_vec(); + let mut stage = Stage::lazy(Trickle(header.clone(), 0), false); + let mut got = vec![0u8; header.len()]; + stage.read_exact(&mut got).unwrap(); + assert_eq!(got, header); + assert_eq!(stage.kind(), Some(Kind::Opaque)); + } + + // Review: head bytes read before a transient error are kept for the retry. + #[test] + fn a_head_read_error_keeps_what_was_read() { + struct Flaky(Vec, usize, bool); + impl Read for Flaky { + fn read(&mut self, buf: &mut [u8]) -> io::Result { + if self.1 == 5 && !self.2 { + self.2 = true; + return Err(io::ErrorKind::ConnectionReset.into()); + } + let n = buf.len().min(self.0.len() - self.1); + buf[..n].copy_from_slice(&self.0[self.1..self.1 + n]); + self.1 += n; + Ok(n) + } + } + let unit = ts_unit(false); + let mut stage = Stage::lazy(Flaky(unit.clone(), 0, false), false); + let mut out = Vec::new(); + assert!(stage.read_to_end(&mut out).is_err()); + stage.read_to_end(&mut out).unwrap(); + assert_eq!(out, unit); + } + + // Review: a refused read leaves none of its raw bytes for a retry to serve. + #[test] + fn a_refused_read_is_refused_again() { + struct Refuses; + impl SectorSource for Refuses { + fn capacity_sectors(&self) -> u32 { + 4 + } + fn read_sectors( + &mut self, + _: u32, + _: u16, + buf: &mut [u8], + _: bool, + ) -> crate::error::Result { + buf.fill(0xEE); + Err(Error::CssKeyMissing) + } + } + let mut bytes = SectorBytes::new(Refuses, 4 * SECTOR_BYTES as u64); + let mut out = [0u8; 100]; + assert!(bytes.read(&mut out).is_err()); + assert!( + bytes.read(&mut out).is_err(), + "never the refused read's bytes" + ); + } + + // N9: a source that ends early is an error, never a silent end of stream. + #[test] + fn a_source_ending_early_is_unexpected_eof() { + struct Empty; + impl SectorSource for Empty { + fn capacity_sectors(&self) -> u32 { + 4 + } + fn read_sectors( + &mut self, + _: u32, + _: u16, + _: &mut [u8], + _: bool, + ) -> crate::error::Result { + Ok(0) + } + } + let e = SectorBytes::new(Empty, 4 * SECTOR_BYTES as u64) + .read(&mut [0u8; 100]) + .unwrap_err(); + assert_eq!(e.kind(), io::ErrorKind::UnexpectedEof); + } + + #[test] + fn sector_bytes_clip_at_the_real_length() { + let data: Vec = noise(3 * SECTOR_BYTES + 700); + let mut padded = data.clone(); + padded.resize(4 * SECTOR_BYTES, 0); + let src = crate::test_util::MemSource::new(padded); + let mut out = Vec::new(); + SectorBytes::new(src, data.len() as u64) + .read_to_end(&mut out) + .unwrap(); + assert_eq!(out, data); + } +} diff --git a/src/test_util.rs b/src/test_util.rs index 649ce26a..c5cb8e64 100644 --- a/src/test_util.rs +++ b/src/test_util.rs @@ -9,6 +9,7 @@ //! - [`CountingSource`]: a [`SectorSource`] wrapper that logs every read. //! - [`decrypt_unit`]: the aligned-unit decrypt, for tests that check ciphertext. //! - [`damage_unit_seed`]: an aligned unit whose first sector read back as garbage. +//! - [`synthetic_bd_clip`]: a muxable clear BD-TS clip (MPEG-2 + two AC-3) for goldens. //! - [`FakeTransport`]: a scripted drive with a state model for Stop tests (stop §5.0). use crate::aacs::content::{ALIGNED_UNIT_LEN, encrypt_unit}; @@ -129,6 +130,7 @@ pub struct BdFile { pub path: String, pub sectors: u32, pub key: Option<[u8; 16]>, + clip: Option>>, } impl BdFile { @@ -137,8 +139,18 @@ impl BdFile { path: path.into(), sectors, key, + clip: None, } } + + /// The file holds `clip` (whole aligned units of clear BD-TS, e.g. + /// [`synthetic_bd_clip`]) instead of the varied junk payload; its length sets the file's + /// sector count. + pub fn with_clip(mut self, clip: Vec) -> Self { + self.sectors = (clip.len() / SECTOR_BYTES) as u32; + self.clip = Some(Arc::new(clip)); + self + } } /// An image built by [`encrypted_bd_image`]. `image` is what a drive serves; `plain` @@ -201,7 +213,8 @@ impl SectorSource for MemSource { /// (KS-3, KS-4). Panics on a fixture error; test use only. pub fn encrypted_bd_image(files: &[BdFile], uk_ro: &[u8]) -> EncryptedBdImage { let dir = FixtureDir(fixture_dir()); - let root = dir.0.clone(); + // The UDF volume id is the root folder's name: fixed, so the image is byte-stable. + let root = dir.0.join("FIXTURE_DISC"); write(&root.join("AACS/Unit_Key_RO.inf"), uk_ro); let unit_sectors = (ALIGNED_UNIT_LEN / SECTOR_BYTES) as u32; for f in files { @@ -236,7 +249,10 @@ pub fn encrypted_bd_image(files: &[BdFile], uk_ro: &[u8]) -> EncryptedBdImage { debug_assert_eq!(sectors % unit_sectors, 0, "{}: whole aligned units", f.path); for u in 0..sectors / unit_sectors { let lba = start + u * unit_sectors; - let mut unit = content_unit(lba, f.key.is_some()); + let mut unit = match &f.clip { + Some(clip) => clip_unit(clip, (lba - start) / unit_sectors, f.key.is_some()), + None => content_unit(lba, f.key.is_some()), + }; let at = lba as usize * SECTOR_BYTES; plain[at..at + ALIGNED_UNIT_LEN].copy_from_slice(&unit); if let Some(key) = &f.key { @@ -265,6 +281,17 @@ fn content_unit(lba: u32, encrypted: bool) -> Vec { u } +// Aligned unit `u` of a clip, CPI-flagged when the file is keyed (KS-5). +fn clip_unit(clip: &[u8], u: u32, encrypted: bool) -> Vec { + let at = u as usize * ALIGNED_UNIT_LEN; + let mut unit = clip[at..at + ALIGNED_UNIT_LEN].to_vec(); + if encrypted { + unit.chunks_mut(BD_SOURCE_PACKET_BYTES) + .for_each(|p| p[0] |= 0xC0); + } + unit +} + // Removes its directory on drop, so a panicking fixture build doesn't leak it. struct FixtureDir(std::path::PathBuf); @@ -396,6 +423,14 @@ pub fn damage_unit_seed(unit: &mut [u8]) { unit[4] = 0x00; } +#[path = "test_util_golden.rs"] +mod golden; +pub use golden::{BLESS_ENV, Golden}; + +#[path = "test_util_media.rs"] +mod media; +pub use media::{CLIP_AUDIO_PIDS, CLIP_VIDEO_PID, synthetic_bd_clip}; + #[path = "test_util_fake.rs"] mod fake; pub use fake::{FakeCdb, FakeHandle, FakeMode, FakeTransport}; diff --git a/src/test_util_golden.rs b/src/test_util_golden.rs new file mode 100644 index 00000000..5ed59be7 --- /dev/null +++ b/src/test_util_golden.rs @@ -0,0 +1,75 @@ +//! Parity goldens: one small text file per cell under `/tests/goldens/`, recording +//! what a path produced today (output hashes, loss counters, gate verdicts). A later change +//! must reproduce every cell exactly, or re-bless it on purpose. +//! +//! Re-bless every cell: `FREEMKV_BLESS_GOLDENS=1 cargo test --lib parity_` (libfreemkv) or +//! `FREEMKV_BLESS_GOLDENS=1 cargo test --test parity_goldens` (engine), then review the +//! `git diff` of `tests/goldens/` and name the reason in the commit message. + +use std::fmt::{Display, Write as _}; +use std::path::PathBuf; + +/// The env var that rewrites goldens instead of comparing against them. +pub const BLESS_ENV: &str = "FREEMKV_BLESS_GOLDENS"; + +/// One cell's observations, built in order and compared (or blessed) by [`Golden::check`]. +pub struct Golden { + file: PathBuf, + cell: String, + text: String, +} + +impl Golden { + /// A cell named `cell` for the crate at `manifest_dir` (pass `env!("CARGO_MANIFEST_DIR")`). + pub fn new(manifest_dir: &str, cell: &str) -> Self { + Self { + file: PathBuf::from(manifest_dir) + .join("tests/goldens") + .join(format!("{cell}.golden")), + cell: cell.to_string(), + text: String::new(), + } + } + + /// Record `key = value`. + pub fn kv(&mut self, key: &str, value: impl Display) -> &mut Self { + let _ = writeln!(self.text, "{key} = {value}"); + self + } + + /// Record the length and SHA-256 of `bytes` under `key`. + pub fn bytes(&mut self, key: &str, bytes: &[u8]) -> &mut Self { + use sha2::{Digest, Sha256}; + let digest: String = Sha256::digest(bytes) + .iter() + .take(16) + .map(|b| format!("{b:02x}")) + .collect(); + self.kv(key, format_args!("{} bytes sha256:{digest}", bytes.len())) + } + + /// The recorded text so far. + pub fn text(&self) -> &str { + &self.text + } + + /// Compare against the committed file; under [`BLESS_ENV`] write it instead. + pub fn check(&self) { + if std::env::var_os(BLESS_ENV).is_some() { + std::fs::create_dir_all(self.file.parent().expect("goldens dir")).expect("mkdir"); + std::fs::write(&self.file, &self.text).expect("write golden"); + return; + } + let want = std::fs::read_to_string(&self.file).unwrap_or_else(|_| { + panic!( + "golden `{}` is missing; run with {BLESS_ENV}=1 to create it", + self.cell + ) + }); + assert_eq!( + self.text, want, + "golden `{}` changed: if intended, re-bless with {BLESS_ENV}=1 and name the reason", + self.cell + ); + } +} diff --git a/src/test_util_media.rs b/src/test_util_media.rs new file mode 100644 index 00000000..a0a2db46 --- /dev/null +++ b/src/test_util_media.rs @@ -0,0 +1,215 @@ +//! Synthetic, muxable BD media for the parity goldens: [`synthetic_bd_clip`] is a clear +//! BD-TS clip (PAT, PMT, one MPEG-2 video and two AC-3 audio tracks) that the demuxer, every +//! sink and the AACS fixtures ([`BdFile::with_clip`](super::BdFile::with_clip)) can carry. +//! Fully deterministic: no clock, no RNG. + +use crate::aacs::content::ALIGNED_UNIT_LEN; +use crate::consts::BD_SOURCE_PACKET_BYTES as PKT; + +/// PID of the clip's MPEG-2 video. +pub const CLIP_VIDEO_PID: u16 = 0x1011; +/// PIDs of the clip's two AC-3 audio tracks. +pub const CLIP_AUDIO_PIDS: [u16; 2] = [0x1100, 0x1101]; +const PMT_PID: u16 = 0x0100; +/// Video frame period (23.976 fps) in nanoseconds. +const FRAME_NS: i64 = 41_708_333; +/// AC-3 frame period (1536 samples at 48 kHz) in nanoseconds. +const AC3_NS: i64 = 32_000_000; + +// One source packet: a zero TP_extra_header (CPI 00) and a TS packet with `body` after +// the 4-byte header, padded to 188 with an adaptation-field stuffing run when short. +fn ts_packet(pid: u16, cc: u8, body: &[u8]) -> Vec { + let mut p = vec![0u8; PKT]; + p[4] = 0x47; + p[5] = (pid >> 8) as u8 & 0x1F; + p[6] = pid as u8; + let room = 184; + if body.len() >= room { + p[7] = 0x10 | (cc & 0x0F); + p[8..8 + room].copy_from_slice(&body[..room]); + } else { + p[7] = 0x30 | (cc & 0x0F); + let af = room - body.len(); + p[8] = (af - 1) as u8; + if af > 1 { + p[9] = 0; + p[10..8 + af].fill(0xFF); + } + p[8 + af..8 + room].copy_from_slice(body); + } + p +} + +// A one-section PSI packet (pointer field 0), the section's CRC left zero (the demuxer +// does not check it). +fn psi(pid: u16, mut section: Vec) -> Vec { + section.extend_from_slice(&[0, 0, 0, 0]); + let mut body = vec![0u8]; + body.extend(section); + let mut p = ts_packet(pid, 0, &body); + p[5] |= 0x40; + p +} + +fn pat() -> Vec { + psi( + 0, + vec![ + 0x00, + 0xB0, + 0x0D, + 0x00, + 0x01, + 0xC1, + 0x00, + 0x00, + 0x00, + 0x01, + 0xE0 | (PMT_PID >> 8) as u8, + PMT_PID as u8, + ], + ) +} + +fn pmt() -> Vec { + let es = [ + (0x02u8, CLIP_VIDEO_PID), + (0x81, CLIP_AUDIO_PIDS[0]), + (0x81, CLIP_AUDIO_PIDS[1]), + ]; + let len = 9 + 5 * es.len() + 4; + let mut s = vec![ + 0x02, + 0xB0, + len as u8, + 0x00, + 0x01, + 0xC1, + 0x00, + 0x00, + 0xE0 | (CLIP_VIDEO_PID >> 8) as u8, + CLIP_VIDEO_PID as u8, + 0xF0, + 0x00, + ]; + for (ty, pid) in es { + s.extend_from_slice(&[ty, 0xE0 | (pid >> 8) as u8, pid as u8, 0xF0, 0x00]); + } + psi(PMT_PID, s) +} + +fn pes_header(stream_id: u8, pts_ns: i64, len: usize) -> Vec { + let pts = ((pts_ns.max(0) as u128 * 90_000) / 1_000_000_000) as u64 + 90_000; + let n = if stream_id == 0xE0 { 0 } else { len + 8 }; + let mut h = vec![0, 0, 1, stream_id, (n >> 8) as u8, n as u8, 0x80, 0x80, 5]; + h.push(0x21 | ((pts >> 29) as u8 & 0x0E)); + h.push((pts >> 22) as u8); + h.push(0x01 | ((pts >> 14) as u8 & 0xFE)); + h.push((pts >> 7) as u8); + h.push(0x01 | ((pts << 1) as u8 & 0xFE)); + h +} + +// A PES (header + `es`) split into source packets on `pid`. +fn pes_packets(pid: u16, cc: &mut u8, stream_id: u8, pts_ns: i64, es: &[u8]) -> Vec> { + let mut pes = pes_header(stream_id, pts_ns, es.len()); + pes.extend_from_slice(es); + let mut out = Vec::new(); + for (i, chunk) in pes.chunks(184).enumerate() { + let mut p = ts_packet(pid, *cc, chunk); + *cc = (*cc + 1) & 0x0F; + if i == 0 { + p[5] |= 0x40; + } + out.push(p); + } + out +} + +// MPEG-2 sequence header (1920x1080, 23.976) + extension, GOP header, then a coded frame. +fn video_frame(n: u32) -> Vec { + let mut v = Vec::new(); + let coding = if n.is_multiple_of(12) { 1u8 } else { 2 }; + if coding == 1 { + v.extend_from_slice(&[ + 0, 0, 1, 0xB3, 0x78, 0x04, 0x38, 0x31, 0xFF, 0xFF, 0xE0, 0x18, + ]); + v.extend_from_slice(&[0, 0, 1, 0xB5, 0x14, 0x8A, 0x00, 0x01, 0x00, 0x00]); + v.extend_from_slice(&[0, 0, 1, 0xB8, 0x00, 0x08, 0x00, 0x40]); + } + let tr = (n % 12) as u8; + v.extend_from_slice(&[ + 0, + 0, + 1, + 0x00, + tr >> 2, + ((tr & 3) << 6) | (coding << 3), + 0xFF, + 0xF8, + ]); + v.extend_from_slice(&[0, 0, 1, 0xB5, 0x8F, 0xFF, 0xF3, 0x80, 0x80]); + v.extend_from_slice(&[0, 0, 1, 0x01, 0x12, 0x34, 0x56]); + // An I frame is long and tail-padded with a constant run, as an encoder pads: a CSS + // crack needs a known run in the scrambled sector. + let size = if coding == 1 { 4000 } else { 300 }; + v.extend((0..size).map(|i| match (coding, i) { + (1, 64..) => 0x55, + _ => ((n.wrapping_mul(131) + i as u32) % 251) as u8 | 1, + })); + v +} + +// A decodable AC-3 syncframe (48 kHz, frmsizecod 8 = 128 words, bsid 8, valid CRC) whose +// body is varied by `n` and `track`. +fn ac3_frame(track: u8, n: u32) -> Vec { + let mut f = vec![0u8; 256]; + f[..2].copy_from_slice(&[0x0B, 0x77]); + f[4] = 8; + f[5] = 0x08 << 3; + for (i, b) in f[6..254].iter_mut().enumerate() { + *b = (n.wrapping_mul(17) as u8) + .wrapping_add(i as u8) + .wrapping_add(track * 61) + | 1; + } + let c = crate::mux::codec::crc::crc16_ansi(&f[2..254]); + f[254..].copy_from_slice(&c.to_be_bytes()); + f +} + +/// A clear BD-TS clip of `units` whole aligned units (6144 bytes each, KS-1): PAT and PMT +/// first, then interleaved video (I frame every 12) and two AC-3 tracks, padded with null +/// packets to the unit boundary. Every packet has CPI `00₂`. Panics if `units` is too few +/// to hold the PSI. +pub fn synthetic_bd_clip(units: usize) -> Vec { + let total = units * ALIGNED_UNIT_LEN / PKT; + let mut pk = vec![pat(), pmt()]; + let (mut cv, mut ca) = (0u8, [0u8; 2]); + let (mut vn, mut an) = (0u32, 0u32); + // Emit by presentation time: one video frame, then the audio due before the next. + while pk.len() + 12 < total { + let t = i64::from(vn) * FRAME_NS; + pk.extend(pes_packets( + CLIP_VIDEO_PID, + &mut cv, + 0xE0, + t, + &video_frame(vn), + )); + vn += 1; + while i64::from(an) * AC3_NS < i64::from(vn) * FRAME_NS && pk.len() + 12 < total { + for (k, pid) in CLIP_AUDIO_PIDS.iter().enumerate() { + let t = i64::from(an) * AC3_NS; + let f = ac3_frame(k as u8, an); + pk.extend(pes_packets(*pid, &mut ca[k], 0xBD, t, &f)); + } + an += 1; + } + } + pk.truncate(total); + while pk.len() < total { + pk.push(ts_packet(0x1FFF, 0, &[0xFF; 184])); + } + pk.concat() +} diff --git a/src/test_util_tests.rs b/src/test_util_tests.rs index 6ecc0aee..3fdcbe8a 100644 --- a/src/test_util_tests.rs +++ b/src/test_util_tests.rs @@ -159,11 +159,7 @@ fn decrypt_unit_inverts_encrypt_unit() { #[test] fn fixture_dir_removed_when_build_panics() { - let bad = [BdFile { - path: "AACS/Unit_Key_RO.inf/x".into(), - sectors: 3, - key: None, - }]; + let bad = [BdFile::new("AACS/Unit_Key_RO.inf/x", 3, None)]; let r = std::panic::catch_unwind(|| encrypted_bd_image(&bad, &[0u8; 4])); assert!(r.is_err()); let dir = crate::test_util::LAST_FIXTURE_DIR diff --git a/src/udf.rs b/src/udf.rs index 150a2a9e..d56a0c20 100644 --- a/src/udf.rs +++ b/src/udf.rs @@ -1388,15 +1388,11 @@ fn read_directory( } if fid_tag != 257 { tracing::warn!(target: "freemkv::udf", fid_tag, icb_abs, "corrupt directory: bad FID tag"); + // A subdirectory lists empty; the root keeps what parsed before the damage. if depth > 0 { entries.clear(); - break; } - return Err(Error::DiscRead { - sector: icb_abs as u64, - status: None, - sense: None, - }); + break; } let file_chars = dir_data[pos + 18]; @@ -1426,13 +1422,8 @@ fn read_directory( tracing::warn!(target: "freemkv::udf", icb_abs, "corrupt directory: name overruns FID data"); if depth > 0 { entries.clear(); - break; } - return Err(Error::DiscRead { - sector: icb_abs as u64, - status: None, - sense: None, - }); + break; } let entry_name = parse_udf_name(&dir_data[name_start..name_end]); @@ -5070,7 +5061,7 @@ mod tests { } #[test] - fn read_directory_rejects_a_descriptor_whose_tag_is_not_a_fid() { + fn read_directory_stops_at_a_descriptor_whose_tag_is_not_a_fid() { // ECMA-167 3/7.2.1: the tag identifier is a 16-bit LE field (257 = FID). A single-byte // compare would accept 0x0201 (513) as 257 (same low byte); it must be an error. let mut fids = Vec::new(); @@ -5092,7 +5083,7 @@ mod tests { reader.put(60, dir); reader.put(7, build_efe_icb(1, 2048, 0)); - let err = read_directory( + let dir = read_directory( &mut reader, &mut 0, &MetaMap::contiguous(0), @@ -5102,8 +5093,14 @@ mod tests { &mut 0, &mut HashSet::new(), ) - .expect_err("a non-FID descriptor inside the declared length is corruption"); - assert!(matches!(err, Error::DiscRead { .. }), "{err:?}"); + .expect("damage stops the listing, never fails it"); + assert_eq!( + dir.entries + .iter() + .map(|e| e.name.as_str()) + .collect::>(), + ["REAL.MPLS"] + ); } #[test] @@ -5140,7 +5137,7 @@ mod tests { } #[test] - fn read_directory_errors_instead_of_panicking_when_a_name_runs_past_the_buffer() { + fn read_directory_stops_instead_of_panicking_when_a_name_runs_past_the_buffer() { // L_IU (ECMA-167 4/14.4.7) is disc-controlled, so a malformed FID can // place the name tens of KB past the buffer end. The scan must error; // slicing there would panic out of the public API on a damaged disc. @@ -5160,7 +5157,7 @@ mod tests { reader.put(60, dir); reader.put(7, build_efe_icb(5, 2048, 0)); - let err = read_directory( + let dir = read_directory( &mut reader, &mut 0, &MetaMap::contiguous(0), @@ -5170,8 +5167,14 @@ mod tests { &mut 0, &mut HashSet::new(), ) - .expect_err("an out-of-range File Identifier is corruption, not a short listing"); - assert!(matches!(err, Error::DiscRead { .. }), "{err:?}"); + .expect("damage stops the listing, never fails it"); + assert_eq!( + dir.entries + .iter() + .map(|e| e.name.as_str()) + .collect::>(), + ["OK.MPLS"] + ); } #[test] diff --git a/src/whole_disc_tests.rs b/src/whole_disc_tests.rs index 27e3d5fc..428a2b20 100644 --- a/src/whole_disc_tests.rs +++ b/src/whole_disc_tests.rs @@ -354,10 +354,10 @@ fn raw_whole_disc_reader_passes_every_sector_through() { let uk_ro = unit_key_ro(crate::aacs::mkb::AacsVersion::V10, &[[0xEE; 16]], &[1]); let files = [BdFile::new("BDMV/STREAM/00001.m2ts", 30, Some([0x11; 16]))]; let mut image = encrypted_bd_image(&files, &uk_ro).image; - // A CSS-scrambled MPEG-2 pack (bits 4-5 of byte 0x14 set) in sector 1. - image[2048..2052].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); - image[2048 + 4] = 0x44; + // A CSS-scrambled DVD-Video pack (bits 4-5 of byte 0x14 set) in sector 1. image[2048 + 0x14] |= 0x30; + crate::css::dvd_pack_header(&mut image[2048..4096], 0xE0); + assert!(crate::css::is_scrambled_pack(&image[2048..4096])); let sectors = (image.len() / 2048) as u32; let mut r = raw_whole_disc_reader(MemSource::new(image.clone())); let mut buf = vec![0u8; image.len()]; diff --git a/tests/crypto_tests.rs b/tests/crypto_tests.rs index 994baad6..6878fda6 100644 --- a/tests/crypto_tests.rs +++ b/tests/crypto_tests.rs @@ -20,13 +20,13 @@ fn css_descramble_sector_roundtrip_via_public_api() { }; let mut sector = vec![0xAAu8; 2048]; - // Real scrambled DVD sectors are MPEG-2 PS packs, and the public API now - // requires the pack start code as well as the flag bits — byte 0x14 means - // something else entirely in an IFO or UDF sector. + // A real scrambled DVD-Video pack: pack header, then a PES whose MPEG-2 flags byte (0x14) + // carries the scramble bits. Byte 0x14 means something else in an IFO or UDF sector. sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); sector[4] = 0x44; // '01': a 13818-1 pack sector[0x0D] = 0xF8; // pack_stuffing_length 0 - sector[0x14] = 0x30; + sector[0x0E..0x12].copy_from_slice(&[0x00, 0x00, 0x01, 0xE0]); + sector[0x14] = 0xB0; sector[0x54..0x59].copy_from_slice(&[0xDE, 0xAD, 0xBE, 0xEF, 0x42]); let original = sector.clone(); diff --git a/tests/goldens/parity_image_bd_bad_sectors.golden b/tests/goldens/parity_image_bd_bad_sectors.golden new file mode 100644 index 00000000..f0468f72 --- /dev/null +++ b/tests/goldens/parity_image_bd_bad_sectors.golden @@ -0,0 +1,25 @@ +resolve-whole status = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +copy failed = E6000 +copy blanked = 0 +extract result = good=18432 unreadable=18432 complete=false halted=false files=[("BDMV/STREAM/00001.m2ts", 18432, 18432, true)] +extract out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:44fee6c3dc35a6f2a58b400893c22b7a +copy-raw failed = E6000 +extract-raw result = good=18432 unreadable=18432 complete=false halted=false files=[("BDMV/STREAM/00001.m2ts", 18432, 18432, true)] +extract-raw out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:48e61cd82e2557534766b163f852bd66 +resolve-title status = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +title-read = 0 bytes sha256:e3b0c44298fc1c149afbf4c8996fb924 +title-read blanked = 0 +title-read failed = E6000 +iso-mkv outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv out o.mkv = 28578 bytes sha256:51db71260cb190c5bafb17544c8976ab +iso-m2ts outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-m2ts out o.m2ts = 39360 bytes sha256:bf6571efb2ada7dc353acd6fb1dca21e +iso-null outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv-raw refused = E6008 +iso-mkv-selection outcome = completed=true opened=true bytes=20016 errors=0 lost=0 streams=2 undelivered=[] +iso-mkv-selection out o.mkv = 21563 bytes sha256:cda2cf35944d0511ffd8de92d4a542fd +live-mkv refused = E6008 +live-mkv out o.mkv = 493 bytes sha256:454f3eb165010aa1646faed1279583cf +live-mkv-raw refused = E6000 +live-mkv-skip outcome = completed=true opened=true bytes=17910 errors=7 lost=6144 streams=3 undelivered=[] +live-mkv-skip out o.mkv = 19715 bytes sha256:732a5c67d1f32079e51a58cd87e8d82e diff --git a/tests/goldens/parity_image_bd_clear_files.golden b/tests/goldens/parity_image_bd_clear_files.golden new file mode 100644 index 00000000..42a80266 --- /dev/null +++ b/tests/goldens/parity_image_bd_clear_files.golden @@ -0,0 +1,28 @@ +resolve-whole status = KeySetStatus { proven: 0, declared: Some(2), keyed: 0, clear: 2, lazy: 0, forensic: None, origin: None, best_effort: false, requests: 0 } +copy written = 8466432 +copy image = 8466432 bytes sha256:98778a03e8af5deb127b504109f70589 +copy blanked = 0 +extract result = good=73728 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 36864, 0, true)] +extract out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +extract out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +copy-raw written = 8466432 +copy-raw image = 8466432 bytes sha256:98778a03e8af5deb127b504109f70589 +extract-raw result = good=73728 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 36864, 0, true)] +extract-raw out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +extract-raw out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +resolve-title status = KeySetStatus { proven: 0, declared: Some(2), keyed: 0, clear: 2, lazy: 0, forensic: None, origin: None, best_effort: false, requests: 0 } +title-read = 73728 bytes sha256:6b12cadf0d7bb63d7decc4a7663d3e52 +title-read blanked = 0 +iso-mkv outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv out o.mkv = 56444 bytes sha256:5acfcd084d22d1df3913475317704196 +iso-m2ts outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +iso-m2ts out o.m2ts = 73920 bytes sha256:75f47cceb62e356629aec9b6bdb722c1 +iso-null outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv-raw outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv-raw out o.mkv = 56444 bytes sha256:5acfcd084d22d1df3913475317704196 +iso-mkv-selection outcome = completed=true opened=true bytes=40032 errors=0 lost=0 streams=2 undelivered=[] +iso-mkv-selection out o.mkv = 42528 bytes sha256:371978c9cf71c3d5c1a01fda23e9bcf9 +live-mkv outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +live-mkv out o.mkv = 56444 bytes sha256:5acfcd084d22d1df3913475317704196 +live-mkv-raw outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +live-mkv-raw out o.mkv = 56444 bytes sha256:5acfcd084d22d1df3913475317704196 diff --git a/tests/goldens/parity_image_bd_damaged_units.golden b/tests/goldens/parity_image_bd_damaged_units.golden new file mode 100644 index 00000000..2853e83d --- /dev/null +++ b/tests/goldens/parity_image_bd_damaged_units.golden @@ -0,0 +1,26 @@ +resolve-whole status = KeySetStatus { proven: 2, declared: Some(2), keyed: 2, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +copy written = 8466432 +copy image = 8466432 bytes sha256:c6d26898508714516541998bf8f33456 +copy blanked = 3 +extract result = good=55296 unreadable=18432 complete=false halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 18432, 18432, true)] +extract out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +extract out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:1cee23a03e507c9e5103d108ada392ce +copy-raw written = 8466432 +copy-raw image = 8466432 bytes sha256:d1ca43ed54cc80c47ce57969c335e303 +extract-raw result = good=73728 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 36864, 0, true)] +extract-raw out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:375559f987f4f85e585d39abe6eb12fa +extract-raw out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:13f1bdfc43aac09ff1c4a0359cd3ccb6 +resolve-title status = KeySetStatus { proven: 2, declared: Some(2), keyed: 2, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +title-read = 73728 bytes sha256:5097bdccd3a8ecd9c3cda375a2153ba6 +title-read blanked = 3 +iso-mkv outcome = completed=true opened=true bytes=37424 errors=8 lost=18432 streams=3 undelivered=[] +iso-mkv out o.mkv = 39926 bytes sha256:339adfed83a9095dc0b4eea03ec907c8 +iso-m2ts outcome = completed=true opened=true bytes=37424 errors=8 lost=18432 streams=3 undelivered=[] +iso-m2ts out o.m2ts = 52992 bytes sha256:eab70dc59a7d7bda01b3368baf5c8cc0 +iso-null outcome = completed=true opened=true bytes=37424 errors=8 lost=18432 streams=3 undelivered=[] +iso-mkv-raw refused = E6008 +iso-mkv-selection outcome = completed=true opened=true bytes=29232 errors=8 lost=18432 streams=2 undelivered=[] +iso-mkv-selection out o.mkv = 30979 bytes sha256:ce3203996e6720690f5de3d739a9ebc6 +live-mkv outcome = completed=true opened=true bytes=37424 errors=8 lost=18432 streams=3 undelivered=[] +live-mkv out o.mkv = 39926 bytes sha256:339adfed83a9095dc0b4eea03ec907c8 +live-mkv-raw refused = E6008 diff --git a/tests/goldens/parity_image_bd_missing_key.golden b/tests/goldens/parity_image_bd_missing_key.golden new file mode 100644 index 00000000..6e84c449 --- /dev/null +++ b/tests/goldens/parity_image_bd_missing_key.golden @@ -0,0 +1,27 @@ +resolve-whole status = KeySetStatus { proven: 1, declared: Some(2), keyed: 1, clear: 0, lazy: 1, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +copy written = 8466432 +copy image = 8466432 bytes sha256:f2dcc9c4d4d4139e9d6efdbdccb5718c +copy blanked = 6 +extract result = good=36864 unreadable=36864 complete=false halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 0, 36864, true)] +extract out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +extract out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:1c0273095382988333e2f2b5ae487cea +copy-raw written = 8466432 +copy-raw image = 8466432 bytes sha256:f0856c74bf5c9f7b6822186cafcaf1d6 +extract-raw result = good=73728 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 36864, 0, true)] +extract-raw out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:375559f987f4f85e585d39abe6eb12fa +extract-raw out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:2d895155c02331f792c4fca0bfb881ea +resolve-title status = KeySetStatus { proven: 1, declared: Some(2), keyed: 1, clear: 0, lazy: 1, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +title-read = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +title-read blanked = 0 +title-read failed = E7022 +iso-mkv refused = E7022 +iso-mkv out o.mkv = 21863 bytes sha256:47337f351b04ebb679c0da200a04e94a +iso-m2ts refused = E7022 +iso-m2ts out o.m2ts = 31488 bytes sha256:3a40b2a87a7e62135df01ead62597601 +iso-null refused = E7022 +iso-mkv-raw refused = E6008 +iso-mkv-selection refused = E7022 +iso-mkv-selection out o.mkv = 15124 bytes sha256:120947d4cc4150b1531e918c11865049 +live-mkv refused = E7022 +live-mkv out o.mkv = 21863 bytes sha256:47337f351b04ebb679c0da200a04e94a +live-mkv-raw refused = E6008 diff --git a/tests/goldens/parity_image_bd_multi_cps.golden b/tests/goldens/parity_image_bd_multi_cps.golden new file mode 100644 index 00000000..e4c7edf7 --- /dev/null +++ b/tests/goldens/parity_image_bd_multi_cps.golden @@ -0,0 +1,26 @@ +resolve-whole status = KeySetStatus { proven: 2, declared: Some(2), keyed: 2, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +copy written = 8466432 +copy image = 8466432 bytes sha256:98778a03e8af5deb127b504109f70589 +copy blanked = 0 +extract result = good=73728 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 36864, 0, true)] +extract out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +extract out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +copy-raw written = 8466432 +copy-raw image = 8466432 bytes sha256:f0856c74bf5c9f7b6822186cafcaf1d6 +extract-raw result = good=73728 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true), ("BDMV/STREAM/00002.m2ts", 36864, 0, true)] +extract-raw out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:375559f987f4f85e585d39abe6eb12fa +extract-raw out BDMV/STREAM/00002.m2ts = 36864 bytes sha256:2d895155c02331f792c4fca0bfb881ea +resolve-title status = KeySetStatus { proven: 2, declared: Some(2), keyed: 2, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +title-read = 73728 bytes sha256:6b12cadf0d7bb63d7decc4a7663d3e52 +title-read blanked = 0 +iso-mkv outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv out o.mkv = 56444 bytes sha256:5acfcd084d22d1df3913475317704196 +iso-m2ts outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +iso-m2ts out o.m2ts = 73920 bytes sha256:75f47cceb62e356629aec9b6bdb722c1 +iso-null outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv-raw refused = E6008 +iso-mkv-selection outcome = completed=true opened=true bytes=40032 errors=0 lost=0 streams=2 undelivered=[] +iso-mkv-selection out o.mkv = 42528 bytes sha256:371978c9cf71c3d5c1a01fda23e9bcf9 +live-mkv outcome = completed=true opened=true bytes=52832 errors=0 lost=0 streams=3 undelivered=[] +live-mkv out o.mkv = 56444 bytes sha256:5acfcd084d22d1df3913475317704196 +live-mkv-raw refused = E6008 diff --git a/tests/goldens/parity_image_bd_one_cps.golden b/tests/goldens/parity_image_bd_one_cps.golden new file mode 100644 index 00000000..e7c2f5be --- /dev/null +++ b/tests/goldens/parity_image_bd_one_cps.golden @@ -0,0 +1,24 @@ +resolve-whole status = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +copy written = 8429568 +copy image = 8429568 bytes sha256:499099b58b2d57445e8d04cea9a7b4a1 +copy blanked = 0 +extract result = good=36864 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true)] +extract out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +copy-raw written = 8429568 +copy-raw image = 8429568 bytes sha256:4372cf8f58195b59c08ba93d47576bc9 +extract-raw result = good=36864 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true)] +extract-raw out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:375559f987f4f85e585d39abe6eb12fa +resolve-title status = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +title-read = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +title-read blanked = 0 +iso-mkv outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv out o.mkv = 28578 bytes sha256:51db71260cb190c5bafb17544c8976ab +iso-m2ts outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-m2ts out o.m2ts = 39360 bytes sha256:bf6571efb2ada7dc353acd6fb1dca21e +iso-null outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv-raw refused = E6008 +iso-mkv-selection outcome = completed=true opened=true bytes=20016 errors=0 lost=0 streams=2 undelivered=[] +iso-mkv-selection out o.mkv = 21563 bytes sha256:cda2cf35944d0511ffd8de92d4a542fd +live-mkv outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +live-mkv out o.mkv = 28578 bytes sha256:51db71260cb190c5bafb17544c8976ab +live-mkv-raw refused = E6008 diff --git a/tests/goldens/parity_image_dvd_css_multi_vts.golden b/tests/goldens/parity_image_dvd_css_multi_vts.golden new file mode 100644 index 00000000..27aac8a5 --- /dev/null +++ b/tests/goldens/parity_image_dvd_css_multi_vts.golden @@ -0,0 +1,3 @@ +extract result = good=860160 unreadable=0 complete=true halted=false files=[("VIDEO_TS/VTS_01_1.VOB", 430080, 0, true), ("VIDEO_TS/VTS_02_1.VOB", 430080, 0, true)] +extract out VIDEO_TS/VTS_01_1.VOB = 430080 bytes sha256:4f62f2019c87c5183c7015aeebdbf241 +extract out VIDEO_TS/VTS_02_1.VOB = 430080 bytes sha256:4f62f2019c87c5183c7015aeebdbf241 diff --git a/tests/goldens/parity_image_fmts_alternate_phase.golden b/tests/goldens/parity_image_fmts_alternate_phase.golden new file mode 100644 index 00000000..0d452249 --- /dev/null +++ b/tests/goldens/parity_image_fmts_alternate_phase.golden @@ -0,0 +1,6 @@ +phase0 status = KeySetStatus { proven: 2, declared: Some(2), keyed: 2, clear: 0, lazy: 0, forensic: Resolved, origin: Some("online"), best_effort: false, requests: 2 } +phase0 forensic-clip = 98304 bytes sha256:2f40ac12464652e99ffb371dc0ed2af6 +phase0 copy = refused E7013 +phase1 status = KeySetStatus { proven: 2, declared: Some(2), keyed: 2, clear: 0, lazy: 0, forensic: Resolved, origin: Some("online"), best_effort: false, requests: 3 } +phase1 forensic-clip = 98304 bytes sha256:5026938565e0cc00c6399f36f8fc1fa2 +phase1 copy = refused E7013 diff --git a/tests/goldens/parity_image_hddvd_clear.golden b/tests/goldens/parity_image_hddvd_clear.golden new file mode 100644 index 00000000..a727432d --- /dev/null +++ b/tests/goldens/parity_image_hddvd_clear.golden @@ -0,0 +1,6 @@ +resolve-whole status = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: true, requests: 1 } +copy written = 8460288 +copy image = 8460288 bytes sha256:438eaf4106ce92a5550c5fa70574a2d2 +extract result = good=61440 unreadable=6144 complete=false halted=false files=[("BDMV/index.bdmv", 0, 6144, true), ("HVDVD_TS/FEATURE.EVO", 61440, 0, true)] +extract out BDMV/index.bdmv = 6144 bytes sha256:fd9243e1ba57263ed469c3bdbd7ade6e +extract out HVDVD_TS/FEATURE.EVO = 61440 bytes sha256:9be03259626dabeb7c985b6930729f99 diff --git a/tests/goldens/parity_image_uhd_bus_flag.golden b/tests/goldens/parity_image_uhd_bus_flag.golden new file mode 100644 index 00000000..e7c2f5be --- /dev/null +++ b/tests/goldens/parity_image_uhd_bus_flag.golden @@ -0,0 +1,24 @@ +resolve-whole status = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +copy written = 8429568 +copy image = 8429568 bytes sha256:499099b58b2d57445e8d04cea9a7b4a1 +copy blanked = 0 +extract result = good=36864 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true)] +extract out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +copy-raw written = 8429568 +copy-raw image = 8429568 bytes sha256:4372cf8f58195b59c08ba93d47576bc9 +extract-raw result = good=36864 unreadable=0 complete=true halted=false files=[("BDMV/STREAM/00001.m2ts", 36864, 0, true)] +extract-raw out BDMV/STREAM/00001.m2ts = 36864 bytes sha256:375559f987f4f85e585d39abe6eb12fa +resolve-title status = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +title-read = 36864 bytes sha256:368a55d7eca3e25f6af377a6f1aec041 +title-read blanked = 0 +iso-mkv outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv out o.mkv = 28578 bytes sha256:51db71260cb190c5bafb17544c8976ab +iso-m2ts outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-m2ts out o.m2ts = 39360 bytes sha256:bf6571efb2ada7dc353acd6fb1dca21e +iso-null outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +iso-mkv-raw refused = E6008 +iso-mkv-selection outcome = completed=true opened=true bytes=20016 errors=0 lost=0 streams=2 undelivered=[] +iso-mkv-selection out o.mkv = 21563 bytes sha256:cda2cf35944d0511ffd8de92d4a542fd +live-mkv outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +live-mkv out o.mkv = 28578 bytes sha256:51db71260cb190c5bafb17544c8976ab +live-mkv-raw refused = E6008 diff --git a/tests/goldens/parity_live_bd_bad_sectors.golden b/tests/goldens/parity_live_bd_bad_sectors.golden new file mode 100644 index 00000000..91c3a7d2 --- /dev/null +++ b/tests/goldens/parity_live_bd_bad_sectors.golden @@ -0,0 +1,4 @@ +strict refused = E6008 +strict out o.mkv = 489 bytes sha256:8628b7bc94f1815e2948a4ce9f32c10f +skip outcome = completed=true opened=true bytes=17910 errors=7 lost=6144 streams=3 undelivered=[] +skip out o.mkv = 19705 bytes sha256:e8bf5f82342443c240a95385c881d3e2 diff --git a/tests/goldens/parity_live_bd_lazy_piece.golden b/tests/goldens/parity_live_bd_lazy_piece.golden new file mode 100644 index 00000000..68a25d32 --- /dev/null +++ b/tests/goldens/parity_live_bd_lazy_piece.golden @@ -0,0 +1,3 @@ +keys = KeySetStatus { proven: 0, declared: Some(1), keyed: 0, clear: 0, lazy: 1, forensic: None, origin: None, best_effort: false, requests: 1 } +lazy outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +lazy out o.mkv = 28567 bytes sha256:bf1a6347d627e65f7580b67322dee4d5 diff --git a/tests/goldens/parity_live_bd_session.golden b/tests/goldens/parity_live_bd_session.golden new file mode 100644 index 00000000..04e6f3e2 --- /dev/null +++ b/tests/goldens/parity_live_bd_session.golden @@ -0,0 +1,8 @@ +scan = titles=1 +keys = KeySetStatus { proven: 1, declared: Some(1), keyed: 1, clear: 0, lazy: 0, forensic: None, origin: Some("keydb"), best_effort: false, requests: 1 } +keyed outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +keyed out o.mkv = 28567 bytes sha256:bf1a6347d627e65f7580b67322dee4d5 +keyless refused = E7022 +keyed-again outcome = completed=true opened=true bytes=26416 errors=0 lost=0 streams=3 undelivered=[] +keyed-again out o.mkv = 28567 bytes sha256:bf1a6347d627e65f7580b67322dee4d5 +raw refused = E6008 diff --git a/tests/goldens/parity_live_bd_stop_mid_read.golden b/tests/goldens/parity_live_bd_stop_mid_read.golden new file mode 100644 index 00000000..43723fa4 --- /dev/null +++ b/tests/goldens/parity_live_bd_stop_mid_read.golden @@ -0,0 +1 @@ +stop = Ok((false, 0, 0)) diff --git a/tests/goldens/parity_network_fmkv_wire.golden b/tests/goldens/parity_network_fmkv_wire.golden new file mode 100644 index 00000000..1eca8b61 --- /dev/null +++ b/tests/goldens/parity_network_fmkv_wire.golden @@ -0,0 +1,3 @@ +wire = 28510 bytes sha256:2b424f1d15a098c9389ff126ab11a83d +frames = 69 +round-trip identical = true diff --git a/tests/goldens/parity_sinks_clear_m2ts_default.golden b/tests/goldens/parity_sinks_clear_m2ts_default.golden new file mode 100644 index 00000000..eb2d10e3 --- /dev/null +++ b/tests/goldens/parity_sinks_clear_m2ts_default.golden @@ -0,0 +1,25 @@ +mkv outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mkv out o.mkv = 58232 bytes sha256:b25ee88e8f6935e99acf42fd476cc71d +mp4 refused = E9048 +mp4 out o.mp4 = 0 bytes sha256:e3b0c44298fc1c149afbf4c8996fb924 +mpg outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mpg out o.mpg = 206852 bytes sha256:1fb67e840822afec9eab312490f39458 +m2ts outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +m2ts out o.m2ts = 80256 bytes sha256:ab5d202fdf7ea5d2e5be98f01840be21 +fvi outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +fvi out o.fvi = 6010 bytes sha256:b2c2689eb46dbb88df82ed4036805429 +json outcome = completed=true opened=true bytes=0 errors=0 lost=0 streams=3 undelivered=[] +json out o.json = 1263 bytes sha256:040a1c5961ab0286333076150fbab37e +chapters outcome = completed=true opened=true bytes=0 errors=0 lost=0 streams=3 undelivered=[] +chapters out o.xml = 147 bytes sha256:1a73b20654e6b734d7388c10049455fb +demux outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +demux out o/title t00 MPEG2.m2v = 27880 bytes sha256:be8990c82ea9024deb38cdfb174bea80 +demux out o/title t01 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:904e9e35a2990b1001d2d99baeb1a633 +demux out o/title t02 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:48694b3013866911b7e0f20507ebb311 +video outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +video out o/title t00 MPEG2.m2v = 27880 bytes sha256:be8990c82ea9024deb38cdfb174bea80 +audio outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +audio out o/title t01 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:904e9e35a2990b1001d2d99baeb1a633 +audio out o/title t02 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:48694b3013866911b7e0f20507ebb311 +sub outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +null outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] diff --git a/tests/goldens/parity_sinks_clear_m2ts_raw.golden b/tests/goldens/parity_sinks_clear_m2ts_raw.golden new file mode 100644 index 00000000..23131941 --- /dev/null +++ b/tests/goldens/parity_sinks_clear_m2ts_raw.golden @@ -0,0 +1,8 @@ +mkv outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mkv out o.mkv = 58232 bytes sha256:b25ee88e8f6935e99acf42fd476cc71d +m2ts outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +m2ts out o.m2ts = 80256 bytes sha256:ab5d202fdf7ea5d2e5be98f01840be21 +demux outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +demux out o/title t00 MPEG2.m2v = 27880 bytes sha256:be8990c82ea9024deb38cdfb174bea80 +demux out o/title t01 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:904e9e35a2990b1001d2d99baeb1a633 +demux out o/title t02 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:48694b3013866911b7e0f20507ebb311 diff --git a/tests/goldens/parity_sinks_clear_m2ts_selection.golden b/tests/goldens/parity_sinks_clear_m2ts_selection.golden new file mode 100644 index 00000000..ffc6d852 --- /dev/null +++ b/tests/goldens/parity_sinks_clear_m2ts_selection.golden @@ -0,0 +1,10 @@ +mkv outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mkv out o.mkv = 58232 bytes sha256:b25ee88e8f6935e99acf42fd476cc71d +m2ts outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +m2ts out o.m2ts = 80256 bytes sha256:ab5d202fdf7ea5d2e5be98f01840be21 +demux outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +demux out o/title t00 MPEG2.m2v = 27880 bytes sha256:be8990c82ea9024deb38cdfb174bea80 +demux out o/title t01 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:904e9e35a2990b1001d2d99baeb1a633 +demux out o/title t02 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:48694b3013866911b7e0f20507ebb311 +json outcome = completed=true opened=true bytes=0 errors=0 lost=0 streams=3 undelivered=[] +json out o.json = 1263 bytes sha256:040a1c5961ab0286333076150fbab37e diff --git a/tests/goldens/parity_sinks_from_mkv_mpg_and_fmkv_sources.golden b/tests/goldens/parity_sinks_from_mkv_mpg_and_fmkv_sources.golden new file mode 100644 index 00000000..a748be81 --- /dev/null +++ b/tests/goldens/parity_sinks_from_mkv_mpg_and_fmkv_sources.golden @@ -0,0 +1,18 @@ +mkv outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mkv out o.mkv = 58232 bytes sha256:b25ee88e8f6935e99acf42fd476cc71d +mkv-src streams = v:Mpeg2:0x1011,a:Ac3:0x1100,a:Ac3:0x1101 +mkv-src frames = 144 +mkv-src frame-digest = 56952 bytes sha256:31b7c2bd3b6567b7ae3e221c83700597 +mkv-src errors = 0 lost=0 +mkv outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mkv out o.mkv = 58240 bytes sha256:45714c430679de5252267609a053a2f5 +mpg-src streams = v:Mpeg2:0xe0,a:Ac3:0xbd80,a:Ac3:0xbd81 +mpg-src frames = 144 +mpg-src frame-digest = 56952 bytes sha256:f0fec739fcfd4189d7be047dde0e4fb5 +mpg-src errors = 0 lost=0 +mkv outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mkv out o.mkv = 58232 bytes sha256:c51c6c8c17bd803887d5024a37952ba2 +m2ts-src streams = v:Mpeg2:0x1011,a:Ac3:0x1100,a:Ac3:0x1101 +m2ts-src frames = 144 +m2ts-src frame-digest = 56952 bytes sha256:885d82d30957a6d1bb04f301c4b3e1f4 +m2ts-src errors = 0 lost=0 diff --git a/tests/goldens/parity_sources_loose_m2ts_aacs.golden b/tests/goldens/parity_sources_loose_m2ts_aacs.golden new file mode 100644 index 00000000..853e9eae --- /dev/null +++ b/tests/goldens/parity_sources_loose_m2ts_aacs.golden @@ -0,0 +1,21 @@ +clear frames = 144 +clear frame-digest = 56808 bytes sha256:bf3c1d39e961e73e299c1724d9b02f13 +clear blanked = 0 +clear-keyed frames = 144 +clear-keyed frame-digest = 56808 bytes sha256:bf3c1d39e961e73e299c1724d9b02f13 +clear-keyed blanked = 0 +stale-cpi frames = 144 +stale-cpi frame-digest = 56808 bytes sha256:bf3c1d39e961e73e299c1724d9b02f13 +stale-cpi blanked = 0 +aacs-held-key frames = 144 +aacs-held-key frame-digest = 56808 bytes sha256:bf3c1d39e961e73e299c1724d9b02f13 +aacs-held-key blanked = 0 +aacs-no-keys refused = E7022 +aacs-wrong-key refused = E7022 +aacs-raw refused = E6009 +aacs-truncated frames = 133 +aacs-truncated frame-digest = 53612 bytes sha256:efce90984f5d5261013c0993a90240df +aacs-truncated blanked = 1 +aacs-damaged-unit frames = 123 +aacs-damaged-unit frame-digest = 49408 bytes sha256:cda35d2f16131e47155e99bf0ab12863 +aacs-damaged-unit blanked = 9 diff --git a/tests/goldens/parity_sources_loose_mpg_css.golden b/tests/goldens/parity_sources_loose_mpg_css.golden new file mode 100644 index 00000000..4e1b7d77 --- /dev/null +++ b/tests/goldens/parity_sources_loose_mpg_css.golden @@ -0,0 +1,14 @@ +clear frames = 304 +clear frame-digest = 114510 bytes sha256:2719568af5ba03412eca1096bdd5913a +clear blanked = 0 +scrambled frames = 304 +scrambled frame-digest = 114510 bytes sha256:2719568af5ba03412eca1096bdd5913a +scrambled blanked = 0 +scrambled-raw frames = 251 +scrambled-raw frame-digest = 107966 bytes sha256:69b84843c0860a5459b23ca497e28366 +scrambled-raw blanked = 0 +scrambled-as-mkv refused = E7023 +mkv outcome = completed=true opened=true bytes=109646 errors=0 lost=0 streams=3 undelivered=[] +mkv out o.mkv = 116792 bytes sha256:a78c043cac5f3aea36f7b6471d335077 +m2ts outcome = completed=true opened=true bytes=109646 errors=0 lost=0 streams=3 undelivered=[] +m2ts out o.m2ts = 163008 bytes sha256:05f5cf7a47850c8a1f8e7cbd15dc7596 diff --git a/tests/lint_stop.rs b/tests/lint_stop.rs index dbc11cf8..b5161a27 100644 --- a/tests/lint_stop.rs +++ b/tests/lint_stop.rs @@ -390,7 +390,7 @@ fn no_thread_sleep_on_op_paths() { ), ( "src/mux/network.rs", - "accept_from_with_halt", + "accept_staged", "halt-checked accept poll; open item: convert to Halt::wait", ), ]; diff --git a/tests/pass_n_patch_fix.rs b/tests/pass_n_patch_fix.rs index f8b0726b..587c3918 100644 --- a/tests/pass_n_patch_fix.rs +++ b/tests/pass_n_patch_fix.rs @@ -28,13 +28,14 @@ fn css_decrypt_of_an_uncrackable_sector_still_descrambles() { // is not a rip failure — `attack_crib` is a heuristic, so this mismatch just // means the cached key stands (CSS recoverability varies per sector, unlike AACS). let mut sector = vec![0xFFu8; 2048]; - // Scrambled DVD sectors are MPEG-2 PS packs. The descramble policy requires - // the pack start code as well as the flag bits, because byte 0x14 means - // something else entirely in an IFO, UDF or ISO 9660 sector. + // Scrambled DVD sectors are DVD-Video packs: the scramble bits are judged in the first + // PES header's MPEG-2 flags byte (0x14), because byte 0x14 means something else + // entirely in an IFO, UDF or ISO 9660 sector. sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); sector[4] = 0x44; // '01': a 13818-1 pack sector[0x0D] = 0xF8; // pack_stuffing_length 0 - sector[0x14] |= 0x30; // CSS scramble flag, bits 4-5 + sector[0x0E..0x12].copy_from_slice(&[0x00, 0x00, 0x01, 0xE0]); + sector[0x14] = 0xB0; // MPEG-2 flags, CSS scramble bits 4-5 let title_key: [u8; 5] = [0x42, 0x13, 0x37, 0xBE, 0xEF]; let mut keys = DecryptKeys::Css { title_key }; diff --git a/tests/spec_sites.txt b/tests/spec_sites.txt index c4193b7e..573e7fcc 100644 --- a/tests/spec_sites.txt +++ b/tests/spec_sites.txt @@ -79,4 +79,4 @@ src/mux/codec/lpcm.rs dvd_header MS-30 src/mux/mpg/scan.rs parse_map MS-8 src/mux/mpg/scan.rs parse_map MS-9 src/mux/mpg/scan.rs scan MS-23 -src/css/mod.rs ps_scrambled_at MS-31 +src/css/mod.rs scrambled_at MS-31