diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ab925bb..e39efe03 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,10 +4,21 @@ ### Breaking -- **AACS keys:** `keys::ResolvedKeySet` is the only door to AACS keys. Removed: crate-root `AacsKeyMap`, `DecryptKeys`, `resolve_and_apply`, `KeyFetch`; `resolve_and_apply_traced`, `keysource::{fetch_unit_keys, fetch_fmts_indexes, key_fetch}`, `sector::{KeyFetch, KeyFetchFn}`, `session::{resolve_keys_for, ResolvedKeys}`, `DiscSession::{resolve_keys, key_fetch}`, `Disc::{decrypt_keys, decrypt_keys_for_title, decrypt_with, ensure_decryptable, ensure_decryptable_keys, ensure_title_decryptable}`, `InputOptions::{unit_keys, key_fetch}`, `DiscStream::with_key_map`. `DecryptKeys` is `#[non_exhaustive]`; `AacsKeyMap::from_ranges*`, `DecryptingSectorSource::{with_key_map, set_key_map}` and `aacs::content::{decrypt_unit, decrypt_bus}` are no longer public. Resolve once with `DiscSession::resolve_key_set` / `ResolvedKeySet::resolve`. -- **Mux entry point:** `mux_stream` and `MuxInput` are replaced by `mux_with_keys(MuxSource, Option<&ResolvedKeySet>, dest_url, opts, halt, events)`; `MuxSource` (`Session`, `Iso`, `Live`, `Url`) carries no key material. `InputOptions::keys: Option` replaces `unit_keys` / `key_fetch`; an AACS image opened without a set is refused before any output. Crate-root `build_iso_pipeline` and `resolve_mux_key_map` are removed. +- **AACS keys:** `keys::KeyRing` is the only door to AACS keys. Removed: crate-root `AacsKeyMap`, `DecryptKeys`, `resolve_and_apply`, `KeyFetch`; `resolve_and_apply_traced`, `keysource::{fetch_unit_keys, fetch_fmts_indexes, key_fetch}`, `sector::{KeyFetch, KeyFetchFn}`, `session::{resolve_keys_for, ResolvedKeys}`, `DiscSession::{resolve_keys, key_fetch}`, `Disc::{decrypt_keys, decrypt_keys_for_title, decrypt_with, ensure_decryptable, ensure_decryptable_keys, ensure_title_decryptable}`, `InputOptions::{unit_keys, key_fetch}`, `DiscStream::with_key_map`. `DecryptKeys` is `#[non_exhaustive]`; `AacsKeyMap::from_ranges*`, `DecryptingSectorSource::{with_key_map, set_key_map}` and `aacs::content::{decrypt_unit, decrypt_bus}` are no longer public. Acquire once with `DiscSession::resolve_key_set` / `KeyRing::acquire`. +- **Mux entry point:** `mux_stream` and `MuxInput` are replaced by `mux_with_keys(Source, Option<&KeyRing>, dest_url, opts, ctx)`; a `Source` carries no key material. `InputOptions::keys: Option` replaces `unit_keys` / `key_fetch`; an AACS image opened without a set is refused before any output. Crate-root `build_iso_pipeline` and `resolve_mux_key_map` are removed. +- **One run context for every stage:** `Ctx { halt, events, stats, diag }` is passed to `input(url, opts, ctx)`, `mux_with_keys(.., ctx)`, `Disc::extract_tree(reader, dest, opts, ctx)`, `write_image(reader, dest, sectors, ctx)` and `PrefetchedSectorSource::new(.., ctx)`. One `Events` trait over one `Event` enum (`BytesRead`, `SectorSkipped`, `UnitBlanked`, `BatchSizeChanged`, `Pass`, `BytesWritten`, `BytesDurable`, `OutputOpened`) replaces `MuxEvents`, `event::{Event, EventKind}`, `progress::Progress`, `EventFn` / `PrefetchedSectorSource::new_with_events`, `DiscStream::{on_event, with_halt}` and `Drive::on_event`; `ExtractOptions::{halt, progress}` are removed (Stop is `ctx.halt`; extract progress is `PassKind::Extract`). `Stats` / `LossReport` count read skips, lost bytes, blanked units and resync drops across the run. `halt::Progress` is renamed `halt::Liveness`. `FREEMKV_SKIP_PARSE` / `FREEMKV_PROFILE` take effect only through `Diag::from_env()` on the caller's `Ctx`. +- **Key acquisition is not bound to a scanned disc:** `ResolvedKeySet` is renamed `KeyRing` and `ResolveKeysOptions` `AcquireOptions` (its `halt` is the run's `Ctx`; `resolve_with_progress` is `AcquireOptions::liveness`). `KeyRing::acquire(&KeyEvidence, &mut Sampler, sources, opts, ctx)` takes what a source can tell (`KeyEvidence`: `MediaId`, AACS structures, `Piece`s in scope, main title, `Detector`), and `KeyRing::acquire_for_disc(disc, reader, scope, sources, opts, ctx)` builds that evidence from a scanned disc (`KeyEvidence::from_disc`). `KeyRing::is_for` takes a `&MediaId` (`Disc::media_id()`), which includes the Volume ID fingerprint. Evidence with no disc hash derives it from the title-key file, so a keydb lookup no longer misses silently. The legacy `aacs::resolve` / `aacs::provider` ladder and `disc::Key` are removed; `KeyOrigin` lives in `aacs::trace` (still re-exported at the crate root). +- **One way to open any input:** `open_source(url, ScanOptions, ctx)` opens every input scheme, `disc://` included (the drive is brought up and scanned; `input("disc://…")` now opens it instead of returning `DiscUrlNotDirect`), and `Source::layout()` is the probe's scanned `Disc` for `disc://`, `iso://` and `dir://`. `MuxSource` is removed: `Source::from_session(&mut DiscSession)`, `Source::from_image(path, ScannedTitle)` (a title scanned elsewhere, never rescanned) and `Source::from_reader(reader, ScannedTitle)` replace `Session`, `Iso` and `Live`, and `mux_url(url, keys, dest, opts, ctx)` replaces `Url`. `MuxOptions` gains `title_index` (it replaces `MuxSource::Session::title_index` and `InputOptions::title_index` for a mux), and `MuxOptions::selection` applies to every source. `scan_iso` / `scan_dir` share the one image probe. +- **Sources and sinks are separate traits:** `pes::Stream` (and the `PesStream` re-export) is replaced by `PesSource` (`read`, `info`, `track_timing`, `codec_private`, `headers_ready`, `config_changes`, `errors`, `lost_bytes`) and `PesSink` (`write`, `finish`, `finish_incomplete`, `info`, `set_track_timing`, `set_codec_private`, `undelivered_streams`). `input()` returns `Box` and `output()` `Box`; a wrong-direction call no longer compiles instead of returning `StreamReadOnly` / `StreamWriteOnly`. `MkvStream`, `StdioStream` and `NetworkStream` implement both (and an inherent `info()`); `CountingStream` wraps a `PesSink`. `SinkCaps::of(&StreamUrl)` (`needs_frames`, `carries_mp2_extensions`) is the one table the mux consults for metadata sinks and MPEG-2 extension tracks. +- **One read loop and one demux for every sector source:** `DiscStream` is removed. A live drive is read by the same prefetch producer and demux as an image, under the drive's Read policy (`sector::read_stage::ReadPolicy::Live`: adaptive batches that shrink on a failing zone and regrow after a clean streak, one ECC recovery read for a unit that still fails, then a zero-filled, counted unit under `skip_errors` or a `DiscRead`); an image keeps fixed batches that stop on the first error (`ReadPolicy::Image`). Live reads now overlap demux and parse on their own thread. Read skips are reported in `MuxOutcome::errors` / `lost_bytes` as before. - **Stop is the only send bound:** `MuxOptions::send_deadline`, `NO_SEND_DEADLINE` and `halt::POLL_INTERVAL` are removed (use `halt::WAIT_SLICE`); a slow sink blocks the mux until it drains or Stop. - `Drive::clear_halt`, `Drive::scsi_mut` and `Drive::is_unlocked` and the path-based `platform::fs_type::detect` are removed; use `Drive::attach` / `detach` / `halt` / `token` and `DiscSession::source_mut`. +- **Block sinks:** `io::BlockSink` (`write_at(lba, bytes)`, `finish(Finish::{Complete, Incomplete})`) with `IsoSink` (a sector-exact image) and `NullBlockSink`, opened from a URL by `io::open_block_sink("iso://…" | "null://")`; `write_image` writes through `IsoSink`. `io::null_device()` / `is_null_device()` name the platform's null device (`NUL` on Windows), and `Disc::mapfile_for` keys its private mapfile on it, so a whole-disc copy to `null://` works on Windows. +- **Tree sink:** `io::TreeSink` is the `dir://` output, opened by `io::open_tree_sink("dir://…", force)`: host-safe names, collision refusal, `.partial` then rename, and the AACS/CERTIFICATE directories left out. `Disc::extract_into(reader, &mut TreeSink, keys, ctx)` runs the extraction into it; `Disc::extract_tree` is that chain over a sink it creates. +- **One decryption-stage constructor:** `DecryptingSectorSource::new(inner, keying)` takes `impl Into` — `DecryptKeys` (as before), a key ring's view, or the content-detected stage — replacing the crate-internal `detecting` constructor and `with_key_map` / `with_arrival` builders. +- **Removed `io::byte_prefetcher`:** `m2ts://` reads through the one Read stage and `PrefetchedSectorSource` (the same reads, in the same ~1 MiB batches, as before); `BytePrefetcher` and its `PrefetchShell` are gone. +- Error variants no path returns any more are removed with their codes: `DiscUrlNotDirect` (E9009; `disc://` opens through `input()`), `DirImageEncrypted` (E9063; an encrypted folder is keyed like its image) and `MultipassRequiresRaw` (E9082; a multipass recovery may decrypt). +- Unused scaffolding is removed: the `io::sink` module (`SequentialSink`, `RandomAccessSink`, `LocalFileSink`, `SocketSink`, `UdpSocketSink`; `WritebackFile` no longer implements them) and the unfinished `fmp4`, `m2ts_mux` and `HevcMux` muxers. - `DvdAudioAttr::sub_stream_id` (positional id) is removed. - `M2tsMeta` gains `timings`, `chapters`, `content_format` and `frame_padding` (all `#[serde(default)]`); a struct-literal build of it must name them. - `css::CrackOutcome` is `#[non_exhaustive]` and gains `Halted` and `Unreadable(Error)`; it no longer derives `Clone`. `css::crack_key` is deprecated for `crack_key_outcome`. @@ -27,6 +38,11 @@ - **`json://`** audio `sample_rate` is a number in Hz (`null` when unknown), not a string such as `"48kHz"`; a `sample_rates` array lists every rate a stream carries. - **FMKV (`network://`, `stdio://`) header version 2** is used when a track carries decoder timing (Opus CodecDelay/SeekPreRoll) and adds per-frame DiscardPadding; streams without timing stay version 1. Headers now also carry display aspect, CICP, stream purpose/qualifier, chapters and content format, and PIDs actually written by M2TS. A header declaring more than 256 streams is refused with E9008. An older receiver omits the MKV Channels element for the new channel-layout strings; keep both ends on one version. - **`dir://` image bytes:** the PVD declares interchange level 3, Unique IDs start at 16 and the volume id round-trips through the d-string encoder; `dir://` sources are stamped as images in the `fvi://` provenance header. +- **One output title name:** a title read from a disc layout (`disc://`, `iso://`, `dir://`, a staged image, a server live rip) is named by the disc's meta title (else its volume id) in every container's title field and in `fvi://` provenance; 1.7.7 named image and server-live rips by the MPLS playlist file. `fvi://` provenance of an `iso://`/`dir://` source now carries the volume id. +- **TrueHD 7.1 / Atmos labels on every path:** the playlist labels a 7.1 or Atmos TrueHD track 5.1 at 48 kHz; every mux (live disc, staged image, `iso://`/`dir://`, containers) now completes the track's channels, sample rate and label from its first major sync before the output opens, and `json://` / `chapters://` get the completed title too. 1.7.7 corrected only `iso://`/`dir://` URL rips, so live and staged-image rips now label such tracks 7.1 / Atmos. +- **Track selection applies to container inputs:** `selection` keeps a subset of the tracks of an `mkv://`, `mp4://`, `m2ts://`, `network://` or `stdio://` input (renumbered in order, video always kept), as it already did for disc titles; an unknown PID is refused at open. +- **A live DVD extracted to `dir://` is descrambled:** `Disc::extract_tree` finds and cracks each scrambled VTS from its content on every DVD-Video layout. A live drive scan records no disc-wide CSS key, so a live DVD's title VOBs were written still scrambled. +- **An encrypted disc folder is an ordinary `dir://` source:** a folder whose sampled content units are AACS-encrypted keeps its AACS verdict and is keyed and decrypted like an image of the same disc, instead of being refused with E9063; with no key it is refused before any output with E7022. - **Frames dropped by the resync gate count as errors** in `MuxOutcome::errors` / `lost_bytes`, together with AACS units blanked as damaged. - **A failed or stopped `network://` / `stdio://` sender resets the connection** (`Stream::finish_incomplete`), so the receiver errors instead of finishing a truncated title. - **Key-service 404 / 422 answers are E7022** (no key opens the title), and a key-service outage or unreachable source is a source failure, never "no key for this disc". @@ -36,14 +52,14 @@ ### Added - `mpg://` as a source and a destination: a DVD-Video / MPEG-2 program stream file reads through the decryption stage and the DVD sector pipeline (a scrambled `.vob` is cracked once, `--raw` keeps the ciphertext; a clear MPEG-1 or stuffed MPEG-2 stream is no longer read as scrambled, and a scrambled pack with no key fails E7023). `MpgSink` writes an H.222.0 program stream with P-STD pack timing, system header and PSM; `Error::MpgNoVideoTrack` (E9074), `Error::MpgUnpacketized` (E9075). `FileSectorSource::open_padded` zero-pads a partial tail sector. -- `keys` module: `ResolvedKeySet::resolve(disc, scope, sources, opts)` (`KeyScope::{None, Titles, WholeDisc}`, `ResolveKeysOptions`, `KeyResolution`) asks the key sources once before any output and proves each held key against the disc's ciphertext per stream file; keys and the Volume ID stay in memory (`Debug` redacts). Also `title_reader`, `whole_disc_reader`, `decrypt_status`, `check_decryptable`, `KeySetStatus`, `ForensicState`, `DecryptStatus`, `ProofCache`. A piece that cannot be proven up front is proven on first read; a unit no held key opens stops a title rip with E7022; in an image or folder it is blanked, warned and counted. `ResolveCtx::halt()` / `progress()` give sources the Stop token; key-service retries back off 1 s to 8 s. `MediaKeyVariantError::code()` returns E7100-E7108. +- `keys` module: `KeyRing::acquire(evidence, sampler, sources, opts, ctx)` (`KeyScope::{None, Titles, WholeDisc}`, `AcquireOptions`, `KeyResolution`) asks the key sources once before any output and proves each held key against the disc's ciphertext per stream file; keys and the Volume ID stay in memory (`Debug` redacts). Also `title_reader`, `whole_disc_reader`, `decrypt_status`, `check_decryptable`, `KeySetStatus`, `ForensicState`, `DecryptStatus`, `ProofCache`. A piece that cannot be proven up front is proven on first read; a unit no held key opens stops a title rip with E7022; in an image or folder it is blanked, warned and counted. `ResolveCtx::halt()` / `progress()` give sources the Stop token; key-service retries back off 1 s to 8 s. `MediaKeyVariantError::code()` returns E7100-E7108. - `disc_root_of(path)`: the AACS disc folder a loose Blu-ray clip sits in (`/BDMV/STREAM/x.m2ts` with `/AACS`), so a caller can resolve the clip's keys on `dir://`. - `mux::fit_report` / `FitReport` / `SkipReason` (`Mp2Extension`, `NoStreamId`, `UnmappableSubtitle`): a container-neutral pre-mux plan of which streams a destination carries. `mp4_fit_report` is unchanged. - `DiscSession::{open_with, scan_with, finish, attach_progress, token, from_drive, source_mut}` and `session::Finish::{Release, Unlock, Eject}`: a session runs under the caller's Stop token and ends on the handle it holds. `Drive::{attach, detach, attach_progress, halt, token}`: every CDB is checked against the op token. -- `halt`: waits re-check every `WAIT_SLICE` (20 ms); `Progress`, `StallTimer`, `join_within`, `spawn_drive_holder`. `halt` is available with only the `scsi` feature. -- `NetworkStream::listen_with_halt` / `accept_from_with_halt`; `MuxEvents::on_flush_progress(bytes_durable, bytes_total)`; `io::{ArtifactLock, durable_sync_file, FlushProgress}` (`.lock` sidecar; E9073 `TimedOut` after 30 s with no change); `Pipeline::{set_op_token, progress}`, `Sink::close_stopped` (a consumer closing after Stop keeps `*.partial`); `ISO_MUX_BATCH_SECTORS` is public. +- `halt`: waits re-check every `WAIT_SLICE` (20 ms); `Liveness`, `StallTimer`, `join_within`, `spawn_drive_holder`. `halt` is available with only the `scsi` feature. +- `NetworkStream::listen_with_halt` / `accept_from_with_halt`; `Event::BytesDurable { bytes, total }` while the output flushes; `io::{ArtifactLock, durable_sync_file, FlushProgress}` (`.lock` sidecar; E9073 `TimedOut` after 30 s with no change); `Pipeline::{set_op_token, progress}`, `Sink::close_stopped` (a consumer closing after Stop keeps `*.partial`); `ISO_MUX_BATCH_SECTORS` is public. - MPEG-2 multichannel extension audio (DVD `0xD0|n`) is a dependent track (`AudioStream::is_mp2_extension`) carried in the IR and over FMKV; sinks with no mapping list it in `MuxOutcome::undelivered_streams`, `mp4://` skips it (`Mp4SkipReason::Mp2Extension`). MPEG-1/2 audio (PES `0xC0|n`) is demuxed on DVD. -- Errors: E9085 `MuxBatchSectorsZero`, E6020 `ImageEndsBeforeRead`, E6021 `BusStreamUnmapped`, E6022 `ImageScoped`, E6023 XPL too large, E7031 `AacsKeyFileUnreadable`, E7032 `WholeDiscKeyMissing`, E7033 `AacsNoUsableHostCert`, E7034 `AacsVidNeedsDisc`, E7100-E7108 MKB variant faults, E9071 `StreamClosed`, E9072 `StreamHeaderWritten`, E9073 `TimedOut { op }`, E9074, E9075, and the remux/preflight set E9077 `RemuxVerifyFailed { kind, path }` (`E9077: `, or `runtime-mismatch /`), E9078 `MuxIncomplete`, E9079 `RemuxStagingInvalid`, E9080 `StagedCopySizeMismatch`, E9081 `WorkerLost { op }`, E9082 `MultipassRequiresRaw`, E9083 `StreamLanguageUnknown`, E9084 `RemuxTargetExists`. +- Errors: E9085 `MuxBatchSectorsZero`, E6020 `ImageEndsBeforeRead`, E6021 `BusStreamUnmapped`, E6022 `ImageScoped`, E6023 XPL too large, E7031 `AacsKeyFileUnreadable`, E7032 `WholeDiscKeyMissing`, E7033 `AacsNoUsableHostCert`, E7034 `AacsVidNeedsDisc`, E7100-E7108 MKB variant faults, E9071 `StreamClosed`, E9072 `StreamHeaderWritten`, E9073 `TimedOut { op }`, E9074, E9075, and the remux/preflight set E9077 `RemuxVerifyFailed { kind, path }` (`E9077: `, or `runtime-mismatch /`), E9078 `MuxIncomplete`, E9079 `RemuxStagingInvalid`, E9080 `StagedCopySizeMismatch`, E9081 `WorkerLost { op }`, E9083 `StreamLanguageUnknown`, E9084 `RemuxTargetExists`. - `whole_disc::{whole_disc_reader, WholeDiscReader}`: the decrypting reader for whole-disc / image to ISO copies; it keys every stream file no title plays (probed at its first unit and up to 32 across it) and blanks, warns and counts the units of a stream file no held key opens (E7032 only when no file opens at all, a source faults, or no key is held). `ScanOptions::raw_copy` is kept but has no effect: an unreadable `Unit_Key_RO.inf` never fails the scan. - `sector::bus_removal::{UnmappedStreamFile, ensure_image_debussable}`, `BusMap::unmapped`, `SectorSource::unmapped_stream_files`, `Disc::{mkv_staging_ranges, stream_content_ranges, inputs_with_samples}`, `DecryptingSectorSource::{blanked_units, clear_unit_base}`, `DiscPresence` / `disc_presence(path)`, `Playlist::duration_ticks`. - `probe_mkv` / `probe_mkv_with_cues` read a Matroska file's Info and Tracks (and last Cues time) without reading a cluster; `parse_freemkv_version`. @@ -56,7 +72,7 @@ - **AACS damage policy (E7013):** a flagged AACS unit no key opens (no TS sync, zero-filled or garbled head, lone FMTS verify failure, off-grid) is zero-filled, warned and counted on every path (ISO mux, live mux, extract, whole-disc reader) instead of failing with E7013, with an end-of-rip "N damaged AACS units blanked" log. A wrong key still fails: E7013 at resolve (or when two or more FMTS units fail under one key with none verifying), E7022 on arrival in a title rip (an image or folder blanks the unit instead). - A BD-TS mux with no AACS key set fails E7022 on the first AACS-flagged unit instead of writing ciphertext; an AACS HD DVD session with no set fails `NoDiscKey`. A key-set stop (E7022) is never shrunk, recovered or skipped. -- A Stop during a mux is always `Halted` / `completed = false`; a stopped read is a stop, not a skipped sector. A read error repeats on every later `read()` of the highway stream. A halted `PrefetchedSectorSource` close is `Err(Halted)`, not EOF. +- A Stop during a mux is always `completed = false, halted = true` (`MuxOutcome::halted`), whether it lands during the open or the pump: `mux_with_keys` no longer returns `Err(Halted)` from a stopped open. A Stop reaches `iso://` and `dir://` sources inside a blocked read, the image scan and the CSS crack (it was seen only between frames), and `iso://`, `dir://` and `mpg://` URL muxes report read progress like the disc arms. A stopped read is a stop, not a skipped sector. A read error repeats on every later `read()` of the highway stream. A halted `PrefetchedSectorSource` close is `Err(Halted)`, not EOF. - `Disc::scan` follows the standard AACS order (UDF, `Unit_Key_RO.inf`, certificate, MKB by plain reads, then the handshake, only on a disc with an AACS directory), no longer reads the MKB from the drive, and passes `None` to `KeySource::host_certs`. A live AACS disc whose `Unit_Key_RO.inf` is missing or unreadable is warned and scanned on, with E7031 recorded in `Disc::aacs_error` and every key refused (titles that need keys then fail E7022 on the normal key path); image and folder scans record it the same way. A handshake transport fault fails with `Drive::init`'s error and a Stop returns `Halted`. `DiscSession::scan` returns the key sources on failure so a retry keeps them. - Host-key (cert-route) bus-encrypted discs: every `/BDMV/STREAM` file is de-bussed (BEE flag and per-unit CPI honoured; CPI 0 passes through). A stream file whose extents cannot be read is re-read with FUA (bounded, 5 s pause a Stop ends), then recorded; `ensure_image_debussable` refuses a whole-disc image with E6021 naming each file and cause, `extract_tree` counts such files lost whole, and `stream_content_ranges` fails E6021. - `DecryptingSectorSource` has no default AACS unit base: a content read before `set_unit_base` fails `DecryptFailed`, as does a flagged BD-TS unit lacking TS sync at byte 4. @@ -68,7 +84,7 @@ - `Pipeline::finish_with_halt` waits on consumer progress (the 600 s join is a stall window). `sync_all` on a `WritebackFile` fails E9056 only after 60 s without flush progress and returns `Halted` on cancel; `WritebackFile` flushes in chunks (64 MiB, down to 4 MiB). Linux `sync_file_range` errors are latched; `write_image` fsyncs the parent directory; reads past EOF of an image return `ImageTruncated`. - `network://` send and receive accept LAN targets: loopback, private (RFC 1918), link-local, ULA and CGNAT addresses connect. Only unspecified, multicast, broadcast and reserved addresses are refused (E9022); a caller that takes targets from untrusted input must vet them itself. - `network://`: send connects to each vetted address in turn (10 s timeout); both ends enable TCP keepalive (a rejected option only warns); receive no longer blocks on a vanished sender or a Stop. `stdio://` input fails E9008 on a non-FMKV or damaged header once bytes were consumed. -- Container headers (MKV, fragmented MP4, M2TS, network) wait (5 s of source PTS, 2048 frames or half the header buffer) for the AAC AudioSpecificConfig and BD LPCM layout byte; a later config is back-patched into a seekable MKV. `MuxEvents::on_output_opened` lists only the streams the sink writes. `DiscStream::new` fails on `batch_sectors == 0`. +- Container headers (MKV, fragmented MP4, M2TS, network) wait (5 s of source PTS, 2048 frames or half the header buffer) for the AAC AudioSpecificConfig and BD LPCM layout byte; a later config is back-patched into a seekable MKV. `Event::OutputOpened` lists only the streams the sink writes. A mux with `batch_sectors == 0` fails E9085. - DVD: titles map through `VTS_PTT_SRPT` (TTN 0 rejected), one angle counted, an unusable IFO falls back to its BUP; MP2 `Channels` follows CRC-verified Layer II frames; the live path shares the file-backed path's access-unit assembly and H.264 second-field merge; unmapped PS streams warn once. - Codec framing: ADTS and MPEG audio resync only on a header confirmed by the next, so a few KB of garbage no longer poisons a track; MPEG-2 field order follows picture_structure; H.264 1080i I/P field pairs are keyframes; HEVC arms CRA-to-BLA on gaps. TS demux skips duplicate packets, resyncs after a byte slip, honours transport_error_indicator. - `mp4://` DTS follows ETSI TS 102 114 Annex E (`ddts` carries the stream's maximum rate; 96 kHz DTS-HD MA reads back as 96 kHz); video CTS is rounded to the nearest tick; `Mp4Reader` labels MPEG-audio `mp4a` as Mp3/Mp2. diff --git a/README.md b/README.md index 6d761eb8..aba254c4 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,8 @@ for title in &disc.titles { // Stream pipeline — read PES frames from any source, write to any output let opts = libfreemkv::InputOptions::default(); -let mut input = libfreemkv::input("iso://Disc.iso", &opts)?; +let ctx = libfreemkv::Ctx::default(); // halt, events, loss counters for the run +let mut input = libfreemkv::input("iso://Disc.iso", &opts, &ctx)?; let title = input.info().clone(); let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?; while let Ok(Some(frame)) = input.read() { @@ -86,15 +87,15 @@ cannot call into it; front-ends get recovery from the engine directly. | Stream | Input | Output | Transport | |--------|-------|--------|-----------| -| DiscStream | Yes | -- | Optical drive via SCSI | -| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) | +| `disc://` | Yes | -- | Optical drive via SCSI (live Read policy: adaptive batches, recovery read, skip or fail) | +| `iso://` / `dir://` | Yes | -- | Disc image file or folder (image Read policy; written by `freemkv_engine::recovery`) | | MkvStream | Yes | Yes | Matroska container | | M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header | | NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header | | StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe | | NullStream | -- | Yes | Discard sink (byte counter for benchmarks) | -Streams implement a single unified `pes::Stream` trait (re-exported as `PesStream`) exposing `read()` and `write()` on one type. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected. +Inputs implement `PesSource` (`read()`) and outputs `PesSink` (`write()`); a format that is both implements both. `open_source()` / `input()` open every input URL (`disc://` included) and `output()` every output URL. All URLs use the `scheme://path` format — bare paths are rejected. ### Keys @@ -119,10 +120,12 @@ Disc — scan titles, streams, AACS/CSS state ├── AACS — key resolution + content decryption └── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack) -Streams — unified PES pipeline - ├── PesStream — pes::Stream: one trait, read()/write() PES frames - ├── DiscStream — sectors → decrypt → TS demux → PES - ├── IsoStream — ISO file → decrypt → TS demux → PES +Streams — one PES pipeline + ├── Source — open_source(url): drive, image, folder, file, socket, pipe + ├── Read stage — extents → chunks under the source's ReadPolicy (live | image) + ├── Decrypt stage — AACS / CSS, content-detected, damage blanked and counted + ├── Demux — TS / PS → PES (one demux for every sector source) + ├── PesSource/PesSink — read()/write() PES frames ├── MkvStream — MKV mux/demux ├── M2tsStream — BD transport stream ├── NetworkStream — TCP with FMKV metadata header diff --git a/src/aacs/mod.rs b/src/aacs/mod.rs index d982f827..8e29c598 100644 --- a/src/aacs/mod.rs +++ b/src/aacs/mod.rs @@ -1,8 +1,5 @@ -//! AACS decryption — Volume Unique Key lookup and title key derivation. -//! -//! Two paths: VUK lookup (disc_hash → KEYDB.cfg → VUK, fast, common) or a full -//! handshake (device_keys + MKB → Media Key → + Volume ID → VUK, fallback). -//! The VUK decrypts title keys from AACS/Unit_Key_RO.inf; title keys decrypt +//! AACS primitives: MKB, title-key file, derivation, unit crypto and the resolution trace. +//! Keys are acquired only through [`crate::keys::KeyRing::acquire`]; title keys decrypt //! m2ts stream content (AES-128-CBC). pub mod content; @@ -14,8 +11,6 @@ pub mod host_certs; pub mod index_select; pub mod inf; pub mod mkb; -pub mod provider; -pub mod resolve; pub mod segment; // No production caller; not the live FMTS path. #[doc(hidden)] diff --git a/src/aacs/provider.rs b/src/aacs/provider.rs deleted file mode 100644 index cb7fd337..00000000 --- a/src/aacs/provider.rs +++ /dev/null @@ -1,387 +0,0 @@ -//! Key source abstraction for the AACS resolve chain. -//! -//! libfreemkv owns all AACS crypto; backends supply raw key material via -//! [`KeyProvider`]. Bulk methods (device/processing/media keys) are unioned -//! and deduped across providers, trying each; disc-keyed lookups -//! (`lookup_disc_by_hash`/`_by_vid`) short-circuit on the first hit, queried -//! in provider-array order. -//! -//! Default impls return empty/`None`; calls may block, so cache per-disc. - -use super::types::{DeviceKey, DiscEntry, HostCert}; - -/// Source of AACS key material. -/// -/// Implementors return raw material only — the resolver in -/// `aacs::resolve` and `aacs::derive` own the crypto (DK→PK walking, PK validation, -/// MK→VUK→TK derivation). See module docs for method semantics. -pub trait KeyProvider: Send + Sync { - /// Device keys (top-of-tree, walked by the resolver). - fn device_keys(&self) -> Vec { - Vec::new() - } - - /// Processing keys — terminal PKs or walk-input PKs. The - /// resolver tries each as a terminal first (cheap validate). - fn processing_keys(&self) -> Vec<[u8; 16]> { - Vec::new() - } - - /// Every Media Key this provider holds, regardless of which disc it was - /// filed under. An MK is MKB-scoped (shared across a pressing/MKB-family), - /// so the resolver can verify each against the disc's MKB (`km_verifies`) - /// and resolve a disc whose own hash/VID isn't directly keyed. - fn media_keys(&self) -> Vec<[u8; 16]> { - Vec::new() - } - - /// AACS host certificates (with their private keys) for drive - /// authentication. Multiple in case some are revoked. - /// - /// NOTE: not consumed by the resolver chain — the handshake reads - /// host certs from the caller-supplied credentials directly, so - /// overriding this method has no effect on drive authentication - /// today. Retained as a forward-looking extension point. - fn host_certs(&self) -> Vec { - Vec::new() - } - - /// Direct per-disc lookup by SHA-1 of `Unit_Key_RO.inf`. Returns - /// `Some(entry)` if this provider has pre-computed material for - /// the disc (paths 4 and 5). Short-circuits the resolver. - fn lookup_disc_by_hash(&self, _disc_hash: &[u8; 20]) -> Option { - None - } - - /// Lookup by Volume ID (path 3 — pre-computed MK + matching - /// VID). Short-circuits the resolver on hit. - fn lookup_disc_by_vid(&self, _volume_id: &[u8; 16]) -> Option { - None - } -} - -// Resolver-side helper that aggregates KeyProvider material across a provider array -// (union-and-dedup for bulk methods, first-hit for disc-keyed lookups). -pub(crate) struct Providers<'a>(pub &'a [&'a dyn KeyProvider]); - -impl Providers<'_> { - /// Union (deduped) — gather DKs from every provider. - pub fn device_keys(&self) -> Vec { - let mut v: Vec = self.0.iter().flat_map(|p| p.device_keys()).collect(); - // DeviceKey has no Ord/Hash; dedup on the value-defining tuple. - v.sort_unstable_by_key(|d| (d.key, d.node, d.uv, d.u_mask_shift)); - v.dedup_by_key(|d| (d.key, d.node, d.uv, d.u_mask_shift)); - v - } - - /// Union (deduped) — gather PKs from every provider. - pub fn processing_keys(&self) -> Vec<[u8; 16]> { - let mut v: Vec<[u8; 16]> = self.0.iter().flat_map(|p| p.processing_keys()).collect(); - v.sort_unstable(); - v.dedup(); - v - } - - /// Union of distinct Media Keys across every provider, for the MK-pool - /// brute (`km_verifies` against the disc's MKB). - pub fn media_keys(&self) -> Vec<[u8; 16]> { - let mut v: Vec<[u8; 16]> = self.0.iter().flat_map(|p| p.media_keys()).collect(); - v.sort_unstable(); - v.dedup(); - v - } - - /// Union — gather host certs from every provider. The SCSI handshake - /// reads host certs from the caller-supplied credentials directly and - /// does not call this, so it is currently unused by the resolver chain. - #[allow(dead_code)] - pub fn host_certs(&self) -> Vec { - self.0.iter().flat_map(|p| p.host_certs()).collect() - } - - /// Short-circuit — query providers in array order, first hit wins. - pub fn lookup_disc_by_hash(&self, disc_hash: &[u8; 20]) -> Option { - self.0.iter().find_map(|p| p.lookup_disc_by_hash(disc_hash)) - } - - /// Short-circuit — query providers in array order, first hit wins. - pub fn lookup_disc_by_vid(&self, volume_id: &[u8; 16]) -> Option { - self.0.iter().find_map(|p| p.lookup_disc_by_vid(volume_id)) - } -} - -// One key's raw material as a KeyProvider, for the resolver tests. Fills only its own -// level (DK/PK/MK/VUK) so the resolver runs the matching path. -#[cfg(test)] -pub(crate) struct SuppliedKey { - pub device_keys: Vec, - pub processing_keys: Vec<[u8; 16]>, - pub media_keys: Vec<[u8; 16]>, - pub disc_entry: Option, -} - -#[cfg(test)] -impl KeyProvider for SuppliedKey { - fn device_keys(&self) -> Vec { - self.device_keys.clone() - } - fn processing_keys(&self) -> Vec<[u8; 16]> { - self.processing_keys.clone() - } - fn media_keys(&self) -> Vec<[u8; 16]> { - self.media_keys.clone() - } - fn lookup_disc_by_hash(&self, _disc_hash: &[u8; 20]) -> Option { - self.disc_entry.clone() - } - fn lookup_disc_by_vid(&self, _volume_id: &[u8; 16]) -> Option { - self.disc_entry.clone() - } -} - -#[cfg(test)] -mod tests { - use super::*; - - fn entry(hash: &str, vuk: u8) -> DiscEntry { - DiscEntry { - disc_hash: hash.to_string(), - title: "t".to_string(), - media_key: None, - disc_id: None, - vuk: Some([vuk; 16]), - unit_keys: Vec::new(), - } - } - - /// A host cert whose (non-secret) certificate body and private key are both - /// filled with `byte`, so a cert is identifiable in an aggregated list. - fn cert(byte: u8) -> HostCert { - HostCert { - private_key: [byte; 20], - certificate: vec![byte; 92], - private_key_v2: None, - certificate_v2: None, - } - } - - fn dk(byte: u8, node: u16) -> DeviceKey { - DeviceKey { - key: [byte; 16], - node, - uv: 1, - u_mask_shift: 0, - } - } - - /// A provider that returns fixed bulk material and an optional disc entry - /// keyed unconditionally (used to test array-order short-circuiting). - #[derive(Default)] - struct Fixed { - dks: Vec, - pks: Vec<[u8; 16]>, - mks: Vec<[u8; 16]>, - hash_hit: Option, - vid_hit: Option, - } - impl KeyProvider for Fixed { - fn device_keys(&self) -> Vec { - self.dks.clone() - } - fn processing_keys(&self) -> Vec<[u8; 16]> { - self.pks.clone() - } - fn media_keys(&self) -> Vec<[u8; 16]> { - self.mks.clone() - } - fn lookup_disc_by_hash(&self, _h: &[u8; 20]) -> Option { - self.hash_hit.clone() - } - fn lookup_disc_by_vid(&self, _v: &[u8; 16]) -> Option { - self.vid_hit.clone() - } - } - - // ── KeyProvider default methods all return empty ─────────────────────── - - #[test] - fn default_provider_methods_return_empty() { - // A bare provider that overrides nothing must yield empty material so - // the resolver simply finds nothing through it (no surprise hits). - struct Empty; - impl KeyProvider for Empty {} - let e = Empty; - assert!(e.device_keys().is_empty()); - assert!(e.processing_keys().is_empty()); - assert!(e.media_keys().is_empty()); - assert!(e.host_certs().is_empty()); - assert!(e.lookup_disc_by_hash(&[0u8; 20]).is_none()); - assert!(e.lookup_disc_by_vid(&[0u8; 16]).is_none()); - } - - // ── Providers::processing_keys: union + dedup ────────────────────────── - - #[test] - fn providers_processing_keys_union_and_dedup() { - // Two providers each carrying overlapping PKs → the aggregate is the - // deduped union (the resolver must not re-validate identical material). - let a = Fixed { - pks: vec![[0x01u8; 16], [0x02u8; 16]], - ..Default::default() - }; - let b = Fixed { - pks: vec![[0x02u8; 16], [0x03u8; 16]], - ..Default::default() - }; - let arr: &[&dyn KeyProvider] = &[&a, &b]; - let mut got = Providers(arr).processing_keys(); - got.sort(); - assert_eq!(got, vec![[0x01u8; 16], [0x02u8; 16], [0x03u8; 16]]); - } - - #[test] - fn providers_media_keys_union_and_dedup() { - let a = Fixed { - mks: vec![[0xAAu8; 16]], - ..Default::default() - }; - let b = Fixed { - mks: vec![[0xAAu8; 16], [0xBBu8; 16]], - ..Default::default() - }; - let arr: &[&dyn KeyProvider] = &[&a, &b]; - let mut got = Providers(arr).media_keys(); - got.sort(); - assert_eq!(got, vec![[0xAAu8; 16], [0xBBu8; 16]]); - } - - #[test] - fn providers_device_keys_dedup_on_value_tuple() { - // DeviceKey has no Hash/Ord; dedup keys on (key,node,uv,u_mask_shift). - // Two identical DKs across providers collapse to one; a DK differing - // only in node is kept. - let a = Fixed { - dks: vec![dk(0x11, 5), dk(0x11, 5)], - ..Default::default() - }; - let b = Fixed { - dks: vec![dk(0x11, 5), dk(0x11, 6)], - ..Default::default() - }; - let arr: &[&dyn KeyProvider] = &[&a, &b]; - let got = Providers(arr).device_keys(); - assert_eq!(got.len(), 2, "identical DKs dedup; differing node kept"); - let nodes: Vec = got.iter().map(|d| d.node).collect(); - assert!(nodes.contains(&5) && nodes.contains(&6)); - } - - #[test] - fn providers_device_keys_dedup_keeps_distinct_uv_and_mask_shift() { - // Same key and node, differing only in uv or u_mask_shift, are distinct positions. - let base = dk(0x22, 7); - let uv = DeviceKey { - uv: 2, - ..dk(0x22, 7) - }; - let shift = DeviceKey { - u_mask_shift: 3, - ..dk(0x22, 7) - }; - let a = Fixed { - dks: vec![base.clone(), uv, shift, base], - ..Default::default() - }; - let arr: &[&dyn KeyProvider] = &[&a]; - assert_eq!(Providers(arr).device_keys().len(), 3); - } - - // ── Disc-keyed lookups: array-order short-circuit ────────────────────── - - #[test] - fn providers_lookup_by_hash_first_hit_wins() { - // Querying providers in array order, the FIRST hit wins (closest / - // fastest first). Provider 0 hits → its entry is returned even though - // provider 1 also has one. - let a = Fixed { - hash_hit: Some(entry("first", 0x01)), - ..Default::default() - }; - let b = Fixed { - hash_hit: Some(entry("second", 0x02)), - ..Default::default() - }; - let arr: &[&dyn KeyProvider] = &[&a, &b]; - let got = Providers(arr).lookup_disc_by_hash(&[0u8; 20]).unwrap(); - assert_eq!(got.disc_hash, "first"); - assert_eq!(got.vuk, Some([0x01u8; 16])); - } - - #[test] - fn providers_lookup_by_hash_falls_through_to_later_provider() { - // Provider 0 misses, provider 1 hits → the later provider's entry is - // used (find_map continues past None). - let a = Fixed::default(); // hash_hit None - let b = Fixed { - hash_hit: Some(entry("second", 0x02)), - ..Default::default() - }; - let arr: &[&dyn KeyProvider] = &[&a, &b]; - let got = Providers(arr).lookup_disc_by_hash(&[0u8; 20]).unwrap(); - assert_eq!(got.disc_hash, "second"); - } - - #[test] - fn providers_lookup_by_vid_first_hit_wins() { - let a = Fixed { - vid_hit: Some(entry("vid-a", 0x07)), - ..Default::default() - }; - let b = Fixed { - vid_hit: Some(entry("vid-b", 0x08)), - ..Default::default() - }; - let arr: &[&dyn KeyProvider] = &[&a, &b]; - let got = Providers(arr).lookup_disc_by_vid(&[0u8; 16]).unwrap(); - assert_eq!(got.disc_hash, "vid-a"); - } - - // `host_certs` unions across the provider array, unwired into the handshake today; asserts - // full concatenation in array order since HostCert has no Ord/Hash to dedup. - #[test] - fn providers_host_certs_unions_every_providers_certs_in_array_order() { - struct Certs(Vec); - impl KeyProvider for Certs { - fn host_certs(&self) -> Vec { - self.0.clone() - } - } - // Distinguish certs by their (non-secret) certificate body, so the - // assertion lands on WHICH certs came back, not merely how many. - let a = Certs(vec![cert(0xA1), cert(0xA2)]); - let b = Certs(vec![cert(0xB1)]); - let arr: &[&dyn KeyProvider] = &[&a, &b]; - - let got = Providers(arr).host_certs(); - let bodies: Vec> = got.iter().map(|c| c.certificate.clone()).collect(); - assert_eq!( - bodies, - vec![vec![0xA1u8; 92], vec![0xA2u8; 92], vec![0xB1u8; 92]], - "every provider's certs must survive the union, in array order" - ); - // The private key travels with the cert — a union that returned default - // certs would still have the right count. - assert_eq!(got[0].private_key, [0xA1u8; 20]); - assert_eq!(got[2].private_key, [0xB1u8; 20]); - } - - #[test] - fn providers_empty_array_yields_nothing() { - let arr: &[&dyn KeyProvider] = &[]; - let p = Providers(arr); - assert!(p.device_keys().is_empty()); - assert!(p.processing_keys().is_empty()); - assert!(p.media_keys().is_empty()); - assert!(p.lookup_disc_by_hash(&[0u8; 20]).is_none()); - assert!(p.lookup_disc_by_vid(&[0u8; 16]).is_none()); - } -} diff --git a/src/aacs/resolve.rs b/src/aacs/resolve.rs deleted file mode 100644 index 7a59b68f..00000000 --- a/src/aacs/resolve.rs +++ /dev/null @@ -1,2248 +0,0 @@ -//! AACS key resolution — VUK derivation, MKB processing, disc hash, unit key parsing. - -use super::crypto::aes_ecb_decrypt; -use super::derive::*; -use super::inf::*; -use super::mkb::*; - -// Canonical form `1003` (`0x1003` fixed marker). Types 3/4/10: AACS -// CCE spec 0.953 §3.2.5.1.1. Category-C 2.0/2.1: standard MKB type constants. - -// ── Full VUK resolution chain ─────────────────────────────────────────────── - -/// Result of resolving a disc's VUK. -pub struct ResolvedKeys { - /// Disc hash (SHA1 of Unit_Key_RO.inf) - pub disc_hash: [u8; 20], - /// Volume Unique Key. `None` for path 5 — the KEYDB unit-keys - /// fallback consumes pre-decrypted unit keys directly and has no - /// VUK to surface. - pub vuk: Option<[u8; 16]>, - /// Decrypted unit keys (CPS unit number, key) - pub unit_keys: Vec<(u32, [u8; 16])>, - /// Title → CPS unit index mapping - pub title_cps_unit: Vec, - /// AACS generation that drove the resolution - pub version: AacsVersion, - /// Whether bus encryption is enabled (from Content Certificate) - pub bus_encryption: bool, - /// Which resolution path succeeded (1=DK, 2=PK, 3=KEYDB derived, - /// 4=KEYDB VUK, 5=KEYDB unit keys) - pub key_source: u8, -} - -// Redacting `Debug`: `vuk` and `unit_keys` are raw key bytes, never printed. -// `disc_hash` is the public per-disc identifier (SHA-1 of the .inf), not secret. -// Guarded by `resolved_keys_debug_is_redacted`. -impl std::fmt::Debug for ResolvedKeys { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.debug_struct("ResolvedKeys") - .field("disc_hash", &self.disc_hash) - .field("vuk", &self.vuk.map(|_| "")) - .field("unit_keys_len", &self.unit_keys.len()) - .field("title_cps_unit", &self.title_cps_unit) - .field("version", &self.version) - .field("bus_encryption", &self.bus_encryption) - .field("key_source", &self.key_source) - .finish() - } -} - -/// Inputs shared by every classical-path resolver. References only — -/// callers retain ownership of all buffers. -pub struct ResolveContext<'a> { - /// `Unit_Key_RO.inf` raw bytes. - pub unit_key_ro: &'a [u8], - /// Content Certificate raw bytes (optional — used for bus-encryption flag). - pub content_cert: Option<&'a [u8]>, - /// 16-byte Volume ID from SCSI handshake. `[0u8; 16]` is the - /// "no VID" sentinel and disables paths 1-3. - pub volume_id: &'a [u8; 16], - /// Key sources — checked in array order for disc-keyed lookups, - /// union'd across all entries for bulk material (DKs, PKs, HCs). - /// A keydb file, a webservice, an OEM provider can all coexist. - pub providers: &'a [&'a dyn super::provider::KeyProvider], - /// MKB raw bytes (optional — paths 1/2 require it). - pub mkb: Option<&'a [u8]>, -} - -// Carries no key bytes and is independent of the decryption math — purely -// the *reason* a resolution returned no key. - -/// Why a key resolution attempt produced no usable key. -/// -/// Distinguishes the two no-key outcomes that an application must report -/// differently: -/// * [`ResolveFailure::VidUnavailable`] — derivation material (device or -/// processing keys) was present, but no VID was available; the fix is -/// recovering the VID (a drive / handshake problem), not adding keys. -/// * [`ResolveFailure::NoMaterial`] — no usable key material was found at -/// all (no DK/PK material, no disc-keyed hit). The fix is to add keys. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum ResolveFailure { - /// Derivation material was present (DKs or PKs) but no VID was available - /// to derive the unit key. Surfaced as [`crate::error::Error::AacsVidUnavailable`]. - VidUnavailable, - /// No usable key material at all. Surfaced as - /// [`crate::error::Error::NoDiscKey`]. - NoMaterial, -} - -/// Version-dispatched resolution that preserves the *reason* on failure. -/// -/// Identical key derivation to the [`resolve_keys_v1`] / [`resolve_keys_v2`] / -/// [`resolve_keys_v21`] chain (it calls straight through to them); the only -/// addition is that an unresolved disc returns a typed [`ResolveFailure`] -/// instead of a bare `None`, so callers can report E7017 (material but no VID) -/// vs E7022 (no material). `version_u8` is the on-disc AACS major (1 → V10, -/// anything else → the V20/V21 chain), matching `AacsState::version`. -pub fn resolve_keys_with_reason( - ctx: &ResolveContext<'_>, - version_u8: u8, -) -> std::result::Result { - let resolved = match version_u8 { - 1 => resolve_keys_v1(ctx), - _ => resolve_keys_v2(ctx).or_else(|| resolve_keys_v21(ctx)), - }; - match resolved { - Some(r) => Ok(r), - None => Err(classify_resolve_failure(ctx)), - } -} - -// Classify why resolution found no key: derivation material present but the -// VID sentinel all-zero -> VidUnavailable; otherwise -> NoMaterial. Reads -// only provider material + the VID sentinel — no derivation, no descramble. -pub(crate) fn classify_resolve_failure(ctx: &ResolveContext<'_>) -> ResolveFailure { - let has_vid = *ctx.volume_id != [0u8; 16]; - let providers = super::provider::Providers(ctx.providers); - // Media keys are also derivation material: with an MK you can derive the VUK - // once you have the VID, so a media-keys-only provider that is merely missing - // the VID is VidUnavailable, not NoMaterial. - let has_derivation_material = !providers.device_keys().is_empty() - || !providers.processing_keys().is_empty() - || !providers.media_keys().is_empty(); - if !has_vid && has_derivation_material { - ResolveFailure::VidUnavailable - } else { - ResolveFailure::NoMaterial - } -} - -/// AACS 1.0 key resolution. Parses `Unit_Key_RO.inf` with 48-byte -/// stride. Tries paths 1 → 5 in order (via `resolve_keys_classical`). -pub fn resolve_keys_v1(ctx: &ResolveContext<'_>) -> Option { - resolve_keys_classical(ctx, AacsVersion::V10) -} - -/// AACS 2.0 key resolution. Parses `Unit_Key_RO.inf` with 64-byte -/// stride. Tries paths 1 → 5 in order. When paths 3/4 succeed against -/// an MKB carrying Variant records (`0x2d` / `0x2f`), the result's -/// `version` is upgraded to [`AacsVersion::V21`] — derivation still -/// runs through the classical V2 path; the V21-specific Variant chain -/// is wired separately via [`resolve_keys_v21`]. -pub fn resolve_keys_v2(ctx: &ResolveContext<'_>) -> Option { - let mut resolved = resolve_keys_classical(ctx, AacsVersion::V20)?; - if let Some(mkb) = ctx.mkb { - let recs = super::mkb::walk_mkb(mkb); - if super::variant::is_variant_mkb(&recs) { - resolved.version = AacsVersion::V21; - } - } - Some(resolved) -} - -/// AACS 2.1 key resolution via the Media Key Variant chain. -/// -/// Paths run in root-of-trust → per-disc-leaf order: -/// 1. Variant chain: device keys → PK → Km → Kvu (needs a covering 2.1 -/// Processing Key) -/// 3. KEYDB MK + matching VID → derived VUK -/// 4. KEYDB disc-hash → VUK -/// 5. KEYDB disc-hash → pre-decrypted unit keys (no VUK) -pub fn resolve_keys_v21(ctx: &ResolveContext<'_>) -> Option { - // Numbering preserves the cross-resolver convention; AACS 2.1 has no - // equivalent of path 2 (no host-side PK derivation against a Variant - // MKB). Path 3's VUK derives identically to V20 once the MK is found. - let uk_file = parse_title_keys(ctx.unit_key_ro, AacsVersion::V20)?; - let hash_hex = disc_hash_hex(&uk_file.disc_hash); - let bus_encryption = ctx - .content_cert - .and_then(parse_content_cert) - .map(|cc| cc.bus_encryption) - .unwrap_or(false); - let has_vid = *ctx.volume_id != [0u8; 16]; - - let derive_uks = |vuk: &[u8; 16]| derive_unit_keys(&uk_file, vuk); - - let build = - |vuk: Option<[u8; 16]>, unit_keys: Vec<(u32, [u8; 16])>, key_source: u8| -> ResolvedKeys { - ResolvedKeys { - disc_hash: uk_file.disc_hash, - vuk, - unit_keys, - title_cps_unit: uk_file.title_cps_unit.clone(), - version: AacsVersion::V21, - bus_encryption, - key_source, - } - }; - - tracing::info!( - target: "freemkv::disc", - phase = "resolve_keys_v21_start", - bus_encryption, - disc_hash = %hash_hex, - has_vid, - mkb_present = ctx.mkb.is_some(), - "resolve_keys_v21: starting" - ); - - let providers = super::provider::Providers(ctx.providers); - - if has_vid { - // Path 1: Variant chain (V21's analogue of classical Path 1's DK - // derivation): DK → PK via variant walk, then PK → Km, then VUK - // from Km + VID. - if let Some(mkb) = ctx.mkb { - let recs = super::mkb::walk_mkb(mkb); - let all_dks = providers.device_keys(); - match super::variant::derive_media_key_from_device_keys(&recs, &all_dks) { - Ok(km) => { - let kvu = derive_vuk(&km, ctx.volume_id); - tracing::debug!( - target: "freemkv::disc", - phase = "resolve_keys_v21_path1_hit", - "Variant chain produced Km + Kvu" - ); - return Some(build(Some(kvu), derive_uks(&kvu), 1)); - } - Err(e) => { - tracing::debug!( - target: "freemkv::disc", - phase = "resolve_keys_v21_path1_miss", - error_code = %e, - "Variant chain failed" - ); - } - } - } - - // Path 3: pre-computed MK + matching VID → derived VUK. - // Short-circuit: first provider with a matching VID wins. - if let Some(entry) = providers.lookup_disc_by_vid(ctx.volume_id) { - // Already matched by VID and derive_vuk needs only mk + volume_id, so a - // provider matching by VID without disc_id (e.g. a webservice) must not - // lose its MK — gate on the MK alone. - if let Some(mk) = entry.media_key { - let vuk = derive_vuk(&mk, ctx.volume_id); - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_v21_path3_hit", "MK+VID entry matched volume_id"); - return Some(build(Some(vuk), derive_uks(&vuk), 3)); - } - } - } else { - tracing::debug!( - target: "freemkv::disc", - phase = "resolve_keys_v21_no_vid", - "VID unavailable; paths 1/3 skipped" - ); - } - - // Paths 4 and 5: hash lookup, prefer V over U on the same entry. - if let Some(entry) = providers.lookup_disc_by_hash(&uk_file.disc_hash) { - if let Some(vuk) = entry.vuk { - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_v21_path4_hit", "VUK from KEYDB"); - return Some(build(Some(vuk), derive_uks(&vuk), 4)); - } else if let Some(unit_keys) = match_keydb_unit_keys(&uk_file, &entry.unit_keys) { - tracing::debug!( - target: "freemkv::disc", - phase = "resolve_keys_v21_path5_hit", - uk_count = unit_keys.len(), - "unit keys from KEYDB (no VUK)" - ); - return Some(build(None, unit_keys, 5)); - } - } - - None -} - -// Classical (single-stage Media Key derivation) path chain, used by both V10 and V20. -fn resolve_keys_classical(ctx: &ResolveContext<'_>, version: AacsVersion) -> Option { - let bus_encryption = ctx - .content_cert - .and_then(parse_content_cert) - .map(|cc| cc.bus_encryption) - .unwrap_or(false); - - // Parse the disc's title-key file (BD/UHD Unit_Key_RO.inf at the - // version-appropriate stride, or HD DVD VTKF000.AACS) → common UnitKeyFile. - let uk_file = parse_title_keys(ctx.unit_key_ro, version)?; - - let hash_hex = disc_hash_hex(&uk_file.disc_hash); - let has_vid = *ctx.volume_id != [0u8; 16]; - - // Decrypt the disc's encrypted unit keys with a freshly-derived VUK. - let derive_uks = |vuk: &[u8; 16]| derive_unit_keys(&uk_file, vuk); - - // Common result constructor — paths 1-4 supply Some(VUK) + derived - // unit keys; path 5 supplies None + pre-decrypted unit keys from - // KEYDB. - let build = - |vuk: Option<[u8; 16]>, unit_keys: Vec<(u32, [u8; 16])>, key_source: u8| -> ResolvedKeys { - ResolvedKeys { - disc_hash: uk_file.disc_hash, - vuk, - unit_keys, - title_cps_unit: uk_file.title_cps_unit.clone(), - version, - bus_encryption, - key_source, - } - }; - - tracing::info!( - target: "freemkv::disc", - phase = "resolve_keys_start", - version = ?version, - bus_encryption, - disc_hash = %hash_hex, - has_vid, - mkb_present = ctx.mkb.is_some(), - "resolve_keys: starting" - ); - - let providers = super::provider::Providers(ctx.providers); - - // Paths 1 and 2 need both MKB and VID. Logged as a single skip when - // either is absent so operators see one reason, not two. - if has_vid { - if let Some(mkb) = ctx.mkb { - // Borrowed views: only the lengths are logged, so copy nothing. - let mk_dv = mkb_find_mk_dv(mkb); - let subdiff = find_record_slice(mkb, REC_SUBSET_DIFFERENCE); - let cvalues = find_record_slice(mkb, REC_MEDIA_KEY_DATA); - tracing::debug!( - target: "freemkv::disc", - phase = "resolve_keys_mkb_records", - mk_dv_found = mk_dv.is_some(), - subdiff_found = subdiff.is_some(), - subdiff_len = subdiff.map_or(0, <[u8]>::len), - cvalues_found = cvalues.is_some(), - cvalues_len = cvalues.map_or(0, <[u8]>::len), - "MKB record scan results" - ); - - // Path 1: MKB + device keys → media key → VUK - let all_dks = providers.device_keys(); - if let Some(mk) = derive_media_key_from_dk(mkb, &all_dks) { - let vuk = derive_vuk(&mk, ctx.volume_id); - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path1_hit", "media key derived from device key"); - return Some(build(Some(vuk), derive_uks(&vuk), 1)); - } - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path1_miss", dk_count = all_dks.len(), "DK derivation failed"); - - // Path 2: MKB + processing keys → media key → VUK - let all_pks = providers.processing_keys(); - if let Some(mk) = derive_media_key_from_pk(mkb, &all_pks) { - let vuk = derive_vuk(&mk, ctx.volume_id); - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_hit", "media key derived from processing key"); - return Some(build(Some(vuk), derive_uks(&vuk), 2)); - } - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_miss", pk_count = all_pks.len(), "PK derivation failed"); - - // Path 2.5: MK-pool brute. An MK is MKB-scoped, so a disc whose own - // hash/VID isn't keyed can still resolve if ANY stored MK verifies (a - // UNIQUE pass via km_verifies is this disc's Km); mk_dv is hoisted out. - let mks = providers.media_keys(); - let chosen_mk = mkb_find_mk_dv(mkb).and_then(|mk_dv| { - unique_verifying_mk(&mks, |mk| aes_ecb_decrypt(mk, &mk_dv)[..8] == VERIFY_MAGIC) - }); - if let Some(mk) = chosen_mk { - let vuk = derive_vuk(&mk, ctx.volume_id); - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_hit", mk_pool = mks.len(), "media key from keydb MK-pool brute (km_verifies)"); - // Same class as path 3 (KEYDB MK → derived VUK). - return Some(build(Some(vuk), derive_uks(&vuk), 3)); - } - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path2_5_miss", mk_pool = mks.len(), "MK-pool brute: no unique verifying MK"); - } else { - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_no_mkb", "no MKB; paths 1/2 skipped"); - } - - // Path 3: pre-computed MK + matching VID → derived VUK. - // Short-circuit: first provider with a matching VID wins. - if let Some(entry) = providers.lookup_disc_by_vid(ctx.volume_id) { - // Already matched by VID and derive_vuk needs only mk + volume_id, so a - // provider matching by VID without disc_id (e.g. a webservice) must not - // lose its MK — gate on the MK alone. - if let Some(mk) = entry.media_key { - let vuk = derive_vuk(&mk, ctx.volume_id); - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path3_hit", "MK+VID entry matched volume_id"); - return Some(build(Some(vuk), derive_uks(&vuk), 3)); - } - } - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path3_miss", "no MK+VID entry matched volume_id"); - } else { - tracing::debug!( - target: "freemkv::disc", - phase = "resolve_keys_no_vid", - "VID unavailable; paths 1/2/3 require VID and are skipped" - ); - } - - // Paths 4 and 5: single hash-keyed lookup, prefer V (path 4) over - // U (path 5). They are not independent checks — path 5 only fires - // because path 4 had no VUK on the same entry. - if let Some(entry) = providers.lookup_disc_by_hash(&uk_file.disc_hash) { - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_keydb_hit_entry", "disc hash found in provider"); - if let Some(vuk) = entry.vuk { - tracing::debug!(target: "freemkv::disc", phase = "resolve_keys_path4_hit", "VUK from provider"); - return Some(build(Some(vuk), derive_uks(&vuk), 4)); - } else if let Some(unit_keys) = match_keydb_unit_keys(&uk_file, &entry.unit_keys) { - tracing::debug!( - target: "freemkv::disc", - phase = "resolve_keys_path5_hit", - uk_count = unit_keys.len(), - "unit keys from provider (no VUK)" - ); - return Some(build(None, unit_keys, 5)); - } - tracing::warn!(target: "freemkv::disc", phase = "resolve_keys_keydb_no_keys", "provider entry has neither VUK nor matching unit keys"); - } else { - tracing::warn!(target: "freemkv::disc", phase = "resolve_keys_keydb_miss", "disc hash NOT found in any provider"); - } - - None -} - -// Path 2.5's MK-pool selection rule: a Media Key must be the EXACTLY ONE distinct verifying -// candidate (dupes across providers are one candidate). -fn unique_verifying_mk(mks: &[[u8; 16]], verifies: impl Fn(&[u8; 16]) -> bool) -> Option<[u8; 16]> { - let mut hits: Vec<[u8; 16]> = Vec::new(); - for mk in mks { - if verifies(mk) && !hits.contains(mk) { - hits.push(*mk); - if hits.len() > 1 { - // Ambiguous — bail rather than pick a Media Key. - return None; - } - } - } - hits.first().copied() -} - -// Path 5: cross-reference the disc's Unit_Key_RO.inf CPS-unit numbering -// against the KEYDB entry's pre-decrypted unit keys. Partial coverage -// returns `None` so the resolver doesn't half-decrypt a disc. -pub(crate) fn match_keydb_unit_keys( - uk_file: &UnitKeyFile, - keydb_unit_keys: &[(u32, [u8; 16])], -) -> Option> { - if keydb_unit_keys.is_empty() { - return None; - } - let mut matched = Vec::with_capacity(uk_file.encrypted_keys.len()); - for (disc_num, _enc_key) in &uk_file.encrypted_keys { - let entry = keydb_unit_keys.iter().find(|(n, _)| n == disc_num)?; - matched.push(*entry); - } - Some(matched) -} - -#[cfg(test)] -mod tests { - // This suite predates the module split; it white-box-tests items now living - // in sibling modules. Pull them all in so the tests keep exercising them. - use super::super::crypto::*; - use super::super::provider::SuppliedKey; - use super::super::types::DiscEntry; - use super::super::types::*; - use super::*; - - /// `ResolvedKeys` carries the disc's VUK and unit keys raw; `Debug` must not - /// leak them. Sentinel 213 (0xD5); non-secret fields are not 213. - #[test] - fn resolved_keys_debug_is_redacted() { - let rk = ResolvedKeys { - disc_hash: [0u8; 20], - vuk: Some([0xD5; 16]), - unit_keys: vec![(1, [0xD5; 16])], - title_cps_unit: vec![0], - version: AacsVersion::V21, - bus_encryption: true, - key_source: 1, - }; - let dbg = format!("{rk:?}"); - assert!(!dbg.contains("213"), "ResolvedKeys leaked keys: {dbg}"); - assert!( - dbg.contains("redacted"), - "ResolvedKeys missing marker: {dbg}" - ); - } - - /// Audit #5: the `major` / `from_major` mapping is load-bearing for the - /// Unit_Key_RO stride, so pin it as a table. V10 ↔ BD; V20/V21 → UHD; any - /// non-BD major selects the V20/V21 64-byte stride (V10 is the only 48-byte). - #[test] - fn aacs_major_round_trips_and_strides_differ() { - assert_eq!(AacsVersion::V10.major(), AACS_MAJOR_BD); - assert_eq!(AacsVersion::V20.major(), AACS_MAJOR_UHD); - assert_eq!(AacsVersion::V21.major(), AACS_MAJOR_UHD); - assert_eq!(AacsVersion::from_major(AACS_MAJOR_BD), AacsVersion::V10); - assert_eq!(AacsVersion::from_major(AACS_MAJOR_UHD), AacsVersion::V20); - assert_eq!(AacsVersion::from_major(99), AacsVersion::V20); // any non-BD → V20 - assert_ne!( - AacsVersion::from_major(AACS_MAJOR_BD).unit_key_stride(), - AacsVersion::from_major(AACS_MAJOR_UHD).unit_key_stride() - ); - } - - // Finding #5 regression: parse_unit_key_ro must REJECT a Unit_Key_RO.inf - // whose declared num_unit_keys exceeds the keys actually present, not - // silently return a short list (which maps titles to nonexistent keys). - #[test] - fn parse_unit_key_ro_rejects_truncated_key_list() { - // V10 layout: stride 48, keys start at uk_pos + 48. - // uk_pos = 32; num_uk = 2; keys at 80 and 128. - let uk_pos = 32usize; - let build = |total_len: usize| -> Vec { - let mut data = vec![0u8; total_len]; - // uk_pos as BE32 at [0..4]. - data[0..4].copy_from_slice(&(uk_pos as u32).to_be_bytes()); - // num_unit_keys = 2 (BE16) at uk_pos. - data[uk_pos] = 0x00; - data[uk_pos + 1] = 0x02; - data - }; - - // Full buffer: room for both keys (keys_start 80, key1 at 128..144). - let full = build(144); - let ok = - parse_unit_key_ro(&full, AacsVersion::V10).expect("a full 2-key buffer must parse"); - assert_eq!(ok.encrypted_keys.len(), 2); - - // Truncated buffer: header still declares 2 keys, but only the first - // fits (len 128 — the second key's 16 bytes run off the end). Must be - // rejected, not silently accepted with one key. - let short = build(128); - assert!( - parse_unit_key_ro(&short, AacsVersion::V10).is_none(), - "a buffer declaring more keys than it contains must be rejected" - ); - } - - #[test] - fn derive_media_key_from_dk_survives_out_of_range_u_mask_shift() { - // Regression: a u_mask_shift of 32..=63 (passes the 0xC0 revoked-marker - // check but overflows `0xFFFF_FFFF << shift`) used to panic in debug / - // compute a wrong mask in release. The walk must skip the bad slot. - let mut mkb: Vec = Vec::new(); - // 0x81 record: 4-byte header + 16-byte mk_dv body (rec_len = 20). - mkb.extend_from_slice(&[0x81, 0x00, 0x00, 0x14]); - mkb.extend_from_slice(&[0xAB; 16]); - // 0x04 Subset-Difference: one 5-byte entry with u_mask_shift = 0x30 - // (48 — out of range, but 0x30 & 0xC0 == 0 so the revoke check passes). - mkb.extend_from_slice(&[0x04, 0x00, 0x00, 0x09]); - mkb.extend_from_slice(&[0x30, 0x00, 0x00, 0x00, 0x01]); - // 0x05 cvalues: one 16-byte entry (rec_len = 20). - mkb.extend_from_slice(&[0x05, 0x00, 0x00, 0x14]); - mkb.extend_from_slice(&[0xCD; 16]); - - let dk = DeviceKey { - key: [0x11; 16], - node: 1, - uv: 1, - u_mask_shift: 0x30, // also out of range on the device-key side - }; - - // Must not panic; no valid derivation is expected from this junk. - let _ = derive_media_key_from_dk(&mkb, &[dk]); - } - - #[test] - fn test_decrypt_unit_key_from_vuk() { - // VUK → encrypted unit key → unit key roundtrip. The keydb-sourced variant - // moved to freemkv-keysources; this exercises the same AES-G primitive - // with directly-constructed material — `aes_ecb_encrypt` is pub(crate). - use super::super::crypto::aes_ecb_encrypt; - let vuk = [0x5Au8; 16]; - // A few representative "decrypted" unit keys. - for expected_uk in [[0x11u8; 16], [0x22u8; 16], [0xCDu8; 16]] { - let encrypted = aes_ecb_encrypt(&vuk, &expected_uk); - let decrypted = decrypt_unit_key(&vuk, &encrypted); - assert_eq!(decrypted, expected_uk, "unit key roundtrip under VUK"); - } - } - - #[test] - fn test_disc_hash() { - // SHA1 of a known byte sequence - let data = b"test unit key ro inf data"; - let hash = disc_hash(data); - assert_ne!(hash, [0u8; 20]); - // Same input → same hash - assert_eq!(hash, disc_hash(data)); - } - - #[test] - fn test_disc_hash_hex() { - let hash = [ - 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, - 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, - ]; - let hex = disc_hash_hex(&hash); - assert_eq!(hex, "0x000102030405060708090A0B0C0D0E0F10111213"); - } - - #[test] - fn test_parse_unit_key_ro_synthetic() { - // Build a synthetic Unit_Key_RO.inf - // Header: uk_pos at offset 0 (BE32), points to key storage - // Keys at uk_pos + 48 (16 bytes each, 48-byte stride for AACS 1.0) - let mut data = vec![0u8; 256]; - - // uk_pos = 0x60 (96) - data[0] = 0x00; - data[1] = 0x00; - data[2] = 0x00; - data[3] = 0x60; - - // Header fields at 16-18 - data[16] = 1; // app_type = BD-ROM - data[17] = 1; // num_bdmv_dir - data[18] = 0; // no SKB - - // Title mapping at 20-25 - data[20] = 0; - data[21] = 1; // first_play = CPS unit 1 - data[22] = 0; - data[23] = 1; // top_menu = CPS unit 1 - data[24] = 0; - data[25] = 1; // num_titles = 1 - // Title 0 entry: 2 bytes pad + CPS unit - data[28] = 0; - data[29] = 1; // CPS unit 1 - - // Key storage at offset 0x60 - let uk_pos = 0x60usize; - data[uk_pos] = 0; - data[uk_pos + 1] = 2; // 2 unit keys - - // Key 1 at uk_pos + 48 - let key1_pos = uk_pos + 48; - for i in 0..16 { - data[key1_pos + i] = 0xAA; - } - - // Key 2 at uk_pos + 48 + 48 - let key2_pos = key1_pos + 48; - for i in 0..16 { - data[key2_pos + i] = 0xBB; - } - - let parsed = parse_unit_key_ro(&data, AacsVersion::V10).unwrap(); - assert_eq!(parsed.app_type, 1); - assert_eq!(parsed.num_bdmv_dir, 1); - assert_eq!(parsed.version, AacsVersion::V10); - assert_eq!(parsed.encrypted_keys.len(), 2); - assert_eq!(parsed.encrypted_keys[0].0, 1); // CPS unit 1 - assert_eq!(parsed.encrypted_keys[0].1, [0xAA; 16]); - assert_eq!(parsed.encrypted_keys[1].0, 2); // CPS unit 2 - assert_eq!(parsed.encrypted_keys[1].1, [0xBB; 16]); - } - - #[test] - fn mkb_version_recognizes_type_0x10() { - // Type-and-Version record: type=0x10, rec_len=12 (BE24). - // Body is 8 bytes; the version u32 sits at offset 8 of the record. - let mkb = [ - 0x10, 0x00, 0x00, 0x0C, 0x48, 0x14, 0x10, 0x03, 0x00, 0x00, 0x00, 0x4D, - ]; - assert_eq!(mkb_version(&mkb), Some(77)); - } - - #[test] - fn mkb_content_len_trims_trailing_padding() { - // Two real records (0x10 type/version + 0x86 verify), then 128 KiB of - // zero padding (the fixed-region tail). Content length must stop at the - // end of the records, not include the padding. - let mut mkb = vec![ - 0x10, 0x00, 0x00, 0x0C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x4D, - ]; - mkb.extend_from_slice(&[0x86, 0x00, 0x00, 0x18]); - mkb.extend_from_slice(&[0xAB; 16]); - mkb.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); - let records_len = mkb.len(); - mkb.extend(std::iter::repeat_n(0u8, 128 * 1024)); // padding - assert_eq!(mkb_content_len(&mkb), records_len); - // No padding → returns the full length. - assert_eq!(mkb_content_len(&mkb[..records_len]), records_len); - // Empty → 0. - assert_eq!(mkb_content_len(&[]), 0); - } - - #[test] - fn trim_mkb_never_zeroes_an_unrecognised_mkb() { - // Regression: 0.31.0 truncated the MKB to mkb_content_len() unconditionally. - // For an unparseable first record, mkb_content_len() returns 0, so autorip - // sent an EMPTY MKB to the key service. trim_mkb must leave it intact. - let unrecognised = vec![0xFFu8; 4096]; // first "rec_type" 0xFF, rec_len huge → content_len 0 - assert_eq!( - mkb_content_len(&unrecognised), - 0, - "precondition: unparseable → 0" - ); - assert_eq!( - trim_mkb(unrecognised.clone()), - unrecognised, - "unrecognised MKB must be returned untouched, never zeroed" - ); - - // A parseable MKB with trailing padding IS trimmed to its records. - let mut mkb = vec![ - 0x10, 0x00, 0x00, 0x0C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x4D, - ]; - mkb.extend_from_slice(&[0x86, 0x00, 0x00, 0x18]); - mkb.extend_from_slice(&[0xAB; 16]); - mkb.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); - let records_len = mkb.len(); - mkb.extend(std::iter::repeat_n(0u8, 1024)); - assert_eq!( - trim_mkb(mkb).len(), - records_len, - "padded MKB trims to records" - ); - - // Empty stays empty (n==0 → untouched). - assert!(trim_mkb(Vec::new()).is_empty()); - } - - #[test] - fn mkb_version_returns_none_on_empty() { - assert_eq!(mkb_version(&[]), None); - assert_eq!(mkb_version(&[0x10, 0x00]), None); - // Type 0x10 record but rec_len < 12 → no version available. - let short = [0x10, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x01]; - assert_eq!(mkb_version(&short), None); - } - - #[test] - fn mkb_find_mk_dv_recognizes_type_0x81() { - // First: type-0x10 type/version record (12 bytes), then type-0x81 verify record. - // Verify record carries a known 16-byte mk_dv at offset 4 of the record body. - let expected: [u8; 16] = [ - 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, - 0xFF, 0x00, - ]; - let mut mkb = vec![ - 0x10, 0x00, 0x00, 0x0C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, - ]; - // type=0x81, rec_len=24 (4-byte header + 16-byte mk_dv + 4-byte trailing zeros) - mkb.extend_from_slice(&[0x81, 0x00, 0x00, 0x18]); - mkb.extend_from_slice(&expected); - mkb.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); - - assert_eq!(mkb_find_mk_dv(&mkb), Some(expected)); - } - - #[test] - fn probe_try_pk_against_tables_accepts_planted_pk_rejects_corrupt() { - // Lock in the terminal PK scan (`try_pk_against_tables`) used by - // `derive_media_key_from_pk`: a planted PK whose derived MK satisfies a - // synthetic verify record must be ACCEPTED; a 1-byte corruption REJECTED. - use super::super::crypto::aes_ecb_encrypt as enc; - - let pk: [u8; 16] = [ - 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, - 0xFF, 0x00, - ]; - let mk: [u8; 16] = [ - 0xA0, 0xA1, 0xA2, 0xA3, 0xA4, 0xA5, 0xA6, 0xA7, 0xA8, 0xA9, 0xAA, 0xAB, 0xAC, 0xAD, - 0xAE, 0xAF, - ]; - let uv: [u8; 4] = [0x00, 0x00, 0x04, 0x00]; - - let mut mk_raw = mk; - for a in 0..4 { - mk_raw[12 + a] ^= uv[a]; - } - let cv = enc(&pk, &mk_raw); // AES-D(pk, cv) == mk_raw - let mut vd = [0u8; 16]; - vd[..8].copy_from_slice(&[0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF]); - let mk_dv = enc(&mk, &vd); // AES-D(mk, mk_dv) starts with sentinel - - // 0x04 SD body: one entry [u_mask_shift=0][uv]. - let mut subdiff = vec![0u8]; - subdiff.extend_from_slice(&uv); - - assert_eq!( - try_pk_against_tables(std::slice::from_ref(&pk), &subdiff, &cv, &mk_dv), - Some(mk), - "planted terminal PK must verify" - ); - let mut bad = pk; - bad[0] ^= 0xFF; - assert_eq!( - try_pk_against_tables(std::slice::from_ref(&bad), &subdiff, &cv, &mk_dv), - None, - "corrupted PK must be rejected" - ); - } - - #[test] - fn validate_processing_key_round_trip_with_nonzero_uv() { - // Synthesise a (pk, uv, mk, cvalue, mk_dv) tuple satisfying the AACS - // PK-validation relation; confirms validate_processing_key recovers mk. - // Catches pre-fix bugs: missing uv XOR, and AES-128E+zero-check vs 128D+magic. - use super::super::crypto::{aes_ecb_decrypt as dec, aes_ecb_encrypt as enc}; - - let pk: [u8; 16] = [ - 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, - 0xFF, 0x00, - ]; - let mk: [u8; 16] = [ - 0xA0, 0xA1, 0xA2, 0xA3, 0xA4, 0xA5, 0xA6, 0xA7, 0xA8, 0xA9, 0xAA, 0xAB, 0xAC, 0xAD, - 0xAE, 0xAF, - ]; - let uv: [u8; 4] = [0x00, 0x00, 0x04, 0x00]; - - // cvalue is AES-128E(pk, mk') where mk' = mk with last-4-bytes-uv XOR - // pre-undone, so the validate step XORs uv back in and recovers mk. - let mut mk_raw = mk; - for a in 0..4 { - mk_raw[12 + a] ^= uv[a]; - } - let cvalue = enc(&pk, &mk_raw); - - // mk_dv is the encryption (under the correct mk) of the verify - // magic, padded with arbitrary bytes — when decrypted with mk we - // recover the magic. - let mut plaintext_vd = [0u8; 16]; - plaintext_vd[..8].copy_from_slice(&[0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF]); - // Trailing 8 bytes are don't-cares in the magic check. - plaintext_vd[8..].copy_from_slice(&[0x11; 8]); - let mk_dv = enc(&mk, &plaintext_vd); - // Sanity: decrypting mk_dv with mk yields the magic. - let _check = dec(&mk, &mk_dv); - - let recovered = validate_processing_key(&pk, &cvalue, &uv, &mk_dv) - .expect("validate_processing_key must accept a correct pk + uv pair"); - assert_eq!(recovered, mk, "recovered mk must match the planted mk"); - - // And a wrong pk must be rejected. - let mut wrong_pk = pk; - wrong_pk[0] ^= 0xFF; - assert!(validate_processing_key(&wrong_pk, &cvalue, &uv, &mk_dv).is_none()); - - // And a uv mismatch must be rejected. - let wrong_uv = [0x00u8, 0x00, 0x00, 0x00]; - assert!(validate_processing_key(&pk, &cvalue, &wrong_uv, &mk_dv).is_none()); - } - - // ── MKB cvalue-record selection (issue #259 / #281): cvalue (Media Key Data) - // is record 0x05, not 0x07 (Explicit SD, small ~96 entries vs 0x05's 181270). - // An earlier `mkb_find_cvalues` wrongly preferred 0x07, breaking DK→walk. - - /// Build a 4-byte MKB record header (type + 3-byte big-endian total - /// length, header included) and append `body`. - fn mkb_record(rec_type: u8, body: &[u8]) -> Vec { - let total = 4 + body.len(); - let mut rec = Vec::with_capacity(total); - rec.push(rec_type); - rec.push(((total >> 16) & 0xFF) as u8); - rec.push(((total >> 8) & 0xFF) as u8); - rec.push((total & 0xFF) as u8); - rec.extend_from_slice(body); - rec - } - - // AACS-2.x-shaped MKB with a small 0x07 decoy (0x07 first) AND the real - // 0x05 cvalue table, so a "0x07-first" selector picks wrong. 0x05 has - // `n` entries (1:1 with 0x04's SD index); the 0x07 decoy has `decoy`. - fn synth_aacs2_mkb(n: usize, decoy: usize) -> Vec { - let mut mkb = Vec::new(); - mkb.extend_from_slice(&mkb_record(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52])); - mkb.extend_from_slice(&mkb_record(0x86, &[0xABu8; 16])); - let mut sd = Vec::with_capacity(n * 5); - for i in 0..n { - sd.push(0x00); // u_mask_shift, top bits clear → not revoked - sd.extend_from_slice(&((i as u32) + 1).to_be_bytes()); - } - mkb.extend_from_slice(&mkb_record(0x04, &sd)); - mkb.extend_from_slice(&mkb_record(0x07, &vec![0x11u8; decoy * 16])); // decoy first - mkb.extend_from_slice(&mkb_record(0x05, &vec![0x22u8; n * 16])); // real cvalues - mkb - } - - #[test] - fn cvalue_selection_prefers_0x05_over_0x07() { - // AACS-2.x layout: large 0x05 (1:1 with 0x04) + smaller decoy 0x07 - // placed earlier in the record stream. - let n = 1500; - let decoy = 96; - let mkb = synth_aacs2_mkb(n, decoy); - - let sd = probe::mkb_subdiff(&mkb).expect("0x04 present"); - let r05 = probe::mkb_record_body(&mkb, 0x05).expect("0x05 present"); - let r07 = probe::mkb_record_body(&mkb, 0x07).expect("0x07 present"); - let selected = mkb_find_cvalues(&mkb).expect("cvalues selected"); - - assert_eq!(sd.len() / 5, n, "0x04 SD index entry count"); - assert_eq!(r05.len() / 16, n, "0x05 cvalue entry count"); - assert_eq!(r07.len() / 16, decoy, "0x07 decoy entry count"); - - // The fix: selection MUST pick 0x05 (the large 1:1 table), NOT the - // 0x07 decoy a "0x07-first" rule would return. - assert_eq!( - selected.len() / 16, - n, - "cvalue selection must use the large 0x05 table, not the {decoy}-entry 0x07 decoy" - ); - assert_eq!( - selected, r05, - "selected body must be the 0x05 record verbatim" - ); - assert_eq!( - selected.len() / 16, - sd.len() / 5, - "cvalue table must be 1:1 with the 0x04 Subset-Difference index" - ); - } - - #[test] - fn cvalue_selection_never_falls_back_to_the_0x07_index() { - // Only a 0x07 record and no 0x05: 0x07 is the Subset-Difference INDEX - // (libaacs reads cvalues from 0x05 only), so there is no cvalue table. - let mut mkb = Vec::new(); - mkb.extend_from_slice(&mkb_record(0x10, &[0, 0, 0, 0x10, 0, 0, 0, 1])); - mkb.extend_from_slice(&mkb_record(0x86, &[0xCDu8; 16])); - mkb.extend_from_slice(&mkb_record(0x04, &[0x00, 0, 0, 0, 1])); - let only07 = vec![0x33u8; 16]; - mkb.extend_from_slice(&mkb_record(0x07, &only07)); - - assert!(probe::mkb_record_body(&mkb, 0x05).is_none()); - assert!(probe::mkb_record_body(&mkb, 0x07) == Some(only07)); - assert!( - mkb_find_cvalues(&mkb).is_none(), - "0x07 is never a cvalue table" - ); - } - - /// Locate a captured MKB sample under the optional `MKB_SAMPLE_DIR`. - /// Returns `None` (skip) when the directory or file is absent. - fn mkb_sample(rel: &str) -> Option { - let dir = std::env::var("MKB_SAMPLE_DIR").ok()?; - let p = std::path::Path::new(&dir).join(rel); - if p.exists() { Some(p) } else { None } - } - - #[test] - fn real_aacs2_samples_select_large_0x05_not_small_0x07() { - // Real AACS 2.x UHD MKBs carry BOTH a small 0x07 record (96 entries) AND - // the large 0x05 cvalue table (181270 entries); selector must return the - // LARGE 0x05 body — the exact regression #259. Skips if no sample dir. - let samples = [ - "sample-a/MKB_RO.inf", - "sample-b/MKB_RO.inf", - "sample-c/MKB_RO.inf", - ]; - let mut checked = 0; - for rel in samples { - let path = match mkb_sample(rel) { - Some(p) => p, - None => continue, - }; - let data = std::fs::read(&path).expect("read sample MKB"); - - let r05 = probe::mkb_record_body(&data, 0x05) - .unwrap_or_else(|| panic!("{rel}: expected a 0x05 Media Key Data record")); - let r07 = probe::mkb_record_body(&data, 0x07) - .unwrap_or_else(|| panic!("{rel}: expected a 0x07 record")); - let sd = probe::mkb_subdiff(&data) - .unwrap_or_else(|| panic!("{rel}: expected a 0x04 Subset-Difference index")); - - let n05 = r05.len() / 16; - let n07 = r07.len() / 16; - - // The discriminating facts the bug report cited. - assert!( - n05 > n07 * 100, - "{rel}: 0x05 ({n05}) must dwarf 0x07 ({n07})" - ); - assert_eq!(n05, 181270, "{rel}: full 0x05 cvalue table size"); - assert_eq!(n07, 96, "{rel}: small 0x07 record size"); - - // Production selection must be the large 0x05 table. - let selected = mkb_find_cvalues(&data) - .unwrap_or_else(|| panic!("{rel}: cvalue selection returned None")); - assert_eq!( - selected, r05, - "{rel}: selector must return the large 0x05 body, not 0x07" - ); - - // And it is 1:1 with the 0x04 SD index the walk iterates: the - // walk's UV count (take_while top-2-bits clear) lines up with - // the cvalue count to within the trailing padding entry. - let uv_entries = sd - .chunks(5) - .take_while(|c| c.len() == 5 && (c[0] & 0xC0) == 0) - .count(); - assert!( - uv_entries >= n05 - 2 && uv_entries <= n05, - "{rel}: 0x04 UV count ({uv_entries}) should match 0x05 cvalue count ({n05})" - ); - - eprintln!( - "{rel}: 0x05={n05} cvalues, 0x07={n07}, 0x04 UVs={uv_entries} — selected 0x05" - ); - checked += 1; - } - if checked == 0 { - eprintln!("no MKB samples present; skipping real-sample assertion"); - } - } - - #[test] - fn mkb_find_mk_dv_recognizes_type_0x86() { - // AACS 2.0 form uses type 0x86 for the verify record. - let expected: [u8; 16] = [ - 0xDE, 0xAD, 0xBE, 0xEF, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, - 0x0B, 0x0C, - ]; - let mut mkb = vec![ - 0x10, 0x00, 0x00, 0x0C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x4D, - ]; - mkb.extend_from_slice(&[0x86, 0x00, 0x00, 0x18]); - mkb.extend_from_slice(&expected); - mkb.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); - - assert_eq!(mkb_find_mk_dv(&mkb), Some(expected)); - } - - /// Build a minimal Unit_Key_RO.inf with `num_unit_keys = 1`. The - /// disc hash won't be in any synthetic keydb so path 1 misses, - /// which lets us isolate the path-2/3/4 short-circuit behavior. - fn minimal_unit_key_ro() -> Vec { - let mut data = vec![0u8; 256]; - // uk_pos = 0x60 - data[3] = 0x60; - data[16] = 1; // app_type = BD-ROM - data[17] = 1; // num_bdmv_dir - let uk_pos = 0x60usize; - data[uk_pos + 1] = 1; // 1 unit key - // Key at uk_pos + 48 — value doesn't matter, just needs to fit. - for i in 0..16 { - data[uk_pos + 48 + i] = 0xCC; - } - data - } - - #[test] - fn resolve_keys_skips_paths_2_through_4_when_vid_is_zero() { - // No VID -> paths 2/3/4 cannot succeed. Must return None WITHOUT touching - // the MKB/device keys, so an MKB that would cause expensive derivation - // work can be passed here and must not be consumed. - let uk_ro = minimal_unit_key_ro(); - let zero_vid = [0u8; 16]; - - // Dummy processing key, NO matching disc entry: `disc_entry: None` keeps - // the path-4/5 hash lookup a miss (SuppliedKey returns disc_entry - // unconditionally, so a planted entry would WRONGLY hit path 4). - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: vec![[0u8; 16]], - media_keys: Vec::new(), - disc_entry: None, - }; - - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &zero_vid, - providers, - mkb: None, - }; - let result = resolve_keys_v1(&ctx); - assert!( - result.is_none(), - "resolve_keys with VID=0 and no matching disc-hash entry must return None" - ); - } - - // Synthetic single-slot MKB keyed by `pk`: cvalue = AES-E(pk, mk ^ uv), verify record - // for `mk`. Returns (mkb, mk). - fn planted_pk_mkb(pk: &[u8; 16]) -> (Vec, [u8; 16]) { - let mk = [0xA5u8; 16]; - let uv: u32 = 0x0000_0400; - let mut raw = mk; - for (a, b) in raw[12..16].iter_mut().zip(uv.to_be_bytes()) { - *a ^= b; - } - let cv = aes_ecb_encrypt(pk, &raw); - let mut vd = [0x5Au8; 16]; - vd[..8].copy_from_slice(&VERIFY_MAGIC); - let mut sub = vec![12u8]; - sub.extend_from_slice(&uv.to_be_bytes()); - let mut mkb = mkb_record(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]); - mkb.extend_from_slice(&mkb_record(0x86, &aes_ecb_encrypt(&mk, &vd))); - mkb.extend_from_slice(&mkb_record(0x04, &sub)); - mkb.extend_from_slice(&mkb_record(0x05, &cv)); - (mkb, mk) - } - - fn resolve_with(keydb: &SuppliedKey, mkb: &[u8], vid: &[u8; 16]) -> Option { - let uk_ro = minimal_unit_key_ro(); - let providers: &[&dyn super::super::provider::KeyProvider] = &[keydb]; - resolve_keys_v1(&ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: vid, - providers, - mkb: Some(mkb), - }) - } - - #[test] - fn resolve_keys_path2_processing_key_derives_vuk() { - let pk = [0x3Cu8; 16]; - let (mkb, mk) = planted_pk_mkb(&pk); - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: vec![pk], - media_keys: Vec::new(), - disc_entry: None, - }; - let vid = [0x22u8; 16]; - let r = resolve_with(&keydb, &mkb, &vid).expect("path 2 resolves"); - assert_eq!(r.key_source, 2); - assert_eq!(r.vuk, Some(derive_vuk(&mk, &vid))); - } - - #[test] - fn resolve_keys_path1_device_key_derives_vuk() { - let dkey = [0x1Du8; 16]; - let (mkb, mk) = planted_pk_mkb(&aesg3(&dkey, 1)); - let dk = super::super::derive::recover_dk_position(&mkb, &dkey) - .expect("planted key applies to the MKB"); - let keydb = SuppliedKey { - device_keys: vec![dk], - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: None, - }; - let vid = [0x22u8; 16]; - let r = resolve_with(&keydb, &mkb, &vid).expect("path 1 resolves"); - assert_eq!(r.key_source, 1); - assert_eq!(r.vuk, Some(derive_vuk(&mk, &vid))); - } - - // With a real MKB and derivation material present, a zero VID must still skip paths 1/2/2.5; - // only the has_vid gate stops them (the MKB itself would resolve). - #[test] - fn resolve_keys_zero_vid_gate_blocks_mkb_paths_even_with_a_resolvable_mkb() { - let pk = [0x3Cu8; 16]; - let (mkb, mk) = planted_pk_mkb(&pk); - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: vec![pk], - media_keys: vec![mk], - disc_entry: None, - }; - assert!(resolve_with(&keydb, &mkb, &[0u8; 16]).is_none()); - assert!(resolve_with(&keydb, &mkb, &[0x22u8; 16]).is_some()); - } - - // Known-answer vector: FIPS 180 SHA-1("abc"), as the standard's H0..H4 words. - #[test] - fn disc_hash_matches_the_sha1_known_answer() { - let h: [u32; 5] = [0xA9993E36, 0x4706816A, 0xBA3E2571, 0x7850C26C, 0x9CD0D89D]; - let want: String = h.iter().map(|w| format!("{w:08X}")).collect(); - assert_eq!(disc_hash_hex(&disc_hash(b"abc")), format!("0x{want}")); - } - - #[test] - fn resolve_keys_path4_still_runs_when_vid_is_zero() { - // Path 4 (disc-hash → VUK) doesn't need VID; confirm the short-circuit - // doesn't block it: install a keydb entry whose disc_hash matches the - // fixture with a known VUK, verify resolve_keys returns key_source = 4. - let uk_ro = minimal_unit_key_ro(); - let hash = disc_hash(&uk_ro); - // `find_disc` lowercases the incoming hash; the entry map is - // keyed lowercase too, so we have to lowercase here. - let hash_hex = disc_hash_hex(&hash).to_lowercase(); - - let known_vuk = [0xABu8; 16]; - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "fixture".to_string(), - media_key: None, - disc_id: None, - vuk: Some(known_vuk), - unit_keys: Vec::new(), - }), - }; - - let vid = [0u8; 16]; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &vid, - providers, - mkb: None, - }; - let resolved = - resolve_keys_v1(&ctx).expect("path 4 must run regardless of VID availability"); - assert_eq!(resolved.vuk, Some(known_vuk)); - assert_eq!(resolved.key_source, 4); - } - - #[test] - fn resolve_keys_path5_uses_keydb_unit_keys_when_vuk_absent() { - // Path 5: an entry with no VUK but pre-decrypted unit keys matching the - // disc's CPS-unit numbering decrypts directly. Covers the ~4,572 U-only - // KEYDB entries (mostly MKBv76+ UHDs) the resolver previously ignored. - let uk_ro = minimal_unit_key_ro(); - let hash = disc_hash(&uk_ro); - let hash_hex = disc_hash_hex(&hash).to_lowercase(); - - // `minimal_unit_key_ro` declares CPS unit 1; supply a matching - // pre-decrypted unit key in the KEYDB entry. - let known_uk = [0xCDu8; 16]; - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "fixture".to_string(), - media_key: None, - disc_id: None, - vuk: None, - unit_keys: vec![(1, known_uk)], - }), - }; - - let vid = [0u8; 16]; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &vid, - providers, - mkb: None, - }; - let resolved = - resolve_keys_v1(&ctx).expect("path 5 must succeed when KEYDB carries unit keys"); - assert_eq!(resolved.vuk, None, "path 5 has no VUK to return"); - assert_eq!(resolved.key_source, 5); - assert_eq!(resolved.unit_keys, vec![(1, known_uk)]); - } - #[test] - fn resolve_keys_path5_rejects_partial_unit_key_coverage() { - // If the disc declares a CPS unit not in the KEYDB entry's unit_keys, path - // 5 must NOT half-decrypt: match returns None and the resolver falls - // through to None overall (no other paths available here). - let uk_ro = minimal_unit_key_ro(); - let hash = disc_hash(&uk_ro); - let hash_hex = disc_hash_hex(&hash).to_lowercase(); - // KEYDB has a key for CPS unit 99, but the disc declares unit 1. - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "fixture".to_string(), - media_key: None, - disc_id: None, - vuk: None, - unit_keys: vec![(99, [0xEEu8; 16])], - }), - }; - - let vid = [0u8; 16]; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &vid, - providers, - mkb: None, - }; - assert!( - resolve_keys_v1(&ctx).is_none(), - "partial CPS-unit coverage must not produce a half-decrypted result" - ); - } - #[test] - fn resolve_keys_path2_5_mk_pool_brute_resolves_unkeyed_disc() { - // The keyless-disc case: this disc's own hash/VID are NOT in keydb, but its - // Media Key IS — filed under a sibling disc that shares its MKB. Path - // 2.5 must km_verifies that MK against the MKB and resolve. - use super::super::crypto::aes_ecb_encrypt as enc; - let km = [0x11u8; 16]; - let vid = [0x22u8; 16]; - // MKB: 0x10 type/version + 0x86 verify record whose mk_dv decrypts under - // km to the AACS verify magic, so km_verifies(mkb, km) == true. - let mut vd = [0u8; 16]; - vd[..8].copy_from_slice(&[0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF]); - let mk_dv = enc(&km, &vd); - let mut mkb = mkb_record(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x4D]); - mkb.extend_from_slice(&mkb_record(0x86, &mk_dv)); - assert!( - probe::km_verifies(&mkb, &km), - "fixture: km must verify the MKB" - ); - // This disc's inf (its hash will NOT be in keydb). - let uk_ro = minimal_unit_key_ro(); - // The sibling's MK is lifted into the MK pool via SuppliedKey's explicit - // media_keys field; `disc_entry: None` keeps this disc's own hash/VID a - // miss, so ONLY the MK-pool brute (km_verifies) can resolve it. - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: vec![km], - disc_entry: None, - }; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &vid, - providers, - mkb: Some(&mkb), - }; - let resolved = resolve_keys_v1(&ctx) - .expect("MK-pool brute (path 2.5) must resolve a disc whose MK is in keydb"); - assert_eq!( - resolved.key_source, 3, - "MK-pool brute is the KEYDB-derived class" - ); - assert_eq!( - resolved.vuk, - Some(derive_vuk(&km, &vid)), - "VUK must derive from the verified Km + this disc's VID" - ); - } - - // Path 2.5's ambiguity guard: >1 distinct pooled MK verifying must yield NO key, never a - // pick (a wrong Km derives a wrong VUK that decrypts to garbage rather than failing - // loudly). - #[test] - fn mk_pool_ambiguity_bails_rather_than_picking_a_media_key() { - let a = [0xAAu8; 16]; - let b = [0xBBu8; 16]; - let c = [0xCCu8; 16]; - - // One verifying candidate → that key. - assert_eq!( - unique_verifying_mk(&[a, b, c], |mk| *mk == b), - Some(b), - "a single verifying MK resolves" - ); - - // The SAME key repeated across providers is one candidate, not an - // ambiguity — the dedup (`!hits.contains`) must keep this resolvable. - assert_eq!( - unique_verifying_mk(&[b, b, b], |mk| *mk == b), - Some(b), - "duplicates of one key are not ambiguity" - ); - - // TWO DISTINCT verifying candidates → bail, no key. - assert_eq!( - unique_verifying_mk(&[a, b], |mk| *mk == a || *mk == b), - None, - "two distinct verifying MKs must yield NO key, not the first one" - ); - - // Ambiguity must still be detected when the second hit is last in the - // pool, i.e. the scan may not stop at the first hit. - assert_eq!( - unique_verifying_mk(&[a, c, [0u8; 16], b], |mk| *mk == a || *mk == b), - None, - "a late second hit is still ambiguous" - ); - - // Every candidate verifying is the degenerate ambiguous case. - assert_eq!(unique_verifying_mk(&[a, b, c], |_| true), None); - - // No candidate verifies → no key (and no panic on an empty pool). - assert_eq!(unique_verifying_mk(&[a, b, c], |_| false), None); - assert_eq!(unique_verifying_mk(&[], |_| true), None); - } - - #[test] - fn test_content_cert_parse() { - // AACS 1.0 cert, bus encryption OFF. Content-cert layout: flag in - // BIT 7 of byte 1, cc_id at bytes 14..20. - let mut data = vec![0u8; 20]; - data[0] = 0x00; // AACS 1.0 - data[1] = 0x00; // bus_encryption flag (bit 7) clear - data[14..20].copy_from_slice(&[0x11, 0x22, 0x33, 0x44, 0x55, 0x66]); - let cc = parse_content_cert(&data).unwrap(); - assert_eq!(cc.version, AacsVersion::V10); - assert!(!cc.bus_encryption); - assert_eq!(cc.cc_id, [0x11, 0x22, 0x33, 0x44, 0x55, 0x66]); - // AACS 2.0 with bus encryption: type 0x10, flag is BIT 7 (0x80) of byte 1 - // — NOT bit 0. A cert with byte1=0x01 must therefore read as bus-OFF. - data[0] = 0x10; // AACS 2.0 - data[1] = 0x80; // bus_encryption_enabled_flag = bit 7 - let cc = parse_content_cert(&data).unwrap(); - assert_eq!(cc.version, AacsVersion::V20); - assert!(cc.bus_encryption); - // Regression guard: bit 0 set, bit 7 clear -> bus OFF (the old bug read this as ON). - data[1] = 0x01; - assert!(!parse_content_cert(&data).unwrap().bus_encryption); - } - // ════════════════ Hardening additions ════════════════ - // ── VUK derivation: spec relation VUK = AES-D(MK, VID) XOR VID ───────── - #[test] - fn derive_vuk_matches_spec_relation_explicitly() { - // Independently compute AES-ECB-D(mk, vid) XOR vid and confirm - // derive_vuk produces the same 16 bytes. A mutation that dropped the - // XOR-VID step, or used encrypt instead of decrypt, fails this. - use super::super::crypto::aes_ecb_decrypt as dec; - let mk = [ - 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, - 0x1E, 0x1F, - ]; - let vid = [ - 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2A, 0x2B, 0x2C, 0x2D, - 0x2E, 0x2F, - ]; - let mut expected = dec(&mk, &vid); - for i in 0..16 { - expected[i] ^= vid[i]; - } - assert_eq!(derive_vuk(&mk, &vid), expected); - } - #[test] - fn decrypt_unit_key_is_plain_aes_ecb_decrypt_under_vuk() { - // The encrypted unit key in Unit_Key_RO.inf is AES-ECB-E(VUK, uk); - // decrypt_unit_key must be the matching ECB-decrypt. Round-trip via - // encrypt to pin the relation. - use super::super::crypto::aes_ecb_encrypt as enc; - let vuk = [0x9Eu8; 16]; - let uk = [0x3Cu8; 16]; - let enc_uk = enc(&vuk, &uk); - assert_eq!(decrypt_unit_key(&vuk, &enc_uk), uk); - } - // ── Unit_Key_RO stride: 48 (V10) vs 64 (V20/V21) ────────────────────── - /// Build a Unit_Key_RO.inf carrying `num_uk` keys at a given stride, - /// where key `i` is filled with byte `0x10 + i`. uk_pos = 0x60. - fn build_unit_key_ro(num_uk: usize, stride: usize) -> Vec { - let uk_pos = 0x60usize; - let size = uk_pos + 48 + stride * num_uk + 64; - let mut data = vec![0u8; size]; - // uk_pos BE32 at [0..4]. - data[0..4].copy_from_slice(&(uk_pos as u32).to_be_bytes()); - data[16] = 1; // app_type - data[17] = 1; // num_bdmv_dir - // num_unit_keys BE16 at uk_pos. - data[uk_pos..uk_pos + 2].copy_from_slice(&(num_uk as u16).to_be_bytes()); - // Keys start at uk_pos + 48, stride apart. - let mut pos = uk_pos + 48; - for i in 0..num_uk { - for b in &mut data[pos..pos + 16] { - *b = 0x10 + i as u8; - } - pos += stride; - } - data - } - #[test] - fn stride_v10_is_48_v20_is_64_and_picks_distinct_keys() { - // AACS 1.0 stride = 48, 2.0/2.1 = 64. Keys laid at 64-byte stride: V20 - // parsing must pick exactly those keys, while V10 (48) parsing the SAME - // bytes would read wrong intermediate bytes for key 2 onward. - let data = build_unit_key_ro(2, 64); - let v20 = parse_unit_key_ro(&data, AacsVersion::V20).unwrap(); - assert_eq!(v20.encrypted_keys.len(), 2); - assert_eq!(v20.encrypted_keys[0].1, [0x10; 16]); - assert_eq!(v20.encrypted_keys[1].1, [0x11; 16]); - // Same buffer, V10 stride: key 1 still lands at uk_pos+48, but key 2 - // is read at +48 (not +64) so it is NOT the planted 0x11 block. - let v10 = parse_unit_key_ro(&data, AacsVersion::V10).unwrap(); - assert_eq!(v10.encrypted_keys[0].1, [0x10; 16]); - assert_ne!( - v10.encrypted_keys[1].1, [0x11; 16], - "48-byte stride must read different bytes than 64-byte stride" - ); - } - #[test] - fn v21_uses_same_64_byte_stride_as_v20() { - // V21 shares V20's 64-byte stride (the enum match groups V20|V21). - let data = build_unit_key_ro(2, 64); - let v20 = parse_unit_key_ro(&data, AacsVersion::V20).unwrap(); - let v21 = parse_unit_key_ro(&data, AacsVersion::V21).unwrap(); - assert_eq!(v20.encrypted_keys, v21.encrypted_keys); - assert_eq!(v21.version, AacsVersion::V21); - } - // ── parse_unit_key_ro: early returns / boundaries ────────────────────── - #[test] - fn parse_unit_key_ro_rejects_too_short_header() { - // < 20 bytes → None (header fields at 16-18 would index OOB). - assert!(parse_unit_key_ro(&[0u8; 19], AacsVersion::V10).is_none()); - } - #[test] - fn parse_unit_key_ro_rejects_uk_pos_past_end() { - // uk_pos points past the buffer → the `uk_pos + 2 > len` guard - // returns None rather than indexing OOB. - let mut data = vec![0u8; 64]; - data[0..4].copy_from_slice(&1000u32.to_be_bytes()); // uk_pos = 1000 - assert!(parse_unit_key_ro(&data, AacsVersion::V10).is_none()); - } - #[test] - fn parse_unit_key_ro_zero_keys_returns_empty_set() { - // num_unit_keys == 0 → a valid file with no encrypted keys (early - // Some(..) branch), NOT None. - let uk_pos = 0x60usize; - let mut data = vec![0u8; uk_pos + 48]; - data[0..4].copy_from_slice(&(uk_pos as u32).to_be_bytes()); - data[16] = 1; - // num_uk left 0. - let parsed = parse_unit_key_ro(&data, AacsVersion::V10).unwrap(); - assert!(parsed.encrypted_keys.is_empty()); - assert_eq!(parsed.app_type, 1); - } - #[test] - fn parse_unit_key_ro_truncated_key_region_returns_none() { - // keys_start + 16 > len → None (the first key can't fit). - let uk_pos = 0x60usize; - let mut data = vec![0u8; uk_pos + 48 + 8]; // only 8 of 16 key bytes - data[0..4].copy_from_slice(&(uk_pos as u32).to_be_bytes()); - data[uk_pos + 1] = 1; // 1 key declared - assert!(parse_unit_key_ro(&data, AacsVersion::V10).is_none()); - } - #[test] - fn parse_unit_key_ro_rejects_when_keys_run_off_end() { - // Finding #5: 3 keys declared but buffer holds only 2 strides + 8 spare - // bytes. Extraction loop breaks at buffer end (never OOB), and the - // post-loop length check rejects the short list rather than accepting fewer. - let uk_pos = 0x60usize; - let stride = 48usize; - // Room for keys at uk_pos+48 and uk_pos+48+48, then only 8 spare bytes - // (key 3 would start at uk_pos+48+96 and need 16, but only 8 remain). - let size = uk_pos + 48 + stride + 16 + 8; - let mut data = vec![0u8; size]; - data[0..4].copy_from_slice(&(uk_pos as u32).to_be_bytes()); - data[uk_pos + 1] = 3; // declare 3 keys - assert!( - parse_unit_key_ro(&data, AacsVersion::V10).is_none(), - "a buffer declaring more keys than it contains must be rejected" - ); - } - #[test] - fn parse_unit_key_ro_app_type_and_skb_flag() { - // app_type at [16], num_bdmv_dir at [17], use_skb_mkb = bit 7 of [18]. - let mut data = build_unit_key_ro(1, 48); - data[16] = 0x02; - data[17] = 0x05; - data[18] = 0x80; // bit 7 set - let p = parse_unit_key_ro(&data, AacsVersion::V10).unwrap(); - assert_eq!(p.app_type, 0x02); - assert_eq!(p.num_bdmv_dir, 0x05); - assert!(p.use_skb_mkb, "bit 7 of byte 18 → use_skb_mkb true"); - // Clearing bit 7 (other bits set) → false. - data[18] = 0x7F; - let p2 = parse_unit_key_ro(&data, AacsVersion::V10).unwrap(); - assert!(!p2.use_skb_mkb); - } - #[test] - fn parse_unit_key_ro_cps_unit_numbers_are_1_based() { - // The disc's CPS unit numbers are emitted as (i+1) — aacs/inf.rs. - let data = build_unit_key_ro(3, 48); - let p = parse_unit_key_ro(&data, AacsVersion::V10).unwrap(); - assert_eq!( - p.encrypted_keys.iter().map(|(n, _)| *n).collect::>(), - vec![1, 2, 3] - ); - } - #[test] - fn parse_unit_key_ro_title_cps_mapping_first_play_top_menu_then_titles() { - // [20..22] first_play, [22..24] top_menu, [24..26] num_titles, then per-title - // 2-byte pad + 2-byte CPS unit at 26 + i*4 + 2. Each on-disc 1-based CPS - // number in `1..=num_uk` converts to a 0-based key index; out-of-range → 0. - let mut data = build_unit_key_ro(4, 64); // num_uk = 4 → CPS 1..=4 valid - data[20..22].copy_from_slice(&1u16.to_be_bytes()); // first_play CPS 1 - data[22..24].copy_from_slice(&2u16.to_be_bytes()); // top_menu CPS 2 - data[24..26].copy_from_slice(&3u16.to_be_bytes()); // num_titles = 3 - data[28..30].copy_from_slice(&3u16.to_be_bytes()); // title 0 CPS 3 - data[32..34].copy_from_slice(&4u16.to_be_bytes()); // title 1 CPS 4 - data[36..38].copy_from_slice(&9u16.to_be_bytes()); // title 2 CPS 9 (> num_uk) - let p = parse_unit_key_ro(&data, AacsVersion::V20).unwrap(); - // 1-based CPS {1,2,3,4} → 0-based {0,1,2,3}; out-of-range 9 → 0. - assert_eq!(p.title_cps_unit, vec![0, 1, 2, 3, 0]); - } - // ── MKB record framing: rec_len is BE24 incl. 4-byte header ──────────── - #[test] - fn mkb_version_uses_be24_length_and_reads_offset_8() { - // Type 0x10, BE24 length 0x0C (12). Body starts at pos+4: Type field - // u32 at body offset 0, version u32 at body offset 4 (pos+8). - // Confirm a length encoded in the high BE24 byte is honored. - let mkb = [ - 0x10, 0x00, 0x00, 0x0C, 0x11, 0x22, 0x33, 0x44, 0x01, 0x02, 0x03, 0x04, - ]; - // version = 0x01020304. - assert_eq!(mkb_version(&mkb), Some(0x0102_0304)); - } - #[test] - fn mkb_type_category_c_20_is_uhd() { - // Type 0x10 record, BE24 length 0x0C (12). MKBType field (body - // offset 0 = pos+4) = MKB_20_CATEGORY_C (0x48141003). - let mkb = [ - 0x10, 0x00, 0x00, 0x0C, 0x48, 0x14, 0x10, 0x03, 0x00, 0x00, 0x00, 0x01, - ]; - assert_eq!(mkb_type_raw(&mkb), Some(MKB_20_CATEGORY_C)); - assert_eq!(mkb_type(&mkb), Some(MkbType::CategoryC20)); - assert_eq!(mkb_is_uhd(&mkb), Some(true)); - assert!(MkbType::CategoryC20.is_uhd()); - assert_eq!(MkbType::CategoryC20.generation(), AacsVersion::V20); - // Sanity on the 2.1 sibling. - assert_eq!(MkbType::from_raw(MKB_21_CATEGORY_C), MkbType::CategoryC21); - assert_eq!(MkbType::CategoryC21.generation(), AacsVersion::V21); - } - #[test] - fn mkb_type_prerecorded_is_bluray_v10() { - // Type 0x10 record with MKB_TYPE_4_PRERECORDED (0x00041003) — a - // standard Blu-ray (AACS 1.0) block, not UHD. - let mkb = [ - 0x10, 0x00, 0x00, 0x0C, 0x00, 0x04, 0x10, 0x03, 0x00, 0x00, 0x00, 0x01, - ]; - assert_eq!(mkb_type(&mkb), Some(MkbType::Prerecorded)); - assert_eq!(mkb_is_uhd(&mkb), Some(false)); - assert!(!MkbType::Prerecorded.is_uhd()); - assert_eq!(MkbType::Prerecorded.generation(), AacsVersion::V10); - } - #[test] - fn mkb_type_none_when_no_0x10_record() { - // A buffer whose only record is a 0x81 (verify-media-key) record and - // no 0x10 Type-and-Version record → mkb_type_raw returns None. - let mkb = [0x81, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00]; - assert_eq!(mkb_type_raw(&mkb), None); - assert_eq!(mkb_type(&mkb), None); - assert_eq!(mkb_is_uhd(&mkb), None); - } - #[test] - fn mkb_find_mk_dv_skips_short_verify_record() { - // A 0x81 record with rec_len < 20 carries no full mk_dv; the finder - // must skip it and keep walking (here to a valid 0x86 after it). - let mut mkb = vec![0x81, 0x00, 0x00, 0x10]; // rec_len 16 (< 20) - mkb.extend_from_slice(&[0x00; 12]); - let expected = [0xC1u8; 16]; - mkb.extend_from_slice(&[0x86, 0x00, 0x00, 0x18]); - mkb.extend_from_slice(&expected); - mkb.extend_from_slice(&[0x00; 4]); - assert_eq!(mkb_find_mk_dv(&mkb), Some(expected)); - } - #[test] - fn mkb_find_mk_dv_stops_on_overrun_length() { - // A rec_len that runs past the buffer ends the walk (break), so no - // mk_dv is found and we get None rather than an OOB slice. - let mkb = [0x81, 0x00, 0xFF, 0xFF, 0x00, 0x00]; // claims 65535 bytes - assert_eq!(mkb_find_mk_dv(&mkb), None); - } - #[test] - fn mkb_find_mk_dv_stops_on_zero_length_record() { - // rec_len < 4 (here 0) breaks the walk — guards against an infinite - // loop on a malformed record (pos would never advance). - let mkb = [0x81, 0x00, 0x00, 0x00, 0x99]; - assert_eq!(mkb_find_mk_dv(&mkb), None); - } - // ── mkb_content_len / trim_mkb ───────────────────────────────────────── - #[test] - fn mkb_content_len_stops_at_zero_type_padding_byte() { - // A type==0 byte marks the start of padding (records done). Two real - // records then a 0x00 type byte → content_len == sum of the two recs. - let mut mkb = vec![0x10, 0x00, 0x00, 0x08, 0, 0, 0, 1]; // 8-byte rec - mkb.extend_from_slice(&[0x05, 0x00, 0x00, 0x08, 9, 9, 9, 9]); // 8-byte rec - let content = mkb.len(); - mkb.extend_from_slice(&[0x00, 0x00, 0x00, 0x08]); // padding starts (type 0) - assert_eq!(mkb_content_len(&mkb), content); - } - #[test] - fn mkb_content_len_returns_full_len_when_no_padding() { - let mut mkb = vec![0x10, 0x00, 0x00, 0x08, 0, 0, 0, 1]; - mkb.extend_from_slice(&[0x05, 0x00, 0x00, 0x08, 9, 9, 9, 9]); - assert_eq!(mkb_content_len(&mkb), mkb.len()); - } - #[test] - fn trim_mkb_leaves_exactly_sized_buffer_untouched() { - // n == mkb.len() (no padding) → the `n < mkb.len()` guard is false, - // so the buffer is returned untouched (no spurious truncate). - let mkb = vec![0x10, 0x00, 0x00, 0x08, 0, 0, 0, 1]; - assert_eq!(trim_mkb(mkb.clone()), mkb); - } - // ── Content Certificate parsing ──────────────────────────────────────── - #[test] - fn parse_content_cert_rejects_short_buffer() { - // < 20 bytes → None (cc_id slice [14..20] would index OOB). - assert!(parse_content_cert(&[0x00; 19]).is_none()); - assert!(parse_content_cert(&[0x00; 20]).is_some()); - } - #[test] - fn parse_content_cert_extracts_cc_id_and_nonzero_type_is_v20() { - // Content-cert layout: [0]=type, [1] bit7=bus-enc, [14..20]=cc_id. Type - // 0x10 → V20. - let mut data = vec![0u8; 20]; - data[0] = 0x10; // AACS2 type marker → V20 - data[1] = 0x00; - data[14..20].copy_from_slice(&[0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF]); - let cc = parse_content_cert(&data).unwrap(); - assert_eq!(cc.version, AacsVersion::V20); - assert_eq!(cc.cc_id, [0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF]); - assert!(!cc.bus_encryption); - } - // Only the two observed cert types parse (0x00 AACS1/HD DVD, 0x10 AACS2): an - // unknown/corrupt type must be None so the bus key falls to keep-key. - #[test] - fn parse_content_cert_rejects_unknown_type_byte() { - let mut data = vec![0u8; 20]; - data[1] = 0x00; - for t in [0x01u8, 0x11, 0x55, 0xFF] { - data[0] = t; - assert!( - parse_content_cert(&data).is_none(), - "type {t:#04x} must not parse" - ); - } - } - #[test] - fn parse_content_cert_bus_encryption_reads_bit7() { - // bus_encryption = (data[1] >> 7) & 1. Low bits set with bit7 - // clear → false; bit7 set → true. Pins the bit, not a truthiness of the byte. - let mut data = vec![0u8; 20]; - data[1] = 0x7F; // bits 0..6 set, bit 7 clear - assert!(!parse_content_cert(&data).unwrap().bus_encryption); - data[1] = 0x80; // bit 7 set - assert!(parse_content_cert(&data).unwrap().bus_encryption); - } - // ── resolve: version → stride wiring + V21 upgrade on variant MKB ────── - #[test] - fn resolve_keys_v2_upgrades_to_v21_on_variant_mkb() { - // resolve_keys_v2 parses with the V20 64-byte stride but upgrades the - // result's version to V21 if the MKB carries a 0x82/0x83 variant - // record. Path 4 (hash→VUK) supplies the actual keys. - let uk_ro = build_unit_key_ro(1, 64); - let hash = disc_hash(&uk_ro); - let hash_hex = disc_hash_hex(&hash).to_lowercase(); - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "fixture".to_string(), - media_key: None, - disc_id: None, - vuk: Some([0x5Au8; 16]), - unit_keys: Vec::new(), - }), - }; - - // MKB with a 0x2f variant record makes is_variant_mkb true. - let mut mkb = vec![0x10, 0x00, 0x00, 0x08, 0, 0, 0, 1]; - mkb.extend_from_slice(&[0x2f, 0x00, 0x00, 0x14]); - mkb.extend_from_slice(&[0x55; 16]); - - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0u8; 16], - providers, - mkb: Some(&mkb), - }; - let resolved = resolve_keys_v2(&ctx).expect("path 4 resolves"); - assert_eq!( - resolved.version, - AacsVersion::V21, - "variant MKB must upgrade V20 result to V21" - ); - } - - #[test] - fn resolve_keys_v2_stays_v20_on_classical_mkb() { - // No variant records → version stays V20. - let uk_ro = build_unit_key_ro(1, 64); - let hash = disc_hash(&uk_ro); - let hash_hex = disc_hash_hex(&hash).to_lowercase(); - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "f".to_string(), - media_key: None, - disc_id: None, - vuk: Some([0x5Au8; 16]), - unit_keys: Vec::new(), - }), - }; - let mkb = vec![0x10, 0x00, 0x00, 0x08, 0, 0, 0, 1]; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0u8; 16], - providers, - mkb: Some(&mkb), - }; - assert_eq!(resolve_keys_v2(&ctx).unwrap().version, AacsVersion::V20); - } - - #[test] - fn resolve_keys_bus_encryption_flag_flows_from_content_cert() { - // The resolved.bus_encryption must reflect the content cert's bit0. - let uk_ro = build_unit_key_ro(1, 48); - let hash = disc_hash(&uk_ro); - let hash_hex = disc_hash_hex(&hash).to_lowercase(); - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "f".to_string(), - media_key: None, - disc_id: None, - vuk: Some([1u8; 16]), - unit_keys: Vec::new(), - }), - }; - // Content cert: AACS2 (type 0x10) + bus encryption enabled (bit 7 of byte 1). - let mut cc = vec![0u8; 20]; - cc[0] = 0x10; - cc[1] = 0x80; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: Some(&cc), - volume_id: &[0u8; 16], - providers, - mkb: None, - }; - assert!(resolve_keys_v1(&ctx).unwrap().bus_encryption); - } - - #[test] - fn resolve_keys_v21_path4_resolves_by_hash() { - // resolve_keys_v21 must hit path 4 (hash→VUK) and stamp version V21, - // deriving unit keys from the VUK. - use super::super::crypto::aes_ecb_encrypt as enc; - let data = build_unit_key_ro(1, 64); - // The single encrypted key in build_unit_key_ro is [0x10;16]. - let hash = disc_hash(&data); - let hash_hex = disc_hash_hex(&hash).to_lowercase(); - let vuk = [0x77u8; 16]; - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "f".to_string(), - media_key: None, - disc_id: None, - vuk: Some(vuk), - unit_keys: Vec::new(), - }), - }; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &data, - content_cert: None, - volume_id: &[0u8; 16], - providers, - mkb: None, - }; - let r = resolve_keys_v21(&ctx).expect("v21 path 4"); - assert_eq!(r.version, AacsVersion::V21); - assert_eq!(r.key_source, 4); - assert_eq!(r.vuk, Some(vuk)); - // Unit key derived: AES-D(vuk, enc_key). enc_key here is [0x10;16]. - assert_eq!(r.unit_keys[0].1, decrypt_unit_key(&vuk, &[0x10u8; 16])); - // Self-consistency: encrypting it back under VUK gives the stored block. - assert_eq!(enc(&vuk, &r.unit_keys[0].1), [0x10u8; 16]); - } - - #[test] - fn resolve_keys_path3_derives_vuk_from_vid_match() { - // Path 3: an entry whose disc_id == ctx.volume_id supplies an MK; - // resolver derives VUK = derive_vuk(mk, vid). No hash match needed. - let uk_ro = minimal_unit_key_ro(); - let vid = [0x42u8; 16]; - let mk = [0x24u8; 16]; - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: "0xnotthishash".to_string(), - title: "sibling".to_string(), - media_key: Some(mk), - disc_id: Some(vid), - vuk: None, - unit_keys: Vec::new(), - }), - }; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &vid, - providers, - mkb: None, // no MKB → paths 1/2/2.5 skipped, path 3 fires - }; - let r = resolve_keys_v1(&ctx).expect("path 3 by VID"); - assert_eq!(r.key_source, 3); - assert_eq!(r.vuk, Some(derive_vuk(&mk, &vid))); - } - - // Zero VID is the "never read" sentinel gating paths 1/3; treating it as real derives a - // WRONG-but-valid VUK (silent garbage), treating a real VID as absent skips a resolvable - // disc. - #[test] - fn resolve_keys_v21_treats_the_all_zero_volume_id_as_no_vid() { - let uk_ro = minimal_unit_key_ro(); - let vid = [0x42u8; 16]; - let mk = [0x24u8; 16]; - // A VID-keyed entry carrying an MK and nothing else: no VUK and no unit - // keys, so paths 4 and 5 cannot fire and ONLY the VID-gated path 3 can - // produce a result. - let entry = DiscEntry { - disc_hash: "not-this-disc".to_string(), - title: "sibling".to_string(), - media_key: Some(mk), - disc_id: Some(vid), - vuk: None, - unit_keys: Vec::new(), - }; - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(entry), - }; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - - // A real VID → path 3 fires and the VUK derives from Km + THIS VID. - let with_vid = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &vid, - providers, - mkb: None, - }; - let r = resolve_keys_v21(&with_vid).expect("a real VID must reach path 3"); - assert_eq!(r.key_source, 3); - assert_eq!( - r.vuk, - Some(derive_vuk(&mk, &vid)), - "VUK must derive from the Media Key and the disc's own VID" - ); - - // The all-zero sentinel → paths 1 and 3 are skipped entirely; with no - // VUK and no unit keys on the entry, nothing resolves. - let no_vid = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0u8; 16], - providers, - mkb: None, - }; - let got = resolve_keys_v21(&no_vid); - assert!( - got.is_none(), - "a zero VID must not be used to derive a VUK; got key_source {:?} vuk {:?}", - got.as_ref().map(|r| r.key_source), - got.as_ref().map(|r| r.vuk.is_some()) - ); - } - - #[test] - fn resolve_keys_returns_none_when_no_provider_has_anything() { - // Empty provider array + VID present + no MKB → all paths miss → None. - let uk_ro = minimal_unit_key_ro(); - let providers: &[&dyn super::super::provider::KeyProvider] = &[]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0x42u8; 16], - providers, - mkb: None, - }; - assert!(resolve_keys_v1(&ctx).is_none()); - } - - #[test] - fn match_keydb_unit_keys_empty_keydb_returns_none() { - // match_keydb_unit_keys with empty keydb keys → None (so path 5 can't - // fire on an entry with no unit keys). - let uk_file = parse_unit_key_ro(&minimal_unit_key_ro(), AacsVersion::V10).unwrap(); - assert!(match_keydb_unit_keys(&uk_file, &[]).is_none()); - } - - // ── derive_media_key_from_dk: revoked-marker stops the uv scan ───────── - - #[test] - fn derive_media_key_from_dk_breaks_on_revoked_marker() { - // A subset-difference entry whose u_mask_shift has bit 0x40/0x80 set - // is a revoke marker; the scan must `break` (not derive a key from it - // and not panic). Pair it with a DK that would otherwise be tempting. - let mut mkb: Vec = Vec::new(); - mkb.extend_from_slice(&[0x81, 0x00, 0x00, 0x14]); - mkb.extend_from_slice(&[0xAB; 16]); - // 0x04 with one entry, u_mask_shift = 0xC0 (both top bits → revoked). - mkb.extend_from_slice(&[0x04, 0x00, 0x00, 0x09]); - mkb.extend_from_slice(&[0xC0, 0x00, 0x00, 0x00, 0x01]); - mkb.extend_from_slice(&[0x05, 0x00, 0x00, 0x14]); - mkb.extend_from_slice(&[0xCD; 16]); - let dk = DeviceKey { - key: [0x11; 16], - node: 1, - uv: 1, - u_mask_shift: 0, - }; - // The 0xC0 entry is filtered by the num_uvs take_while, so the scan - // sees zero usable slots and returns None — never a wrong key. - assert!(derive_media_key_from_dk(&mkb, &[dk]).is_none()); - } - - #[test] - fn derive_media_key_from_dk_returns_none_when_records_missing() { - // No 0x04 / 0x05 records → the `?` short-circuits return None. - let mkb = vec![ - 0x81, 0x00, 0x00, 0x14, /* mk_dv */ 0u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, - ]; - assert!(derive_media_key_from_dk(&mkb, &[]).is_none()); - } - - #[test] - fn find_record_body_returns_none_for_empty_body_record() { - // find_record_body requires rec_len > 4 (non-empty body). A 4-byte - // record (header only, empty body) is treated as absent. - let mkb = [0x05, 0x00, 0x00, 0x04]; // type 0x05, no body - assert!(probe::mkb_record_body(&mkb, 0x05).is_none()); - } - - #[test] - fn derive_media_key_and_pk_from_dk_returns_intermediate_pk() { - // Regression: a classical DK boil must yield the intermediate PK too - // (key service banks DK·PK·MK·VUK·UK; before `_and_pk_` it lost the PK). - // Builds a minimal MKB with a DK whose calc_pk_from_dk walk is a no-op. - use super::super::crypto::aes_ecb_encrypt as enc; - - let dk_bytes: [u8; 16] = [ - 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, - 0xFF, 0x00, - ]; - // Expected Processing Key for the no-op walk. - let expected_pk = aesg3(&dk_bytes, 1); - - // Plant a known Media Key. - let mk: [u8; 16] = [ - 0xA0, 0xA1, 0xA2, 0xA3, 0xA4, 0xA5, 0xA6, 0xA7, 0xA8, 0xA9, 0xAA, 0xAB, 0xAC, 0xAD, - 0xAE, 0xAF, - ]; - // uv (big-endian) = 0x00000002; validate XORs uv into mk[12..16]. - let uv_bytes: [u8; 4] = [0x00, 0x00, 0x00, 0x02]; - - // cvalue C = AES-E(Kp, mk_raw) where mk_raw = mk with the uv XOR - // pre-undone, so the validate step XORs uv back in and recovers mk. - let mut mk_raw = mk; - for a in 0..4 { - mk_raw[12 + a] ^= uv_bytes[a]; - } - let cvalue = enc(&expected_pk, &mk_raw); - - // mk_dv = AES-E(mk, magic || pad); validate decrypts it under mk and - // checks the leading 8 bytes against the verify magic. - let mut plaintext_vd = [0u8; 16]; - plaintext_vd[..8].copy_from_slice(&[0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF]); - plaintext_vd[8..].copy_from_slice(&[0x11; 8]); - let mk_dv = enc(&mk, &plaintext_vd); - - // Assemble the MKB. Type/Version (0x10) header first. - let mut mkb = vec![ - 0x10, 0x00, 0x00, 0x0C, 0x48, 0x14, 0x10, 0x03, 0x00, 0x00, 0x00, 0x4D, - ]; - // 0x04 Subset-Difference: body = u_mask_shift(0x03) || uv(4 bytes). - mkb.extend_from_slice(&[0x04, 0x00, 0x00, 0x09]); - mkb.extend_from_slice(&[0x03]); - mkb.extend_from_slice(&uv_bytes); - // 0x05 cvalues: one 16-byte cvalue (mkb_find_cvalues prefers 0x05). - mkb.extend_from_slice(&[0x05, 0x00, 0x00, 0x14]); - mkb.extend_from_slice(&cvalue); - // 0x86 Verify Media Key: mk_dv. - mkb.extend_from_slice(&[0x86, 0x00, 0x00, 0x14]); - mkb.extend_from_slice(&mk_dv); - - let dk = DeviceKey { - key: dk_bytes, - node: 4, - uv: 2, - u_mask_shift: 3, - }; - - // The new `_and_pk_` form returns BOTH the MK and the intermediate PK. - let dks = [dk]; - let (got_mk, got_pk) = derive_media_key_and_pk_from_dk(&mkb, &dks) - .expect("classical DK boil must derive (mk, pk)"); - assert_eq!(got_mk, mk, "recovered Media Key must match the planted MK"); - assert_eq!( - got_pk, expected_pk, - "returned Processing Key must equal aesg3(dk, 1) for the no-op walk" - ); - - // And the thin wrapper must still return just the MK. - assert_eq!(derive_media_key_from_dk(&mkb, &dks), Some(mk)); - } - - // ── resolve_keys_with_reason / classify_resolve_failure ──────────────── - // Also exercised end-to-end via the decrypt gate (`keys::disc_gate`); these - // pin the *classifier* directly, covering branches that gate test doesn't. - - /// A `SuppliedKey` provider with the given derivation material and no - /// disc-keyed entry. Mirrors the construction the gate test uses, lifted to - /// a helper so each branch reads as one line. - fn material_provider( - device_keys: Vec, - processing_keys: Vec<[u8; 16]>, - ) -> super::super::provider::SuppliedKey { - super::super::provider::SuppliedKey { - device_keys, - processing_keys, - media_keys: Vec::new(), - disc_entry: None, - } - } - - fn one_device_key() -> DeviceKey { - DeviceKey { - key: [0x11; 16], - node: 1, - uv: 1, - u_mask_shift: 0, - } - } - - // Zero VID + PROCESSING keys (not device keys) is still "material - // present, VID missing" -> VidUnavailable; pins the processing-keys arm - // of `has_derivation_material`'s `||` (the gate test only covers device keys). - #[test] - fn classify_processing_keys_zero_vid_is_vid_unavailable() { - let prov = material_provider(Vec::new(), vec![[0u8; 16]]); - let providers: &[&dyn super::super::provider::KeyProvider] = &[&prov]; - let uk_ro = minimal_unit_key_ro(); - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0u8; 16], - providers, - mkb: None, - }; - assert_eq!( - resolve_keys_with_reason(&ctx, 2).err(), - Some(ResolveFailure::VidUnavailable), - "processing keys + zero VID is still material-but-no-VID" - ); - } - - // A non-zero VID present but resolution still fails must classify NoMaterial, never - // VidUnavailable, regardless of how much derivation material is on hand. - #[test] - fn classify_vid_present_with_material_is_no_material_not_vid() { - let prov = material_provider(vec![one_device_key()], vec![[0u8; 16]]); - let providers: &[&dyn super::super::provider::KeyProvider] = &[&prov]; - let uk_ro = minimal_unit_key_ro(); - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0x42u8; 16], // VID IS available - providers, - mkb: None, - }; - assert_eq!( - resolve_keys_with_reason(&ctx, 2).err(), - Some(ResolveFailure::NoMaterial), - "VID present must never be reported as VidUnavailable, however much \ - derivation material is on hand" - ); - } - - /// VID present + NO material → NoMaterial (both conditions for - /// VidUnavailable absent). Distinct from the gate's zero-VID/no-material - /// branch. - #[test] - fn classify_vid_present_no_material_is_no_material() { - let prov = material_provider(Vec::new(), Vec::new()); - let providers: &[&dyn super::super::provider::KeyProvider] = &[&prov]; - let uk_ro = minimal_unit_key_ro(); - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0x42u8; 16], - providers, - mkb: None, - }; - assert_eq!( - resolve_keys_with_reason(&ctx, 2).err(), - Some(ResolveFailure::NoMaterial) - ); - } - - // version_u8 == 1 routes through V10, anything else through V20->V21; - // resolve the SAME path-4 fixture under both to prove the dispatch, and - // that success comes back Ok, never Err(ResolveFailure). - #[test] - fn resolve_with_reason_dispatches_on_version_and_returns_ok() { - let uk_ro = build_unit_key_ro(1, 64); - let hash_hex = disc_hash_hex(&disc_hash(&uk_ro)).to_lowercase(); - let vuk = [0x77u8; 16]; - let keydb = SuppliedKey { - device_keys: Vec::new(), - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: Some(DiscEntry { - disc_hash: hash_hex, - title: "f".to_string(), - media_key: None, - disc_id: None, - vuk: Some(vuk), - unit_keys: Vec::new(), - }), - }; - let providers: &[&dyn super::super::provider::KeyProvider] = &[&keydb]; - let ctx = ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0u8; 16], - providers, - mkb: None, - }; - - // version 1 → V10 resolver. Path 4 succeeds → Ok, version stamped V10. - let v1 = resolve_keys_with_reason(&ctx, 1).expect("v1 dispatch must resolve path 4"); - assert_eq!(v1.vuk, Some(vuk)); - assert_eq!(v1.version, AacsVersion::V10); - - // version 2 → V20→V21 chain. Same fixture resolves; not the V10 stamp. - let v2 = resolve_keys_with_reason(&ctx, 2).expect("non-1 dispatch must resolve path 4"); - assert_eq!(v2.vuk, Some(vuk)); - assert_ne!( - v2.version, - AacsVersion::V10, - "the non-1 arm must run the V20/V21 resolver, not V10" - ); - } -} diff --git a/src/aacs/trace.rs b/src/aacs/trace.rs index 0ba8901e..40fd9674 100644 --- a/src/aacs/trace.rs +++ b/src/aacs/trace.rs @@ -144,6 +144,29 @@ pub enum KeyOutcome { NoKey, } +/// Where an acquired key came from, for the ring's trace and the engine's key status. +/// Variants are ordered root-of-trust → per-disc leaf. +#[derive(Debug, Clone, Copy, PartialEq)] +pub enum KeyOrigin { + /// MKB + device keys → subset-difference tree → VUK + DeviceKey, + /// MKB + processing keys → media key → VUK + ProcessingKey, + /// Media key + Volume ID from KEYDB → derived VUK + KeyDbDerived, + /// VUK found directly in KEYDB by disc hash + KeyDb, + /// Pre-decrypted unit keys taken directly from KEYDB by disc hash. + /// No VUK present in the entry — `AacsState::vuk` is `None`. + KeyDbUnitKeys, + /// Unit key supplied directly by the caller (the external Unit Key path). + /// No keydb, no derivation — `AacsState::vuk` is `None`. + ExternalUk, +} + +// No `KeyOrigin::name()`: the library holds ZERO user-facing English. Callers +// map variants to display text (see freemkv's `disc_info::key_origin_label`). + #[cfg(test)] mod tests { use super::*; diff --git a/src/aacs/variant.rs b/src/aacs/variant.rs index f0433659..3808e8b1 100644 --- a/src/aacs/variant.rs +++ b/src/aacs/variant.rs @@ -119,36 +119,6 @@ fn records_find_mk_dv(records: &[MkbRecord]) -> Option<[u8; 16]> { Some(out) } -/// Try each device key in turn (walk -> Kp -> Km) and return the first Media Key that -/// verifies. On a variant MKB the walk cannot verify a Kp, so a covering-but-wrong key -/// must not stop the search. Errs with the first correction/structural error, else the last -/// plain miss, or `ProcessingKeyUnavailable`. -pub(crate) fn derive_media_key_from_device_keys( - records: &[MkbRecord], - device_keys: &[DeviceKey], -) -> Result<[u8; 16], MediaKeyVariantError> { - let mut last = MediaKeyVariantError::ProcessingKeyUnavailable; - let mut notable: Option = None; - for dk in device_keys { - let Some(pkm) = walk_processing_key(records, std::slice::from_ref(dk)) else { - continue; - }; - match derive_media_key_variant(records, &pkm.kp) { - Ok(km) => return Ok(km), - Err( - e @ (MediaKeyVariantError::SoftCorrectionRequired - | MediaKeyVariantError::OnlineChallengeRequired - | MediaKeyVariantError::MkbIncomplete - | MediaKeyVariantError::VariantsTableUnavailable), - ) => { - notable.get_or_insert(e); - } - Err(e) => last = e, - } - } - Err(notable.unwrap_or(last)) -} - /// Walk an MKB and return the first `(Kp, uv, cvalue)` that /// `device_keys` covers. Returns `None` if no DK walks any uv. /// @@ -1694,7 +1664,6 @@ mod tests { /// A two-slot variant MKB whose SECOND slot is opened by a device key. struct PlantedWalk { - mkb: Vec, records: Vec, /// The device key that covers slot 1 with zero descent. dk: DeviceKey, @@ -1802,7 +1771,6 @@ mod tests { PlantedWalk { records: walk_mkb(&mkb), - mkb, dk: DeviceKey { key: dkey, node: NODE, @@ -1861,62 +1829,6 @@ mod tests { ); } - /// A covering-but-wrong device key ahead of the right one must not hide the Media Key. - #[test] - fn device_key_search_continues_past_a_covering_wrong_key() { - let p = plant_walk_variant_mkb(); - let mut wrong = p.dk.clone(); - wrong.key[0] ^= 0xFF; - assert!(walk_processing_key(&p.records, std::slice::from_ref(&wrong)).is_some()); - assert_eq!( - derive_media_key_from_device_keys(&p.records, &[wrong.clone(), p.dk.clone()]), - Ok(p.km) - ); - assert!(derive_media_key_from_device_keys(&p.records, &[wrong]).is_err()); - assert_eq!( - derive_media_key_from_device_keys(&p.records, &[]), - Err(MediaKeyVariantError::ProcessingKeyUnavailable) - ); - } - - /// The resolver must keep searching past a covering-but-wrong key that sorts first. - #[test] - fn resolve_keys_v21_finds_the_right_key_behind_a_covering_wrong_one() { - use crate::aacs::provider::{KeyProvider, SuppliedKey}; - use crate::aacs::resolve::{ResolveContext, resolve_keys_v21}; - - let p = plant_walk_variant_mkb(); - let mut wrong = p.dk.clone(); - wrong.key[0] = 0x00; - // `Providers::device_keys` sorts by key bytes: the wrong key must come first. - assert!(wrong.key < p.dk.key, "fixture: wrong key sorts first"); - assert!(walk_processing_key(&p.records, std::slice::from_ref(&wrong)).is_some()); - - let mut uk_ro = vec![0u8; 256]; - uk_ro[3] = 0x60; - uk_ro[16] = 1; - uk_ro[17] = 1; - uk_ro[0x60 + 1] = 1; - let keys = SuppliedKey { - device_keys: vec![p.dk.clone(), wrong], - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: None, - }; - let providers: &[&dyn KeyProvider] = &[&keys]; - let vid = [0x22u8; 16]; - let r = resolve_keys_v21(&ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &vid, - providers, - mkb: Some(&p.mkb), - }) - .expect("the right key behind a covering wrong one must resolve"); - assert_eq!(r.key_source, 1); - assert_eq!(r.vuk, Some(crate::aacs::derive::derive_vuk(&p.km, &vid))); - } - /// The `[C]` §3.2.4 subset-difference gate must reject a device key on /// EVERY one of its four coordinates (node/uv/u_mask_shift/key). #[test] diff --git a/src/ctx.rs b/src/ctx.rs new file mode 100644 index 00000000..646823ca --- /dev/null +++ b/src/ctx.rs @@ -0,0 +1,162 @@ +//! The one run context (pipeline design §2.5): every stage of a run reads its stop token, +//! reports through its events and counts loss into its stats. + +use std::sync::Arc; +use std::sync::atomic::{AtomicU64, Ordering}; + +use crate::event::{Event, Events, NoEvents}; +use crate::halt::Halt; + +/// What every stage of one run shares: the stop token, the event sink, the loss counters +/// and the diagnostic switches. Clones share all of them, so a clone observes the same run. +#[derive(Clone)] +pub struct Ctx { + /// The run's one stop token. A stage that observes it returns [`Error::Halted`](crate::Error::Halted). + pub halt: Halt, + /// Where every stage reports progress. + pub events: Arc, + /// Run-wide loss counters, cumulative over every stage built with this context. + pub stats: Arc, + /// Developer diagnostics, fixed for the run. + pub diag: Diag, +} + +impl Ctx { + /// A context over `halt` with no events, fresh stats and diagnostics off. + pub fn new(halt: Halt) -> Self { + Self { + halt, + events: Arc::new(NoEvents), + stats: Arc::default(), + diag: Diag::default(), + } + } + + /// This context reporting to `events`. + pub fn with_events(mut self, events: Arc) -> Self { + self.events = events; + self + } + + /// This context with diagnostics `diag`. + pub fn with_diag(mut self, diag: Diag) -> Self { + self.diag = diag; + self + } + + pub(crate) fn emit(&self, e: Event<'_>) { + self.events.event(&e); + } +} + +impl Default for Ctx { + fn default() -> Self { + Self::new(Halt::new()) + } +} + +impl std::fmt::Debug for Ctx { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("Ctx") + .field("halt", &self.halt) + .field("stats", &self.stats) + .field("diag", &self.diag) + .finish_non_exhaustive() + } +} + +/// Loss counters every source reports through, so a count means the same thing on every +/// path. Atomic: the read producer and the mux pump update them while the run goes. +#[derive(Debug, Default)] +pub struct Stats { + read_skips: AtomicU64, + bytes_lost: AtomicU64, + units_blanked: AtomicU64, + resync_dropped: AtomicU64, +} + +impl Stats { + /// The counters now. + pub fn snapshot(&self) -> LossReport { + LossReport { + read_skips: self.read_skips.load(Ordering::Relaxed), + bytes_lost: self.bytes_lost.load(Ordering::Relaxed), + units_blanked: self.units_blanked.load(Ordering::Relaxed), + resync_dropped: self.resync_dropped.load(Ordering::Relaxed), + } + } + + // One read error zero-filled `bytes` of output. + pub(crate) fn add_skip(&self, bytes: u64) { + self.read_skips.fetch_add(1, Ordering::Relaxed); + self.bytes_lost.fetch_add(bytes, Ordering::Relaxed); + } + + pub(crate) fn add_blanked(&self, units: u64) { + self.units_blanked.fetch_add(units, Ordering::Relaxed); + } + + pub(crate) fn add_resync_dropped(&self, frames: u64) { + self.resync_dropped.fetch_add(frames, Ordering::Relaxed); + } +} + +/// A snapshot of [`Stats`]. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct LossReport { + /// Zero-filled read-error skips: one per skipped read (live mux) or lossy batch + /// (extract); the byte count is `bytes_lost`. + pub read_skips: u64, + /// Bytes those skips zero-filled (blanked AACS units are not included). + pub bytes_lost: u64, + /// Damaged AACS units blanked (each one whole 6144-byte unit of zeros). + pub units_blanked: u64, + /// Video frames dropped after a gap until the next keyframe. + pub resync_dropped: u64, +} + +/// Developer diagnostics a run carries instead of stages reading the environment. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct Diag { + /// Bypass the codec parsers (raw demux throughput profiling). + pub skip_parse: bool, + /// Log per-stage read/feed/parse timings. + pub profile: bool, +} + +impl Diag { + /// `FREEMKV_SKIP_PARSE` and `FREEMKV_PROFILE` (set = on), read once by the caller. + pub fn from_env() -> Self { + Self { + skip_parse: std::env::var_os("FREEMKV_SKIP_PARSE").is_some(), + profile: std::env::var_os("FREEMKV_PROFILE").is_some(), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + // Clones share the token, the events and the counters: a stage holding a clone + // observes the caller's Stop and reports into the caller's totals. + #[test] + fn a_clone_shares_halt_and_stats() { + let ctx = Ctx::default(); + let stage = ctx.clone(); + ctx.halt.cancel(); + stage.stats.add_skip(2048); + stage.stats.add_blanked(2); + stage.stats.add_resync_dropped(3); + assert!(stage.halt.is_cancelled()); + assert_eq!( + ctx.stats.snapshot(), + LossReport { + read_skips: 1, + bytes_lost: 2048, + units_blanked: 2, + resync_dropped: 3, + } + ); + } +} diff --git a/src/decrypt.rs b/src/decrypt.rs index e1ee6815..b4fcd020 100644 --- a/src/decrypt.rs +++ b/src/decrypt.rs @@ -144,7 +144,7 @@ pub enum DecryptKeys { /// Stream, HD-DVD `.evo` = Program Stream); it travels with the keys because /// both are resolved once per disc, and the key SELECTOR (`is_clean`) needs it /// to prove a key structurally against the right container. Only a - /// [`ResolvedKeySet`](crate::keys::ResolvedKeySet) builds one (KU §2.2). + /// [`KeyRing`](crate::keys::KeyRing) builds one (KU §2.2). #[non_exhaustive] Aacs { unit_keys: Vec<(u32, [u8; 16])>, diff --git a/src/dirimage/tests.rs b/src/dirimage/tests.rs index 87c2c4c8..d8e93a8c 100644 --- a/src/dirimage/tests.rs +++ b/src/dirimage/tests.rs @@ -13,7 +13,7 @@ use crate::udf; use std::io::Write; /// A scratch directory that removes itself. -struct Scratch(PathBuf); +pub(crate) struct Scratch(PathBuf); impl Scratch { fn new(tag: &str) -> Self { @@ -34,7 +34,7 @@ impl Scratch { )); p } - fn path(&self) -> &Path { + pub(crate) fn path(&self) -> &Path { &self.0 } fn file(&self, rel: &str, bytes: &[u8]) { @@ -623,7 +623,7 @@ pub(crate) fn minimal_clpi(source_packets: u32) -> Vec { // A BD folder that enumerates a title, so the AACS content probe has an extent to sample; // `scrambled` sets the CPI bits and withholds TS sync. -fn playable_bdmv(tag: &str, scrambled: bool) -> Scratch { +pub(crate) fn playable_bdmv(tag: &str, scrambled: bool) -> Scratch { let s = Scratch::new(tag); let packets = 4096u32; let mut m2ts = vec![0x5Au8; packets as usize * 192]; @@ -675,19 +675,18 @@ fn an_aacs_directory_over_clear_content_is_treated_as_decrypted() { assert!(disc.aacs_error.is_none(), "and no key is demanded"); } -/// The other verdict: a folder whose content units really are flagged and -/// scrambled is a raw encrypted copy, which `dir://` does not support. It must -/// be a typed error, not a rip that emits garbage. +/// The other verdict: a folder whose content units really are flagged and scrambled is +/// a raw encrypted copy. It keeps its AACS verdict, so it is keyed and decrypted like an +/// image of the same disc (every input passes the one decryption stage). #[test] -fn an_aacs_folder_with_scrambled_content_is_rejected() { +fn an_aacs_folder_with_scrambled_content_scans_encrypted() { let s = playable_bdmv("aacsenc", true); s.file("AACS/Unit_Key_RO.inf", &[0u8; 64]); s.file("AACS/MKB_RO.inf", &[0u8; 64]); - let err = match crate::session::scan_dir(s.path(), crate::disc::ScanOptions::default()) { - Ok(_) => panic!("a scrambled folder must not scan clean"), - Err(e) => e, - }; - assert_eq!(err.code(), crate::error::E_DIR_IMAGE_ENCRYPTED); + let (disc, _reader) = + crate::session::scan_dir(s.path(), crate::disc::ScanOptions::default()).unwrap(); + assert!(disc.encrypted, "scrambled content keeps the AACS verdict"); + assert!(disc.aacs.is_some()); } // ── The OTHER door: `dir://` through the PES input path ───────────────────── @@ -722,8 +721,12 @@ fn both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory() { // Door 2 — the PES input path the CLI actually rips through. let url = format!("dir://{}", s.path().display()); - let stream = crate::input(&url, &crate::InputOptions::default()) - .expect("dir:// input must open a clear folder, exactly as scan_dir does"); + let stream = crate::input( + &url, + &crate::InputOptions::default(), + &crate::ctx::Ctx::default(), + ) + .expect("dir:// input must open a clear folder, exactly as scan_dir does"); assert_eq!( stream.info().extents, scanned_extents, @@ -732,23 +735,50 @@ fn both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory() { drop(stream); } -// The other verdict, same door: a truly scrambled folder is refused with the TYPED code, not -// muxed into garbage. Load-bearing half. +// HP3: the `Url` arm reports read progress like every other arm (it used to pass no events). #[test] -fn a_scrambled_folder_is_refused_through_the_dir_url_door_too() { +fn a_dir_url_mux_reports_read_progress_to_the_ctx() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let s = playable_bdmv("direvents", false); + let read = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0)); + let seen = read.clone(); + let ctx = + crate::Ctx::default().with_events(std::sync::Arc::new(move |e: &crate::Event<'_>| { + if let crate::Event::BytesRead { bytes, .. } = *e { + seen.store(bytes, std::sync::atomic::Ordering::Relaxed); + } + })); + let url = format!("dir://{}", s.path().display()); + // The fixture's TS carries no PES, so the drain refuses as NoStreams after reading it all. + let _ = crate::mux_url(&url, None, "null://", &crate::MuxOptions::default(), &ctx); + assert_eq!( + read.load(std::sync::atomic::Ordering::Relaxed), + 4096 * 192, + "every byte of the title's extent was reported read" + ); +} + +// The other verdict, same door: a truly scrambled folder with no key is refused before any +// output with the standard no-key code (E7022), never muxed as ciphertext. +#[test] +fn a_scrambled_folder_with_no_key_is_refused_through_the_dir_url_door() { let s = playable_bdmv("dirdoorenc", true); s.file("AACS/Unit_Key_RO.inf", &[0u8; 64]); s.file("AACS/MKB_RO.inf", &[0u8; 64]); let url = format!("dir://{}", s.path().display()); - let err = match crate::input(&url, &crate::InputOptions::default()) { + let err = match crate::input( + &url, + &crate::InputOptions::default(), + &crate::ctx::Ctx::default(), + ) { Ok(_) => panic!("a scrambled folder must not open as a PES source"), Err(e) => e, }; assert_eq!( crate::error::error_code(&err), - Some(crate::error::E_DIR_IMAGE_ENCRYPTED), - "expected the typed dir-source-encrypted code, got: {err}" + Some(crate::error::E_NO_DISC_KEY), + "expected the no-key code, got: {err}" ); } diff --git a/src/disc/encrypt.rs b/src/disc/encrypt.rs index cfd542fd..1e897d21 100644 --- a/src/disc/encrypt.rs +++ b/src/disc/encrypt.rs @@ -427,13 +427,13 @@ fn cert_unlock_outcome(e: CertUnlockFailure) -> crate::aacs::trace::UnlockOutcom } // The scan's one bus step (§2.3), shared by AACS and CSS: the op token is checked before -// and after, ahead of any result mapping; the Drive's `Progress` is busy throughout, as +// and after, ahead of any result mapping; the Drive's `Liveness` is busy throughout, as // it spans the first keydb parse in `host_certs` (ST4-2). pub(super) fn bus_step_guard( session: &mut crate::drive::Drive, step: impl FnOnce(&mut crate::drive::Drive) -> T, ) -> Result { - let _busy = session.progress().map(crate::halt::Progress::busy); + let _busy = session.progress().map(crate::halt::Liveness::busy); session.check_token()?; let r = step(session); session.check_token()?; diff --git a/src/disc/extract.rs b/src/disc/extract.rs index f064dd57..fbd6a9c8 100644 --- a/src/disc/extract.rs +++ b/src/disc/extract.rs @@ -10,9 +10,14 @@ use super::Disc; use crate::decrypt::DecryptKeys; use crate::error::{Error, Result}; +use crate::io::tree_sink::TreeSink; +#[cfg(test)] +use crate::io::tree_sink::{ + available_space, dir_is_case_insensitive, probe_case_insensitive, sanitize_component, + unique_probe_token, +}; use crate::sector::{DecryptingSectorSource, SectorSource}; use crate::udf::{self, DirEntry, UdfFs}; -use std::io::Write; use std::path::{Path, PathBuf}; use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64}; @@ -31,8 +36,6 @@ const MAX_FAILED_UNIT_READS: u32 = 4; const ECC_BLOCK_SECTORS: u64 = 32; /// Sectors per read while cracking a VTS title key. const CRACK_BATCH_SECTORS: u16 = 64; -/// Attempts to delete the case-probe marker before giving up. -const PROBE_REMOVE_ATTEMPTS: u32 = 3; /// Options for [`Disc::extract_tree`]. #[derive(Default)] @@ -40,28 +43,11 @@ pub struct ExtractOptions<'a> { /// Overwrite into a non-empty destination directory. Without it a /// non-empty target is refused (mixing two discs' trees). pub force: bool, - /// Optional progress sink. `report` returning `false` requests an early - /// stop (the run finalizes whatever completed; in-flight files stay - /// `.partial`). - pub progress: Option<&'a dyn crate::progress::Progress>, - /// Cooperative cancel token. When cancelled (e.g. the CLI bridges its - /// SIGINT flag here), the run stops at the next file / batch boundary and - /// the in-flight file is left as `.partial` — never a half-written file - /// that looks complete. `None` disables cancellation. - pub halt: Option, /// The rip's up-front key set (KU §3.1), scope `WholeDisc`. For an AACS disc every file /// is read through the set's reader: proven files by its map, the rest proven on /// arrival, and a readable unit no held key opens stops the run (E7032). `None` /// decrypts no AACS: the caller's `check_decryptable` gate refuses an AACS disc first. - pub keys: Option<&'a crate::keys::ResolvedKeySet>, -} - -impl ExtractOptions<'_> { - /// Whether the caller asked to stop (halt cancelled or a progress sink - /// returned `false` on its last report). - fn cancelled(&self, progress_continue: bool) -> bool { - !progress_continue || self.halt.as_ref().is_some_and(|h| h.is_cancelled()) - } + pub keys: Option<&'a crate::keys::KeyRing>, } /// Per-file extraction outcome. @@ -129,22 +115,32 @@ impl Disc { /// STRAIGHT IN (no auto-named subfolder). The caller must have run the /// pre-flight decrypt gate ([`check_decryptable`](crate::keys::check_decryptable)). /// - /// Bad sectors become zero-filled holes (the run does not abort); files - /// are written `.partial` and renamed on success. + /// Bad sectors become zero-filled holes; files are written `.partial` and renamed + /// on success. Progress is `PassKind::Extract` events, loss goes to `ctx.stats`; a Stop + /// ends the run at a file or batch boundary, the in-flight file left `.partial`. pub fn extract_tree( &self, reader: &mut dyn SectorSource, dest: &Path, opts: &ExtractOptions, + ctx: &crate::ctx::Ctx, ) -> Result { - // ── Output dir policy (pre-flight, before any read) ────────────── - std::fs::create_dir_all(dest).map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; - if !opts.force && dir_is_non_empty(dest) { - return Err(Error::DirNotEmpty); - } + // Output dir policy (pre-flight, before any read). + let mut sink = TreeSink::create(dest, opts.force)?; + self.extract_into(reader, &mut sink, opts.keys, ctx) + } + /// The `dir://` chain: this disc's file tree read off `reader` (bad sectors zero-filled + /// and counted per file), decrypted through `keys` (AACS) or per VTS (CSS), into + /// `sink` ([`crate::io::open_tree_sink`]). The caller must have run the pre-flight + /// decrypt gate ([`check_decryptable`](crate::keys::check_decryptable)). + pub fn extract_into( + &self, + reader: &mut dyn SectorSource, + sink: &mut TreeSink, + keys: Option<&crate::keys::KeyRing>, + ctx: &crate::ctx::Ctx, + ) -> Result { // ── Phase 1: read the FS structure + all file extents (raw) ────── let fs = udf::read_filesystem(reader)?; let unmapped: Vec = reader @@ -154,11 +150,9 @@ impl Disc { .collect(); let mut planned: Vec = Vec::new(); let mut dirs: Vec = Vec::new(); - let mut seen_hosts: std::collections::HashMap = - std::collections::HashMap::new(); // The collision fold depends on the REAL target volume: fold case only // where the host would (APFS/NTFS), never on a case-sensitive volume. - let case_insensitive = dir_is_case_insensitive(dest); + sink.probe_case(); plan_tree( reader, &fs, @@ -166,11 +160,10 @@ impl Disc { Path::new(""), "", true, - case_insensitive, + sink, &unmapped, &mut planned, &mut dirs, - &mut seen_hosts, )?; // Free-space pre-check: refuse up front if the tree won't fit, before @@ -180,28 +173,16 @@ impl Disc { .iter() .map(|p| p.size) .fold(0u64, |a, b| a.saturating_add(b)); - if let Some(available) = available_space(dest) - && available < required - { - return Err(Error::DirInsufficientSpace { - required, - available, - }); - } + sink.reserve(required)?; // Create directories up-front so a leaf write never races a missing - // parent. The root itself already exists (create_dir_all above). - for d in &dirs { - let abs = dest.join(d); - std::fs::create_dir_all(&abs).map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; - } + // parent. The root itself already exists. + sink.make_dirs(&dirs)?; // Per-VTS CSS key map (DVD only): "VTS_xx" -> DecryptKeys. Built lazily // when a scrambled VOB group needs it. An AACS disc reads through the rip's set. let mut base_keys = self.decrypt_keys(); - let keyed = opts.keys.filter(|s| s.is_aacs()); + let keyed = keys.filter(|s| s.is_aacs()); if let Some(set) = keyed { crate::keys::check_decryptable( self, @@ -221,6 +202,7 @@ impl Disc { } None => DecryptingSectorSource::new(Borrowed(reader), base_keys.clone()), }; + dec.observe(ctx); let mut result = ExtractResult::default(); let total_bytes = required; @@ -229,13 +211,16 @@ impl Disc { // can report the good/unreadable split instead of pinning unreadable at 0. let mut done_unreadable: u64 = 0; - // CSS per-VTS key cache; only consulted for CSS discs. - let is_css = matches!(base_keys, DecryptKeys::Css { .. }); + // CSS per-VTS key cache, consulted for every DVD-Video layout: a scrambled VTS is + // found from its content and cracked, a clear one passes. A live drive scan never + // records a disc-wide CSS key (BUG-1), so the scan's verdict cannot gate this. + let is_css = matches!(base_keys, DecryptKeys::Css { .. }) + || (self.format == crate::disc::DiscFormat::Dvd && self.aacs.is_none()); let mut vts_keys: std::collections::HashMap = std::collections::HashMap::new(); for pf in &planned { - if opts.cancelled(true) { + if ctx.halt.is_cancelled() { result.halted = true; break; } @@ -243,7 +228,7 @@ impl Disc { // cannot be located to de-bus, so it is lost whole rather than written still encrypted. if pf.unmapped { let (fr, halted) = - unmapped_file(pf, total_bytes, &mut done_bytes, &mut done_unreadable, opts); + unmapped_file(pf, total_bytes, &mut done_bytes, &mut done_unreadable, ctx); result.bytes_unreadable = result.bytes_unreadable.saturating_add(fr.bytes_unreadable); result.files.push(fr); @@ -261,7 +246,7 @@ impl Disc { let key = match vts_keys.get(&vts) { Some(k) => k.clone(), None => { - let halt = opts.halt.as_ref(); + let halt = Some(&ctx.halt); let k = match self .resolve_vts_key(&vts, &planned, &mut dec, &base_keys, halt) { @@ -285,12 +270,12 @@ impl Disc { // zero-fills it into bytes_unreadable, the same bucket as media damage. let (fr, halted) = extract_one_file( &mut dec, - dest, + sink, pf, total_bytes, &mut done_bytes, &mut done_unreadable, - opts, + ctx, )?; result.bytes_good = result.bytes_good.saturating_add(fr.bytes_good); @@ -398,69 +383,6 @@ impl SectorSource for Borrowed<'_> { } } -/// Whether `dir` lives on a case-INSENSITIVE filesystem (macOS APFS/HFS+ and -/// Windows NTFS default). Probes the real target: create a lowercase marker -/// and test whether its uppercase spelling resolves to the same file. On a -/// case-SENSITIVE volume (typical Linux ext4, or a case-sensitive APFS) two -/// disc names differing only by case are DISTINCT host files that coexist, so -/// folding them together would wrongly abort a legitimate extract. If the probe -/// can't run (e.g. a read-only dir), assume case-insensitive — the conservative -/// choice that never MISSES a real overwrite collision. -fn dir_is_case_insensitive(dir: &Path) -> bool { - probe_case_insensitive(dir, |p| std::fs::remove_file(p)) -} - -// `dir_is_case_insensitive` with the marker removal injectable (tests fail it). -fn probe_case_insensitive( - dir: &Path, - mut remove: impl FnMut(&Path) -> std::io::Result<()>, -) -> bool { - // Unique per attempt: a FIXED name let concurrent extracts race — one's - // `remove_file` could delete another's marker between create and `exists()` - // (a TOCTOU flip). A per-call token keeps each probe's pair on private paths. - let token = unique_probe_token(); - let lower_name = format!(".fmkv_case_probe_{token}"); - let lower = dir.join(&lower_name); - if std::fs::File::create(&lower).is_err() { - return true; - } - // The upper spelling is the SAME name uppercased end-to-end (hex token - // letters `a-f` fold to `A-F`), so on a case-insensitive volume it resolves - // to the file just created and on a case-sensitive one it does not exist. - let upper = dir.join(lower_name.to_ascii_uppercase()); - let insensitive = upper.exists(); - // Retry: a transient delete failure (Windows AV/indexer handle) would leave the - // marker in the user's target, failing a re-run's non-empty check. - for attempt in 0..PROBE_REMOVE_ATTEMPTS { - match remove(&lower) { - Ok(()) => break, - Err(e) if e.kind() == std::io::ErrorKind::NotFound => break, - Err(_) if attempt + 1 < PROBE_REMOVE_ATTEMPTS => { - std::thread::sleep(std::time::Duration::from_millis(20)); - } - // Still failing: best-effort by design; the probe's answer stands. - Err(_) => {} - } - } - insensitive -} - -/// A process-unique, lowercase-hex token for the case-probe filename. No `rand` -/// dependency: a monotonic counter (distinguishes concurrent same-process -/// probes) mixed with the pid and a nanosecond clock (distinguishes processes / -/// runs). Hex digits are case-foldable, which the probe relies on. -fn unique_probe_token() -> String { - use std::sync::atomic::{AtomicU64, Ordering}; - static COUNTER: AtomicU64 = AtomicU64::new(0); - let n = COUNTER.fetch_add(1, Ordering::Relaxed); - let pid = std::process::id(); - let nanos = std::time::SystemTime::now() - .duration_since(std::time::UNIX_EPOCH) - .map(|d| d.as_nanos()) - .unwrap_or(0); - format!("{pid:x}_{nanos:x}_{n:x}") -} - /// Recursively plan the host tree: collect directories to create and files to /// extract, sanitizing each component and detecting host-path collisions. /// Skips the top-level `AACS/`, `CERTIFICATE/`, and HD DVD's discovered `X!` @@ -473,11 +395,10 @@ fn plan_tree( host_rel: &Path, disc_path: &str, is_root: bool, - case_insensitive: bool, + sink: &mut TreeSink, unmapped: &[u32], files: &mut Vec, dirs: &mut Vec, - seen_hosts: &mut std::collections::HashMap, ) -> Result<()> { // Same discovery the AACS capture reads use, so the two can't drift. let hddvd_aacs_dir = is_root @@ -488,38 +409,14 @@ fn plan_tree( // The "parent" FID (".") has an empty name — skip. continue; } - // Strip AACS / CERTIFICATE at the top level only (a deeper dir of the - // same name is content). - if is_root - && (entry.name.eq_ignore_ascii_case("AACS") - || entry.name.eq_ignore_ascii_case("CERTIFICATE") - || hddvd_aacs_dir.is_some_and(|d| std::ptr::eq(d, entry))) - { + // The sink strips AACS / CERTIFICATE at the top level only (a deeper dir of + // the same name is content). + let aacs_dir = hddvd_aacs_dir.is_some_and(|d| std::ptr::eq(d, entry)); + if is_root && !TreeSink::keeps_top_level(&entry.name, aacs_dir) { continue; } - let safe = sanitize_component(&entry.name); - let child_rel = host_rel.join(&safe); let child_disc = format!("{disc_path}/{}", entry.name); - // Collision: two distinct disc paths → one host FILE. Case-insensitive folds - // `Movie`/`movie`; case-sensitive collides only on EXACT match (else names - // coexist). `.partial` shares the final-name namespace, so `X`/`X.partial` collide. - let mut register = |key: PathBuf| -> Result<()> { - let folded = if case_insensitive { - key.to_string_lossy().to_lowercase() - } else { - key.to_string_lossy().into_owned() - }; - if seen_hosts.insert(folded, child_disc.clone()).is_some() { - return Err(Error::DirNameCollision { - host: key.to_string_lossy().into_owned(), - }); - } - Ok(()) - }; - register(child_rel.clone())?; - if !entry.is_dir { - register(with_partial_suffix(&child_rel))?; - } + let child_rel = sink.claim(host_rel, &entry.name, &child_disc, entry.is_dir)?; if entry.is_dir { dirs.push(child_rel.clone()); plan_tree( @@ -529,11 +426,10 @@ fn plan_tree( &child_rel, &child_disc, false, - case_insensitive, + sink, unmapped, files, dirs, - seen_hosts, )?; } else if unmapped.contains(&entry.meta_lba) { files.push(PlannedFile { @@ -570,7 +466,7 @@ fn unmapped_file( total_bytes: u64, done_bytes: &mut u64, done_unreadable: &mut u64, - opts: &ExtractOptions, + ctx: &crate::ctx::Ctx, ) -> (FileResult, bool) { tracing::warn!( target: "freemkv::extract", @@ -579,14 +475,14 @@ fn unmapped_file( ); *done_bytes = done_bytes.saturating_add(pf.size); *done_unreadable = done_unreadable.saturating_add(pf.size); - let cont = report(opts, *done_bytes, *done_unreadable, total_bytes); + report(ctx, *done_bytes, *done_unreadable, total_bytes); let fr = FileResult { path: pf.host_rel.clone(), bytes_good: 0, bytes_unreadable: pf.size, complete: false, }; - (fr, opts.cancelled(cont)) + (fr, ctx.halt.is_cancelled()) } // Extracts one file via `.partial` (bad sectors -> zero holes), then renames. @@ -594,20 +490,14 @@ fn unmapped_file( // `.partial`; a finished inline file is already renamed). fn extract_one_file( dec: &mut DecryptingSectorSource, - dest: &Path, + sink: &TreeSink, pf: &PlannedFile, total_bytes: u64, done_bytes: &mut u64, done_unreadable: &mut u64, - opts: &ExtractOptions, + ctx: &crate::ctx::Ctx, ) -> Result<(FileResult, bool)> { - let final_path = dest.join(&pf.host_rel); - let partial_path = with_partial_suffix(&final_path); - - let mut writer = crate::io::WritebackFile::create_with_size_hint(&partial_path, pf.size) - .map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; + let mut file = sink.begin(&pf.host_rel, pf.size)?; let mut fr = FileResult { path: pf.host_rel.clone(), @@ -620,17 +510,17 @@ fn extract_one_file( // files are clear). Write verbatim, trimmed to size. if let Some(bytes) = &pf.inline { let n = (pf.size as usize).min(bytes.len()); - write_all(&mut writer, &bytes[..n], &partial_path)?; + file.write(&bytes[..n])?; fr.bytes_good = n as u64; // Data shorter than the declared size: the padded tail is lost, not good. let gap = pf.size - n as u64; fr.bytes_unreadable = gap; *done_unreadable = done_unreadable.saturating_add(gap); - finalize_file(writer, &partial_path, pf.size, &final_path)?; + file.finish()?; fr.complete = true; *done_bytes = done_bytes.saturating_add(pf.size); - let cont = report(opts, *done_bytes, *done_unreadable, total_bytes); - return Ok((fr, opts.cancelled(cont))); + report(ctx, *done_bytes, *done_unreadable, total_bytes); + return Ok((fr, ctx.halt.is_cancelled())); } let mut written: u64 = 0; @@ -656,14 +546,14 @@ fn extract_one_file( } while left > 0 { let n = left.min(buf.len() as u64) as usize; - write_all(&mut writer, &buf[..n], &partial_path)?; + file.write(&buf[..n])?; written = written.saturating_add(n as u64); *done_bytes = done_bytes.saturating_add(n as u64); left -= n as u64; } fr.bytes_good = fr.bytes_good.saturating_add(hole_bytes); - let cont = report(opts, *done_bytes, *done_unreadable, total_bytes); - if opts.cancelled(cont) { + report(ctx, *done_bytes, *done_unreadable, total_bytes); + if ctx.halt.is_cancelled() { return Ok((fr, true)); } if written >= pf.size { @@ -705,7 +595,10 @@ fn extract_one_file( // Unreadable ranges were zero-filled by `read_batch_narrowed`; record the // holes and keep going (no abort, no sweep-skip). let lost = lost.min(usable as u64); - write_all(&mut writer, &buf[..usable], &partial_path)?; + if lost > 0 { + ctx.stats.add_skip(lost); + } + file.write(&buf[..usable])?; // Damaged AACS units the reader blanked are unreadable, not good bytes. let unit = crate::aacs::content::ALIGNED_UNIT_LEN as u64; let blanked = (dec.blanked_units() - blanked_before) * unit; @@ -717,9 +610,9 @@ fn extract_one_file( } written = written.saturating_add(usable as u64); *done_bytes = done_bytes.saturating_add(usable as u64); - let cont = report(opts, *done_bytes, *done_unreadable, total_bytes); + report(ctx, *done_bytes, *done_unreadable, total_bytes); sector_off += batch; - if opts.cancelled(cont) { + if ctx.halt.is_cancelled() { // Leave the `.partial`; do NOT rename — this file stays incomplete. return Ok((fr, true)); } @@ -737,7 +630,7 @@ fn extract_one_file( *done_unreadable = done_unreadable.saturating_add(gap); *done_bytes = done_bytes.saturating_add(gap); } - finalize_file(writer, &partial_path, pf.size, &final_path)?; + file.finish()?; fr.complete = true; Ok((fr, false)) } @@ -856,207 +749,27 @@ fn read_tries( } } -fn write_all(writer: &mut crate::io::WritebackFile, data: &[u8], path: &Path) -> Result<()> { - writer.write_all(data).map_err(|e| { - let _ = std::fs::remove_file(path); - Error::DirWriteFailed { - errno: e.raw_os_error(), - } - }) -} - -/// Flush, sync, set the final length, and rename `.partial` → final. -fn finalize_file( - mut writer: crate::io::WritebackFile, - partial: &Path, - size: u64, - final_path: &Path, -) -> Result<()> { - writer.sync_all().map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; - drop(writer); - // Set the exact declared length (covers both an over-read final sector and - // an under-covered sparse tail). - let f = std::fs::OpenOptions::new() - .write(true) - .open(partial) - .map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; - f.set_len(size).map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; - // `set_len` is a separate kernel op on this second handle; without an fsync - // here, a crash between `set_len` and the rename can leave the file at its - // pre-truncation length. Sync the new metadata (length) before publishing. - f.sync_all().map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; - drop(f); - std::fs::rename(partial, final_path).map_err(|e| Error::DirWriteFailed { - errno: e.raw_os_error(), - })?; - // Durably commit the new dirent: on POSIX filesystems a crash right after a - // rename can lose the directory entry even though the rename returned. Best - // effort (swallowed on failure); no-op on Windows. Matches `write_atomic`. - if let Some(dir) = final_path.parent() { - crate::io::fsync::dir(dir); - } - Ok(()) -} - -/// Emit a progress report. Returns `true` to continue, `false` if the sink -/// requested an early stop (or there is no sink — always continue). -fn report(opts: &ExtractOptions, done: u64, unreadable: u64, total: u64) -> bool { - match opts.progress { - Some(p) => { - let pp = crate::progress::PassProgress { - kind: crate::progress::PassKind::Mux, - work_done: done, - work_total: total, - // `done` counts good AND unreadable bytes together; split them so a - // live-progress-only consumer sees a holed extraction as holed, - // not a clean climb to 100% (this used to pin unreadable at 0). - bytes_good_total: done.saturating_sub(unreadable), - bytes_unreadable_total: unreadable, - bytes_pending_total: 0, - bytes_retryable_total: 0, - bytes_total_disc: total, - disc_duration_secs: None, - bytes_bad_in_main_title: 0, - main_title_duration_secs: None, - main_title_size_bytes: None, - located: Default::default(), - }; - p.report(&pp) - } - None => true, - } -} - -/// Append `.partial` to a path's filename. -fn with_partial_suffix(path: &Path) -> PathBuf { - let mut name = path.file_name().unwrap_or_default().to_os_string(); - name.push(".partial"); - path.with_file_name(name) -} - -/// Available free bytes on the filesystem holding `dir`, or `None` when the -/// platform doesn't expose it (the free-space gate is then skipped). -#[cfg(unix)] -fn available_space(dir: &Path) -> Option { - use std::os::unix::ffi::OsStrExt; - let cpath = std::ffi::CString::new(dir.as_os_str().as_bytes()).ok()?; - // No safe std API for free space. SAFETY: `statvfs` is plain-old-data (all-zero is - // valid); `cpath` is NUL-terminated and outlives the call; `st` is a valid out-pointer. - let mut st: libc::statvfs = unsafe { std::mem::zeroed() }; - let rc = unsafe { libc::statvfs(cpath.as_ptr(), &mut st) }; - if rc != 0 { - return None; - } - // `statvfs` field integer widths differ by platform; cast both to `u64` - // for the product (no-op where already `u64` — allow lint for portability). - #[allow(clippy::unnecessary_cast, clippy::useless_conversion)] - let avail = (st.f_bavail as u64).saturating_mul(st.f_frsize as u64); - Some(avail) -} - -// Windows has no `statvfs`; queries free space via `GetDiskFreeSpaceExW` (declared directly -// against kernel32, matching `scsi::windows`) so the free-space gate still runs. -#[cfg(windows)] -fn available_space(dir: &Path) -> Option { - use std::os::windows::ffi::OsStrExt; - - unsafe extern "system" { - fn GetDiskFreeSpaceExW( - lpDirectoryName: *const u16, - lpFreeBytesAvailableToCaller: *mut u64, - lpTotalNumberOfBytes: *mut u64, - lpTotalNumberOfFreeBytes: *mut u64, - ) -> i32; - } - - // Wide, NUL-terminated. An interior NUL cannot reach the API, so reject it - // rather than silently truncating the path and measuring the wrong volume. - let mut wide: Vec = dir.as_os_str().encode_wide().collect(); - if wide.contains(&0) { - return None; - } - wide.push(0); - - let mut avail: u64 = 0; - // FreeBytesAvailableToCaller honours per-user quotas (unix `f_bavail`). - // SAFETY: `wide` is NUL-terminated and outlives the call; `avail` is a valid - // out-pointer; the two NULL out-params are documented optional. - let ok = unsafe { - GetDiskFreeSpaceExW( - wide.as_ptr(), - &mut avail, - std::ptr::null_mut(), - std::ptr::null_mut(), - ) +/// Emit the run's progress as an [`Event::Pass`](crate::Event::Pass). +fn report(ctx: &crate::ctx::Ctx, done: u64, unreadable: u64, total: u64) { + let pp = crate::progress::PassProgress { + kind: crate::progress::PassKind::Extract, + work_done: done, + work_total: total, + // `done` counts good AND unreadable bytes together; split them so a + // live-progress-only consumer sees a holed extraction as holed, + // not a clean climb to 100% (this used to pin unreadable at 0). + bytes_good_total: done.saturating_sub(unreadable), + bytes_unreadable_total: unreadable, + bytes_pending_total: 0, + bytes_retryable_total: 0, + bytes_total_disc: total, + disc_duration_secs: None, + bytes_bad_in_main_title: 0, + main_title_duration_secs: None, + main_title_size_bytes: None, + located: Default::default(), }; - if ok == 0 { None } else { Some(avail) } -} - -/// Neither unix nor Windows: no way to ask, so the gate is skipped. -#[cfg(not(any(unix, windows)))] -fn available_space(_dir: &Path) -> Option { - None -} - -/// Whether a directory exists and contains any entry. -fn dir_is_non_empty(dir: &Path) -> bool { - std::fs::read_dir(dir) - .map(|mut it| it.next().is_some()) - .unwrap_or(false) -} - -// Sanitizes ONE disc-path component: host-illegal chars and control bytes become `_`, trailing -// dot/space are stripped, a Windows reserved device name gets a `_` prefix. Names that -// collapse together are caught by the collision check. -fn sanitize_component(name: &str) -> String { - let mapped: String = name - .chars() - .map(|c| match c { - '/' | '\\' | ':' | '<' | '>' | '"' | '|' | '?' | '*' => '_', - c if (c as u32) < 0x20 => '_', - c => c, - }) - .collect(); - let trimmed = mapped.trim_end_matches([' ', '.']); - if trimmed.is_empty() { - return "_".to_string(); - } - // The device name is the stem before any extension, ignoring trailing spaces. - let base = trimmed.split('.').next().unwrap_or(trimmed); - if is_windows_reserved(base.trim_end_matches(' ')) { - return format!("_{trimmed}"); - } - trimmed.to_string() -} - -/// Whether `base` (the name component before any extension) matches a Windows -/// reserved device name. These are reserved by the OS regardless of extension -/// and silently alias a device (e.g. `NUL` discards writes). Case-insensitive. -pub(super) fn is_windows_reserved(base: &str) -> bool { - let up = base.trim_end_matches(' ').to_ascii_uppercase(); - if matches!( - up.as_str(), - "CON" | "PRN" | "AUX" | "NUL" | "CONIN$" | "CONOUT$" | "CLOCK$" - ) { - return true; - } - ["COM", "LPT"].iter().any(|p| { - up.strip_prefix(p).is_some_and(|d| { - let mut c = d.chars(); - matches!( - (c.next(), c.next()), - (Some('0'..='9' | '\u{B9}' | '\u{B2}' | '\u{B3}'), None) - ) - }) - }) + ctx.emit(crate::event::Event::Pass(&pp)); } /// DVD VTS group key for a `VTS_xx_*` file name, else `None`. e.g. @@ -1516,7 +1229,12 @@ mod tests { let mut disc = build_disc(root); let out = TmpDir::new("bdmv"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); assert_eq!( @@ -1571,8 +1289,13 @@ mod tests { let out = TmpDir::new("hddvd"); let mut d = clear_disc(); d.content_format = crate::disc::ContentFormat::MpegPs; - d.extract_tree(&mut disc, out.path(), &ExtractOptions::default()) - .expect("extract"); + d.extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) + .expect("extract"); assert_eq!(read_out(out.path(), "HVDVD_TS/MAIN.EVO"), Some(evo)); assert!(!out.path().join("AAC!").exists(), "AAC!/ must be stripped"); } @@ -1607,7 +1330,12 @@ mod tests { let mut d = clear_disc(); d.content_format = crate::disc::ContentFormat::MpegPs; let res = d - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); assert_eq!(read_out(out.path(), "VIDEO_TS/VIDEO_TS.IFO"), Some(ifo)); assert_eq!(read_out(out.path(), "VIDEO_TS/VTS_01_1.VOB"), Some(vob)); @@ -1665,7 +1393,12 @@ mod tests { crack_span: None, }); let res = d - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); let got = read_out(out.path(), "VIDEO_TS/VTS_01_1.VOB").expect("vob"); // Descrambled output matches the plaintext, with the scramble flag @@ -1676,6 +1409,58 @@ mod tests { assert!(res.complete); } + /// BUG-1: a live DVD scan records no disc-wide CSS key (`disc.css` is `None`), yet its + /// scrambled title VOB is still found from content, cracked and descrambled. + #[test] + fn a_live_dvd_with_no_scanned_css_key_still_descrambles() { + let title_key = [0x42u8, 0x13, 0x37, 0xBE, 0xEF]; + let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55]; + let mut plain = vec![0u8; 2048]; + plain[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]); + plain[4] = 0x44; + plain[0x14] = 0x10; + crate::css::dvd_pack_header(&mut plain, 0xE0); + let pat: Vec = (0..8) + .map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A) + .collect(); + for (i, b) in plain.iter_mut().enumerate().skip(0x59) { + *b = pat[i % 8]; + } + plain[0x54..0x59].copy_from_slice(&seed); + let mut scrambled = plain.clone(); + lfsr::scramble_sector(&title_key, &mut scrambled); + let root = DirSpec { + name: String::new(), + icb_lba: 10, + dir_data_lba: 11, + files: Vec::new(), + subdirs: vec![DirSpec { + name: "VIDEO_TS".to_string(), + icb_lba: 20, + dir_data_lba: 21, + files: vec![file("VTS_01_1.VOB", 30, 5000, scrambled, false)], + subdirs: vec![], + }], + }; + let mut disc = build_disc(root); + let out = TmpDir::new("css_live"); + let mut d = clear_disc(); + d.format = crate::disc::DiscFormat::Dvd; + d.content_format = crate::disc::ContentFormat::MpegPs; + assert!(d.css.is_none(), "a live scan's verdict: no disc-wide key"); + d.extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) + .expect("extract"); + let got = read_out(out.path(), "VIDEO_TS/VTS_01_1.VOB").expect("vob"); + let mut expect = plain.clone(); + expect[0x14] = 0x80; + assert_eq!(got, expect, "the scrambled VOB is written descrambled"); + } + /// A bad sector inside a file becomes a recorded zero-filled hole; the run /// does not abort, the file is still written, and loss is accounted. #[test] @@ -1708,7 +1493,12 @@ mod tests { } let out = TmpDir::new("badsector"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract does not abort on bad sectors"); let got = read_out(out.path(), "BDMV/STREAM/00001.m2ts").expect("file written"); assert_eq!( @@ -1755,7 +1545,12 @@ mod tests { } let out = TmpDir::new("decryptfail"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract does not abort on an undecryptable unit"); let got = read_out(out.path(), "BDMV/STREAM/00001.m2ts").expect("file written"); assert_eq!( @@ -1838,7 +1633,12 @@ mod tests { let mut disc = build_disc(root); let out = TmpDir::new("collision"); let err = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect_err("collision must error"); assert!(matches!(err, Error::DirNameCollision { .. })); } @@ -1866,7 +1666,12 @@ mod tests { std::fs::create_dir_all(out.path()).unwrap(); // Expectation from an independent std-only oracle, NOT the probe under test. let insensitive = oracle_case_insensitive(out.path()); - let res = clear_disc().extract_tree(&mut disc, out.path(), &ExtractOptions::default()); + let res = clear_disc().extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ); if insensitive { let err = res.expect_err("two names that fold to one host file must collide"); assert!(matches!(err, Error::DirNameCollision { .. }), "got {err:?}"); @@ -1896,7 +1701,12 @@ mod tests { let mut disc = build_disc(root); let out = TmpDir::new("partial_collision"); let err = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect_err( "X's temp path IS X.partial's final path — one host file for two \ disc files, so it must be refused up front", @@ -1920,7 +1730,12 @@ mod tests { let mut disc = build_disc(root); let err = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect_err("non-empty dir without --force must error"); assert!(matches!(err, Error::DirNotEmpty)); @@ -1937,7 +1752,7 @@ mod tests { ..Default::default() }; let res = clear_disc() - .extract_tree(&mut disc2, out.path(), &opts) + .extract_tree(&mut disc2, out.path(), &opts, &crate::ctx::Ctx::default()) .expect("force proceeds"); assert_eq!(read_out(out.path(), "a.bin"), Some(b"hello".to_vec())); assert!(res.complete); @@ -2009,7 +1824,7 @@ mod tests { let out = TmpDir::new("multiextent_aacs"); let d = aacs_disc(); let (a, b) = (PART_START + DATA_A, PART_START + DATA_B); - let set = crate::keys::ResolvedKeySet::keyed_for_test( + let set = crate::keys::KeyRing::keyed_for_test( &d, key, &[(a, a + SECTORS_EACH), (b, b + SECTORS_EACH)], @@ -2019,7 +1834,7 @@ mod tests { ..Default::default() }; let res = d - .extract_tree(&mut disc, out.path(), &opts) + .extract_tree(&mut disc, out.path(), &opts, &crate::ctx::Ctx::default()) .expect("extract"); let got = read_out(out.path(), "BDMV/STREAM/00001.m2ts").expect("file written"); @@ -2068,7 +1883,12 @@ mod tests { let out = TmpDir::new("unrecorded_extent"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); let got = read_out(out.path(), "INDEX.BDMV").expect("file written"); @@ -2138,41 +1958,17 @@ mod tests { let out = TmpDir::new("inline"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); assert_eq!(read_out(out.path(), "tiny.inf"), Some(payload)); assert!(res.complete); } - // ── Mutation-triage additions ─────────────────────────────────────────── - - /// `cancelled` must stop on EITHER signal alone (a disjunction) — a - /// progress sink asking to stop must cancel even with no halt token, and a - /// cancelled halt token must cancel even when progress says continue. - #[test] - fn cancelled_stops_on_either_signal_alone() { - let opts_no_halt = ExtractOptions::default(); - assert!( - opts_no_halt.cancelled(false), - "a progress sink asking to stop must cancel even with no halt token" - ); - assert!( - !opts_no_halt.cancelled(true), - "neither signal firing must not cancel" - ); - - let halt = crate::halt::Halt::new(); - halt.cancel(); - let opts_halted = ExtractOptions { - halt: Some(halt), - ..Default::default() - }; - assert!( - opts_halted.cancelled(true), - "a cancelled halt token must cancel even when progress says continue" - ); - } - // The free-space pre-check must fire whenever the disc's declared total // exceeds real available space; an exabyte size trips it regardless of // actual free space on whatever machine runs the test. @@ -2196,7 +1992,12 @@ mod tests { let out = TmpDir::new("insufficient_space"); let err = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect_err("an absurdly large declared size must trip the space gate"); assert!(matches!(err, Error::DirInsufficientSpace { .. })); } @@ -2259,7 +2060,12 @@ mod tests { crack_span: None, }); let res = d - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); let got_1 = read_out(out.path(), "VIDEO_TS/VTS_01_1.VOB").expect("vts01 vob"); @@ -2342,7 +2148,12 @@ mod tests { crack_span: None, }); let err = d - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect_err( "a VTS whose key could not be recovered must be a hard error, \ not a silent extract under another VTS's key", @@ -2478,7 +2289,12 @@ mod tests { let out = TmpDir::new("extent_bounds"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); let got = read_out(out.path(), "big.bin").expect("file written"); @@ -2571,17 +2387,23 @@ mod tests { assert_eq!(dec.inner().calls, READ_RETRIES + 1); } - /// `report` must reflect the sink's verdict -- a sink asking to stop must - /// actually halt the run mid-file, not be swallowed. - #[test] - fn progress_sink_stop_halts_run_mid_file() { - struct StopImmediately; - impl crate::progress::Progress for StopImmediately { - fn report(&self, _p: &crate::progress::PassProgress) -> bool { - false - } - } + // A run whose events cancel its own halt at the first pass report (a UI's Stop). + fn stop_on_first_pass() -> crate::ctx::Ctx { + let halt = crate::halt::Halt::new(); + let stop = halt.clone(); + crate::ctx::Ctx::new(halt).with_events(std::sync::Arc::new( + move |e: &crate::event::Event<'_>| { + if let crate::event::Event::Pass(p) = e { + assert_eq!(p.kind, crate::progress::PassKind::Extract); + stop.cancel(); + } + }, + )) + } + /// A Stop raised from the progress events must halt the run mid-file, not be swallowed. + #[test] + fn stop_from_the_progress_events_halts_run_mid_file() { let good = vec![0x66u8; 4 * 2048]; let root = DirSpec { name: String::new(), @@ -2604,16 +2426,16 @@ mod tests { }; let mut disc = build_disc(root); let out = TmpDir::new("progress_stop"); - let sink = StopImmediately; - let opts = ExtractOptions { - progress: Some(&sink), - ..Default::default() - }; let res = clear_disc() - .extract_tree(&mut disc, out.path(), &opts) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &stop_on_first_pass(), + ) .expect("extract does not error on a progress halt"); - assert!(res.halted, "a sink returning false must halt the run"); + assert!(res.halted, "a Stop at the first report must halt the run"); assert!( !res.files[0].complete, "the in-flight file must be left incomplete, not finalized" @@ -2672,7 +2494,12 @@ mod tests { let mut disc = two_clip_disc_with_first_unmapped(); let out = TmpDir::new("unmapped_icb"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("extract"); assert_eq!( read_out(out.path(), "BDMV/STREAM/00002.m2ts"), @@ -2685,23 +2512,18 @@ mod tests { assert!(!res.complete && !res.halted); } - // A progress Stop reported on the lost file ends the run there, like any other file. + // A Stop at the lost file's report ends the run there, like any other file. #[test] fn progress_stop_on_an_unmapped_stream_file_halts_the_run() { - struct StopImmediately; - impl crate::progress::Progress for StopImmediately { - fn report(&self, _p: &crate::progress::PassProgress) -> bool { - false - } - } let mut disc = two_clip_disc_with_first_unmapped(); let out = TmpDir::new("unmapped_stop"); - let opts = ExtractOptions { - progress: Some(&StopImmediately), - ..Default::default() - }; let res = clear_disc() - .extract_tree(&mut disc, out.path(), &opts) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &stop_on_first_pass(), + ) .expect("a progress halt is not an error"); assert!(res.halted); assert_eq!(res.files.len(), 1, "the run stops after the lost file"); @@ -2858,7 +2680,11 @@ mod tests { let mut disc = build_disc(root); let fs = udf::read_filesystem(&mut disc).unwrap(); let plan = |disc: &mut MemDisc, ci: bool| { - let (mut files, mut dirs, mut seen) = (Vec::new(), Vec::new(), HashMap::new()); + let (mut files, mut dirs) = (Vec::new(), Vec::new()); + let out = tempfile::tempdir().unwrap(); + let mut sink = TreeSink::create(out.path(), true) + .unwrap() + .with_case_insensitive(ci); plan_tree( disc, &fs, @@ -2866,11 +2692,10 @@ mod tests { Path::new(""), "", true, - ci, + &mut sink, &[], &mut files, &mut dirs, - &mut seen, ) .map(|()| files.len()) }; @@ -2920,7 +2745,12 @@ mod tests { } let out = TmpDir::new("drive_halt"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("a Stop is a halt, not an error"); assert!(res.halted, "drive-level Halted must halt the run"); assert_eq!(res.files.len(), 1, "no file may start after the Stop"); @@ -2966,15 +2796,14 @@ mod tests { std::fs::create_dir_all(out.path()).unwrap(); let mut dec = DecryptingSectorSource::new(AllBad(Vec::new()), DecryptKeys::None); let (mut done, mut bad) = (0u64, 0u64); - let opts = ExtractOptions::default(); let (fr, halted) = extract_one_file( &mut dec, - out.path(), + &TreeSink::create(out.path(), true).unwrap(), &pf, len as u64, &mut done, &mut bad, - &opts, + &crate::ctx::Ctx::default(), ) .unwrap(); assert!(!halted); @@ -3030,12 +2859,13 @@ mod tests { title_key: [0xFFu8; 5], crack_span: None, }); - let opts = ExtractOptions { - halt: Some(halt.clone()), - ..Default::default() - }; let res = d - .extract_tree(&mut src, out.path(), &opts) + .extract_tree( + &mut src, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::new(halt.clone()), + ) .expect("a Stop during the crack is a halt, not CssKeyMissing"); assert!(res.halted); assert!(res.files.is_empty()); @@ -3044,12 +2874,6 @@ mod tests { // A stop requested on an INLINE file's report must not be dropped. #[test] fn progress_stop_on_inline_file_halts_the_run() { - struct StopOnce(std::cell::Cell); - impl crate::progress::Progress for StopOnce { - fn report(&self, _p: &crate::progress::PassProgress) -> bool { - self.0.replace(true) - } - } let inline_icb = |payload: &[u8]| { let mut icb = [0u8; 2048]; icb[0..2].copy_from_slice(&266u16.to_le_bytes()); @@ -3070,14 +2894,14 @@ mod tests { disc.put_bytes(PART_START + 11, &root_fids); build_udf_skeleton(&mut disc, 10); - let sink = StopOnce(std::cell::Cell::new(false)); - let opts = ExtractOptions { - progress: Some(&sink), - ..Default::default() - }; let out = TmpDir::new("inline_stop"); let res = clear_disc() - .extract_tree(&mut disc, out.path(), &opts) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &stop_on_first_pass(), + ) .expect("extract"); assert!(res.halted, "the inline file's stop request was dropped"); assert_eq!(res.files.len(), 1); @@ -3130,7 +2954,12 @@ mod tests { crack_span: None, }); let res = d - .extract_tree(&mut src, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut src, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect("a drive Stop during the crack is a halt, not CssKeyMissing"); assert!(res.halted); assert!(res.files.is_empty()); @@ -3204,12 +3033,12 @@ mod tests { let (mut done, mut bad) = (0u64, 0u64); let (fr, _) = extract_one_file( &mut dec, - out.path(), + &TreeSink::create(out.path(), true).unwrap(), &pf, len as u64, &mut done, &mut bad, - &ExtractOptions::default(), + &crate::ctx::Ctx::default(), ) .unwrap(); assert_eq!(fr.bytes_unreadable, 6 * SECTOR_BYTES as u64); @@ -3386,12 +3215,12 @@ mod tests { let (mut done, mut bad) = (0u64, 0u64); let (fr, _) = extract_one_file( &mut dec, - out.path(), + &TreeSink::create(out.path(), true).unwrap(), &pf, size, &mut done, &mut bad, - &ExtractOptions::default(), + &crate::ctx::Ctx::default(), ) .unwrap(); assert_eq!(fr.bytes_good, SECTOR_BYTES as u64); @@ -3410,12 +3239,12 @@ mod tests { let (mut done, mut bad) = (0u64, 0u64); let (fr, _) = extract_one_file( &mut dec, - out.path(), + &TreeSink::create(out.path(), true).unwrap(), &pf, 5000, &mut done, &mut bad, - &ExtractOptions::default(), + &crate::ctx::Ctx::default(), ) .unwrap(); assert_eq!(fr.bytes_good, 100); @@ -3455,7 +3284,12 @@ mod tests { let mut disc = build_disc(root); let out = TmpDir::new("dup_names"); let err = clear_disc() - .extract_tree(&mut disc, out.path(), &ExtractOptions::default()) + .extract_tree( + &mut disc, + out.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect_err("duplicate names must collide"); assert!(matches!(err, Error::DirNameCollision { .. })); } diff --git a/src/disc/mod.rs b/src/disc/mod.rs index d81dab8e..8db6c95b 100644 --- a/src/disc/mod.rs +++ b/src/disc/mod.rs @@ -715,11 +715,8 @@ fn bus_map(files: StreamFiles, titles: &[DiscTitle]) -> crate::sector::bus_remov // Corrects a title's TrueHD channels/sample-rate/Atmos by probing the first decrypted major // sync (`reader` must yield DECRYPTED sectors: mux time, not scan). +#[cfg(test)] pub(crate) fn correct_truehd_channels(reader: &mut dyn SectorSource, title: &mut DiscTitle) { - use crate::mux::codec::truehd::{ - truehd_channels, truehd_lfe, truehd_sample_rate_hz, truehd_sync_info_from_stream, - }; - let pids: Vec = title .streams .iter() @@ -770,50 +767,61 @@ pub(crate) fn correct_truehd_channels(reader: &mut dyn SectorSource, title: &mut let Some(payload) = payloads.get(&a.pid) else { continue; }; - // One major-sync read yields channels, sample rate and the Atmos signal. - let Some(info) = truehd_sync_info_from_stream(payload) else { - continue; + correct_truehd_stream(a, payload); + } +} + +/// Correct one TrueHD track from its stream bytes (the playlist understates 7.1/Atmos as +/// 5.1): channels, sample rate and label from the first major sync in `payload`. Returns +/// `false` when `payload` holds no major sync yet. +pub(crate) fn correct_truehd_stream(a: &mut AudioStream, payload: &[u8]) -> bool { + use crate::mux::codec::truehd::{ + truehd_channels, truehd_lfe, truehd_sample_rate_hz, truehd_sync_info_from_stream, + }; + // One major-sync read yields channels, sample rate and the Atmos signal. + let Some(info) = truehd_sync_info_from_stream(payload) else { + return false; + }; + + // Whether the label is still the plain descriptor (no richer editorial + // label). Captured against the CURRENT channels before any correction so + // a label promotion only happens when nothing editorial is present. + let was_basic = + a.label == crate::labels::generate_audio_label(&a.codec, &a.channels, a.secondary); + + // (1) Channels — only when the major sync resolves a different layout. + if let Some(count) = truehd_channels(info.format_info) { + let lfe = truehd_lfe(info.format_info); + let new_ch = match AudioChannels::from_layout(count.saturating_sub(lfe), lfe) { + AudioChannels::Unknown => AudioChannels::from_count(count), + named => named, }; - - // Whether the label is still the plain descriptor (no richer editorial - // label). Captured against the CURRENT channels before any correction so - // a label promotion only happens when nothing editorial is present. - let was_basic = - a.label == crate::labels::generate_audio_label(&a.codec, &a.channels, a.secondary); - - // (1) Channels — only when the major sync resolves a different layout. - if let Some(count) = truehd_channels(info.format_info) { - let lfe = truehd_lfe(info.format_info); - let new_ch = match AudioChannels::from_layout(count.saturating_sub(lfe), lfe) { - AudioChannels::Unknown => AudioChannels::from_count(count), - named => named, - }; - if new_ch != AudioChannels::Unknown && new_ch != a.channels { - a.channels = new_ch; - } - } - - // (2) Sample rate — whitelisted rates only; an unknown nibble or a rate - // that maps to no enum variant leaves the container value untouched - // (never write a wrong SamplingFrequency). - if let Some(hz) = truehd_sample_rate_hz(info.format_info) { - let new_sr = SampleRate::from_hz(hz); - if new_sr != SampleRate::Unknown && new_sr != a.sample_rate { - a.sample_rate = new_sr; - } - } - - // (3) Label — refresh to the corrected channels; promote to the Atmos - // form only when the stream carried the basic descriptor (no editorial - // Atmos already) AND a 4th substream was positively detected. - if was_basic { - a.label = if info.is_atmos == Some(true) { - crate::labels::generate_audio_label_atmos(&a.codec, &a.channels, a.secondary) - } else { - crate::labels::generate_audio_label(&a.codec, &a.channels, a.secondary) - }; + if new_ch != AudioChannels::Unknown && new_ch != a.channels { + a.channels = new_ch; } } + + // (2) Sample rate — whitelisted rates only; an unknown nibble or a rate + // that maps to no enum variant leaves the container value untouched + // (never write a wrong SamplingFrequency). + if let Some(hz) = truehd_sample_rate_hz(info.format_info) { + let new_sr = SampleRate::from_hz(hz); + if new_sr != SampleRate::Unknown && new_sr != a.sample_rate { + a.sample_rate = new_sr; + } + } + + // (3) Label — refresh to the corrected channels; promote to the Atmos + // form only when the stream carried the basic descriptor (no editorial + // Atmos already) AND a 4th substream was positively detected. + if was_basic { + a.label = if info.is_atmos == Some(true) { + crate::labels::generate_audio_label_atmos(&a.codec, &a.channels, a.secondary) + } else { + crate::labels::generate_audio_label(&a.codec, &a.channels, a.secondary) + }; + } + true } /// Calculate how many bytes of bad/unreadable data fall within a title's extents. @@ -1670,31 +1678,6 @@ impl std::fmt::Debug for AacsState { } } -/// How AACS keys were resolved. Variants are ordered root-of-trust → -/// per-disc-leaf, matching the resolver's path-try order: the resolver -/// attempts derivation from the strongest input it has first and falls -/// back toward pre-computed per-disc material. -#[derive(Debug, Clone, Copy, PartialEq)] -pub enum KeyOrigin { - /// MKB + device keys → subset-difference tree → VUK - DeviceKey, - /// MKB + processing keys → media key → VUK - ProcessingKey, - /// Media key + Volume ID from KEYDB → derived VUK - KeyDbDerived, - /// VUK found directly in KEYDB by disc hash - KeyDb, - /// Pre-decrypted unit keys taken directly from KEYDB by disc hash. - /// No VUK present in the entry — `AacsState::vuk` is `None`. - KeyDbUnitKeys, - /// Unit key supplied directly by the caller (the external Unit Key path). - /// No keydb, no derivation — `AacsState::vuk` is `None`. - ExternalUk, -} - -// No `KeyOrigin::name()`: the library holds ZERO user-facing English. Callers -// map variants to display text (see freemkv's `disc_info::key_origin_label`). - // ─── Disc scanning ────────────────────────────────────────────────────────── /// AACS host credentials for the live-drive authenticated handshake. @@ -1714,7 +1697,7 @@ pub struct DriveCredentials { /// Options for disc scanning. /// /// libfreemkv is lookup-free — it resolves no keys. The caller resolves a key -/// out-of-band through [`ResolvedKeySet::resolve`](crate::keys::ResolvedKeySet::resolve). The +/// out-of-band through [`KeyRing::acquire`](crate::keys::KeyRing::acquire). The /// only scan input is the optional drive credentials for the live-drive /// authenticated handshake. #[derive(Default)] @@ -1735,7 +1718,7 @@ pub struct ScanOptions { /// certs into `credentials` may leave it empty too. The library still /// resolves NO keys from these at scan time; they are consulted only for /// their host certs here (key *resolution* stays out-of-band via - /// `ResolvedKeySet::resolve`). + /// `KeyRing::resolve`). pub key_sources: Vec>, /// Optional cooperative-cancellation token. When set, long scan-time /// loops (notably the CSS known-plaintext crack, which can scan up to @@ -2303,7 +2286,7 @@ impl Disc { /// Read a disc's AACS key-input files from an ISO image: returns /// `(Unit_Key_RO.inf, MKB, aacs_major_version)`. For callers that resolve a /// Unit Key out-of-band: the key reaches a read only through - /// [`ResolvedKeySet`](crate::keys::ResolvedKeySet). libfreemkv never makes a network call. + /// [`KeyRing`](crate::keys::KeyRing). libfreemkv never makes a network call. pub fn read_aacs_inputs(iso_path: &std::path::Path) -> Result<(Vec, Vec, u8)> { // Preserve the underlying open error (E5000) instead of collapsing // ENOENT/EPERM into `Error::AacsNoKeys` (E7000): a missing ISO is an @@ -2419,7 +2402,7 @@ impl Disc { /// Same as [`Disc::read_aacs_inputs`] but reads from a live drive. The /// out-of-band Unit Key path fetches the disc's key files from the drive, /// resolves a key from them however it likes, through - /// [`ResolvedKeySet`](crate::keys::ResolvedKeySet). These files are plaintext UDF metadata — no + /// [`KeyRing`](crate::keys::KeyRing). These files are plaintext UDF metadata — no /// AACS handshake or keys are required to read them. pub fn read_aacs_inputs_from_drive(drive: &mut Drive) -> Result<(Vec, Vec, u8)> { // No READ CAPACITY: the key files do not need the disc size. @@ -2458,7 +2441,7 @@ impl Disc { tracing::info!(target: "freemkv::scan", phase = "scan_with", "begin"); let encrypted = aacs.is_some(); // Lookup-free: the state carries the disc's AACS inputs but no key; the caller - // resolves one into a `ResolvedKeySet`. + // resolves one into a `KeyRing`. let (aacs, aacs_error) = match aacs { Some((cap, bus)) => encrypt::resolve_aacs(cap, &bus), None => (None, None), @@ -2901,58 +2884,6 @@ impl Disc { } } -/// A decryption key handed to libfreemkv by the caller. libfreemkv is -/// **lookup-free**: it never reads a keydb, talks to a key server, or -/// searches paths — the application resolves a key however it likes and -/// hands it in here; libfreemkv derives any remaining AACS-chain steps from -/// disc-read inputs (MKB / VID / `Unit_Key_RO.inf`). `#[non_exhaustive]`: -/// AACS is a derivation chain (`DK →(MKB)→ MK →(VID)→ VK →(Unit_Key_RO)→ -/// UK`); each variant enters at one level, and the library -/// derives down from it — new levels can be added without breaking callers. -#[derive(Clone)] -#[non_exhaustive] -pub enum Key { - /// Device key(s) (AACS DK, positioned). libfreemkv walks the MKB - /// (subset-difference tree) to find the one that applies → media key → - /// VUK → unit keys. A source hands in its FULL device-key set, because - /// choosing which one applies *is* the MKB walk (derivation), and all - /// derivation lives here — never in a source. - Device(Vec), - /// Processing key(s) (AACS PK). libfreemkv applies each against the MKB - /// → media key → VUK → unit keys. - Processing(Vec<[u8; 16]>), - /// Media key candidate(s) (Km). A source hands its full pool because an MK - /// is MKB-scoped (shared across a pressing/MKB family) — picking the one - /// that applies is `km_verifies` against this disc's MKB, which is - /// derivation, so it lives here. libfreemkv verifies, then derives the VUK - /// via the Volume ID and the per-CPS-unit keys. - Media(Vec<[u8; 16]>), - /// Volume Unique Key (VK / VUK). libfreemkv decrypts `Unit_Key_RO.inf` - /// into the per-CPS-unit keys. NOT terminal — the chain continues to the - /// unit keys. - Volume([u8; 16]), - /// Final per-CPS-unit AACS keys (`(cps_unit, 16-byte key)`). A key source - /// (keydb / key server) resolved these, or they were cached in the mapfile - /// at sweep; libfreemkv decrypts directly with no further derivation. This - /// is the terminal level every other variant derives down into. - Unit(Vec<(u32, [u8; 16])>), -} - -// Redacting `Debug`: `Key` is a key-transport type; -// every variant carries raw key material. Print only variant name and count — -// never bytes. Guarded by `aacs_state_and_key_debug_are_redacted`. -impl std::fmt::Debug for Key { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { - Key::Device(v) => write!(f, "Key::Device(<{} redacted>)", v.len()), - Key::Processing(v) => write!(f, "Key::Processing(<{} redacted>)", v.len()), - Key::Media(v) => write!(f, "Key::Media(<{} redacted>)", v.len()), - Key::Volume(_) => f.write_str("Key::Volume()"), - Key::Unit(v) => write!(f, "Key::Unit(<{} redacted>)", v.len()), - } - } -} - // Read-ahead confined to one file's recorded extents: a single-sector read inside an // extent fetches up to a batch of THAT extent only (never adjacent essence). Other // single reads (the ICB) are memoised; the caller's `recovery` flag is forwarded. @@ -3078,7 +3009,7 @@ fn is_plain_file_name(name: &str) -> bool { } // Matched on the stem with trailing spaces ignored. let stem = name.split('.').next().unwrap_or(name); - !extract::is_windows_reserved(stem) + !crate::io::tree_sink::is_windows_reserved(stem) } impl Disc { @@ -3098,7 +3029,7 @@ impl Disc { } /// The disc's own decryption keys: CSS, or `None`. AACS keys come only from a - /// [`ResolvedKeySet`](crate::keys::ResolvedKeySet) (KU §2.2), so an AACS disc is `None`. + /// [`KeyRing`](crate::keys::KeyRing) (KU §2.2), so an AACS disc is `None`. pub(crate) fn decrypt_keys(&self) -> crate::decrypt::DecryptKeys { if self.aacs.is_some() { crate::decrypt::DecryptKeys::None @@ -3262,12 +3193,12 @@ pub(crate) fn mapfile_path_for(iso_path: &std::path::Path) -> std::path::PathBuf impl Disc { /// Path to the mapfile for a given output path. /// - /// For `/dev/null` output, returns `{dir}/{volume_id_or_title}.mapfile` + /// For the null device ([`crate::io::null_device`]), returns `{dir}/{volume_id_or_title}.mapfile` /// where `{dir}` is a directory this process created under the temp dir /// (owner-only on Unix, unpredictable name, stable for the process). For /// regular files, returns `{path}.mapfile`. pub fn mapfile_for(&self, path: &std::path::Path) -> std::path::PathBuf { - if path.as_os_str() == "/dev/null" { + if crate::io::is_null_device(path) { let name: String = self .meta_title .as_deref() @@ -4194,23 +4125,6 @@ mod tests { let d = format!("{st:?}"); assert!(!d.contains("213"), "AacsState leaked key bytes: {d}"); assert!(d.contains("redacted"), "AacsState missing marker: {d}"); - - for k in [ - Key::Device(vec![crate::aacs::types::DeviceKey { - key: [0xD5; 16], - node: 0, - uv: 0, - u_mask_shift: 0, - }]), - Key::Unit(vec![(1, [0xD5; 16])]), - Key::Volume([0xD5; 16]), - Key::Processing(vec![[0xD5; 16]]), - Key::Media(vec![[0xD5; 16]]), - ] { - let d = format!("{k:?}"); - assert!(!d.contains("213"), "Key leaked bytes: {d}"); - assert!(d.contains("redacted"), "Key missing marker: {d}"); - } } // ── encrypted-content map (`merged_extents` core) ──────────────────────── @@ -6430,35 +6344,6 @@ mod tests { // AacsVidUnavailable, and the gate surfaces E7017, not the generic E7022. #[test] fn disc_gate_aacs_vid_unavailable_vs_no_key() { - let supplied = crate::aacs::provider::SuppliedKey { - device_keys: vec![crate::aacs::types::DeviceKey { - key: [0x11; 16], - node: 1, - uv: 1, - u_mask_shift: 0, - }], - processing_keys: Vec::new(), - media_keys: Vec::new(), - disc_entry: None, - }; - let provider_refs: [&dyn crate::aacs::provider::KeyProvider; 1] = [&supplied]; - // A parseable Unit_Key_RO.inf (uk_pos=32, zero unit keys), so resolution - // fails for lack of a VID, not because the .inf failed to parse. - let mut uk_ro = vec![0u8; 40]; - uk_ro[0..4].copy_from_slice(&32u32.to_be_bytes()); - let ctx = crate::aacs::resolve::ResolveContext { - unit_key_ro: &uk_ro, - content_cert: None, - volume_id: &[0u8; 16], - providers: &provider_refs, - mkb: None, - }; - assert_eq!( - crate::aacs::resolve::resolve_keys_with_reason(&ctx, 2).err(), - Some(crate::aacs::resolve::ResolveFailure::VidUnavailable), - "device keys + zero VID must classify as VidUnavailable" - ); - let mut disc = make_test_disc(1000, "UHD"); disc.encrypted = true; disc.aacs = Some(aacs_empty()); @@ -8659,7 +8544,12 @@ mod tests { let (mut d, disc) = live_bus_scan(mem, Some(rdk)).expect("scan"); let dest = tempfile::tempdir().unwrap(); let res = disc - .extract_tree(&mut d, dest.path(), &ExtractOptions::default()) + .extract_tree( + &mut d, + dest.path(), + &ExtractOptions::default(), + &crate::ctx::Ctx::default(), + ) .expect(SPEC_BD_3_7_NOTE); let stream = dest.path().join("BDMV/STREAM"); assert_eq!( diff --git a/src/disc/scan_order_tests.rs b/src/disc/scan_order_tests.rs index a3a78b91..d55d47c4 100644 --- a/src/disc/scan_order_tests.rs +++ b/src/disc/scan_order_tests.rs @@ -971,8 +971,8 @@ fn scan_with_final_check() { assert_eq!(fake.log().len(), last); } -// A key source that records whether the drive's Progress was busy while it ran. -struct BusyProbe(crate::halt::Progress, Arc>>); +// A key source that records whether the drive's Liveness was busy while it ran. +struct BusyProbe(crate::halt::Liveness, Arc>>); impl crate::KeySource for BusyProbe { fn get_unit_keys( &self, @@ -986,11 +986,11 @@ impl crate::KeySource for BusyProbe { } } -/// ST4-2: `bus_step_guard` holds `busy()` on the Drive-attached `Progress` for the +/// ST4-2: `bus_step_guard` holds `busy()` on the Drive-attached `Liveness` for the /// whole bus step, so the first keydb parse (in `host_certs`) is never idle time. #[test] fn bus_step_guard_holds_busy_across_host_certs() { - let p = crate::halt::Progress::new(); + let p = crate::halt::Liveness::new(); let seen: Arc>> = Arc::default(); let mut rig = Rig::new(bd_disc(Some(true)), |_| {}); rig.drive.attach_progress(&p); @@ -1036,10 +1036,11 @@ fn cancelled_scan_then_resolve_returns_halted_and_builds_no_set() { assert!(matches!(r, Err(Error::Halted)), "{r:?}"); assert!(s.disc().is_none(), "a stopped scan stores no Disc"); let sources: crate::session::KeySourceFactory = Arc::new(Vec::new); - let r = s.resolve_key_set( + let r = s.acquire_keys( crate::keys::KeyScope::WholeDisc, &sources, Default::default(), + &crate::ctx::Ctx::default(), ); assert!(matches!(r, Err(Error::Halted)), "{:?}", r.err()); let reads = fake.count(is_read10); diff --git a/src/drive/mod.rs b/src/drive/mod.rs index 6bbf7e39..8e508ba8 100644 --- a/src/drive/mod.rs +++ b/src/drive/mod.rs @@ -42,8 +42,7 @@ pub(crate) use macos as platform; pub(crate) use windows as platform; use crate::error::{Error, Result}; -use crate::event::Event; -use crate::halt::{Halt, Progress}; +use crate::halt::{Halt, Liveness}; use crate::identity::DriveId; use crate::scsi::ScsiTransport; use crate::sector::SectorSource; @@ -124,11 +123,9 @@ pub struct Drive { /// The op token every CDB is checked against (§2.2). slot: Slot, /// Bumped on every `exec` completion, busy while one is in flight (T29 feed). - progress: Option, + progress: Option, /// AGIDs allocated through this Drive and not yet invalidated (§2.2 ledger). agids: u8, - /// Event handler — fires for read errors and library-level state changes. - event_fn: Option>, /// The SINGLE AACS bus-encryption removal point. Decided ONCE from the /// unlock/handshake result during [`crate::disc::Disc::scan`] and applied on /// every read below this drive, so every reader above (sampler, mux, sweep, @@ -275,7 +272,6 @@ impl Drive { slot, progress: None, agids: 0, - event_fn: None, bus_stage: crate::sector::bus_removal::BusStage::Passthrough, bus_gate: None, #[cfg(target_os = "linux")] @@ -380,11 +376,6 @@ impl Drive { } } - /// Set an event handler for read recovery events. - pub fn on_event(&mut self, f: impl Fn(Event) + Send + 'static) { - self.event_fn = Some(Box::new(f)); - } - /// The token every CDB is checked against, or `None` once [`detach`](Self::detach)ed. pub fn token(&self) -> Option<&Halt> { self.slot.token() @@ -406,12 +397,12 @@ impl Drive { } /// Report forward progress to `p`: bumped on every CDB completion, and - /// [`busy`](Progress::busy) while a CDB (or a scan bus step) is in flight. - pub fn attach_progress(&mut self, p: &Progress) { + /// [`busy`](Liveness::busy) while a CDB (or a scan bus step) is in flight. + pub fn attach_progress(&mut self, p: &Liveness) { self.progress = Some(p.clone()); } - pub(crate) fn progress(&self) -> Option<&Progress> { + pub(crate) fn progress(&self) -> Option<&Liveness> { self.progress.as_ref() } @@ -532,7 +523,7 @@ impl Drive { buf: &mut [u8], timeout_ms: u32, ) -> Result { - let busy = self.progress.as_ref().map(Progress::busy); + let busy = self.progress.as_ref().map(Liveness::busy); let r = self.scsi.as_mut().execute(cdb, dir, buf, timeout_ms); drop(busy); if let Some(p) = &self.progress { @@ -614,7 +605,7 @@ impl Drive { let ceiling_hit: bool; // T6: the answers seen so far and the highest progress indicator; either // growing re-arms the no-progress window. - let moved = Progress::new(); + let moved = Liveness::new(); let mut stall = crate::halt::StallTimer::new(timing.window, &moved); let mut seen: Vec> = Vec::new(); let mut best: Option = None; diff --git a/src/drive/stop_tests.rs b/src/drive/stop_tests.rs index 382467a6..97e12fae 100644 --- a/src/drive/stop_tests.rs +++ b/src/drive/stop_tests.rs @@ -686,7 +686,7 @@ fn scsi_sense_keeps_three_fields() { /// one is in flight. #[test] fn attached_progress_bumps_per_cdb_and_is_busy_in_flight() { - let p = crate::halt::Progress::new(); + let p = crate::halt::Liveness::new(); let p2 = p.clone(); let t = Script::new(move |_, d| { assert!(p2.is_busy(), "busy mid-CDB"); diff --git a/src/error.rs b/src/error.rs index 74e83b4c..bfab7c94 100644 --- a/src/error.rs +++ b/src/error.rs @@ -171,7 +171,6 @@ pub const E_PES_FRAME_TOO_LARGE: u16 = 9005; pub const E_PES_INVALID_MAGIC: u16 = 9006; pub const E_ISO_TOO_LARGE: u16 = 9007; pub const E_NO_METADATA: u16 = 9008; -pub const E_DISC_URL_NOT_DIRECT: u16 = 9009; /// `--raw` given with a `dir://` destination (raw + decrypted-tree is /// a contradiction; raw bytes go to `iso://`). pub const E_DIR_RAW_REJECTED: u16 = 9019; @@ -211,10 +210,6 @@ pub const E_DIR_IMAGE_SSIF_UNSUPPORTED: u16 = 9061; /// the end of the file that must precede it. Carries the offending file's disc /// path. Also raised for an IFO too short to resolve its placement offsets. pub const E_DIR_IMAGE_PLACEMENT: u16 = 9062; -/// A `dir://` SOURCE folder still carries live AACS-encrypted content: it has -/// an `AACS/` directory AND the sampled content units are genuinely scrambled. -/// `dir://` sources are decrypted-folder only. -pub const E_DIR_IMAGE_ENCRYPTED: u16 = 9063; /// A `dir://` SOURCE folder holds no disc structure the image synthesizer /// understands (no `BDMV/`, no `VIDEO_TS/`). pub const E_DIR_IMAGE_UNSUPPORTED_TREE: u16 = 9064; @@ -327,8 +322,6 @@ pub const E_REMUX_STAGING_INVALID: u16 = 9079; pub const E_STAGED_COPY_SIZE_MISMATCH: u16 = 9080; /// A remux worker thread (`op`: "copy", "verify") was lost before it reported. pub const E_WORKER_LOST: u16 = 9081; -/// A multipass rip was asked to decrypt: multipass recovers a raw whole-disc image. -pub const E_MULTIPASS_REQUIRES_RAW: u16 = 9082; /// A requested audio/subtitle language tag resolves to no known language. pub const E_STREAM_LANGUAGE_UNKNOWN: u16 = 9083; /// A remux target already exists and replacing it was not requested. @@ -778,10 +771,6 @@ pub enum Error { path: String, }, NoMetadata, - /// `disc://` URLs aren't openable through `input()` — callers must use - /// `Drive::open() + Disc::scan() + DiscStream::new()` directly. This - /// is a structural API constraint, not a parse failure. - DiscUrlNotDirect, /// A non-empty `HEVCDecoderConfigurationRecord` (hvcC) was supplied to /// a muxer but failed to parse into any VPS/SPS/PPS NAL — emitting the /// stream without parameter sets would yield an undecodable result. @@ -893,8 +882,6 @@ pub enum Error { DirImagePlacement { path: String, }, - /// A `dir://` SOURCE folder still carries live AACS-encrypted content. - DirImageEncrypted, /// A `dir://` SOURCE folder holds no recognized disc structure. DirImageUnsupportedTree, /// A file in a `dir://` SOURCE folder changed between plan and read. @@ -974,8 +961,6 @@ pub enum Error { WorkerLost { op: &'static str, }, - /// See [`E_MULTIPASS_REQUIRES_RAW`]. - MultipassRequiresRaw, /// `tag` is the caller's unresolvable language tag, verbatim. StreamLanguageUnknown { tag: String, @@ -1121,7 +1106,6 @@ impl Error { Error::PesTrackTooLarge { .. } => E_PES_TRACK_TOO_LARGE, Error::IsoTooLarge { .. } => E_ISO_TOO_LARGE, Error::NoMetadata => E_NO_METADATA, - Error::DiscUrlNotDirect => E_DISC_URL_NOT_DIRECT, Error::HevcParamParse => E_HEVC_PARAM_PARSE, Error::MuxTrackRange { .. } => E_MUX_TRACK_RANGE, Error::Fmp4Unimplemented => E_FMP4_UNIMPLEMENTED, @@ -1144,7 +1128,6 @@ impl Error { Error::DirWriteFailed { .. } => E_DIR_WRITE_FAILED, Error::DirImageSsifUnsupported => E_DIR_IMAGE_SSIF_UNSUPPORTED, Error::DirImagePlacement { .. } => E_DIR_IMAGE_PLACEMENT, - Error::DirImageEncrypted => E_DIR_IMAGE_ENCRYPTED, Error::DirImageUnsupportedTree => E_DIR_IMAGE_UNSUPPORTED_TREE, Error::DirNameTooLong { .. } => E_DIR_NAME_TOO_LONG, Error::DirImageFanout { .. } => E_DIR_IMAGE_FANOUT, @@ -1162,7 +1145,6 @@ impl Error { Error::RemuxStagingInvalid => E_REMUX_STAGING_INVALID, Error::StagedCopySizeMismatch { .. } => E_STAGED_COPY_SIZE_MISMATCH, Error::WorkerLost { .. } => E_WORKER_LOST, - Error::MultipassRequiresRaw => E_MULTIPASS_REQUIRES_RAW, Error::StreamLanguageUnknown { .. } => E_STREAM_LANGUAGE_UNKNOWN, Error::RemuxTargetExists { .. } => E_REMUX_TARGET_EXISTS, Error::MuxBatchSectorsZero => E_MUX_BATCH_SECTORS_ZERO, @@ -1426,9 +1408,6 @@ impl From for std::io::Error { 8000..=8999 => std::io::ErrorKind::Other, 9000..=9001 => std::io::ErrorKind::Unsupported, 9002..=9008 => std::io::ErrorKind::InvalidInput, - // 9009 DiscUrlNotDirect: structurally unsupported entry point, - // not a parse failure — caller used the wrong API. - 9009 => std::io::ErrorKind::Unsupported, // 9010 HevcParamParse: malformed hvcC payload. 9010 => std::io::ErrorKind::InvalidData, // 9011 MuxTrackRange: caller passed a bad track index. @@ -1497,7 +1476,6 @@ impl From for std::io::Error { // tree, too large, name too long, fan-out overflow) — input properties. E_DIR_IMAGE_SSIF_UNSUPPORTED | E_DIR_IMAGE_PLACEMENT - | E_DIR_IMAGE_ENCRYPTED | E_DIR_IMAGE_UNSUPPORTED_TREE | E_DIR_IMAGE_TOO_LARGE | E_DIR_NAME_TOO_LONG @@ -1509,10 +1487,9 @@ impl From for std::io::Error { E_TIMED_OUT => std::io::ErrorKind::TimedOut, // Remux/engine codes keep the kinds their io::Error texts carried. E_REMUX_VERIFY_FAILED | E_STAGED_COPY_SIZE_MISMATCH => std::io::ErrorKind::InvalidData, - E_REMUX_STAGING_INVALID - | E_MULTIPASS_REQUIRES_RAW - | E_STREAM_LANGUAGE_UNKNOWN - | E_MUX_BATCH_SECTORS_ZERO => std::io::ErrorKind::InvalidInput, + E_REMUX_STAGING_INVALID | E_STREAM_LANGUAGE_UNKNOWN | E_MUX_BATCH_SECTORS_ZERO => { + std::io::ErrorKind::InvalidInput + } E_REMUX_TARGET_EXISTS => std::io::ErrorKind::AlreadyExists, _ => std::io::ErrorKind::Other, }; @@ -1830,7 +1807,6 @@ mod tests { } .code(), Error::MapfileInvalid { kind: "hex" }.code(), - Error::DiscUrlNotDirect.code(), Error::ExtentNotUnitAligned.code(), Error::M2tsPacketMalformed.code(), Error::DiscCapacityMalformed.code(), @@ -1847,7 +1823,6 @@ mod tests { Error::DirWriteFailed { errno: Some(28) }.code(), Error::DirImageSsifUnsupported.code(), Error::DirImagePlacement { path: "x".into() }.code(), - Error::DirImageEncrypted.code(), Error::DirImageUnsupportedTree.code(), Error::DirImageFileChanged { path: "x".into() }.code(), Error::DirImageTooLarge.code(), @@ -1893,7 +1868,6 @@ mod tests { Error::RemuxStagingInvalid.code(), Error::StagedCopySizeMismatch { have: 0, want: 1 }.code(), Error::WorkerLost { op: "copy" }.code(), - Error::MultipassRequiresRaw.code(), Error::StreamLanguageUnknown { tag: "x".into() }.code(), Error::RemuxTargetExists { path: "x".into() }.code(), ]; @@ -1954,7 +1928,6 @@ mod tests { }, E_IMAGE_ENDS_BEFORE_READ, ), - (Error::DiscUrlNotDirect, E_DISC_URL_NOT_DIRECT), (Error::ExtentNotUnitAligned, E_EXTENT_NOT_UNIT_ALIGNED), // Both CSS no-key verdicts: numeric-only Display, no English. (Error::CssKeyMissing, E_CSS_KEY_MISSING), @@ -2105,8 +2078,6 @@ mod tests { mapped(Error::MapfileInvalid { kind: "hex" }), ErrorKind::InvalidData ); - // 9009 special-cased to Unsupported - assert_eq!(mapped(Error::DiscUrlNotDirect), ErrorKind::Unsupported); // 9021 special-cased to InvalidData assert_eq!(mapped(Error::M2tsPacketMalformed), ErrorKind::InvalidData); // 9047 DiscCapacityMalformed → InvalidData @@ -2321,7 +2292,6 @@ mod tests { (Error::StreamWriteOnly, E_STREAM_WRITE_ONLY), (Error::PesInvalidMagic, E_PES_INVALID_MAGIC), (Error::NoMetadata, E_NO_METADATA), - (Error::DiscUrlNotDirect, E_DISC_URL_NOT_DIRECT), (Error::HevcParamParse, E_HEVC_PARAM_PARSE), (Error::Fmp4Unimplemented, E_FMP4_UNIMPLEMENTED), (Error::DemuxThreadPanicked, E_DEMUX_THREAD_PANICKED), @@ -2543,12 +2513,6 @@ mod tests { "E9081: verify", ErrorKind::Other, ), - ( - Error::MultipassRequiresRaw, - 9082, - "E9082", - ErrorKind::InvalidInput, - ), ( Error::StreamLanguageUnknown { tag: "xx\nyy".into(), diff --git a/src/event.rs b/src/event.rs index b6dcbede..08e97a67 100644 --- a/src/event.rs +++ b/src/event.rs @@ -1,114 +1,36 @@ -//! Event system for progress and status reporting. +//! The one event vocabulary every stage reports through (pipeline design §2.5). //! -//! The lib fires events during operations like rip(). -//! The app registers a callback to receive them. -//! No display logic, no English text — just data. -//! -//! ```rust,ignore -//! disc.rip(&mut session, 0, output, |event| { -//! match event.kind { -//! EventKind::BytesRead { bytes, total } => update_progress(bytes, total), -//! EventKind::SectorSkipped { sector } => log_skip(sector), -//! EventKind::BatchSizeChanged { new_size, .. } => note_recovery(new_size), -//! _ => {} -//! } -//! }); -//! ``` -//! -//! Note: the library currently emits only `BytesRead`, `SectorSkipped`, -//! and `BatchSizeChanged`. The other [`EventKind`] variants are part of -//! the stable event vocabulary for consumers (and future emit sites) but -//! are not produced by the library today. +//! A stage emits through [`Ctx::events`](crate::Ctx); a consumer implements [`Events`] once +//! and matches the variants it renders. Data only: no display logic, no English text. -use crate::error::Error; +use crate::disc::DiscTitle; +use crate::progress::PassProgress; -/// An event fired by the lib during operations. -#[derive(Debug)] -pub struct Event { - pub kind: EventKind, -} - -/// Types of events the lib can fire. -#[derive(Debug)] -pub enum EventKind { - // ── Init sequence events ──────────────────────────────────────── - /// Drive opened successfully. - DriveOpened { device: String }, - - /// Drive is ready (disc spun up). - DriveReady, - - /// Firmware init completed. - InitComplete { success: bool }, - - /// Disc probe completed. - ProbeComplete { success: bool }, - - /// Disc scan completed. - ScanComplete { titles: usize }, - - // ── Read events ───────────────────────────────────────────────── - /// Bytes successfully read and written to output. - BytesRead { - /// Bytes written so far. - bytes: u64, - /// Total bytes expected (0 if unknown). - total: u64, - }, - - /// A read error occurred. The lib will retry automatically. - ReadError { - /// Sector that failed. - sector: u64, - /// Error code. - error: Error, - }, - - /// Retrying a failed read. - Retry { - /// Current attempt number (1-based). - attempt: u32, - }, - - /// Drive speed changed (error recovery or restoration). - SpeedChange { - /// New speed in KB/s (0xFFFF = max). - speed_kbs: u16, - }, - - /// Starting a new disc extent. - ExtentStart { - /// Extent index (0-based). - index: usize, - /// First sector of extent. - start_sector: u64, - /// Number of sectors in extent. - sector_count: u64, - }, - - /// Sector recovered after a retry (Drive::read multi-phase recovery). - SectorRecovered { sector: u64 }, - - /// Sector unreadable, zero-filled (skip mode). - SectorSkipped { sector: u64 }, - - /// Adaptive batch sizer changed the read size. - /// - /// Fires on shrink (read failed at larger size) and on probe-up - /// (enough clean reads to try larger again). Consumers use this to - /// display a "recovering" state distinct from "ripping normally". +/// What a stage reports while it runs. Borrowed: an [`Events`] impl copies out what it keeps. +#[derive(Debug, Clone, Copy)] +#[non_exhaustive] +pub enum Event<'a> { + /// Source bytes delivered so far by the read stage, of `total` planned (0 if unknown). + BytesRead { bytes: u64, total: u64 }, + /// A read error at `lba` was zero-filled and skipped (skip-errors policy). + SectorSkipped { lba: u64 }, + /// `units` damaged AACS units in the read starting at `lba` were blanked and counted. + UnitBlanked { lba: u64, units: u64 }, + /// The adaptive live read batch changed to `new_size` sectors. BatchSizeChanged { new_size: u16, reason: BatchSizeReason, }, - - /// Operation complete. - Complete { - /// Total bytes written. - bytes: u64, - /// Total read errors encountered. - errors: u32, - }, + /// A recovery, extract or image pass's running totals. + Pass(&'a PassProgress), + /// Bytes handed to the output so far, of the planned `total` (0 if unknown). + BytesWritten { bytes: u64, total: u64 }, + /// While the output is flushed at the end: bytes made durable so far. At most 4 per + /// second and none while nothing moves, so silence means a stalled flush (stop §4.5). + BytesDurable { bytes: u64, total: u64 }, + /// The output opened for `title` as it will be written: streams the sink refused are + /// left out (indices compact); `MuxOutcome::undelivered_streams` keeps source indices. + OutputOpened { title: &'a DiscTitle }, } /// Why the adaptive batch sizer changed size. @@ -120,26 +42,32 @@ pub enum BatchSizeReason { Probed, } -/// A no-op event handler. Ignores all events. -pub fn ignore(_event: Event) {} +/// A consumer of [`Event`]s. Called from stage threads (the read producer, the mux pump): +/// keep it cheap and non-blocking. +pub trait Events: Send + Sync { + /// One event. The default ignores it. + fn event(&self, _e: &Event<'_>) {} +} + +/// An [`Events`] that ignores everything. +#[derive(Debug, Clone, Copy, Default)] +pub struct NoEvents; + +impl Events for NoEvents {} + +impl) + Send + Sync> Events for F { + fn event(&self, e: &Event<'_>) { + self(e) + } +} #[cfg(test)] mod tests { use super::*; // Shrunk != Probed: distinct meanings (error vs. recovery), must not compare equal. - // Mutation: bad PartialEq derive could make distinct variants equal. #[test] fn batch_size_reason_variants_are_not_equal() { assert_ne!(BatchSizeReason::Shrunk, BatchSizeReason::Probed); } - - // Clone + Copy: cloning doesn't move; required since BatchSizeChanged embeds by value. - // Mutation: removing Copy would break callers that pass reason by value. - #[test] - fn batch_size_reason_is_copy() { - let r = BatchSizeReason::Shrunk; - let _r2 = r; // copy, not move - let _r3 = r; // r still usable after copy - } } diff --git a/src/halt.rs b/src/halt.rs index 42f6e8da..fc5de666 100644 --- a/src/halt.rs +++ b/src/halt.rs @@ -5,7 +5,7 @@ //! halt-aware primitive ([`Halt::wait`], [`Halt::recv_timeout`], [`Halt::send_timeout`], //! [`join_within`]) that re-checks the token at least every [`WAIT_SLICE`]. //! -//! Timeouts are stall-only (HR1): [`StallTimer`] fires only when a [`Progress`] +//! Timeouts are stall-only (HR1): [`StallTimer`] fires only when a [`Liveness`] //! counter has not moved for its window, never on total elapsed time. Every //! duration is a parameter at its use site, so tests run in milliseconds. @@ -338,10 +338,10 @@ impl TimedSend for crossbeam_channel::Sender { /// advanced, an item applied, or a response received. Cheap to clone; clones share /// the count. [`busy`](Self::busy) only marks in-flight work: it never bumps. #[derive(Clone, Debug, Default)] -pub struct Progress(Arc); +pub struct Liveness(Arc); #[derive(Debug, Default)] -struct ProgressInner { +struct LivenessInner { count: AtomicU64, // Live `BusyGuard`s, and every guard start or end (so a poll sees a span that // began and ended between two polls). @@ -349,7 +349,7 @@ struct ProgressInner { busy_epoch: AtomicU64, } -impl Progress { +impl Liveness { pub fn new() -> Self { Self::default() } @@ -387,10 +387,10 @@ impl Progress { } } -/// Holds a [`Progress`] busy until dropped. +/// Holds a [`Liveness`] busy until dropped. #[must_use = "the span is busy only while the guard lives"] #[derive(Debug)] -pub struct BusyGuard(Progress); +pub struct BusyGuard(Liveness); impl Drop for BusyGuard { fn drop(&mut self) { @@ -410,10 +410,10 @@ pub enum Stall { Expired, } -/// The HR1 primitive: fires only when a [`Progress`] has not moved for `window`. +/// The HR1 primitive: fires only when a [`Liveness`] has not moved for `window`. /// /// A plain timer ([`new`](Self::new)) counts all time without progress and ignores -/// [`Progress::busy`]; every HR1 failure timer is plain. [`idle_only`](Self::idle_only) +/// [`Liveness::busy`]; every HR1 failure timer is plain. [`idle_only`](Self::idle_only) /// is the opt-in T29 mode: time is not counted while the progress is busy. #[derive(Debug)] pub struct StallTimer { @@ -434,7 +434,7 @@ struct IdleClock { impl StallTimer { /// A plain stall timer: every HR1 failure timer is one of these. - pub fn new(window: Duration, p: &Progress) -> Self { + pub fn new(window: Duration, p: &Liveness) -> Self { Self { window, last: p.get(), @@ -443,9 +443,9 @@ impl StallTimer { } } - /// Opt-in (v5.5, ST3-5): paused while `p` is [`busy`](Progress::busy). The T29 + /// Opt-in (v5.5, ST3-5): paused while `p` is [`busy`](Liveness::busy). The T29 /// launch probe is its only user. - pub fn idle_only(window: Duration, p: &Progress) -> Self { + pub fn idle_only(window: Duration, p: &Liveness) -> Self { let mut t = Self::new(window, p); t.idle = Some(IdleClock { idle: Duration::ZERO, @@ -456,7 +456,7 @@ impl StallTimer { } /// Sample `p`. Any movement re-arms the whole window. - pub fn poll(&mut self, p: &Progress) -> Stall { + pub fn poll(&mut self, p: &Liveness) -> Stall { let now = Instant::now(); let count = p.get(); if count != self.last { diff --git a/src/halt/tests.rs b/src/halt/tests.rs index 9418dedc..15812f0a 100644 --- a/src/halt/tests.rs +++ b/src/halt/tests.rs @@ -349,7 +349,7 @@ fn join_within_finished_cancelled_and_expired() { /// never expires. #[test] fn stall_timer_rearms_on_progress() { - let p = Progress::new(); + let p = Liveness::new(); let mut t = StallTimer::new(WINDOW, &p); let start = Instant::now(); let mut next_bump = start + WINDOW / 5; @@ -368,7 +368,7 @@ fn stall_timer_rearms_on_progress() { /// window + slack; never before the window. #[test] fn stall_timer_expires_without_progress() { - let p = Progress::new(); + let p = Liveness::new(); p.bump(); let start = Instant::now(); let mut t = StallTimer::new(WINDOW, &p); @@ -392,10 +392,10 @@ fn stall_timer_expires_without_progress() { } /// LT5c (ST3-5): with a live `busy()` guard an `idle_only` timer does not expire, -/// while a plain timer on the same `Progress` expires at its window. +/// while a plain timer on the same `Liveness` expires at its window. #[test] fn busy_pauses_only_idle_only_timers() { - let p = Progress::new(); + let p = Liveness::new(); let mut plain = StallTimer::new(WINDOW, &p); let mut idle = StallTimer::idle_only(WINDOW, &p); let guard = p.busy(); @@ -432,7 +432,7 @@ fn busy_pauses_only_idle_only_timers() { /// A busy span that starts and ends between two polls counts as busy, not idle. #[test] fn idle_only_ignores_a_busy_span_between_polls() { - let p = Progress::new(); + let p = Liveness::new(); let mut idle = StallTimer::idle_only(WINDOW, &p); let g = p.busy(); std::thread::sleep(WINDOW * 2); diff --git a/src/io/artifact_lock.rs b/src/io/artifact_lock.rs index f3c66ab2..b603afde 100644 --- a/src/io/artifact_lock.rs +++ b/src/io/artifact_lock.rs @@ -14,7 +14,7 @@ //! (E9073) only after 30 s in which neither the `.partial` nor a watched file changed. use crate::error::{Error, Result}; -use crate::halt::{Halt, Progress, Stall, StallTimer, WAIT_SLICE}; +use crate::halt::{Halt, Liveness, Stall, StallTimer, WAIT_SLICE}; use std::fs::File; use std::io; use std::path::{Path, PathBuf}; @@ -144,7 +144,7 @@ type Ids<'a> = dyn Fn(&File, &Path) -> (io::Result<(u64, u64)>, io::Result<(u64, struct LockWait { watched: Vec, seen: Vec)>>, - progress: Progress, + progress: Liveness, timer: StallTimer, } @@ -153,7 +153,7 @@ impl LockWait { let mut watched = vec![with_suffix(final_path, ".partial")]; watched.extend(watch.iter().map(|p| p.to_path_buf())); let seen = signatures(&watched); - let progress = Progress::new(); + let progress = Liveness::new(); let timer = StallTimer::new(window, &progress); LockWait { watched, diff --git a/src/io/block_sink.rs b/src/io/block_sink.rs new file mode 100644 index 00000000..4e7865c6 --- /dev/null +++ b/src/io/block_sink.rs @@ -0,0 +1,185 @@ +//! Block sinks (pipeline design §2.2, slice 7): where a whole-disc copy's sectors go. A PES +//! sink takes frames; a block sink takes sector-addressed chunks — `iso://` writes a +//! sector-exact image, `null://` discards (a read test of the whole disc). [`open_block_sink`] +//! opens either from a URL, as [`crate::output`] opens a PES sink. + +use crate::consts::SECTOR_BYTES; +use crate::error::{Error, Result}; +use std::fs::File; +use std::io::{BufWriter, Seek, SeekFrom, Write}; +use std::path::{Path, PathBuf}; + +/// The platform's null device: the path a whole-disc copy to `null://` writes to, so the +/// copy runs (and its mapfile records the read) with nothing kept. `/dev/null` on Unix, +/// `NUL` on Windows. +pub fn null_device() -> &'static Path { + if cfg!(windows) { + Path::new("NUL") + } else { + Path::new("/dev/null") + } +} + +/// Whether `path` is the platform's null device (see [`null_device`]). +pub fn is_null_device(path: &Path) -> bool { + path == null_device() || path.as_os_str() == "/dev/null" +} + +/// How a block sink ends. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Finish { + /// Every sector was written: flush and make the output durable. + Complete, + /// The copy stopped or failed early: an image keeps its partial bytes (a resumable + /// copy continues it); nothing is made durable. + Incomplete, +} + +/// A sector-addressed output: `iso://` or `null://`. +pub trait BlockSink: Send { + /// Write whole sectors at sector `lba`. + fn write_at(&mut self, lba: u64, bytes: &[u8]) -> Result<()>; + /// End the output; returns the bytes written. + fn finish(self: Box, how: Finish) -> Result; +} + +/// A sector-exact image file. Sequential writes stream through a buffer; a write at another +/// sector seeks there first. +pub struct IsoSink { + out: BufWriter, + path: PathBuf, + // The next sector the file position is at. + at: u64, + written: u64, + // The parent-directory fsync (a seam so tests can make it fail). + sync_dir: fn(&Path) -> std::io::Result<()>, +} + +impl IsoSink { + /// Create (truncate) the image at `path`. + pub fn create(path: &Path) -> Result { + let file = File::create(path).map_err(|source| Error::IoError { source })?; + Ok(IsoSink { + out: BufWriter::with_capacity(4 * 1024 * 1024, file), + path: path.to_path_buf(), + at: 0, + written: 0, + sync_dir: crate::io::fsync::dir_checked, + }) + } + + #[cfg(test)] + pub(crate) fn with_dir_sync(mut self, f: fn(&Path) -> std::io::Result<()>) -> Self { + self.sync_dir = f; + self + } +} + +impl BlockSink for IsoSink { + fn write_at(&mut self, lba: u64, bytes: &[u8]) -> Result<()> { + let io = |source| Error::IoError { source }; + if lba != self.at { + self.out + .seek(SeekFrom::Start(lba * SECTOR_BYTES as u64)) + .map_err(io)?; + } + self.out.write_all(bytes).map_err(io)?; + self.at = lba + (bytes.len() / SECTOR_BYTES) as u64; + self.written += bytes.len() as u64; + Ok(()) + } + + fn finish(self: Box, how: Finish) -> Result { + // `into_inner` (not `flush`) also surfaces buffered-write errors. + let file = self.out.into_inner().map_err(|e| Error::IoError { + source: e.into_error(), + })?; + if how == Finish::Complete { + // flush() only pushes bytes into the kernel: a crash or yanked volume could + // leave a truncated file reported as complete. + file.sync_all() + .map_err(|source| Error::IoError { source })?; + // The file was just created: its directory entry needs its own fsync. + let dir = match self.path.parent() { + Some(dir) if !dir.as_os_str().is_empty() => dir, + _ => Path::new("."), + }; + (self.sync_dir)(dir).map_err(|source| Error::IoError { source })?; + } + Ok(self.written) + } +} + +/// `null://`: discards every sector, counting them. +#[derive(Default)] +pub struct NullBlockSink { + written: u64, +} + +impl BlockSink for NullBlockSink { + fn write_at(&mut self, _lba: u64, bytes: &[u8]) -> Result<()> { + self.written += bytes.len() as u64; + Ok(()) + } + + fn finish(self: Box, _how: Finish) -> Result { + Ok(self.written) + } +} + +/// Open the block output `url`: `iso://` (an image, created now) or `null://`. +/// `dir://` is the folder output of a whole-disc extraction ([`crate::Disc::extract_tree`]); +/// a PES output scheme is refused with [`Error::StreamWriteOnly`]'s sibling +/// [`Error::StreamUrlInvalid`]. +pub fn open_block_sink(url: &str) -> Result> { + match crate::mux::parse_url(url) { + crate::mux::StreamUrl::Iso { path } => { + crate::mux::resolve::validate_path(&path, "iso")?; + Ok(Box::new(IsoSink::create(&path)?)) + } + crate::mux::StreamUrl::Null => Ok(Box::new(NullBlockSink::default())), + _ => Err(Error::StreamUrlInvalid { + url: url.to_string(), + }), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn an_iso_sink_writes_sectors_where_addressed() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("o.iso"); + let mut sink = open_block_sink(&format!("iso://{}", path.display())).unwrap(); + sink.write_at(0, &[1u8; 2048]).unwrap(); + sink.write_at(2, &[3u8; 2048]).unwrap(); + sink.write_at(1, &[2u8; 2048]).unwrap(); + assert_eq!(sink.finish(Finish::Complete).unwrap(), 3 * 2048); + let got = std::fs::read(&path).unwrap(); + assert_eq!(got.len(), 3 * 2048); + assert!(got[..2048].iter().all(|&b| b == 1)); + assert!(got[2048..4096].iter().all(|&b| b == 2)); + assert!(got[4096..].iter().all(|&b| b == 3)); + } + + #[test] + fn null_discards_and_counts() { + let mut sink = open_block_sink("null://").unwrap(); + sink.write_at(7, &[0u8; 4096]).unwrap(); + assert_eq!(sink.finish(Finish::Complete).unwrap(), 4096); + } + + #[test] + fn a_pes_scheme_is_not_a_block_sink() { + assert!(open_block_sink("mkv:///tmp/x.mkv").is_err()); + } + + #[test] + fn the_null_device_is_recognised() { + assert!(is_null_device(null_device())); + assert!(is_null_device(Path::new("/dev/null"))); + assert!(!is_null_device(Path::new("/tmp/x.iso"))); + } +} diff --git a/src/io/bounded.rs b/src/io/bounded.rs index 80cade48..ae70e382 100644 --- a/src/io/bounded.rs +++ b/src/io/bounded.rs @@ -1,6 +1,6 @@ //! Bounded-syscall primitive: run a (potentially-blocking) operation on a worker thread //! while the caller waits halt-aware, bounded by a deadline (`bounded_syscall`, one -//! call's "no answer" bound) or by a stall window over a [`Progress`] +//! call's "no answer" bound) or by a stall window over a [`Liveness`] //! ([`bounded_syscall_stall`], HR1). The calling thread is never trapped in a kernel call. //! //! Escape hatch for syscalls a cooperative [`Halt`] can't interrupt (`sync_file_range`, @@ -11,7 +11,7 @@ use std::thread; #[cfg(any(target_os = "linux", test))] use std::time::Duration; -use crate::halt::{Halt, Progress, Recv, Stall, StallTimer, WAIT_SLICE}; +use crate::halt::{Halt, Liveness, Recv, Stall, StallTimer, WAIT_SLICE}; /// Failure outcome from a bounded syscall wrapper. #[derive(Debug)] @@ -83,7 +83,7 @@ where // on `progress` (HR1: no progress for its window, never elapsed time). pub(crate) fn bounded_syscall_stall( halt: Option<&Halt>, - progress: &Progress, + progress: &Liveness, timer: &mut StallTimer, tick: &mut dyn FnMut(), op: F, @@ -262,7 +262,7 @@ mod tests { /// A panicking worker is `WorkerLost` in the stall variant, not a stall `Timeout`. #[test] fn stall_variant_reports_worker_lost_on_panic() { - let p = Progress::new(); + let p = Liveness::new(); let mut timer = StallTimer::new(Duration::from_secs(5), &p); let r = bounded_syscall_stall(None, &p, &mut timer, &mut || {}, || -> u8 { panic!("intentional test panic"); @@ -275,7 +275,7 @@ mod tests { #[test] fn stall_bound_rearms_on_progress_and_expires_without() { let w = Duration::from_millis(100); - let p = Progress::new(); + let p = Liveness::new(); let mut timer = StallTimer::new(w, &p); let bump = p.clone(); let r = bounded_syscall_stall(None, &p, &mut timer, &mut || bump.bump(), || { diff --git a/src/io/byte_prefetcher.rs b/src/io/byte_prefetcher.rs deleted file mode 100644 index 8cc02f3e..00000000 --- a/src/io/byte_prefetcher.rs +++ /dev/null @@ -1,660 +0,0 @@ -//! `BytePrefetcher` — `std::io::Read` analogue of [`crate::sector::PrefetchedSectorSource`]. -//! Spawns a producer thread that fills a bounded pool of `Vec` chunks from the underlying -//! reader and ships them through a channel; the consumer recycles emptied buffers back so the -//! producer re-fills in place, for zero allocations and zero cross-thread frees in the hot -//! loop. Works for any stream whose source is an `io::Read`, not just a `SectorSource`. - -use crate::halt::{Halt, Joined, Recv, SendOutcome, join_within}; -use crossbeam_channel::{Receiver, Sender, bounded}; -use std::io::Read; -use std::thread::JoinHandle; -use std::time::Duration; - -/// Items flowing through the forward channel. -pub type Batch = std::io::Result>; - -/// Forward channel depth — how many filled buffers the producer can -/// stay ahead by. Two is enough to absorb a moderate consumer stall -/// without piling up bytes. -const FORWARD_DEPTH: usize = 2; - -/// Recycle channel depth = forward + 1 so the producer always has at -/// least one buffer to fill while the consumer holds one. -const RECYCLE_DEPTH: usize = FORWARD_DEPTH + 1; - -/// Default chunk size — 16 MiB matches the ISO-mux sector batch and -/// is large enough that per-chunk overhead is amortised; small -/// enough that the in-flight memory footprint stays bounded. -pub const DEFAULT_CHUNK_BYTES: usize = 16 * 1024 * 1024; - -/// Returned from [`BytePrefetcher::into_channels`]. Owns the -/// producer-thread join handle so dropping the shell joins the -/// producer. -/// -/// Drop waits a bounded grace for the producer, then detaches it (a `read()` that never -/// returns cannot be interrupted). For a prompt exit, drop the forward receiver and recycle -/// sender first, or cancel the [`Halt`] passed to [`BytePrefetcher::new`]. -pub struct PrefetchShell { - producer: Option>, -} - -// How long Drop waits for the producer before detaching it. -const DROP_GRACE: Duration = if cfg!(test) { - Duration::from_millis(500) -} else { - Duration::from_secs(5) -}; - -// Join the producer within `DROP_GRACE`; a producer still blocked in `read()` is detached. -fn join_or_detach(h: JoinHandle<()>) { - if !matches!(join_within(h, DROP_GRACE, None), Joined::Done(_)) { - tracing::warn!(target: "freemkv::io", "byte prefetch producer blocked in read; detached"); - } -} - -impl Drop for PrefetchShell { - fn drop(&mut self) { - if let Some(h) = self.producer.take() { - join_or_detach(h); - } - } -} - -/// Spawned byte prefetcher. Drop joins the producer thread (bounded; see [`PrefetchShell`]). -pub struct BytePrefetcher { - // Non-`Option`: `into_channels`/`Drop` swap in a disconnected stand-in (dead-channel - // `mem::replace`, as `sector::PrefetchedSectorSource` does), so a missing `rx` can - // never read as a silent, truncating clean EOF. - rx: Receiver, - recycle_tx: Sender>, - producer: Option>, -} - -impl BytePrefetcher { - /// Spawn the producer thread. `reader` must be `Send` because it - /// moves into the thread. `chunk_bytes` is the size of each - /// recycled buffer; pick the natural batch size of the - /// downstream demuxer (16 MiB for the BD-TS mux pipeline). - pub fn new( - mut reader: R, - chunk_bytes: usize, - halt: Option, - ) -> std::io::Result { - // A zero-length chunk would read Ok(0) at once: a silent empty stream. - if chunk_bytes == 0 { - return Err(std::io::ErrorKind::InvalidInput.into()); - } - let (tx, rx) = bounded::(FORWARD_DEPTH); - let (recycle_tx, recycle_rx) = bounded::>(RECYCLE_DEPTH); - - // Seed the recycle pool; otherwise the first `recycle_rx.recv()` blocks - // forever since no consumer has returned a buffer yet. - for _ in 0..RECYCLE_DEPTH { - let _ = recycle_tx.send(vec![0u8; chunk_bytes]); - } - - // A never-cancelled stand-in keeps one halt-aware code path without a token. - let wait = halt.unwrap_or_default(); - let producer = std::thread::Builder::new() - .name("freemkv-byte-prefetch".into()) - .spawn(move || { - // catch_unwind: a clean exit drops `tx` (demux reads RecvError as EOF), - // but a panic sends an error sentinel first so demux gets a typed error - // instead of finalizing a truncated mux as success. - let body = std::panic::AssertUnwindSafe(|| { - // Liveness heartbeat: a stalled consumer or wedged reader shows up - // as the beat going silent. Total is unknown, so `pos` is cumulative. - let mut hb = crate::progress::Heartbeat::new("byte_prefetch"); - let mut produced_bytes: u64 = 0; - loop { - hb.tick(produced_bytes, 0); - // Halt-aware: a cancel does not disconnect the channel, so a - // plain recv() would never re-reach the check. Disconnected = - // the consumer dropped both channels. - let Ok(Recv::Item(mut buf)) = wait.recv_timeout(&recycle_rx, Duration::MAX) - else { - return; - }; - // Regrow to chunk_bytes (a short read may have truncated len): safe - // resize, was `unsafe set_len` (GHSA-j8ww-f5fg-9pmh in `sector::prefetched`). - buf.resize(chunk_bytes, 0); - // Short reads are valid: truncate so the consumer sees only what arrived. - let n = loop { - match reader.read(&mut buf[..]) { - Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {} - r => break r, - } - }; - let n = match n { - Ok(0) => return, // EOF — drop tx, consumer sees RecvError - Ok(n) => n, - Err(e) => { - let _ = wait.send_timeout(&tx, Err(e), Duration::MAX); - return; - } - }; - produced_bytes += n as u64; - buf.truncate(n); - // Hand off the filled buffer, re-polling halt on - // each timeout slice so a cancel can interrupt a - // producer parked on a saturated forward channel. - let sent = wait.send_timeout(&tx, Ok(buf), Duration::MAX); - if !matches!(sent, Ok(SendOutcome::Sent)) { - return; // consumer dropped, or stopped - } - } - }); - if std::panic::catch_unwind(body).is_err() { - // Producer panicked mid-stream — surface a typed terminal - // error so the demux thread does NOT read the dropped channel - // as a clean EOF and truncate output. - let e = crate::error::Error::DemuxThreadPanicked.into(); - let _ = wait.send_timeout(&tx, Err(e), Duration::MAX); - } - })?; - - Ok(Self { - rx, - recycle_tx, - producer: Some(producer), - }) - } - - /// Peel off the channels for zero-copy pipeline consumption. The - /// caller (typically [`crate::mux::demux_thread::DemuxThread`]) - /// drains `rx`, runs the demuxer in place on each filled buffer, - /// and recycles back through `recycle_tx`. - pub fn into_channels(mut self) -> (Receiver, Sender>, PrefetchShell) { - // Same dead-channel swap as `Drop for BytePrefetcher` below: `self` is left - // holding only disconnected placeholders — no `unsafe`, no double-drop, and (being - // non-`Option`) no fallback branch that could hand back a dead `rx`. - let (dead_tx, dead_rx) = bounded::(0); - drop(dead_tx); - let rx = std::mem::replace(&mut self.rx, dead_rx); - let (dead_send, dead_recv) = bounded::>(0); - drop(dead_recv); - let recycle = std::mem::replace(&mut self.recycle_tx, dead_send); - let producer = self.producer.take(); - // `self` drops here: dead endpoints no-op, `producer` is `None` (no join). - (rx, recycle, PrefetchShell { producer }) - } -} - -impl Drop for BytePrefetcher { - fn drop(&mut self) { - // Drop channel endpoints BEFORE joining so the producer observes Disconnected - // (send/recv) and exits promptly. Otherwise a non-EOF source fills the forward - // channel and spins in send_timeout forever since rx is never drained. - let (dead_tx, dead_rx) = bounded::(0); - drop(dead_tx); - drop(std::mem::replace(&mut self.rx, dead_rx)); - let (dead_send, dead_recv) = bounded::>(0); - drop(dead_recv); - drop(std::mem::replace(&mut self.recycle_tx, dead_send)); - if let Some(h) = self.producer.take() { - join_or_detach(h); - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - // RECYCLE_DEPTH must be one MORE than FORWARD_DEPTH so the producer always has a spare - // buffer while the consumer holds the rest in flight. - #[test] - fn recycle_depth_is_forward_depth_plus_one() { - assert_eq!(RECYCLE_DEPTH, FORWARD_DEPTH + 1); - assert_eq!(RECYCLE_DEPTH, 3, "FORWARD_DEPTH is 2, so recycle must be 3"); - } - - // Pins DEFAULT_CHUNK_BYTES (16 MiB) as a literal so a mutation on - // 16 * 1024 * 1024 is caught by a concrete expected value instead - // of by recomputing the same expression. - #[test] - fn default_chunk_bytes_is_16_mib() { - assert_eq!(DEFAULT_CHUNK_BYTES, 16_777_216, "documented as 16 MiB"); - } - - // Endless reader: every read fills the buffer and never hits EOF, so - // the producer keeps pushing until the forward channel disconnects — - // the shape that wedged the pre-1.0.0 clone+mem::forget into_channels. - struct EndlessReader; - impl Read for EndlessReader { - fn read(&mut self, buf: &mut [u8]) -> std::io::Result { - buf.fill(0); - Ok(buf.len()) - } - } - - /// Run `f` on a helper thread and fail if it does not finish within - /// `secs`. Turns a join-deadlock into a test failure instead of a - /// hung CI run. - fn within(secs: u64, f: F) { - let (done_tx, done_rx) = bounded::<()>(1); - std::thread::spawn(move || { - f(); - let _ = done_tx.send(()); - }); - assert!( - done_rx - .recv_timeout(std::time::Duration::from_secs(secs)) - .is_ok(), - "operation did not complete within {secs}s (deadlock)" - ); - } - - // A reader blocked forever in read() cannot be interrupted: both Drops must return - // after the grace (detaching the producer) instead of wedging the dropping thread. - #[test] - fn drop_detaches_producer_blocked_in_read() { - struct Blocked(crossbeam_channel::Receiver<()>); - impl Read for Blocked { - fn read(&mut self, _: &mut [u8]) -> std::io::Result { - let _ = self.0.recv(); - Ok(0) - } - } - let (release, blocked) = bounded::<()>(0); - let pf = BytePrefetcher::new(Blocked(blocked.clone()), 8, None).expect("spawn"); - within(5, move || drop(pf)); - let (_rx, _recycle, shell) = BytePrefetcher::new(Blocked(blocked), 8, None) - .expect("spawn") - .into_channels(); - within(5, move || drop(shell)); - drop(release); - } - - // CRITICAL regression: dropping the forward receiver + recycle sender after into_channels - // must let the producer see disconnection and exit, so dropping PrefetchShell (join) - // returns promptly. - #[test] - fn into_channels_drop_releases_producer() { - within(10, || { - // Small chunk so the producer cycles quickly and fills the - // forward channel without allocating much. - let pf = BytePrefetcher::new(EndlessReader, 4096, None).expect("spawn"); - let (rx, recycle_tx, shell) = pf.into_channels(); - // Consumer goes away early (halt / abort analogue): drop - // both channel endpoints without draining to EOF. - drop(rx); - drop(recycle_tx); - // Joining the producer must not hang. - drop(shell); - }); - } - - // L106 (safe `Option::take` rewrite, no `unsafe`): `shell` drop must join within the - // bound below (a leaked handle hangs it). No-double-drop is structural: an explicit - // second `Drop::drop` call is a compile error (E0040), so it can't be attempted. - #[test] - fn into_channels_then_drop_leaks_nothing() { - within(10, || { - let pf = BytePrefetcher::new(EndlessReader, 4096, None).expect("spawn"); - let (rx, recycle_tx, shell) = pf.into_channels(); - drop(rx); - drop(recycle_tx); - drop(shell); // joins: a leaked/duplicated handle would hang this within(10, ..). - }); - } - - // Channels handed back by `into_channels` must still carry real bytes end-to-end — the - // safe rewrite must not have handed back the dead placeholder channels by mistake. - #[test] - fn into_channels_channels_still_deliver_bytes() { - within(10, || { - let src = vec![7u8; 10_000]; - let pf = BytePrefetcher::new(Cursor::new(src.clone()), 4096, None).expect("spawn"); - let (out, err) = drain_to_vec(pf); - assert!(err.is_none(), "no read error expected: {err:?}"); - assert_eq!( - out, src, - "all bytes delivered, byte-identical, after into_channels" - ); - }); - } - - /// Same property via the halt path: cancel the token, then the - /// producer must exit and the shell join must complete. - #[test] - fn halt_releases_producer() { - within(10, || { - let halt = Halt::new(); - let pf = BytePrefetcher::new(EndlessReader, 4096, Some(halt.clone())).expect("spawn"); - let (_rx, _recycle_tx, shell) = pf.into_channels(); - halt.cancel(); - drop(shell); - }); - } - - // ── Added hardening tests ─────────────────────────────────────── - - use std::io::Cursor; - - // Drain the forward channel, recycling every buffer, and reassemble - // the bytes. Stops on RecvError (EOF) or the first Err batch - // (returned separately). - fn drain_to_vec(pf: BytePrefetcher) -> (Vec, Option) { - let (rx, recycle_tx, shell) = pf.into_channels(); - let mut out = Vec::new(); - let mut err = None; - while let Ok(batch) = rx.recv() { - match batch { - Ok(buf) => { - out.extend_from_slice(&buf); - // Recycle so the producer can refill. Ignore send - // error (producer may have already exited at EOF). - let _ = recycle_tx.send(buf); - } - Err(e) => { - err = Some(e); - break; - } - } - } - drop(rx); - drop(recycle_tx); - drop(shell); - (out, err) - } - - // CORE CONTRACT: every source byte delivered in order, exactly once. 5000-byte source, 1024 - // chunk size forces multiple chunks. - #[test] - fn delivers_all_bytes_in_order_across_chunks() { - within(10, || { - let src: Vec = (0..5000u32).map(|i| (i & 0xff) as u8).collect(); - let pf = BytePrefetcher::new(Cursor::new(src.clone()), 1024, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert!(err.is_none(), "unexpected error batch: {err:?}"); - assert_eq!(got, src, "prefetcher truncated or reordered bytes"); - }); - } - - // Short-read truncation: a reader returning fewer bytes than requested must not leave stale - // tail bytes; 10-byte source with a 4096 chunk must yield exactly 10 bytes. - #[test] - fn short_read_truncates_to_actual_length() { - within(10, || { - let src = vec![0xAB; 10]; - let pf = BytePrefetcher::new(Cursor::new(src.clone()), 4096, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert!(err.is_none()); - assert_eq!(got.len(), 10, "delivered chunk padded past actual read"); - assert_eq!(got, src); - }); - } - - // EOF semantics: an empty source yields read() == Ok(0) on the first - // call, which the producer treats as EOF; consumer sees RecvError - // (zero batches), not an Err or zero-length Ok batch (see docs). - #[test] - fn empty_source_yields_clean_eof_no_batches() { - within(10, || { - let pf = BytePrefetcher::new(Cursor::new(Vec::::new()), 4096, None).expect("spawn"); - let (rx, recycle_tx, shell) = pf.into_channels(); - // No Ok batch should ever arrive; first recv must be Err - // (producer dropped tx at EOF). - let first = rx.recv(); - assert!( - first.is_err(), - "empty source produced a batch instead of clean EOF: {first:?}" - ); - drop(rx); - drop(recycle_tx); - drop(shell); - }); - } - - // Error propagation: a reader that fails mid-stream must surface the io::Error as an Err - // batch, not swallow it — one good chunk then the error. - #[test] - fn read_error_is_propagated_as_err_batch() { - within(10, || { - struct OneThenError { - served: bool, - } - impl Read for OneThenError { - fn read(&mut self, buf: &mut [u8]) -> std::io::Result { - if !self.served { - self.served = true; - let n = buf.len().min(8); - buf[..n].fill(0x11); - Ok(n) - } else { - Err(std::io::Error::other("synthetic mid-stream read failure")) - } - } - } - let pf = BytePrefetcher::new(OneThenError { served: false }, 8, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert_eq!(got, vec![0x11; 8], "good chunk lost"); - let err = err.expect("read error must surface as an Err batch"); - assert_eq!(err.kind(), std::io::ErrorKind::Other); - }); - } - - // PANIC propagation: a reader that PANICS mid-stream must not read as a clean EOF; - // catch_unwind sends an explicit Err sentinel first. - #[test] - fn read_panic_surfaces_as_err_batch_not_clean_eof() { - within(10, || { - struct OneThenPanic { - served: bool, - } - impl Read for OneThenPanic { - fn read(&mut self, buf: &mut [u8]) -> std::io::Result { - if !self.served { - self.served = true; - let n = buf.len().min(8); - buf[..n].fill(0x22); - Ok(n) - } else { - panic!("synthetic mid-stream reader panic"); - } - } - } - let pf = BytePrefetcher::new(OneThenPanic { served: false }, 8, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert_eq!(got, vec![0x22; 8], "good chunk lost before the panic"); - assert!( - err.is_some(), - "a mid-stream producer PANIC must surface as an Err batch, \ - not a clean EOF (which would silently truncate the mux)" - ); - }); - } - - // Exercises the regrow-fill's now-plain `resize` when len is already chunk_bytes (was - // `unsafe set_len`): three full 8-byte chunks back to back, no short read in between. - #[test] - fn full_chunks_back_to_back_exercise_the_no_shrink_resize_path() { - within(10, || { - let src = vec![0x5Cu8; 24]; // 3 whole 8-byte chunks, source ends exactly on a chunk - let pf = BytePrefetcher::new(Cursor::new(src.clone()), 8, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert!(err.is_none()); - assert_eq!( - got, src, - "back-to-back full-size chunks must round-trip exactly" - ); - }); - } - - // Recycle-buffer reuse must not leak stale bytes between chunks of different lengths; - // source 8×0xAA + 3×0xBB, chunk_bytes=8. - #[test] - fn recycled_buffer_carries_no_stale_tail() { - within(10, || { - let mut src = vec![0xAA; 8]; - src.extend_from_slice(&[0xBB; 3]); - let pf = BytePrefetcher::new(Cursor::new(src.clone()), 8, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert!(err.is_none()); - assert_eq!( - got, src, - "stale bytes from recycled buffer leaked into short chunk" - ); - }); - } - - // Exact-multiple boundary: source length an exact multiple of - // chunk_bytes must yield Ok(0) EOF after the last chunk, never a - // spurious empty Ok batch. 12 bytes, chunk_bytes=4 → 3 chunks. - #[test] - fn exact_multiple_length_no_trailing_empty_batch() { - within(10, || { - let src = vec![0x42u8; 12]; - let pf = BytePrefetcher::new(Cursor::new(src.clone()), 4, None).expect("spawn"); - let (rx, recycle_tx, shell) = pf.into_channels(); - let mut total = 0usize; - let mut batch_count = 0usize; - while let Ok(Ok(buf)) = rx.recv() { - assert!(!buf.is_empty(), "producer emitted a zero-length batch"); - total += buf.len(); - batch_count += 1; - let _ = recycle_tx.send(buf); - } - assert_eq!(total, 12); - assert_eq!(batch_count, 3, "expected exactly 3 full chunks"); - drop(rx); - drop(recycle_tx); - drop(shell); - }); - } - - #[test] - fn zero_chunk_is_invalid_input() { - let e = BytePrefetcher::new(Cursor::new(vec![1u8; 4]), 0, None) - .err() - .expect("zero chunk must be rejected"); - assert_eq!(e.kind(), std::io::ErrorKind::InvalidInput); - } - - // EINTR from the reader is retried, not surfaced as a terminal error. - #[test] - fn interrupted_read_is_retried() { - within(10, || { - struct Flaky(u8); - impl Read for Flaky { - fn read(&mut self, buf: &mut [u8]) -> std::io::Result { - self.0 += 1; - match self.0 { - 1 => Err(std::io::ErrorKind::Interrupted.into()), - 2 => { - buf[..3].fill(9); - Ok(3) - } - _ => Ok(0), - } - } - } - let pf = BytePrefetcher::new(Flaky(0), 8, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert!(err.is_none(), "EINTR surfaced: {err:?}"); - assert_eq!(got, vec![9; 3]); - }); - } - - // Dropping BytePrefetcher directly (without into_channels) must join the producer cleanly - // on a finite source: EOF drops tx, Drop's join returns. - #[test] - fn drop_finite_prefetcher_joins_cleanly() { - within(10, || { - let pf = BytePrefetcher::new(Cursor::new(vec![1u8; 100]), 4096, None).expect("spawn"); - // Drop without consuming — producer fills the forward - // channel (capacity 2), reaches EOF on the third read since - // 100 < 4096 (single chunk + EOF), drops tx, exits. - drop(pf); - }); - } - - // Regression: dropping a BytePrefetcher directly with an ENDLESS source must not deadlock. - // Fix drops rx+recycle_tx BEFORE the join. - #[test] - fn drop_endless_prefetcher_joins_cleanly() { - within(10, || { - let pf = BytePrefetcher::new(EndlessReader, 4096, None).expect("spawn"); - // Drop without consuming — the old Drop deadlocked here. - drop(pf); - }); - } - - // A cancel must release a producer parked on the recycle channel (consumer holds every - // buffer). Producer exit shows as the forward channel disconnecting. - #[test] - fn halt_releases_producer_parked_on_recycle() { - let halt = Halt::new(); - let pf = BytePrefetcher::new(EndlessReader, 64, Some(halt.clone())).expect("spawn"); - let (rx, recycle_tx, shell) = pf.into_channels(); - // Take all RECYCLE_DEPTH buffers and never return them. - let held: Vec<_> = (0..RECYCLE_DEPTH) - .map(|_| rx.recv().expect("batch").expect("read ok")) - .collect(); - halt.cancel(); - let r = rx.recv_timeout(Duration::from_secs(5)); - assert!( - matches!(r, Err(crossbeam_channel::RecvTimeoutError::Disconnected)), - "producer still parked on recycle after cancel: {r:?}" - ); - drop((held, recycle_tx, shell)); - } - - // A buffer truncated by a short read must be regrown before its next read, or the chunk - // size shrinks for good. Records the buffer length each read() is offered. - #[test] - fn truncated_buffer_is_regrown_when_reused() { - use std::sync::{Arc, Mutex}; - struct ShortReads { - left: u8, - lens: Arc>>, - } - impl Read for ShortReads { - fn read(&mut self, buf: &mut [u8]) -> std::io::Result { - self.lens.lock().unwrap().push(buf.len()); - if self.left == 0 { - return Ok(0); - } - self.left -= 1; - buf[..3].fill(1); - Ok(3) - } - } - within(10, || { - let lens = Arc::new(Mutex::new(Vec::new())); - // 6 short reads over 3 buffers: every buffer is truncated, then reused. - let r = ShortReads { - left: 6, - lens: lens.clone(), - }; - let pf = BytePrefetcher::new(r, 8, None).expect("spawn"); - let (got, err) = drain_to_vec(pf); - assert!(err.is_none()); - assert_eq!(got.len(), 18); - let lens = lens.lock().unwrap(); - assert_eq!(lens.len(), 7); - assert!(lens.iter().all(|&l| l == 8), "buffer not regrown: {lens:?}"); - }); - } - - /// LP9 (×2, §2.1 "Unbounded Drop joins stay plain joins"): after a cancel, dropping - /// the prefetcher, or its shell with both channel ends still held, returns within - /// 1 s. Guard. - #[test] - fn byte_prefetcher_drop_after_cancel_returns() { - let halt = Halt::new(); - let pf = BytePrefetcher::new(EndlessReader, 4096, Some(halt.clone())).expect("spawn"); - std::thread::sleep(std::time::Duration::from_millis(50)); - halt.cancel(); - within(1, move || drop(pf)); - - let halt = Halt::new(); - let pf = BytePrefetcher::new(EndlessReader, 4096, Some(halt.clone())).expect("spawn"); - let (rx, recycle_tx, shell) = pf.into_channels(); - std::thread::sleep(std::time::Duration::from_millis(50)); - halt.cancel(); - within(1, move || drop(shell)); - drop((rx, recycle_tx)); - } -} diff --git a/src/io/flush.rs b/src/io/flush.rs index 0c3bb8f7..8c5acb23 100644 --- a/src/io/flush.rs +++ b/src/io/flush.rs @@ -13,12 +13,12 @@ use std::time::{Duration, Instant}; use super::bounded::{BoundedError, bounded_syscall_stall}; use crate::error::Error; -use crate::halt::{Halt, Progress, StallTimer}; +use crate::halt::{Halt, Liveness, StallTimer}; /// T12's window: this long with no flush progress is `SyncTimeout` (E9056). pub(crate) const FLUSH_STALL: Duration = Duration::from_secs(60); -/// A shared view of a file's durable-flush progress: the [`Progress`] every accepted +/// A shared view of a file's durable-flush progress: the [`Liveness`] every accepted /// write and completed flush bumps, the bytes made durable, and the file length. /// Clones share the counters; hand one to [`WritebackFile::set_flush_progress`] and /// read it from another thread. @@ -26,14 +26,14 @@ pub(crate) const FLUSH_STALL: Duration = Duration::from_secs(60); /// [`WritebackFile::set_flush_progress`]: crate::io::WritebackFile::set_flush_progress #[derive(Clone, Debug, Default)] pub struct FlushProgress { - progress: Progress, + progress: Liveness, durable: Arc, total: Arc, } impl FlushProgress { /// Counters that bump `progress` (share it with the pipeline consumer, §2.10 item 3). - pub fn new(progress: Progress) -> Self { + pub fn new(progress: Liveness) -> Self { Self { progress, ..Self::default() @@ -41,7 +41,7 @@ impl FlushProgress { } /// The forward-progress counter. - pub fn progress(&self) -> &Progress { + pub fn progress(&self) -> &Liveness { &self.progress } @@ -173,7 +173,7 @@ pub(crate) fn durable_sync_file_with( let len = file.metadata()?.len(); // An owned clone per worker call: a leaked worker keeps a valid fd, never a reused number. let file = Arc::new(file.try_clone()?); - let progress = Progress::new(); + let progress = Liveness::new(); let mut timer = StallTimer::new(timing.stall, &progress); let mut sampler = Sampler::new(timing.sample_every, timing.stall); let mut done = 0u64; diff --git a/src/io/flush/tests.rs b/src/io/flush/tests.rs index 4ef8af1a..6f2acc4b 100644 --- a/src/io/flush/tests.rs +++ b/src/io/flush/tests.rs @@ -143,7 +143,7 @@ fn fake_file( let file = File::create(dir.path().join("out.bin")).unwrap(); let ops: Arc = ops.clone(); let mut w = WritebackFile::with_flush_ops(file, ops, t).unwrap(); - let flush = FlushProgress::new(Progress::new()); + let flush = FlushProgress::new(Liveness::new()); w.set_flush_progress(flush.clone()); (dir, w, flush) } @@ -753,7 +753,7 @@ fn mountstats_parser() { fn real_flush_progress() { let dir = tempfile::tempdir().unwrap(); let mut w = WritebackFile::create(&dir.path().join("real.bin")).unwrap(); - let flush = FlushProgress::new(Progress::new()); + let flush = FlushProgress::new(Liveness::new()); w.set_flush_progress(flush.clone()); let block = vec![9u8; 1024 * 1024]; for _ in 0..8 { diff --git a/src/io/image_writer.rs b/src/io/image_writer.rs index 952699ae..6fedbbd9 100644 --- a/src/io/image_writer.rs +++ b/src/io/image_writer.rs @@ -8,11 +8,11 @@ //! is for everything else. use crate::consts::SECTOR_BYTES; +use crate::ctx::Ctx; use crate::error::{Error, Result}; -use crate::halt::Halt; +use crate::event::Event; +use crate::io::block_sink::{BlockSink, Finish, IsoSink}; use crate::sector::SectorSource; -use std::fs::File; -use std::io::{BufWriter, Write}; use std::path::Path; /// Sectors per read/write batch. 4 MiB — large enough that per-call overhead @@ -24,8 +24,8 @@ const BATCH_SECTORS: u32 = 2048; /// /// Reads 2048-sector batches from LBA 0 in order; decrypts nothing. Batches are not whole /// 3-sector AACS units: wrap an AACS decrypting source to read on the unit grid, or it fails -/// the first misaligned batch ([`Error::DecryptFailed`]). `on_progress` gets cumulative bytes -/// per batch (must not block); `halt` keeps the partial file, any other failure removes it. +/// the first misaligned batch ([`Error::DecryptFailed`]). Each batch emits a cumulative +/// `BytesWritten`; a Stop keeps the partial file, any other failure removes it. /// /// # Errors /// @@ -35,15 +35,13 @@ pub fn write_image( reader: &mut dyn SectorSource, dest: &Path, total_sectors: u32, - halt: &Halt, - on_progress: impl FnMut(u64), + ctx: &Ctx, ) -> Result { write_image_with( reader, dest, total_sectors, - halt, - on_progress, + ctx, crate::io::fsync::dir_checked, ) } @@ -53,24 +51,18 @@ fn write_image_with( reader: &mut dyn SectorSource, dest: &Path, total_sectors: u32, - halt: &Halt, - mut on_progress: impl FnMut(u64), + ctx: &Ctx, sync_dir: fn(&Path) -> std::io::Result<()>, ) -> Result { if total_sectors == 0 { return Err(Error::EmptyImage); } - - let file = File::create(dest).map_err(|source| Error::IoError { source })?; - let r = copy_out( - reader, - file, - dest, - total_sectors, - halt, - &mut on_progress, - sync_dir, - ); + let sink = IsoSink::create(dest)?; + #[cfg(test)] + let sink = sink.with_dir_sync(sync_dir); + #[cfg(not(test))] + let _ = sync_dir; + let r = copy_out(reader, Box::new(sink), total_sectors, ctx); // A failed copy leaves no truncated image at the final name; a halt keeps its partial. if matches!(&r, Err(e) if !matches!(e, Error::Halted)) { let _ = std::fs::remove_file(dest); @@ -78,59 +70,54 @@ fn write_image_with( r } +// Sectors `0..total_sectors` of `reader`, in order, into `sink`. fn copy_out( reader: &mut dyn SectorSource, - file: File, - dest: &Path, + mut sink: Box, total_sectors: u32, - halt: &Halt, - on_progress: &mut impl FnMut(u64), - sync_dir: fn(&Path) -> std::io::Result<()>, + ctx: &Ctx, ) -> Result { - let mut out = BufWriter::with_capacity(BATCH_SECTORS as usize * SECTOR_BYTES, file); - + let total = u64::from(total_sectors) * SECTOR_BYTES as u64; let mut buf = vec![0u8; BATCH_SECTORS as usize * SECTOR_BYTES]; let mut written: u64 = 0; let mut lba: u32 = 0; while lba < total_sectors { - if halt.is_cancelled() { + if ctx.halt.is_cancelled() { + let _ = sink.finish(Finish::Incomplete); return Err(Error::Halted); } let count = BATCH_SECTORS.min(total_sectors - lba); let want = count as usize * SECTOR_BYTES; // `recovery = false`: a file-backed source ignores the flag, and a // retry loop over a local file would only re-read the same bytes. - let got = reader.read_sectors(lba, count as u16, &mut buf[..want], false)?; + let got = match reader.read_sectors(lba, count as u16, &mut buf[..want], false) { + Ok(n) => n, + Err(e) => { + let _ = sink.finish(Finish::Incomplete); + return Err(e); + } + }; if got != want { + let _ = sink.finish(Finish::Incomplete); return Err(Error::ShortImageRead { lba, expected: want as u32, got: got as u32, }); } - out.write_all(&buf[..want]) - .map_err(|source| Error::IoError { source })?; + if let Err(e) = sink.write_at(u64::from(lba), &buf[..want]) { + let _ = sink.finish(Finish::Incomplete); + return Err(e); + } written += want as u64; lba += count; - on_progress(written); + ctx.emit(Event::BytesWritten { + bytes: written, + total, + }); } - - // flush() only pushes bytes into the kernel via write(2), no durability promise — - // a crash or yanked volume could leave a truncated file reported as complete. - // `into_inner` (not `flush`) also surfaces buffered-write errors, not silently drop them. - let file = out.into_inner().map_err(|e| Error::IoError { - source: e.into_error(), - })?; - file.sync_all() - .map_err(|source| Error::IoError { source })?; - // The file was just created: its directory entry needs its own fsync. - let dir = match dest.parent() { - Some(dir) if !dir.as_os_str().is_empty() => dir, - _ => Path::new("."), - }; - sync_dir(dir).map_err(|source| Error::IoError { source })?; - Ok(written) + sink.finish(Finish::Complete) } #[cfg(test)] @@ -227,7 +214,7 @@ mod tests { .with_content_ranges(Arc::from(vec![(3000u32, 30u32)])); dec.set_unit_base(0); // file grid (3000 % 3 == 0): fails loud past the no-base gate let dest = tmp("aacs-batch"); - let r = write_image(&mut dec, &dest, 3100, &Halt::new(), |_| {}); + let r = write_image(&mut dec, &dest, 3100, &Ctx::default()); let _ = std::fs::remove_file(&dest); let e = r.expect_err("batch 2048 is off the unit grid and touches content"); assert_eq!(e.code(), Error::DecryptFailed.code()); @@ -248,7 +235,7 @@ mod tests { sectors: 4096, short_after: Some(2048), }; - let r = write_image(&mut src, &dest, 4096, &Halt::new(), |_| {}); + let r = write_image(&mut src, &dest, 4096, &Ctx::default()); assert!(matches!(r, Err(Error::ShortImageRead { .. })), "{r:?}"); assert!(!dest.exists(), "truncated image left at the final name"); } @@ -265,8 +252,7 @@ mod tests { &mut src, &td.path().join("x.iso"), 4, - &Halt::new(), - |_| {}, + &Ctx::default(), |_| Err(std::io::Error::from(std::io::ErrorKind::Other)), ); assert!(matches!(res, Err(Error::IoError { .. })), "{res:?}"); @@ -281,7 +267,7 @@ mod tests { sectors: 5000, short_after: None, }; - let n = write_image(&mut src, &dest, 5000, &Halt::new(), |_| {}).expect("write"); + let n = write_image(&mut src, &dest, 5000, &Ctx::default()).expect("write"); assert_eq!(n, 5000 * SECTOR_BYTES as u64); let data = std::fs::read(&dest).expect("read back"); @@ -313,7 +299,7 @@ mod tests { sectors: 2049, short_after: None, }; - let n = write_image(&mut src, &dest, 2049, &Halt::new(), |_| {}).expect("write"); + let n = write_image(&mut src, &dest, 2049, &Ctx::default()).expect("write"); assert_eq!(n, 2049 * SECTOR_BYTES as u64); assert_eq!( std::fs::metadata(&dest).expect("stat").len(), @@ -331,7 +317,7 @@ mod tests { sectors: 4096, short_after: Some(2048), }; - let err = write_image(&mut src, &dest, 4096, &Halt::new(), |_| {}).expect_err("must fail"); + let err = write_image(&mut src, &dest, 4096, &Ctx::default()).expect_err("must fail"); assert!( matches!(err, Error::ShortImageRead { lba: 2048, .. }), "got {err:?}" @@ -348,9 +334,10 @@ mod tests { sectors: 100_000, short_after: None, }; - let halt = Halt::new(); + let halt = crate::halt::Halt::new(); halt.cancel(); - let err = write_image(&mut src, &dest, 100_000, &halt, |_| {}).expect_err("must halt"); + let err = + write_image(&mut src, &dest, 100_000, &Ctx::new(halt.clone())).expect_err("must halt"); assert!(matches!(err, Error::Halted), "got {err:?}"); let _ = std::fs::remove_file(&dest); } @@ -365,13 +352,21 @@ mod tests { sectors: 5000, short_after: None, }; - let mut seen: Vec = Vec::new(); - let n = write_image(&mut src, &dest, 5000, &Halt::new(), |b| seen.push(b)).expect("write"); + let seen = std::sync::Arc::new(std::sync::Mutex::new(Vec::::new())); + let rec = seen.clone(); + let ctx = Ctx::default().with_events(std::sync::Arc::new(move |e: &Event<'_>| { + if let Event::BytesWritten { bytes, total } = *e { + assert_eq!(total, 5000 * SECTOR_BYTES as u64); + rec.lock().unwrap().push(bytes); + } + })); + let n = write_image(&mut src, &dest, 5000, &ctx).expect("write"); + let seen = seen.lock().unwrap().clone(); assert!( seen.windows(2).all(|w| w[1] > w[0]), "not monotonic: {seen:?}" ); - assert_eq!(*seen.last().expect("at least one callback"), n); + assert_eq!(*seen.last().expect("at least one event"), n); let _ = std::fs::remove_file(&dest); } @@ -384,7 +379,7 @@ mod tests { sectors: 0, short_after: None, }; - let err = write_image(&mut src, &dest, 0, &Halt::new(), |_| {}).expect_err("must fail"); + let err = write_image(&mut src, &dest, 0, &Ctx::default()).expect_err("must fail"); assert!(matches!(err, Error::EmptyImage), "got {err:?}"); // The destination must not have been created — a failed run leaves no // stub for a later run to mistake for output. diff --git a/src/io/mod.rs b/src/io/mod.rs index eca1dccf..e97e99d2 100644 --- a/src/io/mod.rs +++ b/src/io/mod.rs @@ -4,17 +4,15 @@ //! sequential writes. `FileSectorSource` is the read-side dual, //! implementing [`crate::sector::SectorSource`] for ISO reads. //! `Pipeline` + `Sink` overlaps reads with writes via a bounded -//! channel + consumer thread. `byte_prefetcher` is the read-ahead -//! producer feeding the mux pipeline for `io::Read`-backed sources. +//! channel + consumer thread. pub mod artifact_lock; +pub mod block_sink; pub(crate) mod bounded; -pub mod byte_prefetcher; pub mod file_sector_source; mod flush; pub mod fsync; pub mod image_writer; -pub mod sink; mod writeback; mod writeback_file; @@ -22,9 +20,14 @@ mod writeback_file; pub(crate) mod platform_macos; pub mod pipeline; +pub mod tree_sink; pub use artifact_lock::ArtifactLock; +pub use block_sink::{ + BlockSink, IsoSink, NullBlockSink, is_null_device, null_device, open_block_sink, +}; pub use flush::{FlushProgress, durable_sync_file}; +pub use tree_sink::{TreeSink, open_tree_sink}; pub use writeback_file::WritebackFile; pub use pipeline::{ diff --git a/src/io/pipeline.rs b/src/io/pipeline.rs index 1d75510f..f38ec21a 100644 --- a/src/io/pipeline.rs +++ b/src/io/pipeline.rs @@ -14,7 +14,7 @@ use crossbeam_channel::{Sender, TrySendError, bounded}; use crate::error::Error; use crate::halt::join_within; -use crate::halt::{Halt, Halted, Joined, Progress, SendOutcome, Stall, StallTimer, WAIT_SLICE}; +use crate::halt::{Halt, Halted, Joined, Liveness, SendOutcome, Stall, StallTimer, WAIT_SLICE}; /// `finish_with_halt`'s join window (T7): this long with no consumer progress is /// `PipelineJoinTimeout`. A stall window, not a total (HR1). @@ -96,7 +96,7 @@ fn join_result(r: thread::Result>) -> Result { fn finish_with_grace( handle: thread::JoinHandle>, state: &Arc, - progress: &Progress, + progress: &Liveness, grace: Duration, leak_err: Error, on_poll: &mut dyn FnMut(), @@ -274,7 +274,7 @@ pub struct Pipeline { /// [`Pipeline::send`] users. failed: Arc, /// The consumer's forward progress (T7): shared with the sink's output. - progress: Progress, + progress: Liveness, /// The op's token, read by the consumer when it commits to `close()` (§2.5). op: Arc>, } @@ -297,17 +297,17 @@ impl Pipeline { depth: usize, sink: S, ) -> Result { - Self::spawn_named_with_progress(name, depth, sink, Progress::new()) + Self::spawn_named_with_progress(name, depth, sink, Liveness::new()) } - /// Like [`Pipeline::spawn_named`], with the consumer's [`Progress`] supplied, so the + /// Like [`Pipeline::spawn_named`], with the consumer's [`Liveness`] supplied, so the /// sink's output (a [`WritebackFile`](crate::io::WritebackFile) given the same counter) /// shares it (stop design §2.10 item 3). pub fn spawn_named_with_progress>( name: &str, depth: usize, sink: S, - progress: Progress, + progress: Liveness, ) -> Result { let (tx, rx) = bounded::(depth); let state = Arc::new(AtomicU8::new(state::RUNNING)); @@ -494,7 +494,7 @@ impl Pipeline { /// The consumer's forward-progress counter: bumped per item applied and at /// `close()` entry and exit; [`finish_with_halt`](Self::finish_with_halt) waits on it. - pub fn progress(&self) -> &Progress { + pub fn progress(&self) -> &Liveness { &self.progress } @@ -723,7 +723,7 @@ mod tests { finish_with_grace( handle, &state, - &Progress::new(), + &Liveness::new(), Duration::MAX, Error::Halted, &mut || {}, @@ -1711,7 +1711,7 @@ mod tests { let res = finish_with_grace( handle, &state, - &Progress::new(), + &Liveness::new(), grace, Error::Halted, &mut || {}, diff --git a/src/io/pipeline/stop_tests.rs b/src/io/pipeline/stop_tests.rs index 01cdc56d..a7080955 100644 --- a/src/io/pipeline/stop_tests.rs +++ b/src/io/pipeline/stop_tests.rs @@ -77,7 +77,7 @@ fn wait_until(flag: &AtomicBool) { // `close()` runs a script: sleep `stall` without progress, then `steps` sleeps of // `step`, bumping the shared progress after each. `in_close` flags the entry. struct ScriptedClose { - progress: Progress, + progress: Liveness, in_close: Arc, stall: Duration, step: Duration, @@ -101,7 +101,7 @@ impl Sink for ScriptedClose { } fn scripted(stall: Duration, steps: u32) -> (Pipeline, Arc) { - let progress = Progress::new(); + let progress = Liveness::new(); let in_close = Arc::new(AtomicBool::new(false)); let sink = ScriptedClose { progress: progress.clone(), @@ -331,7 +331,7 @@ struct PartialFile { file: std::fs::File, partial: std::path::PathBuf, out: std::path::PathBuf, - progress: Progress, + progress: Liveness, step: Duration, steps: u32, done: Arc, @@ -381,7 +381,7 @@ fn partial_run(steps: u32) -> PartialRun { let dir = tempfile::tempdir().unwrap(); let out = dir.path().join("Movie.mkv"); let partial = dir.path().join("Movie.mkv.partial"); - let progress = Progress::new(); + let progress = Liveness::new(); let done = Arc::new(AtomicBool::new(false)); let sink = PartialFile { file: std::fs::File::create(&partial).unwrap(), @@ -487,7 +487,7 @@ fn halted_running_consumer_output_stays_partial() { file: std::fs::File::create(&partial).unwrap(), partial: partial.clone(), out: out.clone(), - progress: Progress::new(), + progress: Liveness::new(), step: Duration::ZERO, steps: 0, done: Arc::default(), diff --git a/src/io/platform_macos.rs b/src/io/platform_macos.rs index 3c444363..b9236ffb 100644 --- a/src/io/platform_macos.rs +++ b/src/io/platform_macos.rs @@ -3,8 +3,7 @@ //! The `libc` crate doesn't expose these symbols across all macOS SDK versions, so we define //! them locally with values from `/usr/include/sys/fcntl.h`. //! -//! [`crate::io::writeback_file`] and `crate::io::sink::preallocate` -//! both depend on these constants and the `fstore_t` layout. +//! [`crate::io::writeback_file`] depends on these constants and the `fstore_t` layout. /// `fcntl(F_PREALLOCATE)` command number from `sys/fcntl.h`. pub(crate) const F_PREALLOCATE: libc::c_int = 42; diff --git a/src/io/sink/local_file.rs b/src/io/sink/local_file.rs deleted file mode 100644 index cd516c02..00000000 --- a/src/io/sink/local_file.rs +++ /dev/null @@ -1,211 +0,0 @@ -//! `LocalFileSink` — `BufWriter` for the common local-disk case. -//! -//! 4 MiB internal `BufWriter` coalesces small per-PES muxer writes into -//! kernel-page-aligned flushes. `Seek` flushes the buffer first so a -//! seek can't leapfrog buffered data and corrupt the file — required -//! for MKV's seek-back patches (cluster size, Cues, segment header). -//! -//! [`SequentialSink`](super::SequentialSink) is implemented explicitly -//! so `finish()` flushes and `fsync`s even through a `dyn` trait object; -//! [`RandomAccessSink`](super::RandomAccessSink) is over the `Seek` impl. - -use std::fs::{File, OpenOptions}; -use std::io::{self, BufWriter, Seek, SeekFrom, Write}; -use std::path::Path; - -use super::preallocate; -use super::{RandomAccessSink, SequentialSink}; - -const BUFFER_BYTES: usize = 4 * 1024 * 1024; - -/// Random-access write sink for local disks. -/// -/// Wraps a `BufWriter` with a 4 MiB internal buffer and forwards -/// `Write`/`Seek` so any call site that previously held a `File` or -/// `WritebackFile` can drop this in. `finish()` flushes the buffer and -/// `sync_all`s the underlying file before the caller drops it. -/// -/// Opens the file `create + truncate + read + write`; `read` allows -/// reusing the handle for a post-mux verification re-read. On Linux, -/// `Self::with_size_hint` additionally pre-reserves extents via `fallocate`. -pub struct LocalFileSink { - inner: BufWriter, -} - -impl LocalFileSink { - /// Open `path` for writing, truncating any existing contents. - pub fn create(path: &Path) -> io::Result { - let file = OpenOptions::new() - .read(true) - .write(true) - .create(true) - .truncate(true) - .open(path)?; - Ok(Self { - inner: BufWriter::with_capacity(BUFFER_BYTES, file), - }) - } - - /// Like [`Self::create`] but additionally calls the per-OS - /// preallocate path with `size_bytes`. On Linux this is - /// `fallocate(FALLOC_FL_KEEP_SIZE)` so the on-disk extents are - /// reserved up front (reducing fragmentation for big sequential - /// muxer output); on other OSes it is a no-op today. Failures - /// from the preallocate call are non-fatal — the file is still - /// returned, just without the size reservation. - pub fn with_size_hint(path: &Path, size_bytes: u64) -> io::Result { - let file = OpenOptions::new() - .read(true) - .write(true) - .create(true) - .truncate(true) - .open(path)?; - preallocate::preallocate(&file, size_bytes); - Ok(Self { - inner: BufWriter::with_capacity(BUFFER_BYTES, file), - }) - } - - /// Drain the internal buffer and `fsync` the underlying file. - /// Idempotent with `Drop` (the `BufWriter` also flushes on drop; - /// this call additionally surfaces fsync errors to the caller). - /// [`SequentialSink::finish`](super::SequentialSink::finish) - /// delegates here so the durable flush happens through a trait - /// object too. - pub fn sync_all(&mut self) -> io::Result<()> { - self.inner.flush()?; - self.inner.get_ref().sync_all() - } -} - -impl SequentialSink for LocalFileSink { - /// Flush the 4 MiB `BufWriter` and `fsync` the file. Overriding the - /// trait default is what makes a `dyn SequentialSink` / `dyn - /// RandomAccessSink` `finish()` actually durable instead of a no-op. - fn finish(&mut self) -> io::Result<()> { - self.sync_all() - } -} - -impl RandomAccessSink for LocalFileSink {} - -impl Write for LocalFileSink { - fn write(&mut self, buf: &[u8]) -> io::Result { - self.inner.write(buf) - } - - fn write_all(&mut self, buf: &[u8]) -> io::Result<()> { - self.inner.write_all(buf) - } - - fn flush(&mut self) -> io::Result<()> { - self.inner.flush() - } -} - -impl Seek for LocalFileSink { - fn seek(&mut self, from: SeekFrom) -> io::Result { - // Flush before seeking so buffered bytes land at the offset - // they were written for, not the new one. - self.inner.flush()?; - self.inner.get_mut().seek(from) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use std::io::Read; - - #[test] - fn write_seek_roundtrip() { - let dir = tempfile::tempdir().unwrap(); - let p = dir.path().join("rt.bin"); - let mut s = LocalFileSink::create(&p).unwrap(); - s.write_all(b"AAAA").unwrap(); - s.write_all(b"BBBB").unwrap(); - // Seek back over the second word and overwrite. - s.seek(SeekFrom::Start(4)).unwrap(); - s.write_all(b"CCCC").unwrap(); - s.sync_all().unwrap(); - drop(s); - - let mut f = File::open(&p).unwrap(); - let mut got = Vec::new(); - f.read_to_end(&mut got).unwrap(); - assert_eq!(&got[..], b"AAAACCCC"); - } - - #[test] - fn drop_flushes() { - let dir = tempfile::tempdir().unwrap(); - let p = dir.path().join("drop.bin"); - { - let mut s = LocalFileSink::create(&p).unwrap(); - s.write_all(b"buffered").unwrap(); - // No explicit flush / sync_all — BufWriter drop runs the - // flush and the file should land on disk. - } - let bytes = std::fs::read(&p).unwrap(); - assert_eq!(&bytes[..], b"buffered"); - } - - #[test] - fn with_size_hint_creates_writable_file() { - let dir = tempfile::tempdir().unwrap(); - let p = dir.path().join("sz.bin"); - let mut s = LocalFileSink::with_size_hint(&p, 64 * 1024).unwrap(); - s.write_all(b"hint-ok").unwrap(); - s.sync_all().unwrap(); - drop(s); - let bytes = std::fs::read(&p).unwrap(); - assert_eq!(&bytes[..], b"hint-ok"); - } - - // ── Added hardening tests ─────────────────────────────────────── - - // `create` must truncate: pre-seed a long file, recreate via the sink - // with a shorter payload, and confirm the old tail is gone. - #[test] - fn create_truncates_existing_file() { - let dir = tempfile::tempdir().unwrap(); - let p = dir.path().join("trunc.bin"); - std::fs::write(&p, vec![0xFFu8; 4096]).unwrap(); - let mut s = LocalFileSink::create(&p).unwrap(); - s.write_all(b"short").unwrap(); - s.sync_all().unwrap(); - drop(s); - let bytes = std::fs::read(&p).unwrap(); - assert_eq!( - bytes.len(), - 5, - "create must truncate the pre-existing 4096 bytes" - ); - assert_eq!(&bytes, b"short"); - } - - // Seek must flush the BufWriter first so buffered bytes land at their - // intended offset, not the post-seek one. Removing the flush in `seek` - // would corrupt the file by writing the buffered bytes at the wrong offset. - #[test] - fn seek_flushes_buffer_before_moving() { - let dir = tempfile::tempdir().unwrap(); - let p = dir.path().join("seek-flush.bin"); - let mut s = LocalFileSink::create(&p).unwrap(); - // These bytes sit in the 4 MiB BufWriter, unflushed. - s.write_all(b"HEAD").unwrap(); - // Seek forward to offset 10; the buffered HEAD must be flushed - // to offset 0 BEFORE the position moves. - s.seek(SeekFrom::Start(10)).unwrap(); - s.write_all(b"TAIL").unwrap(); - s.sync_all().unwrap(); - drop(s); - let bytes = std::fs::read(&p).unwrap(); - assert_eq!( - &bytes[0..4], - b"HEAD", - "buffered head landed at the wrong offset" - ); - assert_eq!(&bytes[10..14], b"TAIL"); - } -} diff --git a/src/io/sink/mod.rs b/src/io/sink/mod.rs deleted file mode 100644 index f6046012..00000000 --- a/src/io/sink/mod.rs +++ /dev/null @@ -1,123 +0,0 @@ -//! Output-sink trait split for the buffering architecture. -//! -//! [`SequentialSink`] — anything you can `Write` to in order (no seek; sockets, pipes, -//! M2TS/fMP4/HEVC-ES) — and [`RandomAccessSink`] — `SequentialSink` plus a working `Seek` -//! (local/NFS files; needed for backpatch: MKV cluster sizes, Cues, MP4 moov-at-end). The muxer -//! is generic over whichever it requires, so muxing MKV to a socket is a compile error. -//! Buffering policy belongs to the concrete sink. - -use std::io::{Seek, Write}; - -mod local_file; -mod preallocate; -mod socket; - -pub use local_file::LocalFileSink; -pub use socket::{SocketSink, UdpSocketSink}; - -/// Sequential-only write destination. Sockets, pipes, append-only -/// stores. No seek. Implementations own their write buffering — the -/// trait does not impose or hide any buffering of its own. -/// -/// `finish` drains any internal buffering and signals end-of-stream to the transport -/// (close-write on a socket, flush + fsync on a buffered file, etc.). The default impl flushes -/// via [`Write::flush`]; every concrete sink here overrides it with its own finalisation. There -/// is deliberately no blanket `impl SequentialSink for T`. -pub trait SequentialSink: Write + Send { - fn finish(&mut self) -> std::io::Result<()> { - self.flush() - } -} - -/// Random-access write destination. Local files, NFS files, anything -/// with a working `Seek`. Inherits the `SequentialSink` contract — a -/// random-access sink is always usable as a sequential sink. -pub trait RandomAccessSink: SequentialSink + Seek {} - -#[cfg(test)] -mod tests { - use super::*; - - // Type-level assertion: the concrete sinks satisfy the trait - // objects. These functions never run; they just have to type-check. - fn _assert_is_sequential(_: &mut dyn SequentialSink) {} - fn _assert_is_random_access(_: &mut dyn RandomAccessSink) {} - - #[test] - fn concrete_sinks_satisfy_traits() { - let dir = tempfile::tempdir().unwrap(); - - // `LocalFileSink` is a random-access (and thus sequential) sink. - let mut s = LocalFileSink::create(&dir.path().join("b.bin")).unwrap(); - _assert_is_sequential(&mut s); - _assert_is_random_access(&mut s); - - // `WritebackFile` ditto, via its explicit per-type impls. - let mut wf = crate::io::WritebackFile::create(&dir.path().join("c.bin")).unwrap(); - _assert_is_sequential(&mut wf); - _assert_is_random_access(&mut wf); - } - - // Regression test for the silent-no-op finish() bug. - #[test] - fn finish_through_trait_object_flushes_local_file() { - let dir = tempfile::tempdir().unwrap(); - let p = dir.path().join("trait-finish.bin"); - let sink = LocalFileSink::create(&p).unwrap(); - // Box as the trait object the production path uses. - let mut boxed: Box = Box::new(sink); - boxed.write_all(b"buffered-tail").unwrap(); - // finish() through the vtable must drain the 4 MiB BufWriter and - // fsync; the bytes must be visible to a separate reader BEFORE - // we drop the sink (drop-flush must not be what saves us). - boxed.finish().unwrap(); - let bytes = std::fs::read(&p).unwrap(); - assert_eq!(&bytes[..], b"buffered-tail"); - } - - // ── Added hardening tests ─────────────────────────────────────── - - use std::io::{self, Write}; - use std::sync::Arc; - use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; - - // Minimal SequentialSink that does NOT override finish(), so it exercises the trait's - // default impl. - struct FlushTracker { - flushed: Arc, - bytes: Arc, - } - impl Write for FlushTracker { - fn write(&mut self, buf: &[u8]) -> io::Result { - self.bytes.fetch_add(buf.len(), Ordering::SeqCst); - Ok(buf.len()) - } - fn flush(&mut self) -> io::Result<()> { - self.flushed.store(true, Ordering::SeqCst); - Ok(()) - } - } - // Uses the DEFAULT finish() — deliberately no override. - impl SequentialSink for FlushTracker {} - - #[test] - fn default_finish_flushes() { - let flushed = Arc::new(AtomicBool::new(false)); - let bytes = Arc::new(AtomicUsize::new(0)); - let mut sink = FlushTracker { - flushed: flushed.clone(), - bytes: bytes.clone(), - }; - sink.write_all(b"abc").unwrap(); - assert!( - !flushed.load(Ordering::SeqCst), - "flush should not run before finish" - ); - sink.finish().unwrap(); - assert!( - flushed.load(Ordering::SeqCst), - "default SequentialSink::finish must call Write::flush" - ); - assert_eq!(bytes.load(Ordering::SeqCst), 3); - } -} diff --git a/src/io/sink/preallocate/linux.rs b/src/io/sink/preallocate/linux.rs deleted file mode 100644 index 8d423bea..00000000 --- a/src/io/sink/preallocate/linux.rs +++ /dev/null @@ -1,23 +0,0 @@ -//! Linux `fallocate(FALLOC_FL_KEEP_SIZE)` preallocation. -//! -//! `KEEP_SIZE` reserves extents without changing the apparent file -//! length, which matches the muxer's expectation that writes still grow -//! the file naturally. - -use std::fs::File; -use std::os::unix::io::AsRawFd; - -pub(super) fn preallocate_impl(file: &File, size_bytes: u64) { - let fd = file.as_raw_fd(); - // Clamp to the signed `off_t` range fallocate expects; an unchecked - // `as i64` cast would wrap a >= 2^63 size to a negative length that - // fallocate rejects with EINVAL (silent no-op). - let len = i64::try_from(size_bytes).unwrap_or(i64::MAX); - // FALLOC_FL_KEEP_SIZE = 0x01. - let rc = unsafe { libc::fallocate(fd, libc::FALLOC_FL_KEEP_SIZE, 0, len) }; - tracing::debug!( - target: "mux", - "LocalFileSink fallocate size_hint={size_bytes} rc={rc} ok={}", - rc == 0 - ); -} diff --git a/src/io/sink/preallocate/macos.rs b/src/io/sink/preallocate/macos.rs deleted file mode 100644 index 066ecc4f..00000000 --- a/src/io/sink/preallocate/macos.rs +++ /dev/null @@ -1,41 +0,0 @@ -//! macOS `F_PREALLOCATE` extent reservation. -//! -//! `fcntl(F_PREALLOCATE)` with `F_ALLOCATECONTIG | F_ALLOCATEALL` first -//! (prefer a contiguous run but accept scattered extents to satisfy the -//! full length) and fall back to `F_ALLOCATEALL` alone on failure. -//! Reported file size is unchanged — the muxer's writes still grow it. - -use std::fs::File; -use std::os::unix::io::AsRawFd; - -use crate::io::platform_macos::{ - F_ALLOCATEALL, F_ALLOCATECONTIG, F_PEOFPOSMODE, F_PREALLOCATE, Fstore, -}; - -pub(super) fn preallocate_impl(file: &File, size_bytes: u64) { - let fd = file.as_raw_fd(); - // Clamp to the signed `off_t` range; an unchecked `as off_t` cast - // would wrap a >= 2^63 size to a negative length. - let len = i64::try_from(size_bytes).unwrap_or(i64::MAX) as libc::off_t; - let mut store = Fstore { - // Prefer contiguous but accept scattered extents. Without F_ALLOCATEALL - // the first attempt is best-effort and can return rc=0 with a partial - // allocation, so the fallback below would never fire. Matches writeback_file/macos.rs. - fst_flags: F_ALLOCATECONTIG | F_ALLOCATEALL, - fst_posmode: F_PEOFPOSMODE, - fst_offset: 0, - fst_length: len, - fst_bytesalloc: 0, - }; - let mut rc = unsafe { libc::fcntl(fd, F_PREALLOCATE, &mut store as *mut Fstore) }; - if rc == -1 { - // Fall back to non-contiguous only. - store.fst_flags = F_ALLOCATEALL; - rc = unsafe { libc::fcntl(fd, F_PREALLOCATE, &mut store as *mut Fstore) }; - } - tracing::debug!( - target: "mux", - "LocalFileSink F_PREALLOCATE size_hint={size_bytes} rc={rc} bytesalloc={}", - store.fst_bytesalloc - ); -} diff --git a/src/io/sink/preallocate/mod.rs b/src/io/sink/preallocate/mod.rs deleted file mode 100644 index 1b2f677f..00000000 --- a/src/io/sink/preallocate/mod.rs +++ /dev/null @@ -1,27 +0,0 @@ -//! Per-OS extent preallocation. Best-effort; failures are logged at -//! debug and otherwise swallowed because the file is still usable -//! without the size reservation — only large-file fragmentation gets -//! marginally worse. - -use std::fs::File; - -#[cfg(target_os = "linux")] -mod linux; -#[cfg(target_os = "macos")] -mod macos; -#[cfg(not(any(target_os = "linux", target_os = "macos")))] -mod other; - -#[cfg(target_os = "linux")] -use linux::preallocate_impl; -#[cfg(target_os = "macos")] -use macos::preallocate_impl; -#[cfg(not(any(target_os = "linux", target_os = "macos")))] -use other::preallocate_impl; - -/// Reserve `size_bytes` of disk space for `file`'s on-disk extents. -/// Reported file size is unchanged — writes still grow the file -/// naturally; only the allocator's extent map is primed. -pub(super) fn preallocate(file: &File, size_bytes: u64) { - preallocate_impl(file, size_bytes); -} diff --git a/src/io/sink/preallocate/other.rs b/src/io/sink/preallocate/other.rs deleted file mode 100644 index 22d94b3b..00000000 --- a/src/io/sink/preallocate/other.rs +++ /dev/null @@ -1,10 +0,0 @@ -//! Fallback preallocate impl. No-op. - -use std::fs::File; - -pub(super) fn preallocate_impl(_file: &File, size_bytes: u64) { - tracing::debug!( - target: "mux", - "LocalFileSink preallocate size_hint={size_bytes} skipped (no platform impl)" - ); -} diff --git a/src/io/sink/socket.rs b/src/io/sink/socket.rs deleted file mode 100644 index 82e45746..00000000 --- a/src/io/sink/socket.rs +++ /dev/null @@ -1,343 +0,0 @@ -//! TCP / UDP socket sinks (sequential-only). -//! -//! [`SocketSink`] wraps a `TcpStream` in a 1 MiB `BufWriter`; `finish()` flushes then -//! `shutdown(Write)`s for a clean end-of-stream. [`UdpSocketSink`] wraps a connected -//! `UdpSocket`; each `write` emits one datagram and `finish()` is a no-op. Neither implements -//! `Seek`, so neither satisfies [`RandomAccessSink`]. -//! -//! [`SequentialSink`]: super::SequentialSink -//! [`RandomAccessSink`]: super::RandomAccessSink - -use std::io::{self, BufWriter, Write}; -use std::net::{Shutdown, SocketAddr, TcpStream, ToSocketAddrs, UdpSocket}; - -use super::SequentialSink; - -// `BufWriter` capacity for `SocketSink`. 1 MiB matches the typical kernel send-buffer ceiling. -const TCP_BUF_CAPACITY: usize = 1024 * 1024; - -/// Sequential-only sink over a TCP connection. -/// -/// Wraps a `BufWriter`; the inner `TcpStream` is kept as a -/// clone so [`finish`](Self::finish) can call `shutdown(Write)` after -/// flushing the buffer (the buffered writer doesn't expose the socket -/// directly). -pub struct SocketSink { - /// Buffered write half. All payload bytes go through this. - buf: BufWriter, - /// Shutdown handle — clone of the socket inside `buf`. Used only by - /// `finish()` for `shutdown(Write)`; never read or written through. - shutdown_handle: TcpStream, -} - -impl SocketSink { - /// Open a TCP connection to `addr` and wrap it for sequential - /// writing. `sndbuf_bytes`, when present, is forwarded to - /// `setsockopt(SO_SNDBUF)` as a kernel hint — the OS may clamp it. - /// - /// `addr` accepts anything `ToSocketAddrs` does: `"192.0.2.1:1234"`, - /// `("host", 1234)`, a `SocketAddr`, etc. - pub fn connect(addr: A, sndbuf_bytes: Option) -> io::Result { - let stream = TcpStream::connect(addr)?; - // `set_nodelay(true)` keeps small writes (TS packet trains, fMP4 moof headers) - // from sitting in Nagle's algorithm; BufWriter already absorbs syscall overhead. - // A latency hint, not a correctness requirement — a rejecting platform must not fail connect. - let _ = stream.set_nodelay(true); - if let Some(n) = sndbuf_bytes { - set_send_buffer(&stream, n)?; - } - let shutdown_handle = stream.try_clone()?; - Ok(Self { - buf: BufWriter::with_capacity(TCP_BUF_CAPACITY, stream), - shutdown_handle, - }) - } -} - -impl Write for SocketSink { - fn write(&mut self, buf: &[u8]) -> io::Result { - self.buf.write(buf) - } - fn flush(&mut self) -> io::Result<()> { - self.buf.flush() - } -} - -impl SequentialSink for SocketSink { - // Drains the BufWriter and shuts down writes so a `dyn SequentialSink` - // `finish()` sends the buffered tail and EOF instead of dropping them. - fn finish(&mut self) -> io::Result<()> { - self.buf.flush()?; - // `shutdown(Write)` signals clean EOF to the peer. Errors here - // are non-fatal — the connection may have already been torn down - // by the peer — but we surface them so callers can log. - self.shutdown_handle.shutdown(Shutdown::Write) - } -} - -/// Sequential-only sink over a connected UDP socket. -/// -/// Each [`write`](Write::write) call sends exactly one datagram. The -/// caller is responsible for splitting payload at packet boundaries — -/// for MPEG-TS this means 7 × 188 = 1316 bytes per datagram, the -/// industry standard for MPEG-TS-over-UDP. No buffering happens here; -/// adding it would silently merge datagrams. -/// -/// `finish()` is a no-op: UDP has no end-of-stream marker. Closing the -/// socket happens on drop. -pub struct UdpSocketSink { - socket: UdpSocket, -} - -impl UdpSocketSink { - /// Bind a local UDP socket to an ephemeral port and `connect` it to - /// `peer`. `connect` doesn't open a connection — it just fixes the - /// peer address so subsequent `send` calls don't need to repeat it, - /// and so receive-side filtering rejects packets from other sources. - /// - /// `sndbuf_bytes`, when present, is a hint to `SO_SNDBUF`. - pub fn connect(peer: A, sndbuf_bytes: Option) -> io::Result { - // Resolve the peer first so the local bind matches its address family: - // binding `0.0.0.0:0` (IPv4) then connecting to an IPv6 peer fails - // with EAFNOSUPPORT. - let peer_addr = peer - .to_socket_addrs()? - .next() - .ok_or_else(|| io::Error::from(io::ErrorKind::AddrNotAvailable))?; - let bind_addr = match peer_addr { - SocketAddr::V4(_) => "0.0.0.0:0", - SocketAddr::V6(_) => "[::]:0", - }; - // Bind to the matching wildcard / any port. The kernel picks an - // ephemeral source port and the source IP at first send. - let socket = UdpSocket::bind(bind_addr)?; - socket.connect(peer_addr)?; - if let Some(n) = sndbuf_bytes { - set_udp_send_buffer(&socket, n)?; - } - Ok(Self { socket }) - } -} - -impl Write for UdpSocketSink { - fn write(&mut self, buf: &[u8]) -> io::Result { - // `send` writes the entire datagram or fails — no partial sends - // for UDP. Match `Write::write`'s contract by reporting bytes - // accepted. - self.socket.send(buf) - } - fn flush(&mut self) -> io::Result<()> { - Ok(()) - } -} - -impl SequentialSink for UdpSocketSink { - // No-op: `write` already sent each datagram unbuffered, and UDP has - // no end-of-stream marker to shut down. - fn finish(&mut self) -> io::Result<()> { - Ok(()) - } -} - -// ── Platform `SO_SNDBUF` tuning ───────────────────────────────────────── -// std doesn't expose `SO_SNDBUF`; drop to libc on Linux + macOS. Other -// targets silently ignore the hint — socket still works, kernel picks its own size. - -#[cfg(any(target_os = "linux", target_os = "macos"))] -fn set_send_buffer(stream: &TcpStream, bytes: usize) -> io::Result<()> { - use std::os::unix::io::AsRawFd; - setsockopt_sndbuf(stream.as_raw_fd(), bytes) -} - -#[cfg(any(target_os = "linux", target_os = "macos"))] -fn set_udp_send_buffer(socket: &UdpSocket, bytes: usize) -> io::Result<()> { - use std::os::unix::io::AsRawFd; - setsockopt_sndbuf(socket.as_raw_fd(), bytes) -} - -#[cfg(not(any(target_os = "linux", target_os = "macos")))] -fn set_send_buffer(_stream: &TcpStream, _bytes: usize) -> io::Result<()> { - // Non-Linux-non-macOS targets aren't in Cargo.toml's libc dep list; silently - // ignore the hint rather than failing the connect (intentionally no - // "sndbuf applied" signal — this is a hint, not a guarantee). - Ok(()) -} - -#[cfg(not(any(target_os = "linux", target_os = "macos")))] -fn set_udp_send_buffer(_socket: &UdpSocket, _bytes: usize) -> io::Result<()> { - Ok(()) -} - -#[cfg(any(target_os = "linux", target_os = "macos"))] -fn setsockopt_sndbuf(fd: std::os::unix::io::RawFd, bytes: usize) -> io::Result<()> { - // Clamp into c_int range; SO_SNDBUF takes an `int` argument. - let want: libc::c_int = bytes.try_into().unwrap_or(libc::c_int::MAX); - let ret = unsafe { - libc::setsockopt( - fd, - libc::SOL_SOCKET, - libc::SO_SNDBUF, - &want as *const _ as *const libc::c_void, - std::mem::size_of::() as libc::socklen_t, - ) - }; - if ret != 0 { - return Err(io::Error::last_os_error()); - } - Ok(()) -} - -#[cfg(test)] -mod tests { - use super::*; - use std::io::Read; - use std::net::{TcpListener, UdpSocket}; - use std::thread; - - /// Bind a listener, accept on a thread, return (listener_addr, - /// accepted-bytes future via JoinHandle). - #[test] - fn socket_sink_round_trips_bytes() { - let listener = TcpListener::bind("127.0.0.1:0").unwrap(); - let addr = listener.local_addr().unwrap(); - let accept = thread::spawn(move || { - let (mut sock, _) = listener.accept().unwrap(); - let mut buf = Vec::new(); - sock.read_to_end(&mut buf).unwrap(); - buf - }); - - let mut sink = SocketSink::connect(addr, Some(256 * 1024)).unwrap(); - // Write enough to overflow the BufWriter at least once, then a - // tail that lives in the buffer until `finish` flushes. - let big: Vec = (0..(2 * TCP_BUF_CAPACITY)) - .map(|i| (i & 0xff) as u8) - .collect(); - sink.write_all(&big).unwrap(); - sink.write_all(b"tail\n").unwrap(); - sink.finish().unwrap(); - drop(sink); - - let received = accept.join().unwrap(); - assert_eq!(received.len(), big.len() + 5); - assert_eq!(&received[..big.len()], &big[..]); - assert_eq!(&received[big.len()..], b"tail\n"); - } - - #[test] - fn socket_sink_coerces_to_sequential_sink() { - // Compile-time assertion via dyn — if this ever started - // satisfying `RandomAccessSink`, the trait split would be broken. - fn _assert_seq(_: &mut dyn super::super::SequentialSink) {} - let listener = TcpListener::bind("127.0.0.1:0").unwrap(); - let addr = listener.local_addr().unwrap(); - let _accept = thread::spawn(move || { - let _ = listener.accept(); - }); - let mut sink = SocketSink::connect(addr, None).unwrap(); - _assert_seq(&mut sink); - // Only the positive coercion is checked: `RandomAccessSink` impls are explicit - // (local_file, writeback_file), so a socket needs a deliberate new impl + `Seek`. - } - - #[test] - fn udp_socket_sink_binds_ipv6_for_an_ipv6_peer() { - let Ok(receiver) = UdpSocket::bind("[::1]:0") else { - return; // host has no IPv6 loopback - }; - receiver - .set_read_timeout(Some(std::time::Duration::from_secs(2))) - .unwrap(); - let mut sink = UdpSocketSink::connect(receiver.local_addr().unwrap(), None).unwrap(); - sink.write_all(&[7, 8, 9]).unwrap(); - let mut buf = [0u8; 8]; - let n = receiver.recv(&mut buf).unwrap(); - assert_eq!(&buf[..n], &[7, 8, 9]); - } - - #[test] - fn udp_socket_sink_delivers_datagrams() { - let receiver = UdpSocket::bind("127.0.0.1:0").unwrap(); - receiver - .set_read_timeout(Some(std::time::Duration::from_secs(2))) - .unwrap(); - let addr = receiver.local_addr().unwrap(); - let mut sink = UdpSocketSink::connect(addr, Some(128 * 1024)).unwrap(); - - sink.write_all(&[1, 2, 3, 4, 5]).unwrap(); - sink.write_all(&[9, 9, 9]).unwrap(); - sink.finish().unwrap(); - - let mut buf = [0u8; 64]; - let n1 = receiver.recv(&mut buf).unwrap(); - assert_eq!(&buf[..n1], &[1, 2, 3, 4, 5]); - let n2 = receiver.recv(&mut buf).unwrap(); - assert_eq!(&buf[..n2], &[9, 9, 9]); - } - - // ── Added hardening tests ─────────────────────────────────────── - - // `finish` must `shutdown(Write)` so the peer's `read_to_end` sees EOF instead of blocking - // forever. - #[test] - fn finish_signals_eof_to_peer() { - use std::sync::mpsc; - use std::time::Duration; - let listener = TcpListener::bind("127.0.0.1:0").unwrap(); - let addr = listener.local_addr().unwrap(); - let (tx, rx) = mpsc::channel(); - thread::spawn(move || { - let (mut sock, _) = listener.accept().unwrap(); - let mut buf = Vec::new(); - // Returns only when the peer half-closes (shutdown Write). - sock.read_to_end(&mut buf).unwrap(); - let _ = tx.send(buf); - }); - - let mut sink = SocketSink::connect(addr, None).unwrap(); - sink.write_all(b"unflushed-tail").unwrap(); - sink.finish().unwrap(); - - // read_to_end must complete because finish() shut down writes. - let received = rx - .recv_timeout(Duration::from_secs(3)) - .expect("peer never saw EOF — finish() did not shutdown(Write)"); - assert_eq!(received, b"unflushed-tail"); - } - - // UDP `write` must emit one datagram per call, no coalescing. - #[test] - fn udp_write_is_one_datagram_per_call() { - let receiver = UdpSocket::bind("127.0.0.1:0").unwrap(); - receiver - .set_read_timeout(Some(std::time::Duration::from_secs(2))) - .unwrap(); - let addr = receiver.local_addr().unwrap(); - let mut sink = UdpSocketSink::connect(addr, None).unwrap(); - - // Distinct lengths so a merge would be detectable. - let n_a = sink.write(&[0xAA; 10]).unwrap(); - let n_b = sink.write(&[0xBB; 20]).unwrap(); - assert_eq!(n_a, 10); - assert_eq!(n_b, 20); - - let mut buf = [0u8; 256]; - let first = receiver.recv(&mut buf).unwrap(); - assert_eq!(first, 10, "first datagram must be exactly 10 bytes"); - assert!(buf[..first].iter().all(|&b| b == 0xAA)); - let second = receiver.recv(&mut buf).unwrap(); - assert_eq!(second, 20, "second datagram must be exactly 20 bytes"); - assert!(buf[..second].iter().all(|&b| b == 0xBB)); - } - - // UDP `finish` is a documented no-op; must not error. - #[test] - fn udp_finish_is_noop_ok() { - let receiver = UdpSocket::bind("127.0.0.1:0").unwrap(); - let addr = receiver.local_addr().unwrap(); - let mut sink = UdpSocketSink::connect(addr, None).unwrap(); - assert!(sink.finish().is_ok()); - // A second finish is equally harmless. - assert!(sink.finish().is_ok()); - } -} diff --git a/src/io/tree_sink.rs b/src/io/tree_sink.rs new file mode 100644 index 00000000..88cd4b3a --- /dev/null +++ b/src/io/tree_sink.rs @@ -0,0 +1,431 @@ +//! The tree sink (pipeline design §2.2, slice 7): where a whole-disc copy to `dir://` goes. +//! A block sink takes sectors; a tree sink takes the disc's files, one at a time, and lays +//! them out as the decrypted folder: host-safe names, no two disc paths on one host file, +//! each file written `.partial` and published by rename once whole. The disc's +//! `AACS/`, `CERTIFICATE/` and HD DVD AACS directories are not part of a decrypted tree +//! (X-8): the sink drops them. [`open_tree_sink`] opens one from a URL. + +use crate::error::{Error, Result}; +use std::collections::HashMap; +use std::io::Write; +use std::path::{Path, PathBuf}; + +/// Attempts to delete the case-probe marker before giving up. +const PROBE_REMOVE_ATTEMPTS: u32 = 3; + +/// A decrypted file-tree output (`dir://`). +pub struct TreeSink { + dest: PathBuf, + // Whether the target volume folds case (APFS/NTFS): two names differing only by case + // are one host file there. + case_insensitive: bool, + // Host paths handed out so far (case-folded where the volume folds) → their disc path. + seen: HashMap, +} + +impl TreeSink { + /// Create (or reuse) the folder at `dest`. A non-empty folder is refused + /// ([`Error::DirNotEmpty`]) unless `force`: two discs' trees must not mix. + pub fn create(dest: &Path, force: bool) -> Result { + std::fs::create_dir_all(dest).map_err(|e| Error::DirWriteFailed { + errno: e.raw_os_error(), + })?; + if !force && dir_is_non_empty(dest) { + return Err(Error::DirNotEmpty); + } + Ok(TreeSink { + dest: dest.to_path_buf(), + case_insensitive: false, + seen: HashMap::new(), + }) + } + + /// The folder the tree is written into. + pub fn dest(&self) -> &Path { + &self.dest + } + + // Probe the target volume's case rule; called once the structure is about to be planned. + pub(crate) fn probe_case(&mut self) { + self.case_insensitive = dir_is_case_insensitive(&self.dest); + } + + #[cfg(test)] + pub(crate) fn with_case_insensitive(mut self, yes: bool) -> Self { + self.case_insensitive = yes; + self + } + + /// Whether a top-level disc entry belongs in the decrypted tree (X-8): the AACS and + /// certificate directories (and HD DVD's discovered AACS directory, `aacs_dir`) do not. + pub(crate) fn keeps_top_level(name: &str, aacs_dir: bool) -> bool { + !(name.eq_ignore_ascii_case("AACS") || name.eq_ignore_ascii_case("CERTIFICATE") || aacs_dir) + } + + /// Claim the host path for disc entry `disc_path` (named `name`) under `parent`: the + /// sanitized component, refused ([`Error::DirNameCollision`]) when another disc path + /// already maps to it. A file also claims its `.partial` name. + pub(crate) fn claim( + &mut self, + parent: &Path, + name: &str, + disc_path: &str, + is_dir: bool, + ) -> Result { + let rel = parent.join(sanitize_component(name)); + self.register(&rel, disc_path)?; + if !is_dir { + self.register(&with_partial_suffix(&rel), disc_path)?; + } + Ok(rel) + } + + // Collision: two distinct disc paths → one host FILE. Case-insensitive folds + // `Movie`/`movie`; case-sensitive collides only on EXACT match (else names coexist). + // `.partial` shares the final-name namespace, so `X`/`X.partial` collide. + fn register(&mut self, key: &Path, disc_path: &str) -> Result<()> { + let folded = if self.case_insensitive { + key.to_string_lossy().to_lowercase() + } else { + key.to_string_lossy().into_owned() + }; + if self.seen.insert(folded, disc_path.to_string()).is_some() { + return Err(Error::DirNameCollision { + host: key.to_string_lossy().into_owned(), + }); + } + Ok(()) + } + + /// Refuse up front when `required` bytes will not fit (best-effort: only where the + /// platform reports free space). + pub(crate) fn reserve(&self, required: u64) -> Result<()> { + match available_space(&self.dest) { + Some(available) if available < required => Err(Error::DirInsufficientSpace { + required, + available, + }), + _ => Ok(()), + } + } + + /// Create the host directories `dirs` (relative), parents first. + pub(crate) fn make_dirs(&self, dirs: &[PathBuf]) -> Result<()> { + for d in dirs { + std::fs::create_dir_all(self.dest.join(d)).map_err(|e| Error::DirWriteFailed { + errno: e.raw_os_error(), + })?; + } + Ok(()) + } + + /// Start the file at host path `rel` (declared `size` bytes): written as `.partial`. + pub(crate) fn begin(&self, rel: &Path, size: u64) -> Result { + let final_path = self.dest.join(rel); + let partial = with_partial_suffix(&final_path); + let writer = + crate::io::WritebackFile::create_with_size_hint(&partial, size).map_err(|e| { + Error::DirWriteFailed { + errno: e.raw_os_error(), + } + })?; + Ok(TreeFile { + writer, + partial, + final_path, + size, + }) + } +} + +/// One file of a [`TreeSink`], being written. +pub(crate) struct TreeFile { + writer: crate::io::WritebackFile, + partial: PathBuf, + final_path: PathBuf, + size: u64, +} + +impl TreeFile { + /// Append `data`. A failed write removes the `.partial`. + pub(crate) fn write(&mut self, data: &[u8]) -> Result<()> { + self.writer.write_all(data).map_err(|e| { + let _ = std::fs::remove_file(&self.partial); + Error::DirWriteFailed { + errno: e.raw_os_error(), + } + }) + } + + /// Publish the file: flush, sync, set its exact declared length, rename `.partial` → + /// final, sync the directory entry. A file left unfinished (a Stop) stays `.partial`. + pub(crate) fn finish(self) -> Result<()> { + let TreeFile { + mut writer, + partial, + final_path, + size, + } = self; + writer.sync_all().map_err(|e| Error::DirWriteFailed { + errno: e.raw_os_error(), + })?; + drop(writer); + // Set the exact declared length (covers both an over-read final sector and + // an under-covered sparse tail). + let f = std::fs::OpenOptions::new() + .write(true) + .open(&partial) + .map_err(|e| Error::DirWriteFailed { + errno: e.raw_os_error(), + })?; + f.set_len(size).map_err(|e| Error::DirWriteFailed { + errno: e.raw_os_error(), + })?; + // `set_len` is a separate kernel op on this second handle; without an fsync + // here, a crash between `set_len` and the rename can leave the file at its + // pre-truncation length. Sync the new metadata (length) before publishing. + f.sync_all().map_err(|e| Error::DirWriteFailed { + errno: e.raw_os_error(), + })?; + drop(f); + std::fs::rename(&partial, &final_path).map_err(|e| Error::DirWriteFailed { + errno: e.raw_os_error(), + })?; + // Durably commit the new dirent: on POSIX filesystems a crash right after a + // rename can lose the directory entry even though the rename returned. Best + // effort (swallowed on failure); no-op on Windows. Matches `write_atomic`. + if let Some(dir) = final_path.parent() { + crate::io::fsync::dir(dir); + } + Ok(()) + } +} + +/// Open the tree output `url` (`dir://`); `force` writes into a non-empty folder. +/// Any other scheme is [`Error::StreamUrlInvalid`]. +pub fn open_tree_sink(url: &str, force: bool) -> Result { + match crate::mux::parse_url(url) { + crate::mux::StreamUrl::Dir { path } => TreeSink::create(&path, force), + _ => Err(Error::StreamUrlInvalid { + url: url.to_string(), + }), + } +} + +/// Whether the filesystem holding `dir` folds case: create a lowercase marker, test for its +/// uppercase spelling. Two disc names differing only in case are distinct on a +/// case-sensitive volume, so folding them together would wrongly abort a legitimate +/// extract. If the probe can't run (e.g. a read-only dir), assume case-insensitive — the +/// conservative choice that never MISSES a real overwrite collision. +pub(crate) fn dir_is_case_insensitive(dir: &Path) -> bool { + probe_case_insensitive(dir, |p| std::fs::remove_file(p)) +} + +// `dir_is_case_insensitive` with the marker removal injectable (tests fail it). +pub(crate) fn probe_case_insensitive( + dir: &Path, + mut remove: impl FnMut(&Path) -> std::io::Result<()>, +) -> bool { + // Unique per attempt: a FIXED name let concurrent extracts race — one's + // `remove_file` could delete another's marker between create and `exists()` + // (a TOCTOU flip). A per-call token keeps each probe's pair on private paths. + let token = unique_probe_token(); + let lower_name = format!(".fmkv_case_probe_{token}"); + let lower = dir.join(&lower_name); + if std::fs::File::create(&lower).is_err() { + return true; + } + // The upper spelling is the SAME name uppercased end-to-end (hex token + // letters `a-f` fold to `A-F`), so on a case-insensitive volume it resolves + // to the file just created and on a case-sensitive one it does not exist. + let upper = dir.join(lower_name.to_ascii_uppercase()); + let insensitive = upper.exists(); + // Retry: a transient delete failure (Windows AV/indexer handle) would leave the + // marker in the user's target, failing a re-run's non-empty check. + for attempt in 0..PROBE_REMOVE_ATTEMPTS { + match remove(&lower) { + Ok(()) => break, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => break, + Err(_) if attempt + 1 < PROBE_REMOVE_ATTEMPTS => { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + // Still failing: best-effort by design; the probe's answer stands. + Err(_) => {} + } + } + insensitive +} + +/// A process-unique, lowercase-hex token for the case-probe filename. No `rand` +/// dependency: a monotonic counter (distinguishes concurrent same-process +/// probes) mixed with the pid and a nanosecond clock (distinguishes processes / +/// runs). Hex digits are case-foldable, which the probe relies on. +pub(crate) fn unique_probe_token() -> String { + use std::sync::atomic::{AtomicU64, Ordering}; + static COUNTER: AtomicU64 = AtomicU64::new(0); + let n = COUNTER.fetch_add(1, Ordering::Relaxed); + let pid = std::process::id(); + let nanos = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_nanos()) + .unwrap_or(0); + format!("{pid:x}_{nanos:x}_{n:x}") +} + +/// Append `.partial` to a path's filename. +pub(crate) fn with_partial_suffix(path: &Path) -> PathBuf { + let mut name = path.file_name().unwrap_or_default().to_os_string(); + name.push(".partial"); + path.with_file_name(name) +} + +// Sanitizes ONE disc-path component: host-illegal chars and control bytes become `_`, trailing +// dot/space are stripped, a Windows reserved device name gets a `_` prefix. Names that +// collapse together are caught by the collision check. +pub(crate) fn sanitize_component(name: &str) -> String { + let mapped: String = name + .chars() + .map(|c| match c { + '/' | '\\' | ':' | '<' | '>' | '"' | '|' | '?' | '*' => '_', + c if (c as u32) < 0x20 => '_', + c => c, + }) + .collect(); + let trimmed = mapped.trim_end_matches([' ', '.']); + if trimmed.is_empty() { + return "_".to_string(); + } + // The device name is the stem before any extension, ignoring trailing spaces. + let base = trimmed.split('.').next().unwrap_or(trimmed); + if is_windows_reserved(base.trim_end_matches(' ')) { + return format!("_{trimmed}"); + } + trimmed.to_string() +} + +/// Whether `base` (the name component before any extension) matches a Windows +/// reserved device name. These are reserved by the OS regardless of extension +/// and silently alias a device (e.g. `NUL` discards writes). Case-insensitive. +pub(crate) fn is_windows_reserved(base: &str) -> bool { + let up = base.trim_end_matches(' ').to_ascii_uppercase(); + if matches!( + up.as_str(), + "CON" | "PRN" | "AUX" | "NUL" | "CONIN$" | "CONOUT$" | "CLOCK$" + ) { + return true; + } + ["COM", "LPT"].iter().any(|p| { + up.strip_prefix(p).is_some_and(|d| { + let mut c = d.chars(); + matches!( + (c.next(), c.next()), + (Some('0'..='9' | '\u{B9}' | '\u{B2}' | '\u{B3}'), None) + ) + }) + }) +} + +/// Available free bytes on the filesystem holding `dir`, or `None` when the +/// platform doesn't expose it (the free-space gate is then skipped). +#[cfg(unix)] +pub(crate) fn available_space(dir: &Path) -> Option { + use std::os::unix::ffi::OsStrExt; + let cpath = std::ffi::CString::new(dir.as_os_str().as_bytes()).ok()?; + // No safe std API for free space. SAFETY: `statvfs` is plain-old-data (all-zero is + // valid); `cpath` is NUL-terminated and outlives the call; `st` is a valid out-pointer. + let mut st: libc::statvfs = unsafe { std::mem::zeroed() }; + let rc = unsafe { libc::statvfs(cpath.as_ptr(), &mut st) }; + if rc != 0 { + return None; + } + // `statvfs` field integer widths differ by platform; cast both to `u64` + // for the product (no-op where already `u64` — allow lint for portability). + #[allow(clippy::unnecessary_cast, clippy::useless_conversion)] + let avail = (st.f_bavail as u64).saturating_mul(st.f_frsize as u64); + Some(avail) +} + +// Windows has no `statvfs`; queries free space via `GetDiskFreeSpaceExW` (declared directly +// against kernel32, matching `scsi::windows`) so the free-space gate still runs. +#[cfg(windows)] +pub(crate) fn available_space(dir: &Path) -> Option { + use std::os::windows::ffi::OsStrExt; + + unsafe extern "system" { + fn GetDiskFreeSpaceExW( + lpDirectoryName: *const u16, + lpFreeBytesAvailableToCaller: *mut u64, + lpTotalNumberOfBytes: *mut u64, + lpTotalNumberOfFreeBytes: *mut u64, + ) -> i32; + } + + // Wide, NUL-terminated. An interior NUL cannot reach the API, so reject it + // rather than silently truncating the path and measuring the wrong volume. + let mut wide: Vec = dir.as_os_str().encode_wide().collect(); + if wide.contains(&0) { + return None; + } + wide.push(0); + + let mut avail: u64 = 0; + // FreeBytesAvailableToCaller honours per-user quotas (unix `f_bavail`). + // SAFETY: `wide` is NUL-terminated and outlives the call; `avail` is a valid + // out-pointer; the two NULL out-params are documented optional. + let ok = unsafe { + GetDiskFreeSpaceExW( + wide.as_ptr(), + &mut avail, + std::ptr::null_mut(), + std::ptr::null_mut(), + ) + }; + if ok == 0 { None } else { Some(avail) } +} + +/// Neither unix nor Windows: no way to ask, so the gate is skipped. +#[cfg(not(any(unix, windows)))] +pub(crate) fn available_space(_dir: &Path) -> Option { + None +} + +/// Whether a directory exists and contains any entry. +pub(crate) fn dir_is_non_empty(dir: &Path) -> bool { + std::fs::read_dir(dir) + .map(|mut it| it.next().is_some()) + .unwrap_or(false) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_dir_url_opens_a_tree_sink_and_other_schemes_do_not() { + let dir = tempfile::tempdir().unwrap(); + let out = dir.path().join("tree"); + let sink = open_tree_sink(&format!("dir://{}", out.display()), false).unwrap(); + assert_eq!(sink.dest(), out.as_path()); + assert!(out.is_dir()); + assert!(open_tree_sink("iso:///tmp/x.iso", false).is_err()); + } + + #[test] + fn a_published_file_has_its_declared_length_and_no_partial() { + let dir = tempfile::tempdir().unwrap(); + let sink = TreeSink::create(dir.path(), false).unwrap(); + let mut f = sink.begin(Path::new("a.bin"), 3).unwrap(); + f.write(b"abcdef").unwrap(); + f.finish().unwrap(); + assert_eq!(std::fs::read(dir.path().join("a.bin")).unwrap(), b"abc"); + assert!(!dir.path().join("a.bin.partial").exists()); + } + + #[test] + fn the_aacs_directories_are_not_part_of_the_tree() { + assert!(!TreeSink::keeps_top_level("AACS", false)); + assert!(!TreeSink::keeps_top_level("certificate", false)); + assert!(!TreeSink::keeps_top_level("X!", true)); + assert!(TreeSink::keeps_top_level("BDMV", false)); + } +} diff --git a/src/io/writeback_file/mod.rs b/src/io/writeback_file/mod.rs index 921cb84e..0c47104a 100644 --- a/src/io/writeback_file/mod.rs +++ b/src/io/writeback_file/mod.rs @@ -127,7 +127,7 @@ impl WritebackFile { } /// Share `flush`'s counters (§2.10 item 3): give it the pipeline consumer's - /// [`Progress`](crate::halt::Progress) so a flushing `close()` counts as progress. + /// [`Progress`](crate::halt::Liveness) so a flushing `close()` counts as progress. /// Call before the first write. pub fn set_flush_progress(&mut self, flush: FlushProgress) { flush.note_total(self.flush.bytes_total()); @@ -370,17 +370,6 @@ impl Seek for WritebackFile { } } -impl super::sink::SequentialSink for WritebackFile { - // Same work as sync_all(); implemented explicitly (no blanket impl) so - // a `dyn SequentialSink`/`dyn RandomAccessSink` finish() finalises + - // fsyncs instead of hitting a no-op default. - fn finish(&mut self) -> io::Result<()> { - self.sync_all() - } -} - -impl super::sink::RandomAccessSink for WritebackFile {} - impl Drop for WritebackFile { fn drop(&mut self) { // Run the pipeline's tail finalize (WAIT_AFTER + DONTNEED); otherwise a drop @@ -476,59 +465,6 @@ mod tests { assert_eq!(read_back(&p), b"onetwothree"); } - // finish() through a `dyn RandomAccessSink` must dispatch to - // WritebackFile's override (finalize + durable_sync), not a no-op. - #[test] - fn finish_through_trait_object_persists() { - use crate::io::sink::RandomAccessSink; - let dir = tempfile::tempdir().unwrap(); - let p = dir.path().join("finish-dyn.bin"); - let w = WritebackFile::create(&p).unwrap(); - let mut boxed: Box = Box::new(w); - boxed.write_all(b"durable-tail").unwrap(); - boxed.finish().unwrap(); - assert_eq!(read_back(&p), b"durable-tail"); - } - - // The page cache serves the bytes back whether or not a durable sync ran, so count the - // final flush: `finish()` on a boxed sink must reach it (the trait default does not). - #[test] - fn finish_through_trait_object_runs_the_durable_sync() { - use crate::io::sink::RandomAccessSink; - use std::sync::atomic::{AtomicUsize, Ordering}; - #[derive(Default)] - struct CountFinish(AtomicUsize); - impl FlushOps for CountFinish { - fn chunk(&self, _: &File) -> io::Result<()> { - Ok(()) - } - fn range(&self, _: &File, _: u64, _: u64) -> Option> { - None - } - fn finish(&self, _: &File) -> io::Result<()> { - self.0.fetch_add(1, Ordering::SeqCst); - Ok(()) - } - fn sample(&self) -> Option { - None - } - } - let ops = Arc::new(CountFinish::default()); - let w = WritebackFile::with_flush_ops( - tempfile::tempfile().unwrap(), - ops.clone(), - FlushTiming::default(), - ) - .unwrap(); - let mut boxed: Box = Box::new(w); - boxed.write_all(b"durable-tail").unwrap(); - boxed.finish().unwrap(); - assert!( - ops.0.load(Ordering::SeqCst) > 0, - "finish() skipped the final flush" - ); - } - // A writeback error the pipeline latched (Linux: a failed WAIT_AFTER already // consumed it, so fsync returns 0) must fail sync_all and every later write. #[test] diff --git a/src/keys/arrival.rs b/src/keys/arrival.rs index c2c950d4..dcc7d7c1 100644 --- a/src/keys/arrival.rs +++ b/src/keys/arrival.rs @@ -10,7 +10,7 @@ //! partnerless unit (another LBA) opens under no key before any proof; never in a clear piece. //! An image or folder (`StopKind::Image`) never stops: those units are blanked and counted. -use super::{Proof, ProofCache, ResolvedKeySet, StopKind}; +use super::{KeyRing, Proof, ProofCache, StopKind}; use crate::aacs::content::{ ALIGNED_UNIT_LEN, aacs_unit_encrypted, aacs_unit_on_grid, clear_copy_permission_indicator, decrypt_unit, is_clean, @@ -42,7 +42,7 @@ pub(crate) struct Arrival { } impl Arrival { - pub(crate) fn new(set: &ResolvedKeySet, stop: StopKind) -> Self { + pub(crate) fn new(set: &KeyRing, stop: StopKind) -> Self { let i = &set.0; let mut spans = Vec::new(); let mut pieces = Vec::with_capacity(i.arrival.len()); diff --git a/src/keys/evidence.rs b/src/keys/evidence.rs new file mode 100644 index 00000000..ceb8cfa2 --- /dev/null +++ b/src/keys/evidence.rs @@ -0,0 +1,461 @@ +//! Key evidence (pipeline design §2.6): what a source can tell key acquisition, with no +//! `Disc` in it. [`KeyRing::acquire`](super::KeyRing::acquire) takes evidence plus a +//! [`Sampler`] over the same source. Until the Layout stage exists, evidence comes from a +//! scanned disc through [`KeyEvidence::from_disc`]. + +use super::KeyScope; +use crate::ctx::Ctx; +use crate::disc::{ContentFormat, Disc, DiscFormat, Extent}; +use crate::error::{Error, Result}; +use crate::halt::Halt; +use crate::sector::SectorSource; +use crate::whole_disc::{UNIT, UnitSpan, probe_units, subtract_ranges, unit_head}; + +/// The identity a ring is bound to: [`KeyRing::is_for`](super::KeyRing::is_for) compares +/// it. No key bytes; the Volume ID only as its fingerprint. +#[derive(Clone, Debug, PartialEq)] +pub struct MediaId { + /// SHA-1 of the title-key file (`Unit_Key_RO.inf` / VTKF), hex; empty when the medium + /// has no AACS identity. + pub disc_hash: String, + /// Capacity in sectors; 0 when unknown (then never compared). + pub capacity: u32, + pub format: DiscFormat, + /// [`KeyRing::vid_fingerprint`](super::KeyRing::vid_fingerprint) of the Volume ID, when + /// one is known. + pub vid_fingerprint: Option<[u8; 32]>, +} + +impl Disc { + /// This disc's [`MediaId`]: its AACS disc hash, capacity, format and, when the + /// handshake read one, its Volume ID fingerprint. + pub fn media_id(&self) -> MediaId { + let aacs = self.aacs.as_ref(); + MediaId { + // KA9: a capture with no hash is identified by its title-key file. + disc_hash: aacs.map_or_else(String::new, |a| { + if a.disc_hash.is_empty() && !a.uk_ro.is_empty() { + crate::aacs::inf::disc_hash_hex(&crate::aacs::inf::disc_hash(&a.uk_ro)) + } else { + a.disc_hash.clone() + } + }), + capacity: self.capacity_sectors, + format: self.format, + vid_fingerprint: aacs + .map(|a| a.volume_id) + .filter(|v| *v != [0u8; 16]) + .map(|v| super::vid_fp(&v)), + } + } +} + +/// Whether the medium's encrypted-unit test can be trusted (design X-10). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Detector { + /// BD/UHD CPI flags: keys are proven on ciphertext. + Verified, + /// HD DVD: the test is unverified, so a single declared key is applied without proof. + Unverified, +} + +/// One unit of key proof (design §2.2): a stream file, a title extent no file covers, or a +/// whole loose file. Spans are sectors `(start, count, unit anchor)`, sorted pieces by +/// first sector. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Piece { + pub spans: Vec, + /// The titles (indices) that play it: only pieces sharing a title lend samples (SG23). + pub titles: Vec, + /// Size in bytes of the largest of those titles: pieces are asked largest first. + pub rank: u64, +} + +impl Piece { + /// The piece's first sector, its id. + pub fn id(&self) -> u32 { + self.spans.first().map_or(0, |s| s.0) + } + + pub(crate) fn ranges(&self) -> impl Iterator + '_ { + self.spans.iter().map(|&(s, n, _)| (s, s.saturating_add(n))) + } + + // Unit heads on the piece's grid, as (first head, unit count) per span. A unit that + // would cross its span's end is not a unit of the piece (KS-7, Informative). + pub(crate) fn grid(&self) -> Vec<(u64, u64)> { + self.spans + .iter() + .map(|&(s, n, anchor)| { + let head = unit_head(s, anchor); + let end = s as u64 + n as u64; + (head, end.saturating_sub(head) / UNIT) + }) + .collect() + } + + // A loose clip file's one piece: every aligned unit of the file (KS-1) on its own grid + // from byte 0, title 0. + pub(crate) fn loose_file(capacity: u32) -> Self { + Piece { + spans: vec![(0, capacity, 0)], + titles: vec![0], + rank: 0, + } + } +} + +// The AACS structures a source supplied. Key material: never printed. +pub(crate) struct AacsEvidence { + pub(crate) version: u8, + pub(crate) mkb: Vec, + pub(crate) unit_key_ro: Vec, + // In memory only; `None` when no handshake read one. + pub(crate) vid: Option<[u8; 16]>, + // `Num_of_CPS_Unit` the title-key file declares (KS-14); `None` if unparseable. + pub(crate) n_declared: Option, + pub(crate) volume_label: Option, +} + +// The main feature: the samples a sample-independent request carries. +pub(crate) struct MainTitle { + pub(crate) extents: Vec, + pub(crate) format: ContentFormat, +} + +/// What a source can tell key acquisition (design §2.6), scoped to what the rip decrypts: +/// the medium's identity and AACS structures, the pieces in scope, the main feature, the +/// detector and the forensic layout. Holds key material (the title-key file, MKB, VID), so +/// its `Debug` redacts them. +pub struct KeyEvidence { + pub(crate) scope: KeyScope, + pub(crate) media: MediaId, + pub(crate) aacs: Option, + pub(crate) pieces: Vec, + // No stream file was listed (no `/BDMV/STREAM` or `/HVDVD_TS`, or an empty one). + pub(crate) no_stream_files: bool, + pub(crate) main: Option, + pub(crate) detector: Detector, + pub(crate) container: ContentFormat, + pub(crate) fmts: Option, +} + +impl std::fmt::Debug for KeyEvidence { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("KeyEvidence") + .field("scope", &self.scope) + .field("media", &self.media) + .field("aacs", &self.aacs.as_ref().map(|_| "")) + .field("pieces", &self.pieces.len()) + .field("detector", &self.detector) + .field("container", &self.container) + .field("fmts", &self.fmts.is_some()) + .finish() + } +} + +impl KeyEvidence { + /// The scope this evidence was gathered for. + pub fn scope(&self) -> &KeyScope { + &self.scope + } + + /// The medium's identity. + pub fn media(&self) -> &MediaId { + &self.media + } + + /// The pieces in scope, sorted by first sector. + pub fn pieces(&self) -> &[Piece] { + &self.pieces + } + + /// Evidence from a scanned disc over its raw `reader` (the interim adapter until the + /// Layout stage supplies pieces). Reads the filesystem for the stream files and the + /// forensic layout. An unreadable filesystem fails a `WholeDisc` scope; a title scope + /// falls back to its title extents. A clear disc, or `KeyScope::None`, reads nothing. + pub fn from_disc( + disc: &Disc, + reader: &mut dyn SectorSource, + scope: KeyScope, + ctx: &Ctx, + ) -> Result { + ctx.halt.check()?; + let mut ev = KeyEvidence { + scope, + media: disc.media_id(), + aacs: None, + pieces: Vec::new(), + no_stream_files: false, + main: disc.main_title().map(|t| MainTitle { + extents: t.extents.clone(), + format: t.content_format, + }), + detector: match disc.format { + DiscFormat::HdDvd => Detector::Unverified, + _ => Detector::Verified, + }, + container: disc.content_format, + fmts: None, + }; + let (Some(inputs), false) = (disc.inputs(), ev.scope == KeyScope::None) else { + return Ok(ev); + }; + let sel: Vec = match &ev.scope { + KeyScope::Titles(v) => { + if let Some(&bad) = v.iter().find(|&&t| t >= disc.titles.len()) { + return Err(Error::DiscTitleRange { + index: bad, + count: disc.titles.len(), + }); + } + v.clone() + } + _ => (0..disc.titles.len()).collect(), + }; + let n_declared = disc.declared_cps_units(); + ev.aacs = Some(AacsEvidence { + version: inputs.version, + mkb: inputs.mkb, + unit_key_ro: inputs.unit_key_ro, + vid: Some(inputs.volume_id).filter(|v| *v != [0u8; 16]), + n_declared, + volume_label: inputs.volume_label, + }); + let whole = ev.scope == KeyScope::WholeDisc; + if ev.detector == Detector::Unverified { + // Acquisition refuses a multi-key HD DVD before it lists any file. + if n_declared != Some(1) { + return Ok(ev); + } + let files = match crate::whole_disc::content_files(reader) { + Ok(f) => f, + Err(e) if whole => return Err(e), + Err(e) => { + tracing::warn!(target: "freemkv::keys", error = %e, "no HD DVD file list: keying title extents"); + Vec::new() + } + }; + ev.no_stream_files = files.is_empty(); + ev.pieces = pieces(disc, &files, &sel, whole); + return Ok(ev); + } + // An unreadable filesystem fails a whole-disc copy (it would otherwise ship + // ciphertext); a title rip keys its title extents instead. + let fs = match crate::udf::read_filesystem(reader) { + Ok(fs) => Some(fs), + Err(e) if whole => return Err(e), + Err(e) => { + tracing::warn!(target: "freemkv::keys", error = %e, "no filesystem: keying title extents"); + None + } + }; + let files = match &fs { + Some(fs) => match crate::whole_disc::content_files_in(fs, reader) { + Ok(f) => f, + Err(e) if whole => return Err(e), + Err(e) => { + tracing::warn!(target: "freemkv::keys", error = %e, "no file list: keying title extents"); + Vec::new() + } + }, + None => Vec::new(), + }; + ev.no_stream_files = files.is_empty(); + ev.pieces = pieces(disc, &files, &sel, whole); + ev.fmts = match &fs { + Some(fs) => super::fmts::layout(fs, reader)?, + None => None, + }; + Ok(ev) + } +} + +pub(crate) fn sorted_ranges(mut v: Vec<(u32, u32)>) -> Vec<(u32, u32)> { + v.sort_unstable(); + let mut out: Vec<(u32, u32)> = Vec::with_capacity(v.len()); + for (s, e) in v { + match out.last_mut() { + Some(last) if s <= last.1 => last.1 = last.1.max(e), + _ => out.push((s, e)), + } + } + out +} + +// The pieces of `sel`: each content file the scope touches (on its own unit grid, KS-1), +// minus sectors an earlier file claimed (an SSIF re-lists m2ts sectors), then title extents +// no file covers (anchored at the extent start). +pub(crate) fn pieces( + disc: &Disc, + files: &[Vec<(u32, u32)>], + sel: &[usize], + whole: bool, +) -> Vec { + let title_ranges: Vec<(usize, u32, u32)> = sel + .iter() + .flat_map(|&t| { + disc.titles[t] + .extents + .iter() + .filter(|e| e.sector_count > 0) + .map(move |e| (t, e.start_lba, e.start_lba.saturating_add(e.sector_count))) + }) + .collect(); + let touches = |ranges: &[(u32, u32)]| -> Vec { + let mut t: Vec = title_ranges + .iter() + .filter(|&&(_, s, e)| super::overlaps(ranges, s, e)) + .map(|&(t, _, _)| t) + .collect(); + t.sort_unstable(); + t.dedup(); + t + }; + let mut claimed: Vec<(u32, u32)> = Vec::new(); + let mut out = Vec::new(); + let push = |spans: Vec, out: &mut Vec| { + let ranges: Vec<(u32, u32)> = spans + .iter() + .map(|&(s, n, _)| (s, s.saturating_add(n))) + .collect(); + let titles = touches(&ranges); + let rank = titles + .iter() + .map(|&t| disc.titles[t].size_bytes) + .max() + .unwrap_or(0); + out.push(Piece { + spans, + titles, + rank, + }); + }; + for file in files { + let ends: Vec<(u32, u32)> = file + .iter() + .map(|&(s, n)| (s, s.saturating_add(n))) + .collect(); + if !whole && touches(&ends).is_empty() { + continue; + } + let mut spans = Vec::new(); + let mut off = 0u64; + for &(lba, n) in file { + let anchor = (lba as u64).saturating_sub(off % UNIT); + for (s, e) in subtract_ranges(&[(lba, n)], &claimed) { + spans.push((s, e - s, anchor)); + } + off += n as u64; + } + claimed = sorted_ranges(claimed.into_iter().chain(ends).collect()); + if !spans.is_empty() { + push(spans, &mut out); + } + } + for &(_, s, e) in &title_ranges { + let left = subtract_ranges(&[(s, e - s)], &claimed); + if left.is_empty() { + continue; + } + let spans = left.iter().map(|&(a, b)| (a, b - a, s as u64)).collect(); + claimed = sorted_ranges(claimed.into_iter().chain(left).collect()); + push(spans, &mut out); + } + out.sort_by_key(Piece::id); + out +} + +/// The one way key acquisition reads ciphertext (design KA6): whole aligned units from the +/// raw (never decrypted) source the evidence describes. Random access is required. +pub struct Sampler<'a> { + reader: &'a mut dyn SectorSource, +} + +/// A sampled unit: `Enc` ciphertext, `Clear`, or a `Fault` (a soft read error). +pub(crate) enum Sample { + Enc(Vec), + Clear, + Fault, +} + +impl<'a> Sampler<'a> { + /// A sampler over the raw source `reader`. + pub fn new(reader: &'a mut dyn SectorSource) -> Self { + Sampler { reader } + } + + pub(crate) fn source(&mut self) -> &mut dyn SectorSource { + self.reader + } + + // The aligned unit at `lba`, or `None` on a short or failed read. A Stop, a gone source + // or a transport failure is `Err`: not a soft fault. + pub(crate) fn unit(&mut self, lba: u32) -> Result>> { + read_unit(self.reader, lba) + } + + // Up to 32 units on `p`'s grid (KU §2.3 step 7), skipping FMTS segment units. + pub(crate) fn probe( + &mut self, + p: &Piece, + segments: &[(u32, u32)], + format: ContentFormat, + halt: &Halt, + ) -> Result<(u64, Vec)> { + let grid = p.grid(); + let units: u64 = grid.iter().map(|g| g.1).sum(); + let mut out = Vec::new(); + for idx in probe_units(units) { + halt.check()?; + let mut k = idx; + let Some(&(head, _)) = grid.iter().find(|g| { + let hit = k < g.1; + if !hit { + k -= g.1; + } + hit + }) else { + continue; + }; + let Ok(lba) = u32::try_from(head + k * UNIT) else { + continue; + }; + if in_segment(segments, lba) { + continue; + } + out.push(match self.unit(lba)? { + Some(u) if crate::aacs::content::aacs_unit_encrypted(&u, format) => Sample::Enc(u), + Some(_) => Sample::Clear, + None => Sample::Fault, + }); + } + Ok((units, out)) + } + + // Up to `n` encrypted units spread over the main feature. + pub(crate) fn main_samples(&mut self, main: Option<&MainTitle>, n: usize) -> Vec> { + main.map(|m| crate::keysource::encrypted_units_in(self.reader, &m.extents, m.format, n)) + .unwrap_or_default() + } +} + +pub(crate) fn read_unit(reader: &mut dyn SectorSource, lba: u32) -> Result>> { + let mut buf = vec![0u8; crate::aacs::content::ALIGNED_UNIT_LEN]; + match reader.read_sectors(lba, UNIT as u16, &mut buf, false) { + Ok(n) if n == buf.len() => Ok(Some(buf)), + Err(e) if fatal_read(&e) => Err(e), + _ => Ok(None), + } +} + +fn fatal_read(e: &Error) -> bool { + matches!(e, Error::Halted) + || e.is_source_terminated() + || (e.is_scsi_transport_failure() && !matches!(e, Error::IoError { .. })) +} + +fn in_segment(segments: &[(u32, u32)], lba: u32) -> bool { + let i = segments.partition_point(|s| s.0 <= lba); + i > 0 && lba < segments[i - 1].1 +} diff --git a/src/keys/fmts.rs b/src/keys/fmts.rs index c5cbeeac..81eaf5a1 100644 --- a/src/keys/fmts.rs +++ b/src/keys/fmts.rs @@ -9,7 +9,6 @@ use crate::disc::{ContentFormat, Extent}; use crate::error::{Error, Result}; use crate::halt::Halt; use crate::sector::SectorSource; -use crate::whole_disc::UNIT; use std::collections::HashMap; use std::io; @@ -107,11 +106,8 @@ fn read_unit( ) -> Option> { let byte = seg.start_spn as u64 * SPN_BYTES + index as u64 * ALIGNED_UNIT_LEN as u64; let lba = clip_byte_to_lba(clip, byte)?; - let mut unit = vec![0u8; ALIGNED_UNIT_LEN]; - match reader.read_sectors(lba, UNIT as u16, &mut unit, false) { - Ok(n) if n == ALIGNED_UNIT_LEN => Some(unit), - _ => None, - } + // Any failed read is a fault here, a Stop included: the anchor goes Pending. + super::evidence::read_unit(reader, lba).ok().flatten() } /// Sends one anchor batch to the sources: `Ok(None)` for an empty answer. diff --git a/src/keys/mod.rs b/src/keys/mod.rs index 221e7394..f25ba551 100644 --- a/src/keys/mod.rs +++ b/src/keys/mod.rs @@ -1,7 +1,10 @@ -//! Keys up front, memory only (keys-upfront design, KU). +//! Keys up front, memory only (keys-upfront design, KU; pipeline design §2.6). //! -//! [`ResolvedKeySet::resolve`] is the one place that asks a [`crate::KeySource`] for keys. -//! It runs once per rip, before any output, over the rip's [`KeyScope`]. It proves each +//! [`KeyRing::acquire`] is the one place that asks a [`crate::KeySource`] for keys. It +//! takes [`KeyEvidence`] (what the source can tell: identity, AACS structures, pieces) and +//! a [`Sampler`] over that source, never a `Disc`; [`KeyEvidence::from_disc`] builds the +//! evidence from a scanned disc. It runs once per rip, before any output, over the rip's +//! [`KeyScope`]. It proves each //! held key against real ciphertext of each stream file (a *piece*), counting CPS units by //! the DECLARED `Num_of_CPS_Unit` (KS-14), never by the keys held. The result is an //! immutable, in-memory set: keys and the disc's Volume ID live in it for this rip only, @@ -13,6 +16,7 @@ //! readable unit is never withheld; one that no held key opens is a loud stop. mod arrival; +mod evidence; mod fmts; #[cfg(doctest)] mod public_api_cannot_decrypt_aacs; @@ -21,30 +25,22 @@ mod resolve; mod tests; pub(crate) use arrival::Arrival; +pub use evidence::{Detector, KeyEvidence, MediaId, Piece, Sampler}; #[cfg(test)] pub(crate) use resolve::whole_disc_pieces; use crate::aacs::trace::ResolutionTrace; +use crate::ctx::Ctx; use crate::decrypt::{AacsKeyMap, DecryptKeys}; use crate::disc::{ContentFormat, Disc, DiscFormat}; use crate::error::{Error, Result}; -use crate::halt::Halt; +use crate::halt::{Halt, Liveness}; use crate::sector::{DecryptingSectorSource, SectorSource}; use crate::session::KeySourceFactory; use crate::whole_disc::{UnitSpan, WholeDiscReader}; use std::collections::HashMap; use std::sync::{Arc, Mutex}; -// A title's `DiscStream` read through `map`: the read plan, then each unit's mapped key. -pub(crate) fn install_key_map( - mut stream: crate::mux::DiscStream, - map: Arc, -) -> crate::mux::DiscStream { - stream.plan_reads(&map); - stream.reader_mut().set_key_map(map); - stream -} - // A test's own phased map; map construction stays in `keys` (KU §2.2). #[cfg(test)] pub(crate) fn test_key_map( @@ -59,10 +55,10 @@ pub(crate) fn test_keyed_source( src: DecryptingSectorSource, map: Arc, ) -> DecryptingSectorSource { - src.with_key_map(map) + src.keyed_for_test(map) } -/// What a rip decrypts, and so what `resolve` must key (KU §2.5). +/// What a rip decrypts, and so what `acquire` must key (KU §2.5). #[derive(Clone, Debug, PartialEq, Eq)] pub enum KeyScope { /// Nothing is decrypted (a raw copy): no key-source call at all. @@ -73,28 +69,29 @@ pub enum KeyScope { WholeDisc, } -/// Options for [`ResolvedKeySet::resolve`]. +/// Options for [`KeyRing::acquire`]. The stop token is the run's [`Ctx`]. #[derive(Default)] -pub struct ResolveKeysOptions<'a> { - /// Stop token: checked before each piece, probe and request; interrupts retry waits. - pub halt: Option<&'a Halt>, - /// A set this rip already holds (e.g. from `info` or the scan): its keys join the pool - /// first, and its in-memory VID is used when the disc has none (KU §5.4). - pub seed: Option<&'a ResolvedKeySet>, +pub struct AcquireOptions<'a> { + /// A ring this rip already holds (e.g. from `info` or the scan): its keys join the pool + /// first, and its in-memory VID is used when the evidence has none (KU §5.4). + pub seed: Option<&'a KeyRing>, /// An in-memory VID the caller re-read from the drive (KU §4.2). Never written. pub vid: Option<[u8; 16]>, - /// Set by `resolve` when no VID was in hand and one would help (KU J23): a source - /// reported a Media Key it could not finish without the VID (`KeyNode::NoVid`), or a - /// configured source [`uses_vid`](crate::KeySource::uses_vid). Never cleared. + /// Set when no VID was in hand and one would help (KU J23): a source reported a Media + /// Key it could not finish without the VID (`KeyNode::NoVid`), or a configured source + /// [`uses_vid`](crate::KeySource::uses_vid). Never cleared. pub vid_would_help: Option<&'a std::sync::atomic::AtomicBool>, /// Filled with the per-source walk on `Ok` and on `Err` alike, so a refusal still says /// which source answered what. It holds labels, node enums and counts, never key bytes. pub trace: Option<&'a Mutex>, + /// The op's progress (stop design §2.1, T29): every source call runs + /// [`busy`](Liveness::busy) on it, and each source's ctx hands it out. + pub liveness: Option<&'a Liveness>, } -/// The outcome of [`ResolvedKeySet::resolve`]: the set, plus the per-source walk. +/// The outcome of [`KeyRing::acquire`]: the ring, plus the per-source walk. pub struct KeyResolution { - pub keys: ResolvedKeySet, + pub keys: KeyRing, pub trace: ResolutionTrace, } @@ -287,19 +284,19 @@ impl Inner { } } -/// The rip's keys, resolved up front and held in memory only (KU §2.1). +/// The rip's keys, acquired up front and held in memory only (KU §2.1). /// -/// Immutable and cheap to clone (an `Arc`); `Send + Sync`. Bound to one disc +/// Immutable and cheap to clone (an `Arc`); `Send + Sync`. Bound to one medium /// ([`is_for`](Self::is_for)) and one [`KeyScope`]. Holds no key-source factory, so nothing -/// can ask a source after [`resolve`](Self::resolve) returns. No `Serialize`, no accessor +/// can ask a source after [`acquire`](Self::acquire) returns. No `Serialize`, no accessor /// for key bytes or the Volume ID, and a redacting `Debug`. #[derive(Clone)] -pub struct ResolvedKeySet(pub(crate) Arc); +pub struct KeyRing(pub(crate) Arc); -impl std::fmt::Debug for ResolvedKeySet { +impl std::fmt::Debug for KeyRing { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { let i = &self.0; - f.debug_struct("ResolvedKeySet") + f.debug_struct("KeyRing") .field("aacs", &i.aacs) .field("disc_hash", &i.disc_hash) .field("scope", &i.scope) @@ -319,6 +316,11 @@ pub(super) fn overlaps(a: &[(u32, u32)], s: u32, e: u32) -> bool { a.iter().any(|&(x, y)| x < e && s < y) } +// `SHA-256("freemkv-vid-fp-v1" ‖ VID)`, the only form of a VID that may be written. +pub(crate) fn vid_fp(vid: &[u8; 16]) -> [u8; 32] { + sha256(b"freemkv-vid-fp-v1", vid) +} + // SHA-256 over a domain tag and bytes (KU §4.1 fingerprints). fn sha256(tag: &[u8], bytes: &[u8]) -> [u8; 32] { use sha2::Digest; @@ -328,49 +330,42 @@ fn sha256(tag: &[u8], bytes: &[u8]) -> [u8; 32] { h.finalize().into() } -impl ResolvedKeySet { - /// Resolve the keys `scope` needs, once, before any output (KU §2.3). The only code in - /// the product that asks a [`crate::KeySource`] for keys; `sources` is called once and - /// every source it builds is dropped before this returns (LK7). +impl KeyRing { + /// Acquire the keys `ev.scope()` needs, once, before any output (KU §2.3). The only + /// code in the product that asks a [`crate::KeySource`] for keys; `sources` is called + /// once and every source it builds is dropped before this returns (LK7). `sampler` + /// reads the evidence's pieces from the same raw source; `ctx.halt` stops it. /// /// `Err` refuses the rip before any output: E7022 (a title piece is Missing), E7032 (an /// image or folder piece is Missing), E7013 (a single declared key opens none of the /// ciphertext), E7026 (forensic keys Missing), E7028–30 (a source failed), `Halted`. - pub fn resolve( - disc: &Disc, - reader: &mut dyn SectorSource, - scope: KeyScope, + pub fn acquire( + ev: &KeyEvidence, + sampler: &mut Sampler, sources: &KeySourceFactory, - opts: ResolveKeysOptions, + opts: AcquireOptions, + ctx: &Ctx, ) -> Result { - resolve::resolve( - disc, - reader, - scope, - sources, - opts, - &resolve::RealClock::new(), - ) + resolve::acquire(ev, sampler, sources, opts, ctx, &resolve::RealClock::new()) } - /// [`resolve`](Self::resolve), reporting to the op's `progress` (stop design §2.1, - /// T29): every source call runs [`busy`](crate::halt::Progress::busy) on it, and each - /// source's ctx hands it out as [`ResolveCtx::progress`](crate::keysource::ResolveCtx::progress). - pub fn resolve_with_progress( + /// [`from_disc`](KeyEvidence::from_disc) then [`acquire`](Self::acquire) over the same + /// raw `reader`: the scanned-disc form until the Layout stage supplies evidence. + pub fn acquire_for_disc( disc: &Disc, reader: &mut dyn SectorSource, scope: KeyScope, sources: &KeySourceFactory, - opts: ResolveKeysOptions, - progress: &crate::halt::Progress, + opts: AcquireOptions, + ctx: &Ctx, ) -> Result { - let clock = resolve::RealClock::new(); - resolve::resolve_observed(disc, reader, scope, sources, opts, &clock, Some(progress)) + let ev = KeyEvidence::from_disc(disc, reader, scope, ctx)?; + Self::acquire(&ev, &mut Sampler::new(reader), sources, opts, ctx) } /// A set holding no keys: for a raw copy, or a disc with no AACS. - pub fn none() -> ResolvedKeySet { - ResolvedKeySet(Arc::new(Inner::empty())) + pub fn none() -> KeyRing { + KeyRing(Arc::new(Inner::empty())) } /// Whether this set keys AACS content: `false` for [`none`](Self::none), which a caller @@ -379,27 +374,22 @@ impl ResolvedKeySet { self.0.aacs } - /// Whether this set was resolved for `disc`: the same disc hash, capacity and format, + /// Whether this ring was acquired for `media`: the same disc hash, capacity and format, /// and the same VID fingerprint when both are known. [`none`](Self::none) holds nothing - /// and fits any disc. - pub fn is_for(&self, disc: &Disc) -> bool { + /// and fits any medium. + pub fn is_for(&self, media: &MediaId) -> bool { let i = &self.0; if !i.aacs { return true; } - let Some(aacs) = disc.aacs.as_ref() else { - return false; - }; - if norm_hash(&aacs.disc_hash) != norm_hash(&i.disc_hash) - || disc.format != i.format - || (disc.capacity_sectors != 0 - && i.capacity != 0 - && disc.capacity_sectors != i.capacity) + if norm_hash(&media.disc_hash) != norm_hash(&i.disc_hash) + || media.format != i.format + || (media.capacity != 0 && i.capacity != 0 && media.capacity != i.capacity) { return false; } - match (i.vid, aacs.volume_id) { - (Some(held), vid) if vid != [0u8; 16] => held == vid, + match (self.vid_fingerprint(), media.vid_fingerprint) { + (Some(held), Some(fp)) => held == fp, _ => true, } } @@ -440,7 +430,7 @@ impl ResolvedKeySet { /// `SHA-256("freemkv-vid-fp-v1" ‖ VID)`: the only form of the VID that may be written /// (KU §4.1). `None` when the set holds no VID. pub fn vid_fingerprint(&self) -> Option<[u8; 32]> { - self.0.vid.map(|v| sha256(b"freemkv-vid-fp-v1", &v)) + self.0.vid.as_ref().map(vid_fp) } /// `SHA-256("freemkv-key-fp-v1" ‖ key)[..8]` of each proven base key: the legacy @@ -597,8 +587,9 @@ impl ResolvedKeySet { ) -> Result> { let i = &self.0; self.gate(inner.random_access(), extents, allow_pending)?; - let mut dec = - DecryptingSectorSource::new(inner, self.decrypt_keys()).with_key_map(self.key_map()); + let keying = + crate::sector::Keying::ring(self.decrypt_keys(), self.key_map(), self.arrival(stop)); + let mut dec = DecryptingSectorSource::new(inner, keying); if self.forensic_pending() && self.touches_clip(extents) { // Pending segments stay ciphertext (never a stop, KU §2.4): outside content. let spans: Vec<(u32, u32)> = i.spans.iter().map(|&(s, n, _)| (s, n)).collect(); @@ -608,9 +599,6 @@ impl ResolvedKeySet { .collect(); dec = dec.with_content_ranges(Arc::from(crate::whole_disc::merge_ranges(content))); } - if let Some(a) = self.arrival(stop) { - dec = dec.with_arrival(a); - } Ok(dec) } @@ -692,12 +680,12 @@ impl ResolvedKeySet { i.map = Arc::new(AacsKeyMap::from_ranges( ranges.iter().map(|&(s, e)| (s, e, 0)).collect(), )); - ResolvedKeySet(Arc::new(i)) + KeyRing(Arc::new(i)) } // A loose clip file's set (no CPS-unit map): `held`'s base keys, proven on arrival over the // file's one piece. Title scope, so a unit no held key opens stops E7022 (KU §2.4). - pub(crate) fn loose_file(held: Option<&ResolvedKeySet>, capacity: u32) -> Self { + pub(crate) fn loose_file(held: Option<&KeyRing>, capacity: u32) -> Self { let mut i = Inner::empty(); i.aacs = true; i.scope = KeyScope::Titles(vec![0]); @@ -725,7 +713,7 @@ impl ResolvedKeySet { i.scope = KeyScope::WholeDisc; i.pool = keys.to_vec(); i.proven = (0..keys.len()).collect(); - ResolvedKeySet(Arc::new(i)) + KeyRing(Arc::new(i)) } /// The decrypting reader for title `idx` of `disc` over the raw, random-access `inner`: @@ -738,7 +726,7 @@ impl ResolvedKeySet { idx: usize, inner: S, ) -> Result> { - if !self.is_for(disc) { + if !self.is_for(&disc.media_id()) { return Err(Self::caller_bug("key set is not for this disc")); } let title = disc.titles.get(idx).ok_or(Error::DiscTitleRange { @@ -778,7 +766,7 @@ impl ResolvedKeySet { if halt.is_some_and(|h| h.is_cancelled()) { return Err(Error::Halted); } - if !self.is_for(disc) { + if !self.is_for(&disc.media_id()) { return Err(Self::caller_bug("key set is not for this disc")); } let mut content = disc.encrypted_content_ranges(); @@ -819,7 +807,7 @@ impl ResolvedKeySet { /// Whether `disc` can be decrypted with `keys` (KU §3.5). CSS is read from `disc.css` / /// `css_error`; an AACS disc needs a set for it, so `keys == None` is `AacsKeysMissing`. -pub fn decrypt_status(disc: &Disc, keys: Option<&ResolvedKeySet>) -> DecryptStatus { +pub fn decrypt_status(disc: &Disc, keys: Option<&KeyRing>) -> DecryptStatus { if disc.css_error.is_some() { return DecryptStatus::CssNotCracked(Error::CssNoDiscKey); } @@ -831,7 +819,7 @@ pub fn decrypt_status(disc: &Disc, keys: Option<&ResolvedKeySet>) -> DecryptStat } match keys { Some(set) if set.is_aacs() => { - if !set.is_for(disc) { + if !set.is_for(&disc.media_id()) { DecryptStatus::AacsKeysMissing(Error::DecryptFailed) } else if set.forensic_pending() { DecryptStatus::ForensicPending @@ -852,7 +840,7 @@ pub fn decrypt_status(disc: &Disc, keys: Option<&ResolvedKeySet>) -> DecryptStat pub fn check_decryptable( disc: &Disc, raw: bool, - keys: Option<&ResolvedKeySet>, + keys: Option<&KeyRing>, scope: &KeyScope, ) -> Result<()> { if raw || matches!(scope, KeyScope::None) { @@ -860,11 +848,11 @@ pub fn check_decryptable( } match keys { Some(set) if set.is_aacs() && disc.css.is_none() && disc.css_error.is_none() => { - if !set.is_for(disc) { - return Err(ResolvedKeySet::caller_bug("key set is not for this disc")); + if !set.is_for(&disc.media_id()) { + return Err(KeyRing::caller_bug("key set is not for this disc")); } if !set.covers(scope) { - return Err(ResolvedKeySet::caller_bug("scope outside the key set")); + return Err(KeyRing::caller_bug("scope outside the key set")); } if set.forensic_pending() { return Err(Error::FmtsKeyMissing); diff --git a/src/keys/public_api_cannot_decrypt_aacs.rs b/src/keys/public_api_cannot_decrypt_aacs.rs index 3e4a0bb8..0f0f518a 100644 --- a/src/keys/public_api_cannot_decrypt_aacs.rs +++ b/src/keys/public_api_cannot_decrypt_aacs.rs @@ -1,4 +1,4 @@ -//! LK19 (KU §2.2): "within libfreemkv's public API, `ResolvedKeySet` is the only source of +//! LK19 (KU §2.2): "within libfreemkv's public API, `KeyRing` is the only source of //! AACS-decrypted data". Each door below compiled before KU-X2 and must not compile now. //! Per spec; do not change without a spec citation proving otherwise. //! @@ -18,12 +18,6 @@ //! use libfreemkv::decrypt::{AacsKeyMap, Phase}; //! let _ = AacsKeyMap::from_ranges_phased(vec![(0, 3, 0, Phase::All)]); //! ``` -//! `Disc::decrypt_with` (§2.2: "`pub(crate)`"): -//! ```compile_fail -//! fn f(d: &mut libfreemkv::Disc) { -//! let _ = d.decrypt_with(libfreemkv::disc::Key::Unit(Vec::new()), &[]); -//! } -//! ``` //! `DecryptingSectorSource::with_key_map` (§2.2: "`pub(crate)`"): //! ```compile_fail //! use libfreemkv::{DecryptingSectorSource, FileSectorSource, decrypt::AacsKeyMap}; diff --git a/src/keys/resolve.rs b/src/keys/resolve.rs index f723e26d..ee2d081d 100644 --- a/src/keys/resolve.rs +++ b/src/keys/resolve.rs @@ -1,19 +1,19 @@ -//! [`ResolvedKeySet::resolve`] (KU §2.3): pieces, probes, sources, verdicts. +//! [`KeyRing::acquire`] (KU §2.3): pieces, probes, sources, verdicts, over [`KeyEvidence`]. +use super::evidence::{Detector, KeyEvidence, Piece, Sample, Sampler, sorted_ranges}; use super::{ - ArrivalPiece, ForensicState, Inner, KeyResolution, KeyScope, ProofCache, ResolveKeysOptions, - ResolvedKeySet, overlaps, + AcquireOptions, ArrivalPiece, ForensicState, Inner, KeyResolution, KeyRing, KeyScope, + ProofCache, overlaps, }; -use crate::aacs::content::{ALIGNED_UNIT_LEN, aacs_unit_encrypted, decrypt_unit, is_clean}; +use crate::aacs::content::{decrypt_unit, is_clean}; use crate::aacs::trace::{KeyNode, KeyOutcome, KeyStep, ResolutionTrace}; +use crate::ctx::Ctx; use crate::decrypt::{AacsKeyMap, Phase}; -use crate::disc::{ContentFormat, Disc, DiscFormat, Extent}; +use crate::disc::{ContentFormat, Extent}; use crate::error::{Error, Result}; -use crate::halt::{Halt, Progress}; +use crate::halt::{Halt, Liveness}; use crate::keysource::{DiscInputs, DiscInputsCtx, KeySource, MIN_SAMPLE_UNITS}; -use crate::sector::SectorSource; use crate::session::KeySourceFactory; -use crate::whole_disc::{UNIT, UnitSpan, probe_units, subtract_ranges, unit_head}; use std::collections::HashMap; use std::sync::Arc; use std::time::{Duration, Instant}; @@ -35,7 +35,7 @@ const MAX_POOL_KEYS: usize = 256; pub(crate) trait Clock { fn now(&self) -> Duration; /// Wait `d`, returning `Err(Halted)` as soon as `halt` is cancelled. - fn sleep(&self, d: Duration, halt: Option<&Halt>) -> Result<()>; + fn sleep(&self, d: Duration, halt: &Halt) -> Result<()>; } /// The wall clock; waits are halt-aware ([`Halt::wait`]). @@ -51,14 +51,8 @@ impl Clock for RealClock { fn now(&self) -> Duration { self.0.elapsed() } - fn sleep(&self, d: Duration, halt: Option<&Halt>) -> Result<()> { - match halt { - Some(h) => h.wait(d), - None => { - std::thread::sleep(d); - Ok(()) - } - } + fn sleep(&self, d: Duration, halt: &Halt) -> Result<()> { + halt.wait(d) } } @@ -72,37 +66,32 @@ enum Verdict { Ask, } -// One Clip AV stream file (or an uncovered title extent) in scope, with its probes. -struct Piece { - spans: Vec, - titles: Vec, - rank: u64, +// One piece in scope with its probes. +struct Probed { + piece: Piece, units: u64, enc: Vec>, faults: usize, verdict: Verdict, } -impl Piece { +impl Probed { + fn new(piece: Piece) -> Self { + Probed { + piece, + units: 0, + enc: Vec::new(), + faults: 0, + verdict: Verdict::Ask, + } + } + fn id(&self) -> u32 { - self.spans[0].0 + self.piece.id() } fn ranges(&self) -> impl Iterator + '_ { - self.spans.iter().map(|&(s, n, _)| (s, s.saturating_add(n))) - } - - // Unit heads on the piece's grid, as (first head, unit count) per span. A unit that - // would cross its span's end is not a unit of the piece (KS-7, Informative). - fn grid(&self) -> Vec<(u64, u64)> { - self.spans - .iter() - .map(|&(s, n, anchor)| { - let head = unit_head(s, anchor); - let end = s as u64 + n as u64; - (head, end.saturating_sub(head) / UNIT) - }) - .collect() + self.piece.ranges() } } @@ -117,9 +106,9 @@ enum Asked { // The state of one `resolve` run. struct Run<'a> { - halt: Option<&'a Halt>, + halt: &'a Halt, // The op's progress: busy around each source call (stop design §2.1 item 2). - progress: Option<&'a Progress>, + progress: Option<&'a Liveness>, clock: &'a dyn Clock, format: ContentFormat, sources: Vec>, @@ -138,10 +127,7 @@ struct Run<'a> { impl Run<'_> { fn check_halt(&self) -> Result<()> { - match self.halt { - Some(h) => h.check(), - None => Ok(()), - } + self.halt.check() } // Add keys to the pool (deduplicated): the pool only grows during `resolve`. @@ -173,7 +159,7 @@ impl Run<'_> { self.requests += 1; let mut inputs = self.inputs.clone(); inputs.samples = samples.to_vec(); - let ctx = DiscInputsCtx::new(&inputs).with_stop(self.halt, self.progress); + let ctx = DiscInputsCtx::new(&inputs).with_stop(Some(self.halt), self.progress); let start = self.clock.now(); let mut wait = RETRY_FIRST; let mut attempted = false; @@ -189,7 +175,7 @@ impl Run<'_> { let who = src.label().to_string(); // Stop §2.1 item 2 (ST4-2): a source call (a keydb parse, a key-service call) // moves no CDB, so it is busy for the idle-only T29 probe. - let busy = self.progress.map(Progress::busy); + let busy = self.progress.map(Liveness::busy); let answer = if forensic { src.get_fmts_indexes(&ctx).map(|k| (k, None)) } else { @@ -286,7 +272,7 @@ impl Run<'_> { // KU §2.3 step 9, rules 1 and 3–5, against the current pool. `Ask` means rule 2 applies: // an `Enc` probe no held key opens. - fn judge(&self, p: &Piece) -> Verdict { + fn judge(&self, p: &Probed) -> Verdict { if p.units == 0 { return Verdict::Clear; } @@ -316,113 +302,10 @@ impl Run<'_> { } } -fn sorted_ranges(mut v: Vec<(u32, u32)>) -> Vec<(u32, u32)> { - v.sort_unstable(); - let mut out: Vec<(u32, u32)> = Vec::with_capacity(v.len()); - for (s, e) in v { - match out.last_mut() { - Some(last) if s <= last.1 => last.1 = last.1.max(e), - _ => out.push((s, e)), - } - } - out -} - -// The pieces of `scope`: each content file the scope touches (on its own unit grid, KS-1), -// minus sectors an earlier file claimed (an SSIF re-lists m2ts sectors), then title extents -// no file covers (anchored at the extent start). -fn pieces(disc: &Disc, files: &[Vec<(u32, u32)>], sel: &[usize], whole: bool) -> Vec { - let title_ranges: Vec<(usize, u32, u32)> = sel - .iter() - .flat_map(|&t| { - disc.titles[t] - .extents - .iter() - .filter(|e| e.sector_count > 0) - .map(move |e| (t, e.start_lba, e.start_lba.saturating_add(e.sector_count))) - }) - .collect(); - let touches = |ranges: &[(u32, u32)]| -> Vec { - let mut t: Vec = title_ranges - .iter() - .filter(|&&(_, s, e)| overlaps(ranges, s, e)) - .map(|&(t, _, _)| t) - .collect(); - t.sort_unstable(); - t.dedup(); - t - }; - let mut claimed: Vec<(u32, u32)> = Vec::new(); - let mut out = Vec::new(); - let push = |spans: Vec, out: &mut Vec| { - let ranges: Vec<(u32, u32)> = spans - .iter() - .map(|&(s, n, _)| (s, s.saturating_add(n))) - .collect(); - let titles = touches(&ranges); - let rank = titles - .iter() - .map(|&t| disc.titles[t].size_bytes) - .max() - .unwrap_or(0); - out.push(Piece { - spans, - titles, - rank, - units: 0, - enc: Vec::new(), - faults: 0, - verdict: Verdict::Ask, - }); - }; - for file in files { - let ends: Vec<(u32, u32)> = file - .iter() - .map(|&(s, n)| (s, s.saturating_add(n))) - .collect(); - if !whole && touches(&ends).is_empty() { - continue; - } - let mut spans = Vec::new(); - let mut off = 0u64; - for &(lba, n) in file { - let anchor = (lba as u64).saturating_sub(off % UNIT); - for (s, e) in subtract_ranges(&[(lba, n)], &claimed) { - spans.push((s, e - s, anchor)); - } - off += n as u64; - } - claimed = sorted_ranges(claimed.into_iter().chain(ends).collect()); - if !spans.is_empty() { - push(spans, &mut out); - } - } - for &(_, s, e) in &title_ranges { - let left = subtract_ranges(&[(s, e - s)], &claimed); - if left.is_empty() { - continue; - } - let spans = left.iter().map(|&(a, b)| (a, b - a, s as u64)).collect(); - claimed = sorted_ranges(claimed.into_iter().chain(left).collect()); - push(spans, &mut out); - } - out.sort_by_key(|p| p.id()); - out -} - -// A loose clip file's one piece, in the shape `pieces` builds for a disc: every aligned unit -// of the file (KS-1) on its own grid from byte 0, title 0, proven on arrival. -pub(crate) fn loose_file_piece(capacity: u32) -> super::ArrivalPiece { - let p = Piece { - spans: vec![(0, capacity, 0)], - titles: vec![0], - rank: 0, - units: 0, - enc: Vec::new(), - faults: 0, - verdict: Verdict::Lazy(None), - }; - super::ArrivalPiece { +// A loose clip file's one piece, proven on arrival. +pub(crate) fn loose_file_piece(capacity: u32) -> ArrivalPiece { + let p = Piece::loose_file(capacity); + ArrivalPiece { id: p.id(), spans: p.spans, candidate: None, @@ -432,152 +315,98 @@ pub(crate) fn loose_file_piece(capacity: u32) -> super::ArrivalPiece { // Each whole-disc piece's unit spans, for the unit-grid guards in `whole_disc_tests`. #[cfg(test)] -pub(crate) fn whole_disc_pieces(disc: &Disc, files: &[Vec<(u32, u32)>]) -> Vec> { +pub(crate) fn whole_disc_pieces( + disc: &crate::disc::Disc, + files: &[Vec<(u32, u32)>], +) -> Vec> { let sel: Vec = (0..disc.titles.len()).collect(); - pieces(disc, files, &sel, true) + super::evidence::pieces(disc, files, &sel, true) .into_iter() .map(|p| p.spans) .collect() } -fn in_segment(segments: &[(u32, u32)], lba: u32) -> bool { - let i = segments.partition_point(|s| s.0 <= lba); - i > 0 && lba < segments[i - 1].1 -} - // KU §2.3 step 7: up to 32 units per piece on its grid, each `Enc`, `Clr` or `Fault`. // FMTS segment units are not base probes (KU §2.3 step 6). fn probe( - reader: &mut dyn SectorSource, - p: &mut Piece, + sampler: &mut Sampler, + p: &mut Probed, segments: &[(u32, u32)], format: ContentFormat, - halt: Option<&Halt>, + halt: &Halt, ) -> Result<()> { - let grid = p.grid(); - p.units = grid.iter().map(|g| g.1).sum(); - let mut buf = vec![0u8; ALIGNED_UNIT_LEN]; - for idx in probe_units(p.units) { - if halt.is_some_and(|h| h.is_cancelled()) { - return Err(Error::Halted); - } - let mut k = idx; - let Some(&(head, _)) = grid.iter().find(|g| { - let hit = k < g.1; - if !hit { - k -= g.1; - } - hit - }) else { - continue; - }; - let Ok(lba) = u32::try_from(head + k * UNIT) else { - continue; - }; - if in_segment(segments, lba) { - continue; - } - match reader.read_sectors(lba, UNIT as u16, &mut buf, false) { - Ok(n) if n == buf.len() => { - if aacs_unit_encrypted(&buf, format) { - p.enc.push(buf.clone()); - } - } - // A gone source or a Stop is not a soft fault: report it, not a missing key. - Err(e) if fatal_read(&e) => return Err(e), - _ => p.faults += 1, + let (units, samples) = sampler.probe(&p.piece, segments, format, halt)?; + p.units = units; + for s in samples { + match s { + Sample::Enc(u) => p.enc.push(u), + Sample::Clear => {} + Sample::Fault => p.faults += 1, } } Ok(()) } -fn fatal_read(e: &Error) -> bool { - matches!(e, Error::Halted) - || e.is_source_terminated() - || (e.is_scsi_transport_failure() && !matches!(e, Error::IoError { .. })) -} - -fn missing_error(scope: &KeyScope, disc: &Disc) -> Error { +fn missing_error(scope: &KeyScope, disc_hash: &str) -> Error { match scope { KeyScope::WholeDisc => Error::WholeDiscKeyMissing, _ => Error::NoDiscKey { - disc_hash: disc.aacs_disc_hash(), + disc_hash: crate::hex::strip_hex_prefix(disc_hash).to_string(), }, } } -/// The body of [`ResolvedKeySet::resolve`] with an injected clock. -pub(crate) fn resolve( - disc: &Disc, - reader: &mut dyn SectorSource, - scope: KeyScope, +/// The body of [`KeyRing::acquire`] with an injected clock. +pub(crate) fn acquire( + ev: &KeyEvidence, + sampler: &mut Sampler, sources: &KeySourceFactory, - opts: ResolveKeysOptions, + opts: AcquireOptions, + ctx: &Ctx, clock: &dyn Clock, ) -> Result { - resolve_observed(disc, reader, scope, sources, opts, clock, None) -} - -/// [`resolve`] reporting to the op's `progress` (stop design §2.1, T29). -pub(crate) fn resolve_observed( - disc: &Disc, - reader: &mut dyn SectorSource, - scope: KeyScope, - sources: &KeySourceFactory, - opts: ResolveKeysOptions, - clock: &dyn Clock, - progress: Option<&Progress>, -) -> Result { - let halt = opts.halt; - if let Some(h) = halt { - h.check()?; - } + let halt = &ctx.halt; + halt.check()?; // Step 1: not AACS, or nothing decrypted: no source call. - let (Some(aacs), false) = (disc.aacs.as_ref(), scope == KeyScope::None) else { + let (Some(aacs), false) = (ev.aacs.as_ref(), ev.scope == KeyScope::None) else { return Ok(KeyResolution { - keys: ResolvedKeySet::none(), + keys: KeyRing::none(), trace: ResolutionTrace::new(), }); }; - let sel: Vec = match &scope { - KeyScope::Titles(v) => { - if let Some(&bad) = v.iter().find(|&&t| t >= disc.titles.len()) { - return Err(Error::DiscTitleRange { - index: bad, - count: disc.titles.len(), - }); - } - v.clone() - } - _ => (0..disc.titles.len()).collect(), - }; // Step 3: KS-14 [BD] §3.9.3 "Num_of_CPS_Unit field (16 bits) indicates the number of CPS // Units on the disc" — the declared count; `None` (unparseable) fails closed. - let n_decl = disc.declared_cps_units(); - let seed = opts.seed.filter(|s| s.is_aacs() && s.is_for(disc)); + let n_decl = aacs.n_declared; + // KA9: evidence with no captured hash is still looked up by its title-key file. + let disc_hash = if ev.media.disc_hash.is_empty() && !aacs.unit_key_ro.is_empty() { + crate::aacs::inf::disc_hash_hex(&crate::aacs::inf::disc_hash(&aacs.unit_key_ro)) + } else { + ev.media.disc_hash.clone() + }; + let media = super::MediaId { + disc_hash: disc_hash.clone(), + ..ev.media.clone() + }; + let seed = opts.seed.filter(|s| s.is_aacs() && s.is_for(&media)); // Step 4: the VID, in memory only. - let vid = Some(aacs.volume_id) - .filter(|v| *v != [0u8; 16]) - .or(opts.vid) - .or(seed.and_then(|s| s.0.vid)); - let mut inputs = disc.inputs().unwrap_or_else(|| DiscInputs { - disc_hash: aacs.disc_hash.clone(), - volume_id: [0u8; 16], + let vid = aacs.vid.or(opts.vid).or(seed.and_then(|s| s.0.vid)); + let inputs = DiscInputs { + disc_hash: disc_hash.clone(), + volume_id: vid.unwrap_or([0u8; 16]), version: aacs.version, - mkb: Vec::new(), - unit_key_ro: Vec::new(), + mkb: aacs.mkb.clone(), + unit_key_ro: aacs.unit_key_ro.clone(), samples: Vec::new(), - volume_label: None, - }); - inputs.volume_id = vid.unwrap_or([0u8; 16]); + volume_label: aacs.volume_label.clone(), + }; let built = sources(); let n_src = built.len(); let vid_consumer = built.iter().any(|s| s.uses_vid()); let mut run = Run { halt, - progress, + progress: opts.liveness, clock, - format: disc.content_format, + format: ev.container, sources: built, dead: vec![false; n_src], inputs, @@ -591,12 +420,11 @@ pub(crate) fn resolve_observed( if let Some(s) = seed { run.add_keys(&s.0.pool, "seed"); } - let result = if disc.format == DiscFormat::HdDvd { - resolve_hddvd(disc, reader, &scope, &sel, n_decl, &mut run) - } else { - resolve_bd(disc, reader, &scope, &sel, n_decl, seed, &mut run) + let result = match ev.detector { + Detector::Unverified => resolve_hddvd(ev, sampler, &disc_hash, n_decl, &mut run), + Detector::Verified => resolve_bd(ev, sampler, &disc_hash, n_decl, seed, &mut run), }; - // Every source is dropped here: nothing can ask after `resolve` (LK7). + // Every source is dropped here: nothing can ask after `acquire` (LK7). let trace = std::mem::take(&mut run.trace); drop(run.sources); if let Some(out) = opts.trace { @@ -611,17 +439,15 @@ pub(crate) fn resolve_observed( } // The final check (Stop §6 ST-L3): a Stop during the last source call ends `Halted`, // never a refusal or a set. - if let Some(h) = halt { - h.check()?; - } + halt.check()?; let mut inner = result?; - inner.disc_hash = aacs.disc_hash.clone(); - inner.capacity = disc.capacity_sectors; - inner.format = disc.format; - inner.content_format = disc.content_format; + inner.disc_hash = disc_hash; + inner.capacity = media.capacity; + inner.format = media.format; + inner.content_format = ev.container; inner.vid = vid; inner.n_decl = n_decl; - inner.scope = scope; + inner.scope = ev.scope.clone(); inner.requests = run.requests; tracing::info!( target: "freemkv::keys", @@ -634,7 +460,7 @@ pub(crate) fn resolve_observed( "keys: resolved up front" ); Ok(KeyResolution { - keys: ResolvedKeySet(Arc::new(inner)), + keys: KeyRing(Arc::new(inner)), trace, }) } @@ -651,30 +477,20 @@ fn aacs_inner(run: &Run) -> Inner { // KU §2.6: HD DVD is best effort. Not probed (its encrypted flag is unverified, KS-27): // a single declared key is applied to every piece; several, or none parseable, refuse. fn resolve_hddvd( - disc: &Disc, - reader: &mut dyn SectorSource, - scope: &KeyScope, - sel: &[usize], + ev: &KeyEvidence, + sampler: &mut Sampler, + disc_hash: &str, n_decl: Option, run: &mut Run, ) -> Result { if n_decl != Some(1) { tracing::error!(target: "freemkv::keys", declared = ?n_decl, "multi-key HD DVD cannot be matched reliably"); return Err(Error::NoDiscKey { - disc_hash: disc.aacs_disc_hash(), + disc_hash: crate::hex::strip_hex_prefix(disc_hash).to_string(), }); } - let files = match crate::whole_disc::content_files(reader) { - Ok(f) => f, - Err(e) if *scope == KeyScope::WholeDisc => return Err(e), - Err(e) => { - tracing::warn!(target: "freemkv::keys", error = %e, "no HD DVD file list: keying title extents"); - Vec::new() - } - }; - let ps = pieces(disc, &files, sel, *scope == KeyScope::WholeDisc); if run.pool.is_empty() { - let samples = disc.content_samples(reader, MIN_SAMPLE_UNITS); + let samples = sampler.main_samples(ev.main.as_ref(), MIN_SAMPLE_UNITS); for i in 0..run.sources.len() { if let Asked::Keys(k) = run.ask(i, &samples, false)? { let who = run.sources[i].label(); @@ -684,17 +500,17 @@ fn resolve_hddvd( } if run.pool.is_empty() { let failure = run.first_failure.take(); - return Err(failure.unwrap_or_else(|| missing_error(scope, disc))); + return Err(failure.unwrap_or_else(|| missing_error(&ev.scope, disc_hash))); } } let mut inner = aacs_inner(run); - inner.no_stream_files = files.is_empty(); + inner.no_stream_files = ev.no_stream_files; inner.best_effort = true; inner.origin = run.origin.first().copied(); inner.proven = vec![0]; - inner.keyed = ps.len(); + inner.keyed = ev.pieces.len(); let mut ranges = Vec::new(); - for p in &ps { + for p in &ev.pieces { ranges.extend(p.ranges().map(|(s, e)| (s, e, 0usize))); inner.spans.extend(p.spans.iter().copied()); } @@ -704,44 +520,20 @@ fn resolve_hddvd( // KU §2.3 steps 5–14 for BD/UHD content. fn resolve_bd( - disc: &Disc, - reader: &mut dyn SectorSource, - scope: &KeyScope, - sel: &[usize], + ev: &KeyEvidence, + sampler: &mut Sampler, + disc_hash: &str, n_decl: Option, - seed: Option<&ResolvedKeySet>, + seed: Option<&KeyRing>, run: &mut Run, ) -> Result { - let format = disc.content_format; - // Step 5: pieces. An unreadable filesystem fails a whole-disc copy (it would otherwise - // ship ciphertext); a title rip keys its title extents instead. - let fs = match crate::udf::read_filesystem(reader) { - Ok(fs) => Some(fs), - Err(e) if *scope == KeyScope::WholeDisc => return Err(e), - Err(e) => { - tracing::warn!(target: "freemkv::keys", error = %e, "no filesystem: keying title extents"); - None - } - }; - let files = match &fs { - Some(fs) => match crate::whole_disc::content_files_in(fs, reader) { - Ok(f) => f, - Err(e) if *scope == KeyScope::WholeDisc => return Err(e), - Err(e) => { - tracing::warn!(target: "freemkv::keys", error = %e, "no file list: keying title extents"); - Vec::new() - } - }, - None => Vec::new(), - }; - let mut ps = pieces(disc, &files, sel, *scope == KeyScope::WholeDisc); + let format = ev.container; + let scope = &ev.scope; + // Step 5: the evidence's pieces. + let mut ps: Vec = ev.pieces.iter().cloned().map(Probed::new).collect(); // Step 6: FMTS, when the forensic clip is in scope. - let layout = match &fs { - Some(fs) => super::fmts::layout(fs, reader)?, - None => None, - }; + let layout = ev.fmts.as_ref(); let clip: Vec<(u32, u32)> = layout - .as_ref() .map(|l| { l.clip .iter() @@ -754,19 +546,18 @@ fn resolve_bd( .any(|p| p.ranges().any(|(s, e)| overlaps(&clip, s, e))) }); let segments: Vec<(u32, u32)> = layout - .as_ref() .map(|l| sorted_ranges(l.ranges.iter().map(|&(s, e, _)| (s, e)).collect())) .unwrap_or_default(); // Step 7: probes. for p in &mut ps { - probe(reader, p, &segments, format, run.halt)?; + probe(sampler, p, &segments, format, run.halt)?; } // Step 8: sample-independent sources, once, with the main title's samples. let needs_keys = ps .iter() .any(|p| p.units > 0 && (!p.enc.is_empty() || p.faults > 0)); if needs_keys { - let main = disc.main_title().map(|t| { + let main = ev.main.as_ref().map(|t| { t.extents .iter() .map(|e| (e.start_lba, e.start_lba.saturating_add(e.sector_count))) @@ -797,7 +588,7 @@ fn resolve_bd( let mut order: Vec = (0..ps.len()) .filter(|&i| ps[i].verdict == Verdict::Ask) .collect(); - order.sort_by_key(|&i| (std::cmp::Reverse(ps[i].rank), ps[i].id())); + order.sort_by_key(|&i| (std::cmp::Reverse(ps[i].piece.rank), ps[i].id())); // Step 11: sample-dependent requests ≤ n_decl (or the number of pieces). let mut budget = n_decl.unwrap_or(ps.len()); for &i in &order { @@ -853,32 +644,32 @@ fn resolve_bd( if let Some(i) = ps.iter().position(|p| p.verdict == Verdict::Missing) { let lba = ps[i].id(); let failure = run.first_failure.take(); - let err = failure.unwrap_or_else(|| missing_error(scope, disc)); + let err = failure.unwrap_or_else(|| missing_error(scope, disc_hash)); tracing::error!(target: "freemkv::keys", lba, code = err.code(), "a stream file in scope has no key; refusing before any output"); return Err(err); } // KU §2.7: an empty pool on an encrypted scope is today's keyless case. if run.pool.is_empty() && ps.iter().any(|p| matches!(p.verdict, Verdict::Lazy(_))) { let failure = run.first_failure.take(); - return Err(failure.unwrap_or_else(|| missing_error(scope, disc))); + return Err(failure.unwrap_or_else(|| missing_error(scope, disc_hash))); } let mut inner = aacs_inner(run); - inner.no_stream_files = files.is_empty(); + inner.no_stream_files = ev.no_stream_files; // Step 6 (continued): forensic keys, reused from the seed or anchored once. - let forensic = match &layout { + let forensic = match layout { None => None, - Some(l) => Some(resolve_forensic(reader, l, seed, run, format)?), + Some(l) => Some(resolve_forensic(sampler, l, seed, run, format)?), }; - build(&mut inner, &ps, layout.as_ref(), forensic, run, clip); + build(&mut inner, &ps, layout, forensic, run, clip); Ok(inner) } // Step 9.2: this piece's unopened `Enc` units first, topped up from other unopened pieces // of the same playlist, never across playlists (SG23). -fn borrow_samples(run: &Run, ps: &[Piece], i: usize) -> Vec> { +fn borrow_samples(run: &Run, ps: &[Probed], i: usize) -> Vec> { // KS-10 [BD] §3.9.2: "All AV stream files that are referred to by one Title are included // in the same CPS Unit" — so only pieces sharing a title may lend units. - let unopened = |p: &Piece| -> Vec> { + let unopened = |p: &Probed| -> Vec> { p.enc .iter() .filter(|u| !(0..run.pool.len()).any(|s| run.opens(u, s))) @@ -890,7 +681,13 @@ fn borrow_samples(run: &Run, ps: &[Piece], i: usize) -> Vec> { if out.len() >= SAMPLE_CAP { break; } - if j == i || q.verdict != Verdict::Ask || !q.titles.iter().any(|t| ps[i].titles.contains(t)) + if j == i + || q.verdict != Verdict::Ask + || !q + .piece + .titles + .iter() + .any(|t| ps[i].piece.titles.contains(t)) { continue; } @@ -901,7 +698,7 @@ fn borrow_samples(run: &Run, ps: &[Piece], i: usize) -> Vec> { } // Step 10: the `n_decl == 1` rule. -fn apply_single_unit_rule(run: &Run, ps: &mut [Piece], n_decl: Option) -> Result<()> { +fn apply_single_unit_rule(run: &Run, ps: &mut [Probed], n_decl: Option) -> Result<()> { // KS-14 [BD] §3.9.3: "Num_of_CPS_Unit field (16 bits) indicates the number of CPS Units // on the disc" — with one declared, a key proven on one piece keys them all. if n_decl != Some(1) { @@ -949,9 +746,9 @@ enum Forensic { } fn resolve_forensic( - reader: &mut dyn SectorSource, + sampler: &mut Sampler, layout: &super::fmts::Layout, - seed: Option<&ResolvedKeySet>, + seed: Option<&KeyRing>, run: &mut Run, format: ContentFormat, ) -> Result { @@ -974,7 +771,7 @@ fn resolve_forensic( } Ok(None) }; - let keys = match super::fmts::anchor(reader, layout, halt, &mut ask)? { + let keys = match super::fmts::anchor(sampler.source(), layout, Some(halt), &mut ask)? { super::fmts::Anchor::Keys(k) => k, super::fmts::Anchor::Pending => { tracing::warn!(target: "freemkv::keys", "fmts: every index-1 anchor read faulted: forensic keys pending"); @@ -993,7 +790,7 @@ fn resolve_forensic( { return Err(Error::FmtsKeyMissing); } - match super::fmts::phases(reader, layout, &keys, format, halt)? { + match super::fmts::phases(sampler.source(), layout, &keys, format, Some(halt))? { Some(p) => Ok(Forensic::Resolved(keys, p)), None => Err(Error::FmtsKeyMissing), } @@ -1002,7 +799,7 @@ fn resolve_forensic( // Step 14: maps from Keyed pieces (and the forensic ranges); arrival pieces for the rest. fn build( inner: &mut Inner, - ps: &[Piece], + ps: &[Probed], layout: Option<&super::fmts::Layout>, forensic: Option, run: &Run, @@ -1032,7 +829,7 @@ fn build( } let mut proven = Vec::new(); for p in ps { - inner.spans.extend(p.spans.iter().copied()); + inner.spans.extend(p.piece.spans.iter().copied()); match p.verdict { Verdict::Keyed(slot) => { inner.keyed += 1; @@ -1062,7 +859,7 @@ fn build( } inner.arrival.push(ArrivalPiece { id: p.id(), - spans: p.spans.clone(), + spans: p.piece.spans.clone(), candidate: match p.verdict { Verdict::Lazy(c) => c, _ => None, diff --git a/src/keys/tests.rs b/src/keys/tests.rs index c1f4ab80..e55cebf5 100644 --- a/src/keys/tests.rs +++ b/src/keys/tests.rs @@ -250,7 +250,7 @@ impl Clock for FakeClock { fn now(&self) -> Duration { *self.now.lock().unwrap() } - fn sleep(&self, d: Duration, halt: Option<&Halt>) -> Result<()> { + fn sleep(&self, d: Duration, halt: &Halt) -> Result<()> { let mut now = self.now.lock().unwrap(); *now += d; if let Some((at, h)) = &self.cancel_at @@ -258,10 +258,7 @@ impl Clock for FakeClock { { h.cancel(); } - match halt { - Some(h) => h.check(), - None => Ok(()), - } + halt.check() } } @@ -452,20 +449,35 @@ fn resolve_with( reader: &mut dyn SectorSource, scope: KeyScope, specs: &[Spec], - opts: ResolveKeysOptions, + opts: AcquireOptions, clock: &dyn Clock, -) -> Result { - let f = factory(specs); - super::resolve::resolve(&fx.disc, reader, scope, &f, opts, clock).map(|r| r.keys) +) -> Result { + resolve_halted(fx, reader, scope, specs, opts, clock, &Halt::new()) } -fn resolve(fx: &Fx, scope: KeyScope, specs: &[Spec]) -> Result { +// `resolve_with` under the run's stop token `halt`. +fn resolve_halted( + fx: &Fx, + reader: &mut dyn SectorSource, + scope: KeyScope, + specs: &[Spec], + opts: AcquireOptions, + clock: &dyn Clock, + halt: &Halt, +) -> Result { + let f = factory(specs); + let ctx = crate::ctx::Ctx::new(halt.clone()); + let ev = KeyEvidence::from_disc(&fx.disc, reader, scope, &ctx)?; + super::resolve::acquire(&ev, &mut Sampler::new(reader), &f, opts, &ctx, clock).map(|r| r.keys) +} + +fn resolve(fx: &Fx, scope: KeyScope, specs: &[Spec]) -> Result { resolve_with( fx, &mut fx.source(), scope, specs, - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) } @@ -587,7 +599,7 @@ fn single_declared_unit_keys_lazy_pieces() { &mut src.clone(), KeyScope::Titles(vec![0]), &[Spec::keydb(&[K1], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -658,7 +670,7 @@ fn damaged_extent_never_inherits_neighbour_key() { &mut src.clone(), KeyScope::Titles(vec![0]), &[Spec::keydb(&pool, &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -704,7 +716,7 @@ fn each_piece_request_has_eight_units_and_the_vid() { let mut image = two_units(); image.disc.aacs.as_mut().unwrap().volume_id = [0u8; 16]; let calls = Calls::default(); - let opts = ResolveKeysOptions { + let opts = AcquireOptions { seed: Some(&seed), ..Default::default() }; @@ -824,7 +836,7 @@ fn one_opened_probe_is_lazy_never_keyed() { &mut src.clone(), KeyScope::Titles(vec![0]), &[Spec::keydb(&[K1, K2], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -867,7 +879,7 @@ fn transient_source_failure_retried_until_it_answers() { &mut fx.source(), KeyScope::Titles(vec![0]), &[online], - ResolveKeysOptions::default(), + AcquireOptions::default(), clock.as_ref(), ) .unwrap(); @@ -884,17 +896,15 @@ fn transient_source_failure_retried_until_it_answers() { fn transient_source_gives_up_after_60s_without_an_answer() { let fx = fixture(&[stream(1, 10, Some(K1))], 1, &[&[0]]); let run = |specs: &[Spec], clock: &FakeClock, halt: Option<&Halt>| { - let opts = ResolveKeysOptions { - halt, - ..Default::default() - }; - resolve_with( + let idle = Halt::new(); + resolve_halted( &fx, &mut fx.source(), KeyScope::Titles(vec![0]), specs, - opts, + AcquireOptions::default(), clock, + halt.unwrap_or(&idle), ) }; let calls = Calls::default(); @@ -975,16 +985,16 @@ fn key_set_bound_to_disc_and_debug_redacts_keys_and_vid() { let fx = two_units(); let calls = Calls::default(); let set = resolve(&fx, KeyScope::WholeDisc, &[Spec::keydb(&[K1, K2], &calls)]).unwrap(); - assert!(set.is_for(&fx.disc)); + assert!(set.is_for(&fx.disc.media_id())); let mut other = two_units(); other.disc.aacs.as_mut().unwrap().disc_hash = "0xffff".into(); - assert!(!set.is_for(&other.disc)); + assert!(!set.is_for(&other.disc.media_id())); let mut other = two_units(); other.disc.aacs.as_mut().unwrap().volume_id = [1; 16]; - assert!(!set.is_for(&other.disc)); + assert!(!set.is_for(&other.disc.media_id())); let mut other = two_units(); other.disc.capacity_sectors += 3; - assert!(!set.is_for(&other.disc)); + assert!(!set.is_for(&other.disc.media_id())); caller_bug(|| set.title_reader(&other.disc, 0, fx.source())); assert!(set.vid_fingerprint().is_some()); assert_eq!(set.proven_key_fingerprints().len(), 2); @@ -1006,34 +1016,34 @@ fn stop_during_resolve_builds_no_set() { let mut src = fx.source(); src.halt_after = Some((halt.clone(), Arc::new(Mutex::new(40)))); let calls = Calls::default(); - let opts = ResolveKeysOptions { - halt: Some(&halt), + let opts = AcquireOptions { ..Default::default() }; - let r = resolve_with( + let r = resolve_halted( &fx, &mut src, KeyScope::WholeDisc, &[Spec::keydb(&[K1, K2], &calls)], opts, &FakeClock::default(), + &halt, ); assert!(matches!(r, Err(Error::Halted))); let halt = Halt::new(); let mut online = Spec::online(&[K1, K2], &calls); online.cancel = Some(halt.clone()); - let opts = ResolveKeysOptions { - halt: Some(&halt), + let opts = AcquireOptions { ..Default::default() }; - let r = resolve_with( + let r = resolve_halted( &fx, &mut fx.source(), KeyScope::WholeDisc, &[online], opts, &FakeClock::default(), + &halt, ); assert!(matches!(r, Err(Error::Halted))); } @@ -1041,7 +1051,7 @@ fn stop_during_resolve_builds_no_set() { // ── §2.4: proof on first read ─────────────────────────────────────────────── // File B (K2) Lazy: every probe of it faulted at resolve time. Pool K1 + K2. -fn lazy_b(pool: &[[u8; 16]]) -> (Fx, ResolvedKeySet, Faulty) { +fn lazy_b(pool: &[[u8; 16]]) -> (Fx, KeyRing, Faulty) { let fx = fixture( &[stream(1, 10, Some(K1)), stream(2, 10, Some(K2))], 2, @@ -1056,7 +1066,7 @@ fn lazy_b(pool: &[[u8; 16]]) -> (Fx, ResolvedKeySet, Faulty) { &mut src.clone(), KeyScope::WholeDisc, &[Spec::keydb(pool, &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -1163,7 +1173,7 @@ fn on_arrival_proof() { &mut src.clone(), KeyScope::WholeDisc, &[Spec::keydb(&[K1, K2], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -1185,7 +1195,7 @@ fn on_arrival_proof() { } // File B (K2) Lazy as in `lazy_b`, its image first damaged by `damage`. -fn lazy_b_damaged(pool: &[[u8; 16]], damage: impl Fn(&mut Fx)) -> (Fx, ResolvedKeySet, Faulty) { +fn lazy_b_damaged(pool: &[[u8; 16]], damage: impl Fn(&mut Fx)) -> (Fx, KeyRing, Faulty) { let mut fx = fixture( &[stream(1, 10, Some(K1)), stream(2, 10, Some(K2))], 2, @@ -1201,7 +1211,7 @@ fn lazy_b_damaged(pool: &[[u8; 16]], damage: impl Fn(&mut Fx)) -> (Fx, ResolvedK &mut src.clone(), KeyScope::WholeDisc, &[Spec::keydb(pool, &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -1485,7 +1495,10 @@ fn extract_tree_blanks_clustered_damage() { keys: Some(&set), ..Default::default() }; - let res = fx.disc.extract_tree(&mut src, dest.path(), &opts).unwrap(); + let res = fx + .disc + .extract_tree(&mut src, dest.path(), &opts, &crate::ctx::Ctx::default()) + .unwrap(); assert!( !res.halted && res.files.iter().all(|f| f.complete), "every file is written" @@ -1695,7 +1708,7 @@ fn fmts_all_anchors_unreadable_is_pending() { &mut src.clone(), KeyScope::WholeDisc, &[Spec::keydb(&[K1], &calls), fmts_online(&calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -1748,7 +1761,7 @@ fn on_arrival_proof_skips_forensic_segment_units() { &mut src.clone(), KeyScope::WholeDisc, &[Spec::keydb(&[K1, K2], &calls), fmts_online(&calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -1813,7 +1826,10 @@ fn extract_tree_blanks_a_file_no_key_opens() { keys: Some(&set), ..Default::default() }; - let res = fx.disc.extract_tree(&mut src, dest.path(), &opts).unwrap(); + let res = fx + .disc + .extract_tree(&mut src, dest.path(), &opts, &crate::ctx::Ctx::default()) + .unwrap(); let got = std::fs::read(dest.path().join("BDMV/STREAM/00002.m2ts")).unwrap(); assert!(got.iter().all(|&b| b == 0), "B is blanked, not ciphertext"); assert!(!res.complete, "blanked units are not good bytes"); @@ -1824,7 +1840,10 @@ fn extract_tree_blanks_a_file_no_key_opens() { keys: Some(&set), ..Default::default() }; - let res = fx.disc.extract_tree(&mut src, dest.path(), &opts).unwrap(); + let res = fx + .disc + .extract_tree(&mut src, dest.path(), &opts, &crate::ctx::Ctx::default()) + .unwrap(); assert!(res.complete); for (i, name) in [(0, "00001"), (1, "00002")] { let got = std::fs::read(dest.path().join(format!("BDMV/STREAM/{name}.m2ts"))).unwrap(); @@ -1848,12 +1867,13 @@ fn resolve_retains_no_source_and_nothing_asks_after() { src.kill(b, b + n); let calls = Calls::default(); let f = factory(&[Spec::keydb(&[K1, K2], &calls)]); - let set = ResolvedKeySet::resolve( + let set = KeyRing::acquire_for_disc( &fx.disc, &mut src.clone(), KeyScope::WholeDisc, &f, - ResolveKeysOptions::default(), + AcquireOptions::default(), + &crate::ctx::Ctx::default(), ) .unwrap() .keys; @@ -1884,19 +1904,17 @@ fn resolve_retains_no_source_and_nothing_asks_after() { measured_cicp: None, })); let r = crate::mux::mux_with_keys( - crate::mux::MuxSource::Live { - reader: Box::new(src.clone()), - title, - format: ContentFormat::BdTs, - }, + crate::mux::Source::from_reader( + Box::new(src.clone()), + crate::mux::ScannedTitle::new(title, ContentFormat::BdTs), + ), Some(&set), "null://", &crate::mux::MuxOptions { batch_sectors: 30, ..Default::default() }, - &Halt::new(), - Arc::new(crate::mux::driver::NoopEvents), + &crate::ctx::Ctx::default(), ); // The fixture TS carries no muxable frames, so a mux that read every unit through the // held keys ends E6008 (MkvInvalid); E6009 would mean it never read. @@ -1939,12 +1957,13 @@ fn iso_input_reads_through_the_key_set() { let (img, disc) = scannable_image(); let calls = Calls::default(); let f = factory(&[Spec::keydb(&[K1], &calls)]); - let set = ResolvedKeySet::resolve( + let set = KeyRing::acquire_for_disc( &disc, &mut MemSource::new(img.image.clone()), KeyScope::Titles(vec![0]), &f, - ResolveKeysOptions::default(), + AcquireOptions::default(), + &crate::ctx::Ctx::default(), ) .unwrap() .keys; @@ -1952,13 +1971,18 @@ fn iso_input_reads_through_the_key_set() { let path = dir.path().join("disc.iso"); std::fs::write(&path, &img.image).unwrap(); let url = format!("iso://{}", path.display()); - let keyless = crate::input(&url, &crate::InputOptions::default()); + let keyless = crate::input( + &url, + &crate::InputOptions::default(), + &crate::ctx::Ctx::default(), + ); assert_eq!(crate::error_code(&keyless.err().unwrap()), Some(E7022)); let opts = crate::InputOptions { keys: Some(set), ..Default::default() }; - let stream = crate::input(&url, &opts).expect("the set keys the title"); + let stream = + crate::input(&url, &opts, &crate::ctx::Ctx::default()).expect("the set keys the title"); assert_eq!(stream.info().extents, disc.titles[0].extents); } @@ -1974,7 +1998,12 @@ fn session_resolves_a_key_set_through_its_reader() { Some(Box::new(fx.source())), ); let r = session - .resolve_key_set(KeyScope::WholeDisc, &f, ResolveKeysOptions::default()) + .acquire_keys( + KeyScope::WholeDisc, + &f, + AcquireOptions::default(), + &crate::ctx::Ctx::default(), + ) .unwrap(); assert_eq!(r.keys.status().keyed, 2); assert_eq!(calls.len(), 1); @@ -2041,26 +2070,24 @@ impl SectorSource for RecoveryCount { /// never zero-filled as a bad sector. #[test] fn live_stream_stops_on_an_unkeyed_piece_without_recovery() { - use crate::pes::Stream; + use crate::pes::PesSource; for skip in [true, false] { let (fx, set, src) = lazy_b(&[K1]); let recovery = Arc::new(Mutex::new(0u32)); let reader = RecoveryCount(src, recovery.clone()); - let mut stream = super::install_key_map( - crate::mux::DiscStream::new( - Box::new(reader), - fx.disc.titles[0].clone(), - set.decrypt_keys(), - 30, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(), - set.key_map(), + let _serial = crate::sector::prefetched::holder_test_lock(); + let policy = crate::sector::read_stage::ReadPolicy::Live { + batch: 30, + skip_errors: skip, + }; + let mut stream = crate::mux::resolve::build_keyed_pipeline( + reader, + fx.disc.titles[0].clone(), + &set, + policy, + &crate::ctx::Ctx::default(), ) - .with_arrival(set.arrival(set.title_stop()).expect("B is Lazy")); - stream.skip_errors = skip; + .unwrap(); let got = loop { match stream.read() { Ok(Some(_)) => continue, @@ -2114,7 +2141,7 @@ fn single_unit_rule_never_overrides_a_piece_another_key_opened() { &mut src.clone(), KeyScope::Titles(vec![0]), &[Spec::keydb(&[K1, K2], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -2187,7 +2214,7 @@ fn single_unit_rule_keys_only_pieces_the_proven_key_opens() { &mut src.clone(), KeyScope::Titles(vec![0]), &[Spec::keydb(&[K1, K2], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -2210,7 +2237,7 @@ fn single_unit_rule_keys_only_pieces_the_proven_key_opens() { /// DecryptFailed handled as bad media, in both skip modes. #[test] fn keyless_set_over_overlapping_extents_stops_e7022() { - use crate::pes::Stream; + use crate::pes::PesSource; let fx = fixture(&[stream(1, 10, Some(K1))], 1, &[&[0]]); let a = fx.file(0).0; let mut title = fx.disc.titles[0].clone(); @@ -2224,24 +2251,22 @@ fn keyless_set_over_overlapping_extents_stops_e7022() { sector_count: 6, }, ]; - let set = ResolvedKeySet::keyless_for(&title, ContentFormat::BdTs); + let set = KeyRing::keyless_for(&title, ContentFormat::BdTs); for skip in [false, true] { let recovery = Arc::new(Mutex::new(0u32)); - let mut stream = super::install_key_map( - crate::mux::DiscStream::new( - Box::new(RecoveryCount(fx.source(), recovery.clone())), - title.clone(), - set.decrypt_keys(), - 3, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(), - set.key_map(), + let _serial = crate::sector::prefetched::holder_test_lock(); + let policy = crate::sector::read_stage::ReadPolicy::Live { + batch: 3, + skip_errors: skip, + }; + let mut stream = crate::mux::resolve::build_keyed_pipeline( + RecoveryCount(fx.source(), recovery.clone()), + title.clone(), + &set, + policy, + &crate::ctx::Ctx::default(), ) - .with_arrival(set.arrival(set.title_stop()).expect("every extent is lazy")); - stream.skip_errors = skip; + .unwrap(); // Start past the inner extent: its first read is unit 4 of the outer one. let mut buf = vec![0u8; ALIGNED_UNIT_LEN]; let mut r = set @@ -2286,7 +2311,7 @@ fn single_unit_rule_never_keys_a_piece_with_no_readable_probe() { &mut src.clone(), KeyScope::Titles(vec![0]), &[Spec::keydb(&[K1], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -2313,7 +2338,7 @@ fn keyless_set_keeps_adjacent_clips_on_their_own_grids() { sector_count, }) .collect(); - let set = ResolvedKeySet::keyless_for(&title, ContentFormat::BdTs); + let set = KeyRing::keyless_for(&title, ContentFormat::BdTs); assert_eq!(set.0.spans, [(100, 31, 100), (131, 30, 131)]); let anchors: Vec = set.0.arrival.iter().map(|p| p.spans[0].2).collect(); assert_eq!(anchors, [100, 131]); @@ -2325,14 +2350,14 @@ fn keyless_set_keeps_adjacent_clips_on_their_own_grids() { #[test] fn keyless_session_set_is_bound_to_its_disc() { let fx = two_units(); - let set = ResolvedKeySet::keyless_for_disc(&fx.disc, 0).expect("title 0"); - assert!(set.is_for(&fx.disc)); + let set = KeyRing::keyless_for_disc(&fx.disc, 0).expect("title 0"); + assert!(set.is_for(&fx.disc.media_id())); let mut other = two_units(); other.disc.aacs.as_mut().unwrap().disc_hash = "0xffff".into(); - assert!(!set.is_for(&other.disc)); - assert!(ResolvedKeySet::keyless_for_disc(&fx.disc, 9).is_none()); - let bare = ResolvedKeySet::keyless_for(&fx.disc.titles[0], ContentFormat::BdTs); - assert!(!bare.is_for(&fx.disc), "no identity, no bypass"); + assert!(!set.is_for(&other.disc.media_id())); + assert!(KeyRing::keyless_for_disc(&fx.disc, 9).is_none()); + let bare = KeyRing::keyless_for(&fx.disc.titles[0], ContentFormat::BdTs); + assert!(!bare.is_for(&fx.disc.media_id()), "no identity, no bypass"); } /// J22 with J21 parity (review r4). KS-14 [BD] §3.9.3: "Num_of_CPS_Unit field (16 bits) @@ -2359,7 +2384,7 @@ fn single_unit_rule_needs_two_probes_unless_one_unit_long() { &mut src.clone(), KeyScope::Titles(vec![0]), &[Spec::keydb(&[K1], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ) .unwrap(); @@ -2379,13 +2404,13 @@ fn single_unit_rule_needs_two_probes_unless_one_unit_long() { // Resolve `Titles([1])` of `two_units` (K2 held by no source) with the disc's VID zeroed or // not; returns the refusal and the `vid_would_help` flag. -fn vid_help(zero_vid: bool, specs: &[Spec]) -> (Result, bool) { +fn vid_help(zero_vid: bool, specs: &[Spec]) -> (Result, bool) { let mut fx = two_units(); if zero_vid { fx.disc.aacs.as_mut().unwrap().volume_id = [0u8; 16]; } let flag = std::sync::atomic::AtomicBool::new(false); - let opts = ResolveKeysOptions { + let opts = AcquireOptions { vid_would_help: Some(&flag), ..Default::default() }; @@ -2445,13 +2470,14 @@ fn a_matched_miss_keeps_its_reason_in_the_trace() { ..Spec::keydb(&[], &calls) }; let f = factory(&[km_no_vid, Spec::keydb(&[K1], &calls)]); - let opts = ResolveKeysOptions::default(); - let r = ResolvedKeySet::resolve( + let opts = AcquireOptions::default(); + let r = KeyRing::acquire_for_disc( &fx.disc, &mut fx.source(), KeyScope::Titles(vec![0]), &f, opts, + &crate::ctx::Ctx::default(), ) .unwrap(); assert_eq!( @@ -2502,7 +2528,7 @@ fn an_unreachable_service_is_never_coded_missing() { &mut fx.source(), KeyScope::Titles(vec![0, 1]), &[down], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ); assert_eq!(code(r), E7028); @@ -2528,18 +2554,18 @@ fn a_stop_during_backoff_keeps_the_asked_step() { dead.clock = Some(clock.clone()); dead.down_until = Some(Duration::MAX); let out = Mutex::new(crate::aacs::trace::ResolutionTrace::new()); - let opts = ResolveKeysOptions { - halt: Some(&halt), + let opts = AcquireOptions { trace: Some(&out), ..Default::default() }; - let r = resolve_with( + let r = resolve_halted( &fx, &mut fx.source(), KeyScope::Titles(vec![0]), &[dead], opts, &*clock, + &halt, ); assert!(matches!(r, Err(Error::Halted)), "{r:?}"); let t = out.lock().unwrap().clone(); @@ -2550,13 +2576,13 @@ fn a_stop_during_backoff_keeps_the_asked_step() { // ── KU-E1: the trace on Ok and Err; `is_aacs` ────────────────────────────── /// The per-source walk ("keydb > matched disc > online > …") answers "why no key", so -/// `resolve` hands it back through `ResolveKeysOptions::trace` on a refusal too. +/// `resolve` hands it back through `AcquireOptions::trace` on a refusal too. #[test] fn resolve_returns_the_trace_on_ok_and_on_err() { let fx = two_units(); let calls = Calls::default(); let out = Mutex::new(crate::aacs::trace::ResolutionTrace::new()); - let opts = ResolveKeysOptions { + let opts = AcquireOptions { trace: Some(&out), ..Default::default() }; @@ -2574,17 +2600,18 @@ fn resolve_returns_the_trace_on_ok_and_on_err() { assert_eq!(t.keys[0].who, "keydb"); let out = Mutex::new(crate::aacs::trace::ResolutionTrace::new()); - let opts = ResolveKeysOptions { + let opts = AcquireOptions { trace: Some(&out), ..Default::default() }; let f = factory(&[Spec::keydb(&[K1], &calls)]); - let ok = ResolvedKeySet::resolve( + let ok = KeyRing::acquire_for_disc( &fx.disc, &mut fx.source(), KeyScope::Titles(vec![0]), &f, opts, + &crate::ctx::Ctx::default(), ) .unwrap(); assert_eq!(*out.lock().unwrap(), ok.trace); @@ -2607,12 +2634,13 @@ fn the_resolution_trace_holds_no_key_vid_or_mkb_bytes() { Spec::keydb(&[K1, K2], &calls), Spec::online(&[K1, K2], &calls), ]); - let r = ResolvedKeySet::resolve( + let r = KeyRing::acquire_for_disc( &fx.disc, &mut fx.source(), KeyScope::Titles(vec![0, 1]), &f, - ResolveKeysOptions::default(), + AcquireOptions::default(), + &crate::ctx::Ctx::default(), ) .unwrap(); let text = format!("{:?} {:#?}", r.trace, r.trace); @@ -2631,7 +2659,7 @@ fn the_resolution_trace_holds_no_key_vid_or_mkb_bytes() { #[test] fn is_aacs_tells_none_from_a_resolved_set() { let fx = two_units(); - assert!(!ResolvedKeySet::none().is_aacs()); + assert!(!KeyRing::none().is_aacs()); let set = resolve( &fx, KeyScope::Titles(vec![0]), @@ -2650,7 +2678,7 @@ fn a_stop_seen_before_the_retry_keeps_the_asked_step() { fn now(&self) -> Duration { Duration::ZERO } - fn sleep(&self, _d: Duration, _halt: Option<&Halt>) -> Result<()> { + fn sleep(&self, _d: Duration, _halt: &Halt) -> Result<()> { self.0.cancel(); Ok(()) } @@ -2661,19 +2689,19 @@ fn a_stop_seen_before_the_retry_keeps_the_asked_step() { let mut dead = Spec::online(&[K1], &calls); dead.down_until = Some(Duration::MAX); let out = Mutex::new(crate::aacs::trace::ResolutionTrace::new()); - let opts = ResolveKeysOptions { - halt: Some(&halt), + let opts = AcquireOptions { trace: Some(&out), ..Default::default() }; let clock = WakeOnStop(halt.clone()); - let r = resolve_with( + let r = resolve_halted( &fx, &mut fx.source(), KeyScope::Titles(vec![0]), &[dead], opts, &clock, + &halt, ); assert!(matches!(r, Err(Error::Halted)), "{r:?}"); assert_eq!( @@ -2730,7 +2758,7 @@ fn a_source_halted_mid_request_keeps_the_asked_step() { let mut stopped = Spec::online(&[K1], &calls); stopped.fails = Some(|| Error::Halted); let out = Mutex::new(crate::aacs::trace::ResolutionTrace::new()); - let opts = ResolveKeysOptions { + let opts = AcquireOptions { trace: Some(&out), ..Default::default() }; @@ -2769,7 +2797,7 @@ fn decrypt_clears_cpi_on_every_source_packet() { let dir = tempfile::tempdir().unwrap(); let dest = dir.path().join("out.iso"); let cap = fx.disc.capacity_sectors; - crate::write_image(&mut w, &dest, cap, &Halt::new(), |_| {}).unwrap(); + crate::write_image(&mut w, &dest, cap, &crate::ctx::Ctx::default()).unwrap(); let bytes = std::fs::read(&dest).unwrap(); for i in 0..2 { let (start, n) = fx.file(i); @@ -2985,8 +3013,57 @@ fn probe_reports_a_terminated_source() { &mut src, KeyScope::WholeDisc, &[Spec::keydb(&[K1, K2], &calls)], - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ); assert!(matches!(r, Err(Error::SourceTerminated)), "{r:?}"); } + +// ── Pipeline 3a: evidence with no captured identity (KA9) ───────────────── + +/// KA9: evidence whose disc hash was never captured is still looked up by its title-key +/// file: the source is sent SHA-1(`Unit_Key_RO.inf`), the hash a keydb keys entries by, +/// and the ring binds to that hash. +#[test] +fn evidence_without_a_hash_is_looked_up_by_its_title_key_file() { + struct HashSpy(Arc>>); + impl KeySource for HashSpy { + fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result> { + self.0.lock().unwrap().push(ctx.disc_hash().to_string()); + Ok(vec![UnitKey::new(1, K1), UnitKey::new(2, K2)]) + } + fn answer_depends_on_samples(&self) -> bool { + false + } + } + let mut fx = two_units(); + let aacs = fx.disc.aacs.as_mut().unwrap(); + let want = crate::aacs::inf::disc_hash_hex(&crate::aacs::inf::disc_hash(&aacs.uk_ro)); + aacs.disc_hash = String::new(); + let seen = Arc::new(Mutex::new(Vec::new())); + let spy = seen.clone(); + let f: crate::session::KeySourceFactory = + Arc::new(move || vec![Box::new(HashSpy(spy.clone())) as Box]); + let ring = KeyRing::acquire_for_disc( + &fx.disc, + &mut fx.source(), + KeyScope::Titles(vec![0]), + &f, + AcquireOptions::default(), + &crate::ctx::Ctx::default(), + ) + .unwrap() + .keys; + assert_eq!(*seen.lock().unwrap(), std::slice::from_ref(&want)); + let media = MediaId { + disc_hash: want, + ..fx.disc.media_id() + }; + assert!(ring.is_for(&media)); + assert_eq!( + fx.disc.media_id().disc_hash, + media.disc_hash, + "the disc derives it too" + ); + assert!(ring.is_for(&fx.disc.media_id())); +} diff --git a/src/keys/tests/parity.rs b/src/keys/tests/parity.rs index 67aa40a1..fa3c8d48 100644 --- a/src/keys/tests/parity.rs +++ b/src/keys/tests/parity.rs @@ -4,9 +4,10 @@ //! see [`Golden`](crate::test_util::Golden) for the re-bless command. use super::*; -use crate::mux::driver::{MuxOptions, MuxSource, NoopEvents, mux_with_keys}; +use crate::mux::driver::{MuxOptions, mux_with_keys}; use crate::mux::parity_tests::{golden, record_run, record_tree}; use crate::mux::select::{PidFilter, StreamSelection}; +use crate::mux::source::{ScannedTitle, Source}; use crate::test_util::{CLIP_AUDIO_PIDS, Golden, synthetic_bd_clip}; fn refused(g: &mut Golden, key: &str, r: &Result) -> bool { @@ -48,7 +49,7 @@ fn copy_image(g: &mut Golden, tag: &str, fx: &Fx, reader: &mut dyn SectorSource) let dir = tempfile::tempdir().unwrap(); let dest = dir.path().join("out.iso"); let cap = fx.disc.capacity_sectors; - let r = crate::io::image_writer::write_image(reader, &dest, cap, &Halt::new(), |_| {}); + let r = crate::io::image_writer::write_image(reader, &dest, cap, &crate::ctx::Ctx::default()); match r { Ok(n) => { g.kv(&format!("{tag} written"), n); @@ -60,14 +61,17 @@ fn copy_image(g: &mut Golden, tag: &str, fx: &Fx, reader: &mut dyn SectorSource) } } -fn extract(g: &mut Golden, tag: &str, fx: &Fx, keys: Option<&ResolvedKeySet>, src: Faulty) { +fn extract(g: &mut Golden, tag: &str, fx: &Fx, keys: Option<&KeyRing>, src: Faulty) { let dest = tempfile::tempdir().unwrap(); let opts = crate::disc::ExtractOptions { keys, ..Default::default() }; let mut src = src; - match fx.disc.extract_tree(&mut src, dest.path(), &opts) { + match fx + .disc + .extract_tree(&mut src, dest.path(), &opts, &crate::ctx::Ctx::default()) + { Ok(r) => { g.kv( &format!("{tag} result"), @@ -96,7 +100,7 @@ fn extract(g: &mut Golden, tag: &str, fx: &Fx, keys: Option<&ResolvedKeySet>, sr } } -fn mux_cells(g: &mut Golden, fx: &Fx, set: Option<&ResolvedKeySet>, dead: Option<(u32, u32)>) { +fn mux_cells(g: &mut Golden, fx: &Fx, set: Option<&KeyRing>, dead: Option<(u32, u32)>) { let _serial = crate::sector::prefetched::holder_test_lock(); let dir = tempfile::tempdir().unwrap(); let iso = write_iso(fx, dir.path()); @@ -136,19 +140,11 @@ fn mux_cells(g: &mut Golden, fx: &Fx, set: Option<&ResolvedKeySet>, dead: Option if let Some((a, b)) = dead { src.kill(a, b); } - MuxSource::Live { - reader: Box::new(src), - title: title.clone(), - format, - } + Source::from_reader(Box::new(src), ScannedTitle::new(title.clone(), format)) } else { - MuxSource::Iso { - path: &iso, - title: title.clone(), - format, - } + Source::from_image(&iso, ScannedTitle::new(title.clone(), format)) }; - let r = mux_with_keys(source, set, &dest, opts, &Halt::new(), Arc::new(NoopEvents)); + let r = mux_with_keys(source, set, &dest, opts, &crate::ctx::Ctx::default()); record_run(g, tag, &r); record_tree(g, tag, &out); }; @@ -182,7 +178,7 @@ fn image_cell(name: &str, fx: &Fx, pool: &[[u8; 16]], dead: Option<(u32, u32)>) &mut fx.source(), KeyScope::WholeDisc, &specs, - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ); if !refused(&mut g, "resolve-whole", &whole) { @@ -207,7 +203,7 @@ fn image_cell(name: &str, fx: &Fx, pool: &[[u8; 16]], dead: Option<(u32, u32)>) &mut fx.source(), KeyScope::Titles(vec![0]), &specs, - ResolveKeysOptions::default(), + AcquireOptions::default(), &FakeClock::default(), ); if !refused(&mut g, "resolve-title", &titles) { @@ -438,7 +434,12 @@ fn parity_bug1_live_dvd_folder_is_descrambled() { let dest = tempfile::tempdir().unwrap(); let mut src = fx.source(); fx.disc - .extract_tree(&mut src, dest.path(), &Default::default()) + .extract_tree( + &mut src, + dest.path(), + &Default::default(), + &crate::ctx::Ctx::default(), + ) .unwrap(); for vts in ["VTS_01_1.VOB", "VTS_02_1.VOB"] { let got = std::fs::read(dest.path().join("VIDEO_TS").join(vts)).unwrap(); diff --git a/src/keys/tests/parity_live.rs b/src/keys/tests/parity_live.rs index e9013286..4ff0cda6 100644 --- a/src/keys/tests/parity_live.rs +++ b/src/keys/tests/parity_live.rs @@ -6,8 +6,9 @@ use super::*; use crate::dirimage::tests::{minimal_clpi, one_item_mpls}; use crate::disc::ScanOptions; use crate::drive::Drive; -use crate::mux::driver::{MuxOptions, MuxSource, NoopEvents, mux_with_keys}; +use crate::mux::driver::{MuxOptions, mux_with_keys}; use crate::mux::parity_tests::{golden, record_run, record_tree}; +use crate::mux::source::Source; use crate::session::{DiscSession, KeySpec}; use crate::test_util::{FakeMode, FakeTransport, Golden, synthetic_bd_clip}; @@ -72,22 +73,18 @@ fn mux( g: &mut Golden, tag: &str, s: &mut DiscSession, - set: Option<&ResolvedKeySet>, + set: Option<&KeyRing>, opts: &MuxOptions, halt: &Halt, ) { let dir = tempfile::tempdir().unwrap(); let dest = format!("mkv://{}", dir.path().join("o.mkv").display()); let r = mux_with_keys( - MuxSource::Session { - session: s, - title_index: 0, - }, + Source::from_session(s), set, &dest, opts, - halt, - Arc::new(NoopEvents), + &crate::ctx::Ctx::new(halt.clone()), ); record_run(g, tag, &r); record_tree(g, tag, dir.path()); @@ -101,11 +98,16 @@ fn opts(skip: bool) -> MuxOptions { } } -fn keys(s: &mut DiscSession, kdb: &[[u8; 16]]) -> crate::error::Result { +fn keys(s: &mut DiscSession, kdb: &[[u8; 16]]) -> crate::error::Result { let calls = Calls::default(); let f = factory(&[Spec::keydb(kdb, &calls)]); - s.resolve_key_set(KeyScope::Titles(vec![0]), &f, ResolveKeysOptions::default()) - .map(|r| r.keys) + s.acquire_keys( + KeyScope::Titles(vec![0]), + &f, + AcquireOptions::default(), + &crate::ctx::Ctx::default(), + ) + .map(|r| r.keys) } #[test] @@ -165,12 +167,13 @@ fn parity_live_bd_lazy_piece() { let disc = session(t2.with_image(img.image.clone()), &halt, &clip) .take_disc() .unwrap(); - let set = ResolvedKeySet::resolve( + let set = KeyRing::acquire_for_disc( &disc, &mut dead.clone(), KeyScope::Titles(vec![0]), &f, - ResolveKeysOptions::default(), + AcquireOptions::default(), + &crate::ctx::Ctx::default(), ) .unwrap() .keys; @@ -226,18 +229,14 @@ fn parity_live_bd_stop_mid_read() { armed.store(true, std::sync::atomic::Ordering::Relaxed); let dir = tempfile::tempdir().unwrap(); let r = mux_with_keys( - MuxSource::Session { - session: &mut s, - title_index: 0, - }, + Source::from_session(&mut s), Some(&set), &format!("mkv://{}", dir.path().join("o.mkv").display()), &MuxOptions { batch_sectors: 2, ..Default::default() }, - &halt, - Arc::new(NoopEvents), + &crate::ctx::Ctx::new(halt.clone()), ); let o = r.as_ref().map(|o| (o.completed, o.errors, o.lost_bytes)); g.kv("stop", format_args!("{o:?}")); diff --git a/src/keys/tests/stop_tests.rs b/src/keys/tests/stop_tests.rs index 9e5b885b..d99261d0 100644 --- a/src/keys/tests/stop_tests.rs +++ b/src/keys/tests/stop_tests.rs @@ -2,7 +2,7 @@ //! the op's token and progress (§2.12, :575), and each source call is busy (§2.1 item 2). use super::*; -use crate::halt::Progress; +use crate::halt::Liveness; // What one spy call saw: `ctx.halt()` is the op token, it was cancelled, and // `ctx.progress()` was present and busy. @@ -30,7 +30,7 @@ impl KeySource for Spy { .halt() .map(|h| Arc::ptr_eq(h.as_arc(), self.op.as_arc())), cancelled: ctx.halt().is_some_and(Halt::is_cancelled), - busy: ctx.progress().map(Progress::is_busy), + busy: ctx.progress().map(Liveness::is_busy), }); Ok(Vec::new()) } @@ -59,24 +59,21 @@ fn spies(op: &Halt, cancel_last: bool) -> (KeySourceFactory, Arc (f, seen) } -/// LSe4 (stop design §5.1, retargeted by §2.12 onto `ResolvedKeySet::resolve`): "a spy -/// sees `ctx.halt()` cancelled". Every source's ctx carries the op token itself, so a -/// Stop reaches a source mid-call; `resolve` then ends `Halted` and builds no set. +/// LSe4 (stop design §5.1, retargeted by §2.12 onto `KeyRing::acquire`): "a spy +/// sees `ctx.halt()` cancelled". Every source's ctx carries the run context's token +/// itself, so a Stop reaches a source mid-call; `acquire` then ends `Halted`, no ring. #[test] fn resolve_passes_halt_to_every_source_ctx() { let fx = two_units(); let op = Halt::new(); let (f, seen) = spies(&op, true); - let opts = ResolveKeysOptions { - halt: Some(&op), - ..Default::default() - }; - let r = ResolvedKeySet::resolve( + let r = KeyRing::acquire_for_disc( &fx.disc, &mut fx.source(), KeyScope::Titles(vec![0]), &f, - opts, + AcquireOptions::default(), + &crate::ctx::Ctx::new(op.clone()), ); assert!(matches!(r, Err(Error::Halted)), "{:?}", r.err()); let seen = seen.lock().unwrap().clone(); @@ -93,19 +90,23 @@ fn resolve_passes_halt_to_every_source_ctx() { /// ST4-2 / §2.1 item 2: "Each source call and each CSS crack in KU's `resolve` holds /// `busy()` on `ResolveCtx::progress()`" — so the idle-only T29 probe never counts a -/// slow keydb parse as idle. The op's `Progress` is the ctx's, busy inside each call only. +/// slow keydb parse as idle. The op's `Liveness` is the ctx's, busy inside each call only. #[test] fn resolve_holds_busy_around_every_source_call() { let fx = two_units(); - let (op, p) = (Halt::new(), Progress::new()); + let (op, p) = (Halt::new(), Liveness::new()); let (f, seen) = spies(&op, false); - let r = ResolvedKeySet::resolve_with_progress( + let opts = AcquireOptions { + liveness: Some(&p), + ..Default::default() + }; + let r = KeyRing::acquire_for_disc( &fx.disc, &mut fx.source(), KeyScope::Titles(vec![0]), &f, - ResolveKeysOptions::default(), - &p, + opts, + &crate::ctx::Ctx::new(op.clone()), ); assert!(r.is_err(), "no source holds a key: a refusal"); let seen = seen.lock().unwrap().clone(); @@ -117,25 +118,22 @@ fn resolve_holds_busy_around_every_source_call() { assert!(!p.is_busy(), "no busy span outlives resolve"); } -/// Guard: a ctx built with no token or progress hands out `None` for both (defaults). +/// Guard: with no liveness in the options, a source's ctx hands out no progress. #[test] -fn resolve_without_stop_hands_out_no_token_or_progress() { +fn resolve_without_liveness_hands_out_no_progress() { let fx = two_units(); let (f, seen) = spies(&Halt::new(), false); - let r = ResolvedKeySet::resolve( + let r = KeyRing::acquire_for_disc( &fx.disc, &mut fx.source(), KeyScope::Titles(vec![0]), &f, - ResolveKeysOptions::default(), + AcquireOptions::default(), + &crate::ctx::Ctx::default(), ); assert!(r.is_err()); let seen = seen.lock().unwrap().clone(); - assert!( - seen.iter() - .all(|s| s.halt_is_op.is_none() && s.busy.is_none()), - "{seen:?}" - ); + assert!(seen.iter().all(|s| s.busy.is_none()), "{seen:?}"); } /// Review minor 1 (§2.2 alias rule, as `Drive::alias`): under `open_with` the session's @@ -150,11 +148,13 @@ fn session_token_wins_over_the_callers_in_resolve() { let mut s = crate::session::DiscSession::from_parts_for_test(Some(fx.disc), Some(Box::new(reader))); s.set_halt_for_test(&session_tok); - let opts = ResolveKeysOptions { - halt: Some(&caller_tok), - ..Default::default() - }; - let r = s.resolve_key_set(KeyScope::Titles(vec![0]), &f, opts); + let ctx = crate::ctx::Ctx::new(caller_tok.clone()); + let r = s.acquire_keys( + KeyScope::Titles(vec![0]), + &f, + AcquireOptions::default(), + &ctx, + ); assert!(matches!(r, Err(Error::Halted)), "{:?}", r.err()); let seen = seen.lock().unwrap().clone(); assert!( diff --git a/src/keysource.rs b/src/keysource.rs index 55f3d9e1..7811c32f 100644 --- a/src/keysource.rs +++ b/src/keysource.rs @@ -159,10 +159,10 @@ pub trait ResolveCtx { fn halt(&self) -> Option<&crate::halt::Halt> { None } - /// The op's [`Progress`](crate::halt::Progress) (§2.7, T29): a source bumps it per - /// byte moved and holds [`busy`](crate::halt::Progress::busy) while a call is in + /// The op's [`Liveness`](crate::halt::Liveness) (§2.7, T29): a source bumps it per + /// byte moved and holds [`busy`](crate::halt::Liveness::busy) while a call is in /// flight. `None` when nothing watches the op. Defaulted like [`Self::halt`]. - fn progress(&self) -> Option<&crate::halt::Progress> { + fn progress(&self) -> Option<&crate::halt::Liveness> { None } } @@ -177,7 +177,7 @@ pub struct DiscInputsCtx<'a> { inner: &'a DiscInputs, enc_keys: Vec<[u8; 16]>, halt: Option<&'a crate::halt::Halt>, - progress: Option<&'a crate::halt::Progress>, + progress: Option<&'a crate::halt::Liveness>, } impl<'a> DiscInputsCtx<'a> { @@ -211,7 +211,7 @@ impl<'a> DiscInputsCtx<'a> { pub(crate) fn with_stop( self, halt: Option<&'a crate::halt::Halt>, - progress: Option<&'a crate::halt::Progress>, + progress: Option<&'a crate::halt::Liveness>, ) -> Self { Self { halt, @@ -250,7 +250,7 @@ impl ResolveCtx for DiscInputsCtx<'_> { fn halt(&self) -> Option<&crate::halt::Halt> { self.halt } - fn progress(&self) -> Option<&crate::halt::Progress> { + fn progress(&self) -> Option<&crate::halt::Liveness> { self.progress } } @@ -341,7 +341,7 @@ pub trait KeySource { } /// Whether this source's answer depends on the content samples it is sent. `true` (the - /// default): `ResolvedKeySet::resolve` asks it once per unopened piece, with that piece's + /// default): `KeyRing::resolve` asks it once per unopened piece, with that piece's /// samples. `false` (a keydb keyed by disc hash): asked once per resolve (KU §2.3 step 8). fn answer_depends_on_samples(&self) -> bool { true @@ -373,6 +373,16 @@ pub fn read_encrypted_units( reader: &mut dyn crate::sector::SectorSource, title: &crate::disc::DiscTitle, n: usize, +) -> Vec> { + encrypted_units_in(reader, &title.extents, title.content_format, n) +} + +// `read_encrypted_units` over bare extents (the key sampler's main-feature samples). +pub(crate) fn encrypted_units_in( + reader: &mut dyn crate::sector::SectorSource, + extents: &[crate::disc::Extent], + format: crate::disc::ContentFormat, + n: usize, ) -> Vec> { use crate::aacs::content::{ALIGNED_UNIT_LEN, ALIGNED_UNIT_SECTORS, aacs_unit_encrypted}; const CHUNK_UNITS: u32 = 15; // 45 sectors/read — under the drive transfer cap @@ -385,7 +395,7 @@ pub fn read_encrypted_units( if n == 0 { return out; } - for ext in &title.extents { + for ext in extents { let total_units = ext.sector_count / ALIGNED_UNIT_SECTORS; if total_units == 0 { continue; @@ -426,7 +436,7 @@ pub fn read_encrypted_units( break; } let u = &buf[o..o + ALIGNED_UNIT_LEN]; - if aacs_unit_encrypted(u, title.content_format) { + if aacs_unit_encrypted(u, format) { out.push(u.to_vec()); if out.len() >= n { return out; diff --git a/src/lib.rs b/src/lib.rs index 32bd83d1..8a644305 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -17,15 +17,16 @@ //! } //! ``` //! -//! Muxing runs through the PES pipeline. `input()` rejects live `disc://` -//! URLs with [`Error::DiscUrlNotDirect`] (use `Drive` + `Disc::scan` + -//! `DiscStream::new` instead); file-backed sources (`iso://`, `m2ts://`) open -//! directly: +//! Muxing runs through the PES pipeline. `input()` opens every input URL, a live +//! `disc://` drive included, and file-backed sources (`iso://`, `m2ts://`) the same +//! way: //! //! ```no_run //! # fn run() -> std::io::Result<()> { //! let opts = libfreemkv::InputOptions::default(); -//! let mut input = libfreemkv::input("iso://disc.iso", &opts)?; +//! // One run context: its halt stops every stage, its events hear the progress. +//! let ctx = libfreemkv::Ctx::default(); +//! let mut input = libfreemkv::input("iso://disc.iso", &opts, &ctx)?; //! let title = input.info().clone(); //! let mut output = libfreemkv::output("mkv://Movie.mkv", &title, None)?; //! // Propagate read errors instead of silently stopping on the first one. @@ -76,6 +77,8 @@ pub mod consts; #[cfg(feature = "rip")] pub mod css; #[cfg(feature = "rip")] +pub mod ctx; +#[cfg(feature = "rip")] pub mod decrypt; #[cfg(feature = "rip")] pub mod diag; @@ -178,15 +181,17 @@ pub use io::WritebackFile; #[cfg(feature = "rip")] pub use io::image_writer::write_image; -// ─── Drive events (low-level callbacks) ───────────────────────────────────── +// ─── Run context: halt, events, loss counters (pipeline design §2.5) ──────── #[cfg(feature = "rip")] -pub use event::{BatchSizeReason, Event, EventKind}; +pub use ctx::{Ctx, Diag, LossReport, Stats}; +#[cfg(feature = "rip")] +pub use event::{BatchSizeReason, Event, Events, NoEvents}; #[cfg(feature = "rip")] pub use identity::DriveId; // ─── Unlock seam: drive/disc unlocking (firmware, AACS cert, CSS bus-auth) lives entirely in `freemkv-unlock`; consumed via `unlock_bridge`, exposes nothing. ─── -// ─── Decryption (AACS/CSS): AACS keys come only from `keys::ResolvedKeySet` (KU §2.2); `decrypt_sectors()` is for raw sector buffers (ISO patching). ─── +// ─── Decryption (AACS/CSS): AACS keys come only from `keys::KeyRing` (KU §2.2); `decrypt_sectors()` is for raw sector buffers (ISO patching). ─── #[cfg(feature = "rip")] pub use decrypt::{decrypt_sectors, decrypt_threads, set_decrypt_threads}; @@ -194,27 +199,27 @@ pub use decrypt::{decrypt_sectors, decrypt_threads, set_decrypt_threads}; // `Disc::scan()` fully populates `Disc`; `Disc::identify()` is a UDF-only fast path // for name/format display. Codec enums are canonical; never compare display strings. #[cfg(feature = "rip")] +pub use aacs::trace::KeyOrigin; +#[cfg(feature = "rip")] pub use dirimage::DirImage; #[cfg(feature = "rip")] pub use disc::{ AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc, DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult, - FileResult, FrameRate, HdrFormat, Key, KeyOrigin, LabelPurpose, LabelQualifier, Resolution, - SampleRate, ScanOptions, Stream, SubtitleStream, VideoStream, + FileResult, FrameRate, HdrFormat, LabelPurpose, LabelQualifier, Resolution, SampleRate, + ScanOptions, Stream, SubtitleStream, VideoStream, }; #[cfg(feature = "rip")] pub use keysource::{DiscInputs, KeySource, read_encrypted_units}; // ─── Streams ──────────────────────────────────────────────────────────────── -// All types implement `pes::Stream` (re-exported `PesStream` to avoid colliding -// with `disc::Stream`). Prefer `input()`/`output()` URL resolvers over direct construction. +// Inputs implement `PesSource`, outputs `PesSink`. Prefer `input()`/`output()` URL +// resolvers over direct construction. #[cfg(feature = "rip")] pub use pes::PesFrame; #[cfg(feature = "rip")] -pub use pes::Stream as PesStream; +pub use pes::{PesSink, PesSource}; -#[cfg(feature = "rip")] -pub use mux::DiscStream; #[cfg(feature = "rip")] pub use mux::M2tsStream; #[cfg(feature = "rip")] @@ -230,7 +235,7 @@ pub use mux::WriteSeek; #[cfg(feature = "rip")] pub use mux::{FitReport, SkipReason, fit_report}; #[cfg(feature = "rip")] -pub use mux::{InputOptions, StreamUrl, disc_root_of, input, output, parse_url}; +pub use mux::{InputOptions, SinkCaps, StreamUrl, disc_root_of, input, output, parse_url}; #[cfg(feature = "rip")] pub use mux::{Medium, SourceInfo}; #[cfg(feature = "rip")] @@ -246,7 +251,7 @@ pub use mux::{Mp4FitReport, Mp4Sink, Mp4SkipReason, mp4_fit_report}; #[cfg(feature = "rip")] pub use mux::select::{PidFilter, StreamSelection}; #[cfg(feature = "rip")] -pub use mux::{MuxEvents, MuxOptions, MuxOutcome, MuxSource, mux_with_keys}; +pub use mux::{MuxOptions, MuxOutcome, ScannedTitle, Source, mux_url, mux_with_keys, open_source}; pub use scsi::{ DiscPresence, DriveInfo, ScsiSense, ScsiTransport, SenseFamily, disc_presence, drive_has_disc, list_drives, diff --git a/src/mux/codec/pgs_tests.rs b/src/mux/codec/pgs_tests.rs index 191994cb..d7880b3d 100644 --- a/src/mux/codec/pgs_tests.rs +++ b/src/mux/codec/pgs_tests.rs @@ -6,7 +6,7 @@ use super::*; use crate::disc::{Codec, SubtitleStream}; use crate::mux::mkv::{MkvMuxer, MkvTrack}; use crate::mux::mkvstream::MkvStream; -use crate::pes::{PesFrame, SourcePos, Stream}; +use crate::pes::{PesFrame, PesSource, SourcePos}; use std::io::Cursor; use std::path::Path; use std::process::Command; diff --git a/src/mux/demux_sink.rs b/src/mux/demux_sink.rs index 3f330028..0061b814 100644 --- a/src/mux/demux_sink.rs +++ b/src/mux/demux_sink.rs @@ -2,7 +2,7 @@ //! stream file (per-codec ES, PGS `.sup`, VobSub `.idx`/`.sub`, LPCM raw PCM), //! plus a chapters file and per-audio-track delay metadata. //! -//! This is a write-only [`crate::pes::Stream`] that routes each frame's payload to the file for +//! This is a write-only [`crate::pes::PesSink`] that routes each frame's payload to the file for //! `frame.track`, post-processing where the codec's internal `Frame` form differs from the //! on-disk ES form (HEVC/H.264 Annex-B, PGS `.sup`, VobSub `.idx`). @@ -11,7 +11,7 @@ use crate::mux::hevc::{ append_length_prefixed_as_annex_b_sized, avcc_to_annex_b, hvcc_to_annex_b, nal_length_size, }; use crate::mux::timeline::TimelineContinuity; -use crate::pes::{PesFrame, Stream}; +use crate::pes::{PesFrame, PesSink}; use std::fs::File; use std::io::{self, BufWriter, Write}; use std::path::{Path, PathBuf}; @@ -863,16 +863,11 @@ impl DemuxSink { } } -impl Stream for DemuxSink { +impl PesSink for DemuxSink { fn undelivered_streams(&self) -> Vec { self.excluded.seen() } - fn read(&mut self) -> io::Result> { - // Write-only sink, per the Stream trait contract. - Err(crate::error::Error::StreamWriteOnly.into()) - } - fn write(&mut self, frame: &PesFrame) -> io::Result<()> { if self.excluded.drop_frame(frame.track) { return Ok(()); @@ -1974,10 +1969,10 @@ mod tests { data: vec![0x00, 0x00, 0x00, 0x01, 0x09, 0x10, i as u8], duration_ns: None, }; - let _ = Stream::write(&mut sink, &f); + let _ = PesSink::write(&mut sink, &f); } - let err = Stream::finish(&mut sink).expect_err("an emptied export must fail"); + let err = PesSink::finish(&mut sink).expect_err("an emptied export must fail"); assert_eq!( crate::error::error_code(&err), Some(crate::error::E_SINK_WROTE_NOTHING), @@ -2026,7 +2021,7 @@ mod tests { data: vec![0x00, 0x00, 0x00, 0x01, 0x09, 0x10, i as u8], duration_ns: None, }; - let _ = Stream::write(&mut sink, &f); + let _ = PesSink::write(&mut sink, &f); } // … and one frame inside the first clip (150 s, in ns) that DOES persist, // so frames_mapped > 0 while seam_dropped (2) still exceeds it (1). @@ -2040,9 +2035,9 @@ mod tests { data: vec![0x00, 0x00, 0x00, 0x01, 0x09, 0x10, 0x42], duration_ns: None, }; - Stream::write(&mut sink, &inside).unwrap(); + PesSink::write(&mut sink, &inside).unwrap(); - let err = Stream::finish(&mut sink).expect_err("a mostly-emptied export must fail"); + let err = PesSink::finish(&mut sink).expect_err("a mostly-emptied export must fail"); assert_eq!( crate::error::error_code(&err), Some(crate::error::E_SEAM_PLAN_DROPPED_MOST), @@ -2108,10 +2103,10 @@ mod tests { data: vec![0x00, 0x00, 0x00, 0x01, 0x09, 0x10, i as u8], duration_ns: None, }; - let _ = Stream::write(&mut sink, &f); + let _ = PesSink::write(&mut sink, &f); } // One AUDIO frame inside the first clip (150 s): placed and persisted. - Stream::write( + PesSink::write( &mut sink, &PesFrame { discard_padding_ns: 0, @@ -2127,7 +2122,7 @@ mod tests { .unwrap(); // Gate must PASS: the only drops were on the filtered-out video track. - Stream::finish(&mut sink) + PesSink::finish(&mut sink) .expect("a filtered export must not fail on drops it never persisted"); // And the audio file was actually written with its payload. let wrote_audio = std::fs::read_dir(&dir) @@ -2183,12 +2178,12 @@ mod tests { duration_ns: None, }; for s in [150i64, 151, 152] { - let _ = Stream::write(&mut sink, &frame(0, s * 1_000_000_000)); + let _ = PesSink::write(&mut sink, &frame(0, s * 1_000_000_000)); } for s in [0i64, 1] { - let _ = Stream::write(&mut sink, &frame(1, s * 1_000_000_000)); + let _ = PesSink::write(&mut sink, &frame(1, s * 1_000_000_000)); } - let err = Stream::finish(&mut sink).expect_err("no audio was written"); + let err = PesSink::finish(&mut sink).expect_err("no audio was written"); assert_eq!( crate::error::error_code(&err), Some(crate::error::E_SINK_WROTE_NOTHING) @@ -2196,16 +2191,6 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } - #[test] - fn sink_read_returns_write_only() { - let dir = tempdir(); - let title = title_with(vec![video_stream(Codec::Mpeg2)], vec![None]); - let mut sink = DemuxSink::create(&dir, &title, &DemuxOptions::default()).unwrap(); - let err = Stream::read(&mut sink).expect_err("sink read must error"); - assert_eq!(err.kind(), io::ErrorKind::Unsupported); - let _ = std::fs::remove_dir_all(&dir); - } - /// Tiny unique temp dir helper (avoids a dev-dependency on `tempfile`). fn tempdir() -> PathBuf { use std::sync::atomic::{AtomicU64, Ordering}; diff --git a/src/mux/demux_thread.rs b/src/mux/demux_thread.rs index 143e8ebd..44abdf1e 100644 --- a/src/mux/demux_thread.rs +++ b/src/mux/demux_thread.rs @@ -56,18 +56,19 @@ impl DemuxThread { /// returned to `recycle_tx` for the producer to re-fill. /// /// `producer_shell` is an opaque handle that outlives the demux thread and joins the - /// upstream producer on drop. Accepts a shell from - /// [`crate::sector::PrefetchedSectorSource::into_channels`] or - /// [`crate::io::byte_prefetcher::BytePrefetcher::into_channels`]. + /// upstream producer on drop: the shell from + /// [`crate::sector::PrefetchedSectorSource::into_channels`]. pub fn spawn_zero_copy( prefetch_rx: Receiver>>, recycle_tx: Sender>, producer_shell: S, - halt: Option, + ctx: &crate::ctx::Ctx, ts: Option, ps: Option, ) -> crate::error::Result<(Self, Receiver)> { let (tx, rx) = bounded::(DEMUX_CHANNEL_DEPTH); + let halt = Some(ctx.halt.clone()); + let prof = ctx.diag.profile; let mut ts = ts; let mut ps = ps; @@ -77,7 +78,6 @@ impl DemuxThread { let spawn_result = std::thread::Builder::new() .name("freemkv-demux".into()) .spawn(move || { - let prof = std::env::var_os("FREEMKV_PROFILE").is_some(); let mut prof_started = std::time::Instant::now(); let mut prof_last_dump = prof_started; let mut prof_read_ns: u128 = 0; @@ -301,8 +301,15 @@ mod tests { let (rc_tx, _rc_rx) = bounded::>(4); let pid = 0x1011; let ts = super::super::ts::TsDemuxer::new(&[pid]); - let (_dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (_dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); pf_tx.send(Ok(bdts_pes_packet(pid, &[0xDE, 0xAD]))).unwrap(); drop(pf_tx); // producer done → EOF @@ -331,8 +338,15 @@ mod tests { let (rc_tx, _rc_rx) = bounded::>(4); let pid = 0x1011; let ts = super::super::ts::TsDemuxer::new(&[pid]); - let (_dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (_dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); pf_tx .send(Ok(bdts_pes_packet(pid, &[0x11, 0x22, 0x33]))) @@ -365,9 +379,15 @@ mod tests { halt.cancel(); } let ts = super::super::ts::TsDemuxer::new(&[0x1011]); - let (dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), Some(halt.clone()), Some(ts), None) - .unwrap(); + let (dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::new(halt.clone()), + Some(ts), + None, + ) + .unwrap(); if close_first { halt.cancel(); drop(pf_tx); @@ -401,8 +421,15 @@ mod tests { let (rc_tx, _rc_rx) = bounded::>(4); let pid = 0x1011; let ts = super::super::ts::TsDemuxer::new(&[pid]); - let (_dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (_dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); pf_tx.send(Ok(bdts_pes_packet(pid, &[0x01]))).unwrap(); let mut second = bdts_pes_packet(pid, &[0x02]); second[7] |= 1; // next CC: a same-CC packet is a duplicate @@ -429,8 +456,15 @@ mod tests { let (pf_tx, pf_rx) = bounded::>>(4); let (rc_tx, _rc_rx) = bounded::>(4); let ts = super::super::ts::TsDemuxer::new(&[0x1011]); - let (_dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (_dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); pf_tx.send(Err(std::io::Error::other("boom"))).unwrap(); drop(pf_tx); @@ -456,8 +490,15 @@ mod tests { let (rc_tx, rc_rx) = bounded::>(4); let pid = 0x1011; let ts = super::super::ts::TsDemuxer::new(&[pid]); - let (_dt, _rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (_dt, _rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); pf_tx.send(Ok(bdts_pes_packet(pid, &[0xAA]))).unwrap(); let recycled = rc_rx.recv_timeout(Duration::from_secs(5)); @@ -474,8 +515,15 @@ mod tests { let (pf_tx, pf_rx) = bounded::>>(4); let (rc_tx, _rc_rx) = bounded::>(4); let ps = super::super::ps::PsDemuxer::new(); - let (_dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, None, Some(ps)).unwrap(); + let (_dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + None, + Some(ps), + ) + .unwrap(); // PES (video 0xE0, bounded length 5) + program-end delimiter. let mut buf = vec![ @@ -503,7 +551,9 @@ mod tests { // and terminate with Eof — also forward an empty batch per buffer for disconnect detection. let (pf_tx, pf_rx) = bounded::>>(4); let (rc_tx, rc_rx) = bounded::>(4); - let (_dt, rx) = DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, None, None).unwrap(); + let (_dt, rx) = + DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), &crate::ctx::Ctx::default(), None, None) + .unwrap(); pf_tx.send(Ok(vec![0u8; 192])).unwrap(); assert!( @@ -534,8 +584,15 @@ mod tests { let (rc_tx, _rc_rx) = bounded::>(4); let pid = 0x1011; let ts = super::super::ts::TsDemuxer::new(&[pid]); - let (_dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (_dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); // A non-PUSI packet on a tracked PID with header_remaining 0 and no // active PES: process_packet pushes nothing (asm inactive), so feed @@ -573,8 +630,15 @@ mod tests { let (pf_tx, pf_rx) = bounded::>>(4); let (rc_tx, _rc_rx) = bounded::>(4); let ts = super::super::ts::TsDemuxer::new(&[tracked_pid]); - let (dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); // An endless producer: it stops only once the worker is gone. std::thread::spawn(move || while pf_tx.send(Ok(empty_pkt.clone())).is_ok() {}); drop(rx); @@ -597,8 +661,15 @@ mod tests { let (rc_tx, _rc_rx) = bounded::>(16); let pid = 0x1011; let ts = super::super::ts::TsDemuxer::new(&[pid]); - let (dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), None, Some(ts), None).unwrap(); + let (dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::default(), + Some(ts), + None, + ) + .unwrap(); for i in 0..8u8 { let mut chunk = bdts_pes_packet(pid, &[i]); chunk.extend(bdts_pes_packet(pid, &[i, i])); @@ -638,9 +709,15 @@ mod tests { let pid = 0x1011; let ts = super::super::ts::TsDemuxer::new(&[pid]); let halt = Halt::new(); - let (dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rc_tx, (), Some(halt.clone()), Some(ts), None) - .unwrap(); + let (dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rc_tx, + (), + &crate::ctx::Ctx::new(halt.clone()), + Some(ts), + None, + ) + .unwrap(); // Rebound after `dt` so a failing assert drops the input first (no hung join). let pf_tx = pf_tx; for i in 0..16u8 { diff --git a/src/mux/disc.rs b/src/mux/disc.rs deleted file mode 100644 index 184ecee3..00000000 --- a/src/mux/disc.rs +++ /dev/null @@ -1,3554 +0,0 @@ -//! DiscStream — read any disc (physical drive or ISO file) → PES frames. -//! -//! One stream type for all disc sources. The source is a SectorSource — -//! Drive (hardware) or FileSectorSource (file). DiscStream doesn't care. -//! -//! Read-only. For disc→ISO (raw sector copy), use `freemkv_engine::recovery::copy`. - -use super::ts::PesPacket; -use crate::disc::{DiscTitle, Extent}; -use crate::drive::extract_scsi_context; -use crate::event::{BatchSizeReason, Event, EventKind}; -use crate::halt::Halt; -use crate::sector::{DecryptingSectorSource, SectorSource}; -use std::io; -#[cfg(test)] -use std::sync::Arc; -#[cfg(test)] -use std::sync::atomic::AtomicBool; - -// Ramp back to the preferred batch size after this many clean sectors (100 -// MiB = 51,200 sectors) — long enough that noisy zones can't trigger a -// premature probe, short enough an isolated failure doesn't lock size 1. -const PROBE_THRESHOLD_SECTORS: u32 = 100 * 1024 * 1024 / 2048; - -/// Halve a batch size, keeping 3-sector alignment when >= 6 -/// (3-sector alignment = one AACS unit). At sizes < 6 we descend -/// through 3 → 1 without intermediate unaligned sizes. -fn halve_batch_size(size: u16) -> u16 { - let h = (size / 2).max(1); - if h >= 6 { h - (h % 3) } else { h } -} - -/// Double a batch size toward a preferred max, keeping 3-sector alignment -/// when the result is >= 6. -fn double_batch_size(size: u16, preferred: u16) -> u16 { - let d = size.saturating_mul(2).min(preferred); - if d >= 6 { d - (d % 3) } else { d } -} - -/// Adaptive batch sizer. Shrinks on read failure, grows after a sustained -/// clean streak. Amortizes the cost of entering a bad zone — descent happens -/// once, not once per bad sector. -#[derive(Debug)] -struct AdaptiveBatch { - preferred: u16, - current: u16, - streak_sectors: u32, -} - -impl AdaptiveBatch { - fn new(preferred: u16) -> Self { - Self { - preferred, - current: preferred, - streak_sectors: 0, - } - } - - fn current(&self) -> u16 { - self.current - } - - /// Record a successful read of `sectors`. Returns an event if the - /// sizer probed up to a larger batch size. - fn on_success(&mut self, sectors: u16) -> Option { - self.streak_sectors = self.streak_sectors.saturating_add(sectors as u32); - if self.current < self.preferred && self.streak_sectors >= PROBE_THRESHOLD_SECTORS { - let new_size = double_batch_size(self.current, self.preferred); - if new_size != self.current { - self.current = new_size; - self.streak_sectors = 0; - return Some(EventKind::BatchSizeChanged { - new_size, - reason: BatchSizeReason::Probed, - }); - } - } - None - } - - /// Record a read failure. Returns an event if the sizer shrank. - /// Does nothing at size 1 (caller handles skip/error). - fn on_failure(&mut self) -> Option { - self.streak_sectors = 0; - if self.current <= 1 { - return None; - } - let new_size = halve_batch_size(self.current); - self.current = new_size; - Some(EventKind::BatchSizeChanged { - new_size, - reason: BatchSizeReason::Shrunk, - }) - } -} - -/// Disc stream. Reads sectors from any source → PES frames. -/// -/// Sources: physical drive, ISO file, or any SectorSource. -/// Decrypt, demux, and codec parsing happen internally. -pub struct DiscStream { - /// Underlying sector source wrapped in the 0.18 - /// [`DecryptingSectorSource`] decorator. Every `read_sectors` - /// call yields plaintext, so `fill_extents` no longer needs an - /// inline `decrypt::decrypt_sectors` step. `DecryptKeys::None` - /// (raw / unencrypted disc) makes the decorator a pass-through. - reader: DecryptingSectorSource>, - title: DiscTitle, - /// Mirror of the keys handed in at construction. The decorator - /// owns the cryptographic state; this field is kept for - /// metadata-side callers (`info()` and friends) that want to - /// know whether the disc was encrypted, without reaching through - /// the wrapper. - decrypt_keys: crate::decrypt::DecryptKeys, - - /// Sector granularity the decrypt step requires each read buffer to start - /// on and span a multiple of. AACS decrypts whole 6144-byte (3-sector) - /// units keyed off the buffer's first 16 bytes, so every `read_sectors` - /// buffer must begin on a real on-disc unit boundary — hence reads and - /// error-skips must stay aligned to this. `3` for AACS, `1` for CSS / - /// unencrypted (per-sector, self-synchronizing). Mirrors the file-backed - /// highway's `PrefetchedSectorSource` guard; this is the inline live path. - unit_align: u16, - - // Extents to read - extents: Vec, - - // Position - current_extent: usize, - current_offset: u32, - - // Buffer - read_buf: Vec, - buf_valid: usize, - - // Adaptive batch sizer — preferred comes from the caller - // (detect_max_batch_sectors), shrinks/grows based on read outcomes. - adaptive: AdaptiveBatch, - errors: u64, - /// Cumulative bytes actually skipped (zero-filled) on read error. - /// Distinct from `errors`, which counts skip *events*: one event can - /// cover a whole AACS unit (`unit_align` sectors = 6144 bytes), so - /// `errors * 2048` understates real loss by the alignment factor. - /// Consumers estimating lost video time must scale by this, not by - /// the event count. - lost_bytes: u64, - pub skip_errors: bool, - /// When set and the token is cancelled, fill_extents returns Err(Halted) - /// at the next retry boundary. Unlike skip_errors, this propagates the - /// error up so the rip terminates cleanly. Construct with - /// [`DiscStream::with_halt`], passing the same `Halt` clone handed to - /// sweep / patch / mux so every phase observes one Stop signal. - halt: Option, - event_fn: Option>, - eof: bool, - /// Count of dropped DVD navigation packets (private_stream_2, 0xBF) — these - /// are expected on every disc; tallied and summarised once at EOF instead of - /// a per-packet WARN. - dropped_nav_packets: u64, - /// Packets of MPEG-2 audio extension streams (`0xD0|n`) with no declared extension track, - /// reported once at EOF (see `ps::warn_undeclared_extensions`). - mpeg_extension_packets: [u64; 8], - dropped_ps: super::ps::DroppedPs, - - // Cumulative bytes successfully read from the source. Drives - // EventKind::BytesRead emission and autorip's per-device progress. - bytes_read_total: u64, - // Pre-computed total of all extents in bytes (or 0 if extents are - // empty). Carried in EventKind::BytesRead.total so consumers can show - // a percent without a separate API call. - bytes_total_extents: u64, - - // PES output — single-threaded inline demux + codec parse. Pipeline-mode - // mux lives in [`super::pipelined_stream::PipelinedPesStream`]; this is - // the legacy in-thread path where bad-sector adaptive retry lives. - ts_demuxer: Option, - ps_demuxer: Option, - parsers: Vec<(u16, Box)>, - pending_frames: std::collections::VecDeque, - pid_to_track: Vec<(u16, usize)>, - /// Cached `FREEMKV_SKIP_PARSE` profiling flag. The env var cannot - /// change at runtime, and `std::env::var_os` takes a process-wide - /// lock; reading it once at construction keeps it out of the - /// per-batch read() hot loop. - skip_parse: bool, - /// Cached `FREEMKV_PROFILE` presence, read once at construction. When - /// false, the read() loop skips the four `Instant::now()` captures and the - /// `prof_tick` calls entirely, so profiling-off runs pay no per-iteration - /// timestamp cost or `prof_active()` env-var lookup (which takes a - /// process-wide lock). - profiling: bool, - /// Cumulative bytes fed to the demuxer across all read buffers, - /// used to stamp byte-exact provenance (`SourcePos`) onto demuxed packets. - fed_bytes: u64, - /// B1 drop-to-keyframe resync gate, one per stream — mirrors - /// `PipelinedPesStream`. After an AACS-concealed (undecryptable) gap stamps - /// a frame `discontinuity`, the gate drops forward inter-coded video frames - /// until the next keyframe so no dangling-reference frame reaches the muxer. - /// Without this the live-drive single-pass path emitted decode-broken output - /// on an undecryptable unit (the file-backed highway path is already gated). - resync: Vec, - /// Whether each stream (by track index) is video — audio/subtitle frames are - /// independent and always admit. - is_video: Vec, - /// Per-track PS access-unit assembler and H.264 second-field merge, as in - /// `PipelinedPesStream`; PS video AUs span many PES fragments. - au_asm: Vec, - field_merge: Vec>, - /// Bounds the header pump's wait for in-band codec configs (AAC). - header_gate: super::header_gate::HeaderGate, -} - -impl DiscStream { - /// Create a disc stream from any sector reader. - /// - /// Works with physical drives and ISO files — both implement SectorSource. - /// The caller opens the source, scans for titles/keys, and passes them in. - /// The stream handles demuxing, decryption, and codec parsing internally. - pub fn new( - mut reader: Box, - title: DiscTitle, - mut decrypt_keys: crate::decrypt::DecryptKeys, - batch_sectors: u16, - content_format: crate::disc::ContentFormat, - raw: bool, - halt: Option, - ) -> std::io::Result { - // A zero batch reads 0 sectors and never advances (endless loop until - // Stop); `MuxOptions::default()` carries 0. Same refusal as the highway. - if batch_sectors == 0 { - return Err(crate::error::Error::MuxBatchSectorsZero.into()); - } - let mut title = title; - let extents = title.extents.clone(); - - // Resolve this title's CSS key here (same step build_iso_pipeline uses, - // so passes descramble identically); no-op for AACS/clear/raw. `halt` is - // passed now, not via `with_halt`, since the scan runs before attach. - crate::css::resolve_dvd_title_key( - &mut *reader, - &extents, - &mut decrypt_keys, - batch_sectors, - content_format, - raw, - halt.as_ref(), - )?; - let bytes_total_extents: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum(); - - // Debug log reader type — helps diagnose mux reading from drive instead - // of ISO. `type_name_of_val` resolves the CONCRETE type behind the box, - // unlike `type_name::()` which prints the trait name. - tracing::debug!( - target: "mux", - "DiscStream constructed with reader type: {}", - std::any::type_name_of_val(&*reader) - ); - - // CSS/unencrypted content needs a decrypting wrapper to yield plaintext - // VOB bytes before the AC-3 probe reads real `acmod`s. A unit that - // decrypts to broken TS is the muxer's concern, not loss to conceal. - let mut reader = DecryptingSectorSource::new(reader, decrypt_keys.clone()); - - // Diagnostics only: logs each physical AC-3 sub-stream's real channel count - // at --log-level 3. Routing is the scanner's PGC AST_CTL map. - crate::disc::dvd_audio_probe::probe_and_remap(&mut reader, &mut title); - - // Use the CANONICAL builder shared with the file-backed highway - // (`resolve::build_demux_state`): an open-coded copy of this once - // drifted and mis-routed Blu-ray 3D MVC dependent views on this path. - let (parsers, pid_to_track, ts_demuxer, ps_demuxer) = - super::resolve::build_demux_state(&title, content_format); - - // AACS decrypts whole 6144-byte (3-sector) units keyed off each read - // buffer's first bytes, so reads/skips must stay 3-sector aligned; CSS/ - // unencrypted is per-2048-byte, self-sync (align 1). Same rule as resolve.rs. - let unit_align: u16 = match &decrypt_keys { - crate::decrypt::DecryptKeys::Aacs { .. } => 3, - _ => 1, - }; - - // `reader` is already wrapped above (DecryptKeys::None makes it a - // pass-through). Reset the unit base the probe read advanced so the - // first fill_extents read starts cleanly. - reader.clear_unit_base(); - - // B1 resync gates: one per stream, video flagged so the gate only - // drop-to-keyframes video (audio/subtitle always admit). Computed before - // `title` is moved into the struct below. - let is_video: Vec = title - .streams - .iter() - .map(|s| matches!(s, crate::disc::Stream::Video(_))) - .collect(); - let resync = (0..title.streams.len()) - .map(|_| super::resync::ResyncGate::new()) - .collect(); - let au_asm = title - .streams - .iter() - .map(|s| { - super::au_assembly::AuAssembler::for_codec(super::pipelined_stream::stream_codec(s)) - }) - .collect(); - let field_merge = title - .streams - .iter() - .map(|s| { - (super::pipelined_stream::stream_codec(s) == crate::disc::Codec::H264) - .then(super::au_assembly::SecondFieldMerge::default) - }) - .collect(); - - Ok(Self { - reader, - title, - decrypt_keys, - unit_align, - extents, - current_extent: 0, - current_offset: 0, - read_buf: Vec::with_capacity(batch_sectors as usize * 2048), - buf_valid: 0, - adaptive: AdaptiveBatch::new(batch_sectors), - errors: 0, - lost_bytes: 0, - skip_errors: false, - halt, - event_fn: None, - eof: false, - dropped_nav_packets: 0, - mpeg_extension_packets: [0; 8], - dropped_ps: Default::default(), - bytes_read_total: 0, - bytes_total_extents, - ts_demuxer, - ps_demuxer, - parsers, - pending_frames: std::collections::VecDeque::new(), - pid_to_track, - skip_parse: std::env::var_os("FREEMKV_SKIP_PARSE").is_some(), - profiling: std::env::var_os("FREEMKV_PROFILE").is_some(), - fed_bytes: 0, - resync, - is_video, - au_asm, - field_merge, - header_gate: super::header_gate::HeaderGate::default(), - }) - } - - /// Set event handler for sector-level events (binary search, skip, recover). - pub fn on_event(&mut self, f: impl Fn(Event) + Send + 'static) { - self.event_fn = Some(Box::new(f)); - } - - /// Constructor-time builder: attach a [`Halt`] token so that when - /// any clone is cancelled, the next read-retry boundary inside - /// `fill_extents` returns `Err(Halted)`. Required for Stop to work - /// during dense bad-sector regions (where the outer PES read() loop - /// can spend minutes inside fill_extents before emitting a frame). - /// - /// Pass the same `Halt` clone you hand to sweep / patch / mux so every - /// phase observes a single Stop signal. - pub fn with_halt(mut self, halt: Halt) -> Self { - self.halt = Some(halt); - self - } - - // The extent walk becomes `map`'s read plan: an FMTS forensic segment keeps only our-phase - // units (a non-forensic map leaves the extents unchanged). The key map itself is - // installed by `keys::install_key_map`. - pub(crate) fn plan_reads(&mut self, map: &crate::decrypt::AacsKeyMap) { - self.extents = map.read_plan(&self.extents, self.unit_align.max(1) as u32); - self.bytes_total_extents = self - .extents - .iter() - .map(|e| e.sector_count as u64 * 2048) - .sum(); - } - - pub(crate) fn reader_mut(&mut self) -> &mut DecryptingSectorSource> { - &mut self.reader - } - - /// Install a [`ResolvedKeySet`](crate::keys::ResolvedKeySet)'s on-arrival proof on the - /// inline reader (KU §2.4): a piece the set left unproven is proven when first read. - pub(crate) fn with_arrival(mut self, arrival: crate::keys::Arrival) -> Self { - self.reader.set_arrival(arrival); - self - } - - fn is_halted(&self) -> bool { - self.halt - .as_ref() - .map(|h| h.is_cancelled()) - .unwrap_or(false) - } - - // A failed read that Stop caused: the reader reports Halted, or our token fired. - fn stopped(&self, e: &crate::error::Error) -> bool { - matches!(e, crate::error::Error::Halted) || self.is_halted() - } - - fn emit(&self, kind: EventKind) { - if let Some(ref f) = self.event_fn { - f(Event { kind }); - } - } - - /// Skip decryption — return raw encrypted bytes. Updates both - /// the metadata-side key field and the wrapped reader's keys so - /// subsequent `read_sectors` calls become a pass-through. - pub fn set_raw(&mut self) { - self.decrypt_keys = crate::decrypt::DecryptKeys::None; - self.reader.set_keys(crate::decrypt::DecryptKeys::None); - } - - // Commit a successful read_sectors, zero-filling+accounting a short read under skip_errors - // (never a silent hole) instead of trusting the count. - fn commit_read(&mut self, lba: u32, got: usize, bytes: usize) -> io::Result<()> { - if got < bytes { - if !self.skip_errors { - return Err(crate::error::Error::DiscRead { - sector: lba as u64, - status: None, - sense: None, - } - .into()); - } - // Clear the tail the source did not write — it holds stale bytes - // from a previous batch, which would mux as plausible garbage. - self.read_buf[got..bytes].fill(0); - self.errors += 1; - self.lost_bytes = self.lost_bytes.saturating_add((bytes - got) as u64); - self.emit(EventKind::SectorSkipped { sector: lba as u64 }); - } - self.buf_valid = bytes; - // `bytes` is always a whole number of 2048-byte logical sectors (it is - // `sectors * 2048`), so this is the requested sector count — the cursor - // advances a whole unit span either way, keeping AACS alignment. - self.current_offset += (bytes / 2048) as u32; - // Only the bytes the source actually delivered count as read; the - // zero-filled tail is loss, already charged to `lost_bytes`. - self.bytes_read_total = self.bytes_read_total.saturating_add(got as u64); - self.emit(EventKind::BytesRead { - bytes: self.bytes_read_total, - total: self.bytes_total_extents, - }); - Ok(()) - } - - fn fill_extents(&mut self) -> io::Result { - // Skip exhausted/zero-sector extents iteratively, not recursively: a - // malformed disc can declare thousands of empty extents, and a - // self-recursive version overflowed the stack, aborting the process. - let (ext_start, ext_sectors, remaining) = loop { - if self.current_extent >= self.extents.len() { - return Ok(false); - } - let ext = &self.extents[self.current_extent]; - let remaining = ext.sector_count.saturating_sub(self.current_offset); - if remaining > 0 { - break (ext.start_lba, ext.sector_count, remaining); - } - self.current_extent += 1; - self.current_offset = 0; - }; - - // start_lba comes from UDF/MPLS extents; a malformed extent near - // u32::MAX would overflow (debug panic / release wrap to a wrong LBA). - // Saturate for consistency with the rest of the file's arithmetic. - let lba = ext_start.saturating_add(self.current_offset); - - // AACS aligned units are anchored at this extent's start LBA — gate the - // decrypt-on-read source relative to it (clip-anchored), not absolute - // disc LBA 0. No-op for CSS / None. - self.reader.set_unit_base(ext_start); - - // Adaptive sizer shrinks on failure, grows on success; one 5s attempt - // per try, no retry loops. Halt is checked every iteration since a bad - // zone can spend minutes here, else Stop waits for the outer read() loop. - - let start_time = std::time::Instant::now(); - - loop { - if self.is_halted() { - return Err(crate::error::Error::Halted.into()); - } - - // Debug: log slow reads during mux — helps diagnose stalls - if cfg!(debug_assertions) && start_time.elapsed().as_secs() > 5 { - tracing::debug!(target: "mux", "fill_extents waiting at LBA {} ({}s elapsed, sectors={})", lba, start_time.elapsed().as_secs(), remaining); - } - - // Every read buffer must start on a real on-disc unit boundary: AACS - // (align=3) decrypts whole units keyed off the buffer's first bytes, - // so a sub-unit mid-extent read would desync the rest of the title. - let align = self.unit_align.max(1) as u32; - let want = remaining.min(self.adaptive.current() as u32); - let sectors: u16 = if align <= 1 { - want as u16 - } else if remaining < align { - remaining as u16 - } else if want < align { - align as u16 - } else { - (want - want % align) as u16 - }; - let bytes = sectors as usize * 2048; - self.read_buf.resize(bytes, 0); - - let res = self - .reader - .read_sectors(lba, sectors, &mut self.read_buf[..bytes], false); - // The key set's loud stop (KU §2.4, §6): a readable unit no held key opens is not - // bad media — never shrunk, recovered or skipped. - if let Err(e) = res.as_ref() - && is_key_stop(e) - { - return Err(res.unwrap_err().into()); - } - // A read failed by Stop (cancelled drive, or a stop mid bad zone) is - // not bad media: never shrink, recover or skip it. - if let Err(e) = res.as_ref() - && self.stopped(e) - { - return Err(crate::error::Error::Halted.into()); - } - - if let Ok(&got) = res.as_ref() { - // read_sectors returns bytes written into buf. All in-tree - // sources are full-or-error, but a short count would leave - // stale/zeroed tail bytes in read_buf. - debug_assert!(got <= bytes, "read_sectors over-reported byte count"); - if let Some(ev) = self.adaptive.on_success(sectors) { - self.emit(ev); - } - let got = got.min(bytes); - self.commit_read(lba, got, bytes)?; - break; - } - - // Transport failure (status=0xFF: bridge crash/disconnect) is NOT a - // skippable bad sector — the bridge is wedged and every subsequent - // read fails, so skipping just marches the disc producing no MKV. - if let Some(e) = res.as_ref().err() - && e.is_scsi_transport_failure() - { - let (status, sense) = extract_scsi_context(e); - return Err(crate::error::Error::DiscRead { - sector: lba as u64, - status: Some(status), - sense, - } - .into()); - } - - // The read SOURCE is gone (e.g. a terminated prefetch thread), not - // bad media — skip_errors would zero-fill the rest of the title and - // report success. Abort with the terminal error, not a fake status. - if let Some(e) = res.as_ref().err() - && e.is_source_terminated() - { - return Err(crate::error::Error::SourceTerminated.into()); - } - - // The image file ends before this read: every later sector is missing - // too, so neither shrink/retry nor skip_errors applies. - if let Err(e @ crate::error::Error::ImageEndsBeforeRead { .. }) = res { - return Err(e.into()); - } - - if (sectors as u32) <= align { - // Bottomed out with no later pass to recover — give the drive one - // full ECC recovery read (~60s) before skip/bail. A single bounded - // read, never a loop (hard rule #2: retry loops fast-fail the BU40N). - tracing::debug!( - target: "mux", - "fill_extents last-chance recovery read at LBA {} ({} sectors, 60s ECC)", - lba, - sectors - ); - let rec = self - .reader - .read_sectors(lba, sectors, &mut self.read_buf[..bytes], true); - if let Err(e) = rec.as_ref() - && is_key_stop(e) - { - return Err(rec.unwrap_err().into()); - } - if let Err(e) = rec.as_ref() - && self.stopped(e) - { - return Err(crate::error::Error::Halted.into()); - } - if let Ok(&got) = rec.as_ref() { - debug_assert!(got <= bytes, "recovery read over-reported byte count"); - if let Some(ev) = self.adaptive.on_success(sectors) { - self.emit(ev); - } - let got = got.min(bytes); - // Same short-read rule as the first-attempt read above — - // one shared commit, so the two cannot drift apart. - self.commit_read(lba, got, bytes)?; - break; - } - - // Recovery read also failed: re-check `rec` for a transport - // failure (bridge crash) same as `res` above — the 60s recovery - // read can wedge the bridge on its own; abort rather than skip. - if let Some(e) = rec.as_ref().err() - && e.is_scsi_transport_failure() - { - let (status, sense) = extract_scsi_context(e); - return Err(crate::error::Error::DiscRead { - sector: lba as u64, - status: Some(status), - sense, - } - .into()); - } - - // Same rule as the first-attempt read: the recovery read can - // also be the call that discovers the source is dead. Skipping - // would zero-fill the rest of the title as fabricated content. - if let Some(e) = rec.as_ref().err() - && e.is_source_terminated() - { - return Err(crate::error::Error::SourceTerminated.into()); - } - - // Recovery read also failed. Skip the WHOLE failed unit, not one - // sector: advancing by the full unit keeps current_offset - // AACS-aligned (a partial skip here would desync the rest). - if self.skip_errors { - let zb = sectors as usize * 2048; - self.read_buf.resize(zb, 0); - self.read_buf[..zb].fill(0); - self.buf_valid = zb; - self.errors += 1; - // `errors` counts skip events, `lost_bytes` the bytes - // zero-filled. One AACS event can skip a whole 6144-byte - // unit, so loss estimates must use lost_bytes, not errors*2048. - self.lost_bytes = self.lost_bytes.saturating_add(zb as u64); - self.emit(EventKind::SectorSkipped { sector: lba as u64 }); - self.current_offset += sectors as u32; - break; - } else { - // Build the error from the recovery failure we now hold - // (falling back to the original 10s-read failure). - let err = rec.err().or(res.err()); - let (status, sense) = - err.as_ref().map(extract_scsi_context).unwrap_or((0, None)); - return Err(crate::error::Error::DiscRead { - sector: lba as u64, - status: Some(status), - sense, - } - .into()); - } - } - - // Shrink and retry at the same LBA with a smaller batch. - if let Some(ev) = self.adaptive.on_failure() { - self.emit(ev); - } - } - - if self.current_offset >= ext_sectors { - self.current_extent += 1; - self.current_offset = 0; - } - Ok(true) - } -} - -// The key set's on-arrival loud stop: E7022 (a title) or E7032 (an image or folder). -fn is_key_stop(e: &crate::error::Error) -> bool { - matches!( - e, - crate::error::Error::NoDiscKey { .. } | crate::error::Error::WholeDiscKeyMissing - ) -} - -// Per-stage profiling state, populated only when FREEMKV_PROFILE is set. -// Logs a percentage breakdown via tracing (target "mux") every -// PROFILE_INTERVAL. Zero overhead otherwise. -struct StageProf { - started: std::time::Instant, - last_dump: std::time::Instant, - fill_ns: u128, - feed_ns: u128, - consume_ns: u128, - bytes_in: u64, -} - -const PROFILE_INTERVAL: std::time::Duration = std::time::Duration::from_secs(5); - -thread_local! { - static STAGE_PROF: std::cell::RefCell> = const { std::cell::RefCell::new(None) }; -} - -fn prof_active() -> bool { - std::env::var_os("FREEMKV_PROFILE").is_some() -} - -fn prof_tick(stage: &str, ns: u128, bytes: u64) { - STAGE_PROF.with(|cell| { - let mut slot = cell.borrow_mut(); - if slot.is_none() { - if !prof_active() { - return; - } - let now = std::time::Instant::now(); - *slot = Some(StageProf { - started: now, - last_dump: now, - fill_ns: 0, - feed_ns: 0, - consume_ns: 0, - bytes_in: 0, - }); - } - let p = slot.as_mut().unwrap(); - match stage { - "fill" => p.fill_ns += ns, - "feed" => p.feed_ns += ns, - "consume" => p.consume_ns += ns, - _ => {} - } - p.bytes_in += bytes; - let now = std::time::Instant::now(); - if now.duration_since(p.last_dump) < PROFILE_INTERVAL { - return; - } - let elapsed_ms = now.duration_since(p.started).as_millis().max(1); - let fill_pct = p.fill_ns / 10_000 / elapsed_ms; - let feed_pct = p.feed_ns / 10_000 / elapsed_ms; - let consume_pct = p.consume_ns / 10_000 / elapsed_ms; - let mbps = p.bytes_in as u128 * 1000 / 1_000_000 / elapsed_ms; - tracing::debug!( - target: "mux", - "[profile] elapsed={}ms in={}MB/s fill={}% feed={}% consume={}%", - elapsed_ms, mbps, fill_pct, feed_pct, consume_pct, - ); - p.last_dump = now; - }); -} - -// B1 gate for one parsed frame: `true` to emit, `false` to drop. Streams without a gate admit. -fn admit( - resync: &mut [super::resync::ResyncGate], - is_video: &[bool], - track: usize, - frame: &super::codec::Frame, -) -> bool { - match resync.get_mut(track) { - Some(gate) => gate.admit( - is_video.get(track).copied().unwrap_or(false), - frame.discontinuity, - frame.keyframe, - ), - None => true, - } -} - -impl DiscStream { - // Route one PS packet by its real DVD PID through the AU assembler to the codec parser, - // the same path as `PipelinedPesStream::consume_ps`. Takes the packet by value. - fn route_ps_packet(&mut self, ps: super::ps::PsPacket) { - let Some(pid) = ps.dvd_pid() else { - if ps.is_nav() { - self.dropped_nav_packets += 1; - } else { - self.dropped_ps.drop_packet(&ps, None); - } - return; - }; - let Some((_, track)) = self.pid_to_track.iter().find(|(p, _)| *p == pid).copied() else { - if let Some(base) = super::ps::dvd_mpeg_audio_extension_base(pid) { - // Counted and reported once at EOF: no extension track was declared. - self.mpeg_extension_packets[(base & 0x07) as usize] += 1; - return; - } - self.dropped_ps.drop_packet(&ps, Some(pid)); - return; - }; - let (pts, dts, src) = ( - ps.pts.map(|p| p as i64), - ps.dts.map(|d| d as i64), - ps.source, - ); - // PS path: no AACS conceal, no gap flag. - let pkts: Vec = match self.au_asm.get_mut(track) { - Some(asm) => asm - .push_owned(ps.data, pts, dts, src, false) - .into_iter() - .map(|au| PesPacket { - source: au.source, - pid, - pts: au.pts, - dts: au.dts, - data: au.data, - discontinuity: au.discontinuity, - }) - .collect(), - None => vec![PesPacket { - source: src, - pid, - pts, - dts, - data: ps.data, - discontinuity: false, - }], - }; - let pkts = match self.field_merge.get_mut(track).and_then(Option::as_mut) { - Some(m) => pkts.into_iter().flat_map(|p| m.push(p)).collect(), - None => pkts, - }; - if let Some((_, parser)) = self.parsers.iter_mut().find(|(p, _)| *p == pid) { - for pes in &pkts { - for frame in parser.parse(pes) { - self.pending_frames - .push_back(crate::pes::PesFrame::from_codec_frame(track, frame)); - } - } - } - } - - fn read_frame(&mut self) -> io::Result> { - if let Some(frame) = self.pending_frames.pop_front() { - return Ok(Some(frame)); - } - - if self.eof { - return Ok(None); - } - - loop { - // Profiling timestamps only when FREEMKV_PROFILE is set; otherwise - // these stay None and no Instant::now() is taken in the hot loop. - let t0 = self.profiling.then(std::time::Instant::now); - if !self.fill_extents()? { - self.eof = true; - if self.dropped_nav_packets > 0 { - tracing::debug!( - target: "mux", - "dropped {} DVD navigation packets (private_stream_2/0xBF) — expected, carry no elementary stream", - self.dropped_nav_packets - ); - } - // Flush demuxer — last PES packet may still be in the assembler - if let Some(ref mut demuxer) = self.ts_demuxer { - for pes in &demuxer.flush() { - if let Some((_, track)) = - self.pid_to_track.iter().find(|(pid, _)| *pid == pes.pid) - && let Some((_, parser)) = - self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid) - { - let resync = &mut self.resync; - let is_video = &self.is_video; - let pending = &mut self.pending_frames; - for frame in parser.parse(pes) { - // Same B1 gate — a concealed gap can leave a - // post-gap frame in the demuxer's final flush. - let emit = admit(resync, is_video, *track, &frame); - if emit { - pending.push_back(crate::pes::PesFrame::from_codec_frame( - *track, frame, - )); - } - } - } - } - } - // PS demuxer flush (DVD) - let flushed = self - .ps_demuxer - .as_mut() - .map(|d| d.flush()) - .unwrap_or_default(); - for ps in flushed { - self.route_ps_packet(ps); - } - super::ps::warn_undeclared_extensions(&self.mpeg_extension_packets); - self.dropped_ps.report(); - // Drain any access unit a codec parser buffered past the last - // PES (DTS-HD's final core+extension unit, assembled across - // PES boundaries). - let pid_to_track = &self.pid_to_track; - let pending = &mut self.pending_frames; - let resync = &mut self.resync; - let is_video = &self.is_video; - let au_asm = &mut self.au_asm; - let field_merge = &mut self.field_merge; - for (pid, parser) in self.parsers.iter_mut() { - let Some(&(_, track)) = pid_to_track.iter().find(|(p, _)| p == pid) else { - continue; - }; - // Flush frames go through the SAME B1 gate as the in-stream path. The - // PS assembler's held AU(s) come first, then the parser's own tail. - let mut tail: Vec = au_asm - .get_mut(track) - .map(|a| a.flush()) - .unwrap_or_default() - .into_iter() - .map(|au| PesPacket { - source: au.source, - pid: *pid, - pts: au.pts, - dts: au.dts, - data: au.data, - discontinuity: au.discontinuity, - }) - .collect(); - if let Some(m) = field_merge.get_mut(track).and_then(Option::as_mut) { - tail = tail.into_iter().flat_map(|p| m.push(p)).collect(); - tail.extend(m.flush()); - } - let mut frames = Vec::new(); - for pes in &tail { - frames.extend(parser.parse(pes)); - } - frames.extend(parser.flush()); - for frame in frames { - let emit = admit(resync, is_video, track, &frame); - if emit { - pending.push_back(crate::pes::PesFrame::from_codec_frame(track, frame)); - } - } - } - // A gate still armed at EOF dropped post-gap frames that never - // reached a keyframe (a concealed gap in the final GOP). Surface - // it once so the loss is visible, not silent. - for (track, gate) in self.resync.iter().enumerate() { - if gate.is_armed() { - tracing::warn!( - target: "mux", - track, - dropped = gate.dropped_in_run(), - "B1: stream ended while dropping to a keyframe after a concealed gap (no trailing keyframe)" - ); - } - } - return Ok(self.pending_frames.pop_front()); - } - - let bytes = self.buf_valid; - let t1 = self.profiling.then(std::time::Instant::now); - if let (Some(t0), Some(t1)) = (t0, t1) { - prof_tick("fill", t1.duration_since(t0).as_nanos(), bytes as u64); - } - // Plaintext already: DecryptingSectorSource applied AACS/CSS - // in-place during fill_extents' read_sectors call (the pre-0.18 - // inline decrypt step used to live here). - - let buf_base = self.fed_bytes; - self.fed_bytes = self.fed_bytes.saturating_add(bytes as u64); - - if let Some(ref mut demuxer) = self.ts_demuxer { - let packets = demuxer.feed_at(buf_base, &self.read_buf[..bytes]); - let t2 = self.profiling.then(std::time::Instant::now); - if let (Some(t1), Some(t2)) = (t1, t2) { - prof_tick("feed", t2.duration_since(t1).as_nanos(), 0); - } - let skip_parse = self.skip_parse; - for pes in packets { - if let Some((_, track)) = self - .pid_to_track - .iter() - .find(|(pid, _)| *pid == pes.pid) - .copied() - { - if skip_parse { - // Profiling escape hatch — bypass the codec parser, - // pass raw PES bytes through as one PesFrame, so we - // can attribute time to demux+framing vs codec parse. - self.pending_frames.push_back(crate::pes::PesFrame { - discard_padding_ns: 0, - coding: None, - source: None, - track, - pts: pes.pts.map(super::codec::pts_to_ns).unwrap_or(0), - keyframe: false, - data: pes.data, - duration_ns: None, - }); - } else if let Some((_, parser)) = - self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid) - { - // Disjoint field borrows: the resync gate + is_video - // reads coexist with the mutable `self.parsers` drain. - let resync = &mut self.resync; - let is_video = &self.is_video; - let pending = &mut self.pending_frames; - for frame in parser.parse(&pes) { - // B1: after a concealed/lost gap, drop forward to - // the next keyframe on a video track so no frame - // with a dangling reference is emitted. - let emit = admit(resync, is_video, track, &frame); - if emit { - pending.push_back(crate::pes::PesFrame::from_codec_frame( - track, frame, - )); - } - } - } - } - } - let t3 = self.profiling.then(std::time::Instant::now); - if let (Some(t2), Some(t3)) = (t2, t3) { - prof_tick("consume", t3.duration_since(t2).as_nanos(), 0); - } - } else if let Some(ref mut demuxer) = self.ps_demuxer { - let packets = demuxer.feed_at(buf_base, &self.read_buf[..bytes]); - for ps in packets { - self.route_ps_packet(ps); - } - } - - self.buf_valid = 0; - - if let Some(frame) = self.pending_frames.pop_front() { - return Ok(Some(frame)); - } - } - } -} - -impl crate::pes::Stream for DiscStream { - fn read(&mut self) -> io::Result> { - let read = self.read_frame(); - match &read { - Ok(Some(frame)) => self.header_gate.observe(frame), - Ok(None) => self.header_gate.expire(), - Err(_) => {} - } - read - } - - fn write(&mut self, _frame: &crate::pes::PesFrame) -> io::Result<()> { - Err(crate::error::Error::StreamReadOnly.into()) - } - - fn finish(&mut self) -> io::Result<()> { - Ok(()) - } - - fn info(&self) -> &DiscTitle { - &self.title - } - - fn codec_private(&self, track: usize) -> Option> { - let pid = self - .pid_to_track - .iter() - .find(|(_, idx)| *idx == track) - .map(|(pid, _)| *pid)?; - self.parsers - .iter() - .find(|(p, _)| *p == pid) - .and_then(|(_, parser)| parser.codec_private()) - } - - fn config_changes(&self) -> Vec<(usize, u64)> { - self.pid_to_track - .iter() - .filter_map(|&(pid, track)| { - let (_, parser) = self.parsers.iter().find(|(p, _)| *p == pid)?; - let n = parser.config_changes(); - (n > 0).then_some((track, n)) - }) - .collect() - } - - fn headers_ready(&self) -> bool { - // FREEMKV_SKIP_PARSE bypasses codec parsers entirely, so - // codec_private is never populated — pretend headers are ready - // immediately in that mode so the CLI loop doesn't hang. - if self.skip_parse { - return true; - } - self.header_gate - .ready(&self.title, |idx| self.codec_private(idx)) - } - - fn errors(&self) -> u64 { - // Read errors PLUS every frame the B1 resync gates discarded. Summed - // here rather than at each of three `admit` call sites (one place can't - // drift); `dropped_total` catches gaps that RESOLVE, which most do. - self.errors - + self.reader.blanked_units() - + self - .resync - .iter() - .map(super::resync::ResyncGate::dropped_total) - .sum::() - } - - fn lost_bytes(&self) -> u64 { - // Read-error zero-fill plus the damaged AACS units the reader blanked (one whole - // aligned unit each): both are real missing content. - let unit = crate::aacs::content::ALIGNED_UNIT_LEN as u64; - self.lost_bytes + self.reader.blanked_units() * unit - } -} - -#[cfg(test)] -mod tests { - //! `DiscStream` is the only read-only `Stream` impl in tree (every - //! other concrete impl in `mux/*` is bidirectional or write-only). - //! These tests lock down a static `Send` assertion plus a - //! `Box` round trip exercising every method through the - //! trait object, so future Send-breaking edits to `DiscStream`'s - //! interior types fail at compile time. - use super::*; - use crate::disc::{ContentFormat, DiscTitle}; - use crate::pes::Stream; - - // Static-assert DiscStream: Send (Stream has Send as a supertrait) — a - // future non-Send field fails this at compile time, before the runtime - // trait-object test below. - fn _assert_disc_stream_is_send() { - fn requires_send() {} - requires_send::(); - } - - // Trivial SectorSource yielding zeroed sectors. Empty title -> no PES - // frames -> read() walks to EOF -> Ok(None); enough to exercise the - // trait-object dispatch (the goal is the bridge, not the demuxer). - struct ZeroReader { - capacity: u32, - } - - impl crate::sector::SectorSource for ZeroReader { - fn read_sectors( - &mut self, - _lba: u32, - count: u16, - buf: &mut [u8], - _recovery: bool, - ) -> crate::error::Result { - let bytes = count as usize * 2048; - buf[..bytes].fill(0); - Ok(bytes) - } - - fn capacity_sectors(&self) -> u32 { - self.capacity - } - } - - // A SectorSource that UNDER-reports: writes the whole requested span but - // reports only the first sector valid, modeling a non-full-or-error - // source — the case fill_extents handled inconsistently pre-fix. - struct ShortReader { - capacity: u32, - } - - impl crate::sector::SectorSource for ShortReader { - fn read_sectors( - &mut self, - _lba: u32, - count: u16, - buf: &mut [u8], - _recovery: bool, - ) -> crate::error::Result { - let bytes = count as usize * 2048; - // Stale marker across the whole span; only the first sector is - // reported as actually delivered. - buf[..bytes].fill(0xAB); - Ok(2048usize.min(bytes)) - } - - fn capacity_sectors(&self) -> u32 { - self.capacity - } - } - - fn short_read_stream(skip_errors: bool) -> DiscStream { - let mut s = DiscStream::new( - Box::new(ShortReader { capacity: 64 }), - synthetic_title(64), - crate::decrypt::DecryptKeys::None, - 8, // request 8 sectors (16384 B); the source delivers 1 (2048 B) - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - s.skip_errors = skip_errors; - s - } - - // A short read must never become a silent gap: advancing current_offset - // by the full requested count would drop the undelivered tail with no - // error. Without skip_errors it's a hard read error: E6000. - #[test] - fn short_read_without_skip_errors_is_reported_not_silently_skipped() { - let mut s = short_read_stream(false); - let err = s - .fill_extents() - .expect_err("a short read must not be reported as a clean fill"); - assert_eq!(err.kind(), std::io::ErrorKind::InvalidData); - assert!( - err.to_string().contains("E6000"), - "expected the disc-read code E6000, got {err}" - ); - } - - // With skip_errors the undelivered tail is zero-filled and accounted, not - // left stale or dropped silently. current_offset still advances by the - // full request to stay on an AACS unit boundary. - #[test] - fn short_read_with_skip_errors_is_zero_filled_and_accounted() { - let mut s = short_read_stream(true); - assert!( - s.fill_extents() - .expect("skip_errors absorbs the short read") - ); - - // 8 sectors requested at 2048 B/sector (the ECMA-167 / UDF logical - // sector size) = 16384 B of buffer; the source delivered 2048 B. - assert_eq!(s.buf_valid, 16_384, "the whole requested span stays valid"); - assert_eq!(s.current_offset, 8, "the cursor advances a whole unit span"); - assert_eq!( - s.lost_bytes, 14_336, - "16384 requested - 2048 delivered = 14336 lost bytes must be counted" - ); - assert_eq!(s.errors, 1, "the gap is counted as one skip event"); - // The delivered sector survives; the undelivered tail is zeroed, not - // the stale 0xAB the source left behind. - assert!(s.read_buf[..2048].iter().all(|&b| b == 0xAB)); - assert!( - s.read_buf[2048..16_384].iter().all(|&b| b == 0), - "the undelivered tail must be zero-filled, not stale bytes" - ); - } - - // A malformed disc can declare thousands of zero-sector extents; the old self-recursive - // skip overflowed the stack (fatal, unlike iterative EOF). - #[test] - fn a_long_run_of_empty_extents_does_not_recurse_per_extent() { - let title = DiscTitle { - extents: (0..50_000) - .map(|_| crate::disc::Extent { - start_lba: 0, - sector_count: 0, - }) - .collect(), - ..DiscTitle::empty() - }; - let handle = std::thread::Builder::new() - .stack_size(256 * 1024) - .spawn(move || { - let mut s = DiscStream::new( - Box::new(ZeroReader { capacity: 0 }), - title, - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - s.fill_extents() - }) - .unwrap(); - assert!( - !handle.join().unwrap().unwrap(), - "an all-empty extent list is EOF, not an error" - ); - } - - fn synthetic_title(sector_count: u32) -> DiscTitle { - DiscTitle { - extents: vec![crate::disc::Extent { - start_lba: 0, - sector_count, - }], - ..DiscTitle::empty() - } - } - - // MuxOptions::default() carries batch_sectors 0: a zero batch reads 0 sectors - // and never advances, so it must be refused up front, not spin until Stop. - #[test] - fn zero_batch_is_rejected_not_an_endless_zero_read() { - let dir = tempfile::tempdir().unwrap(); - let iso = dir.path().join("t.iso"); - std::fs::write(&iso, vec![0u8; 4 * 2048]).unwrap(); - let src = crate::io::file_sector_source::FileSectorSource::open(&iso).unwrap(); - let res = DiscStream::new( - Box::new(src), - synthetic_title(4), - crate::decrypt::DecryptKeys::None, - 0, - ContentFormat::BdTs, - false, - None, - ); - let err = res.err().expect("batch_sectors 0 must be rejected"); - assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput); - assert_eq!(crate::error::error_code(&err), Some(9085)); - } - - // A truncated ISO is not a bad sector: even with skip_errors the missing tail - // must abort with the image error, not be zero-filled and reported as success. - #[test] - fn truncated_image_aborts_even_with_skip_errors() { - let dir = tempfile::tempdir().unwrap(); - let iso = dir.path().join("short.iso"); - std::fs::write(&iso, vec![0u8; 4 * 2048]).unwrap(); - let src = crate::io::file_sector_source::FileSectorSource::open(&iso).unwrap(); - let mut stream = DiscStream::new( - Box::new(src), - synthetic_title(16), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - stream.skip_errors = true; - let mut outcome = Ok(true); - for _ in 0..64 { - outcome = stream.fill_extents(); - if !matches!(outcome, Ok(true)) { - break; - } - } - let err = outcome.expect_err("a truncated image must abort the read"); - assert!( - err.to_string() - .starts_with(&format!("E{}", crate::error::E_IMAGE_ENDS_BEFORE_READ)), - "got {err}" - ); - assert_eq!(stream.errors, 0, "must not count as a skipped sector"); - assert_eq!(stream.lost_bytes, 0, "must not zero-fill the missing tail"); - } - - /// Smallest credible witness that `DiscStream` flows through `dyn Stream`: - /// build a `Box`, drive `read()` to EOF, exercise `info()` / - /// `headers_ready()` / `codec_private()` through the trait object. - #[test] - fn stream_via_dyn_object() { - let reader = ZeroReader { capacity: 8 }; - let title = synthetic_title(8); - let stream = DiscStream::new( - Box::new(reader), - title, - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - - let mut src: Box = Box::new(stream); - - // Empty-title fixture has no streams configured, so headers are - // trivially ready and codec_private() yields nothing on track 0. - assert!(src.headers_ready()); - assert!(src.codec_private(0).is_none()); - let _ = src.info(); - - // Drive read() to EOF through the trait object — empty-title fixture - // produces no frames, but the call still routes through the blanket - // dispatch into Stream::read. - let mut frames = 0usize; - while src.read().expect("read").is_some() { - frames += 1; - if frames > 1024 { - panic!("unexpected unbounded frame stream from empty title"); - } - } - assert_eq!(frames, 0); - } - - /// A Blu-ray 3D title: an AVC base view plus the MVC dependent (right-eye) - /// view, marked by `MVC_DEPENDENT_LABEL`. - fn mvc_title() -> DiscTitle { - use crate::disc::{Codec, ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream}; - let view = |pid: u16, label: &str| { - crate::disc::Stream::Video(VideoStream { - pid, - codec: Codec::H264, - resolution: Resolution::R1080p, - frame_rate: FrameRate::F23_976, - hdr: HdrFormat::Sdr, - color_space: ColorSpace::Bt709, - display_aspect: None, - secondary: false, - label: label.to_string(), - measured_cicp: None, - }) - }; - let mut t = synthetic_title(8); - t.streams = vec![ - view(0x1011, ""), // base view - view(0x1012, crate::disc::MVC_DEPENDENT_LABEL), // dependent view - ]; - t.content_format = ContentFormat::BdTs; - t - } - - // A dependent-view access unit: PPS (NAL 8) + coded-slice-extension (NAL - // 20), no IDR. MVC passthrough keeps every parameter set in-band (ISO/IEC - // 14496-10 Annex H); the base-view parser strips PPS from non-keyframes. - fn mvc_dependent_au_pes(pid: u16) -> crate::mux::ts::PesPacket { - let nal = |t: u8, body: &[u8]| { - let mut v = vec![0x00, 0x00, 0x01, t]; - v.extend_from_slice(body); - v - }; - let mut data = Vec::new(); - data.extend_from_slice(&nal(0x68, &[0xCE, 0x01])); // PPS (8) - data.extend_from_slice(&nal(0x74, &[0x11, 0x22])); // slice extension (20) - crate::mux::ts::PesPacket { - source: None, - pid, - pts: Some(90_000), - dts: None, - data, - discontinuity: false, - } - } - - /// Whether a parser's output for a dependent-view AU still carries the PPS — - /// the observable signature of the MVC param-set-passthrough parser. - fn keeps_pps_inband(parser: &mut dyn super::super::codec::CodecParser, pid: u16) -> bool { - let frames = parser.parse(&mvc_dependent_au_pes(pid)); - assert_eq!(frames.len(), 1, "one access unit in, one frame out"); - // Frame payload is length-prefixed NALs (4-byte BE length + NAL). - let d = &frames[0].data; - let mut i = 0; - while i + 4 <= d.len() { - let len = u32::from_be_bytes([d[i], d[i + 1], d[i + 2], d[i + 3]]) as usize; - i += 4; - if i + len > d.len() { - break; - } - if len > 0 && d[i] & 0x1F == 8 { - return true; - } - i += len; - } - false - } - - // The live disc:// path must dispatch the Blu-ray 3D MVC dependent view - // to the passthrough parser, same as the ISO path (build_demux_state) — - // before sharing this, the same 3D disc muxed wrong from a drive. - #[test] - fn live_path_dispatches_mvc_dependent_view_to_passthrough_parser() { - let title = mvc_title(); - let mut stream = DiscStream::new( - Box::new(ZeroReader { capacity: 8 }), - title.clone(), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - - let idx_of = |parsers: &Vec<(u16, Box)>, pid: u16| { - parsers.iter().position(|(p, _)| *p == pid).unwrap() - }; - - // Live path: dependent view keeps its PPS in-band, base view does not. - let dep = idx_of(&stream.parsers, 0x1012); - assert!( - keeps_pps_inband(stream.parsers[dep].1.as_mut(), 0x1012), - "live disc:// path must give the MVC dependent view the passthrough parser" - ); - let base = idx_of(&stream.parsers, 0x1011); - assert!( - !keeps_pps_inband(stream.parsers[base].1.as_mut(), 0x1011), - "base view keeps the ordinary parser (discriminator is real, not vacuous)" - ); - - // And it must agree with the ISO path, which is the canonical builder. - let (mut iso_parsers, _, _, _) = - crate::mux::resolve::build_demux_state(&title, ContentFormat::BdTs); - let iso_dep = idx_of(&iso_parsers, 0x1012); - assert!( - keeps_pps_inband(iso_parsers[iso_dep].1.as_mut(), 0x1012), - "ISO path reference behaviour" - ); - } - - /// `is_halted()` must observe a cancellation signal installed via - /// `with_halt(Halt)` — flipping the token must cause the next - /// `fill_extents` retry boundary to bail. - #[test] - fn halt_via_with_halt_observed_by_is_halted() { - let halt = Halt::new(); - let stream = DiscStream::new( - Box::new(ZeroReader { capacity: 8 }), - synthetic_title(8), - crate::decrypt::DecryptKeys::None, - 8, - crate::disc::ContentFormat::BdTs, - false, - None, - ) - .unwrap() - .with_halt(halt.clone()); - assert!(!stream.is_halted()); - halt.cancel(); - assert!( - stream.is_halted(), - "with_halt token cancellation must be observed by is_halted()" - ); - } - - // Every read fails; the `stop_at`-th read cancels `halt` (Stop pressed inside a - // bad zone) and fails with `Halted` (a cancelled drive) or a media error. - struct StopInBadZoneReader { - halt: Halt, - stop_at: usize, - reader_halted: bool, - log: std::sync::Arc>>, - } - - impl crate::sector::SectorSource for StopInBadZoneReader { - fn read_sectors( - &mut self, - lba: u32, - _count: u16, - _buf: &mut [u8], - recovery: bool, - ) -> crate::error::Result { - let mut log = self.log.lock().unwrap(); - log.push(recovery); - if log.len() >= self.stop_at { - self.halt.cancel(); - if self.reader_halted { - return Err(crate::error::Error::Halted); - } - } - Err(crate::error::Error::DiscRead { - sector: lba as u64, - status: Some(0x02), - sense: None, - }) - } - - fn capacity_sectors(&self) -> u32 { - 64 - } - } - - fn stop_in_bad_zone( - batch: u16, - stop_at: usize, - reader_halted: bool, - share_halt: bool, - skip_errors: bool, - ) -> (io::Result, DiscStream, Vec) { - let halt = Halt::new(); - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = StopInBadZoneReader { - halt: halt.clone(), - stop_at, - reader_halted, - log: log.clone(), - }; - let mut s = DiscStream::new( - Box::new(reader), - synthetic_title(64), - crate::decrypt::DecryptKeys::None, - batch, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - if share_halt { - s = s.with_halt(halt.clone()); - } - s.skip_errors = skip_errors; - if stop_at == 0 { - halt.cancel(); - } - let res = s.fill_extents(); - let log = log.lock().unwrap().clone(); - (res, s, log) - } - - // A drive cancelled by Stop fails its read with Halted: that is a stop, not a - // bad sector — no 60s recovery read, no skip accounting, no DiscRead. - #[test] - fn a_halted_read_at_the_bottomed_out_unit_is_a_stop_not_a_bad_sector() { - for share_halt in [true, false] { - for skip_errors in [true, false] { - let (res, s, log) = stop_in_bad_zone(1, 1, true, share_halt, skip_errors); - let err = res.expect_err("a halted read must not fill"); - assert!( - crate::error::is_halt(&err), - "share={share_halt} skip={skip_errors}: got {err}" - ); - assert_eq!(log, vec![false], "no recovery read after a halted read"); - assert_eq!((s.errors, s.lost_bytes), (0, 0), "no bogus skip"); - } - } - } - - // Stop pressed while the read stalls in a failing region returns Halted at once, - // never walking the shrink/recovery ladder first. - #[test] - fn stop_in_a_failing_region_returns_halted_without_more_reads() { - let (res, s, log) = stop_in_bad_zone(32, 0, false, true, true); - assert!(crate::error::is_halt(&res.expect_err("halted before read"))); - assert!(log.is_empty(), "no read once already halted: {log:?}"); - assert_eq!(s.errors, 0); - - let (res, s, log) = stop_in_bad_zone(32, 2, false, true, true); - let err = res.expect_err("a stop mid bad zone must not fill"); - assert!(crate::error::is_halt(&err), "got {err}"); - assert_eq!(log, vec![false, false], "no read after the stop"); - assert_eq!((s.errors, s.lost_bytes), (0, 0), "no bogus skip"); - } - - /// A key map on the inline live-drive path applies the same FMTS read plan - /// the file-backed highway uses: within a forensic segment only our-phase units - /// survive the extent walk, so the alternate device-group units are never read. - #[test] - fn a_key_map_reads_only_our_phase_units() { - use crate::decrypt::{DecryptKeys, Phase}; - // AACS keys → unit_align = 3, so a unit is 3 sectors and the phase filter - // engages. Key contents are irrelevant to the read plan. - let aacs = DecryptKeys::Aacs { - unit_keys: vec![(0, [0u8; 16]), (1, [1u8; 16])], - format: ContentFormat::BdTs, - }; - // 100 units (300 sectors). A 10-unit Even forensic segment at LBA [30,60): - // even units (30,36,42,48,54) are ours; odd (33,39,45,51,57) are dropped. - let map = crate::keys::test_key_map(vec![(30, 60, 1, Phase::Even)]); - let stream = DiscStream::new( - Box::new(ZeroReader { capacity: 300 }), - synthetic_title(300), - aacs, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - let stream = crate::keys::install_key_map(stream, map); - let total: u32 = stream.extents.iter().map(|e| e.sector_count).sum(); - assert_eq!( - total, - 300 - 5 * 3, - "exactly the 5 alternate-phase units (15 sectors) are dropped from the read walk" - ); - assert!( - stream.extents.len() > 1, - "the forensic segment split the single extent into our-phase-only runs" - ); - // The progress denominator tracks the reduced read set. - assert_eq!(stream.bytes_total_extents, total as u64 * 2048); - } - - // Recording SectorSource: logs every (lba, count) request, errors when the - // range covers bad_sector. Successful reads return zeroed sectors (not - // flagged scrambled, so DecryptingSectorSource passes them through as-is). - struct RecordingReader { - capacity: u32, - bad_sector: u32, - log: std::sync::Arc>>, - } - - impl crate::sector::SectorSource for RecordingReader { - fn read_sectors( - &mut self, - lba: u32, - count: u16, - buf: &mut [u8], - _recovery: bool, - ) -> crate::error::Result { - self.log.lock().unwrap().push((lba, count)); - let end = lba + count as u32; - if self.bad_sector >= lba && self.bad_sector < end { - return Err(crate::error::Error::DiscRead { - sector: self.bad_sector as u64, - status: Some(0x02), - sense: None, - }); - } - let bytes = count as usize * 2048; - buf[..bytes].fill(0); - Ok(bytes) - } - - fn capacity_sectors(&self) -> u32 { - self.capacity - } - } - - // SectorSource that fails every read covering bad_sector with a SCSI - // transport failure (status=0xFF, the USB-bridge-crash sentinel Drive::read - // surfaces). Logs each (lba, count) so a test can prove it wasn't retried. - struct TransportFailReader { - capacity: u32, - bad_sector: u32, - log: std::sync::Arc>>, - } - - impl crate::sector::SectorSource for TransportFailReader { - fn read_sectors( - &mut self, - lba: u32, - count: u16, - buf: &mut [u8], - _recovery: bool, - ) -> crate::error::Result { - self.log.lock().unwrap().push((lba, count)); - let end = lba + count as u32; - if self.bad_sector >= lba && self.bad_sector < end { - return Err(crate::error::Error::DiscRead { - sector: self.bad_sector as u64, - status: Some(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE), - sense: None, - }); - } - let bytes = count as usize * 2048; - buf[..bytes].fill(0); - Ok(bytes) - } - - fn capacity_sectors(&self) -> u32 { - self.capacity - } - } - - // SectorSource mirroring a marginal sector recoverable only with the - // drive's full ECC budget: fails while recovery=false, succeeds once - // recovery=true. Exercises fill_extents' last-chance recovery-read branch. - struct RecoverableReader { - capacity: u32, - bad_sector: u32, - /// `(lba, count, recovery)` for every issued read. - log: std::sync::Arc>>, - } - - impl crate::sector::SectorSource for RecoverableReader { - fn read_sectors( - &mut self, - lba: u32, - count: u16, - buf: &mut [u8], - recovery: bool, - ) -> crate::error::Result { - self.log.lock().unwrap().push((lba, count, recovery)); - let end = lba + count as u32; - let covers_bad = self.bad_sector >= lba && self.bad_sector < end; - // Fail on the fast (non-recovery) pass; the 60s ECC recovery read - // succeeds. Distinct non-0x02 sense byte so a transport-failure - // re-check (status 0xFF) is provably NOT triggered here. - if covers_bad && !recovery { - return Err(crate::error::Error::DiscRead { - sector: self.bad_sector as u64, - status: Some(0x02), - sense: None, - }); - } - let bytes = count as usize * 2048; - buf[..bytes].fill(0); - Ok(bytes) - } - - fn capacity_sectors(&self) -> u32 { - self.capacity - } - } - - // Coverage for the recovery-read SUCCESS branch (rc.5.2 audit #3): a - // sector failing the fast read but clean on the 60s ECC pass must mux the - // RECOVERED data — cursor advances, no skip counted. unit_align=1 (None). - #[test] - fn recovery_read_success_muxes_recovered_data_no_skip() { - const COUNT: u32 = 10; - let bad = 4u32; - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = RecoverableReader { - capacity: COUNT, - bad_sector: bad, - log: log.clone(), - }; - let mut stream = DiscStream::new( - Box::new(reader), - synthetic_title(COUNT), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - // skip_errors=false: if the recovery read did NOT succeed, fill_extents - // would return Err — so reaching EOF cleanly proves recovery worked. - stream.skip_errors = false; - - let mut guard = 0; - loop { - match stream.fill_extents() { - Ok(true) => {} - Ok(false) => break, - Err(e) => panic!("recovery read should have succeeded, got: {e}"), - } - guard += 1; - assert!(guard < 1000, "fill_extents did not reach EOF"); - } - - // No skip counted: the recovered unit was muxed, not zero-filled. - assert_eq!( - stream.errors, 0, - "a successful recovery read must not count as a skipped sector" - ); - assert_eq!( - stream.lost_bytes, 0, - "a successful recovery read loses no bytes" - ); - // All COUNT sectors' worth of bytes were read through to the cursor end. - assert_eq!( - stream.bytes_read_total, - COUNT as u64 * 2048, - "every sector (including the recovered one) must be counted as read" - ); - - // The bad sector was retried with recovery=true and that read SUCCEEDED. - let reads = log.lock().unwrap(); - assert!( - reads - .iter() - .any(|&(lba, count, rec)| rec && lba == bad && count == 1), - "expected a recovery=true single-sector read at the bad sector; got {reads:?}" - ); - // And the fast pass at the bad sector did happen with recovery=false. - assert!( - reads.iter().any(|&(lba, _c, rec)| !rec && lba == bad), - "expected a non-recovery read to have first failed at the bad sector" - ); - } - - // SectorSource that fails the fast read at bad_sector with status 0x02, - // then fails the 60s ECC recovery read with transport failure (0xFF) — - // models a bridge wedging during the last-chance recovery read. - struct RecoveryTransportFailReader { - capacity: u32, - bad_sector: u32, - log: std::sync::Arc>>, - } - - impl crate::sector::SectorSource for RecoveryTransportFailReader { - fn read_sectors( - &mut self, - lba: u32, - count: u16, - buf: &mut [u8], - recovery: bool, - ) -> crate::error::Result { - self.log.lock().unwrap().push((lba, count, recovery)); - let end = lba + count as u32; - if self.bad_sector >= lba && self.bad_sector < end { - let status = if recovery { - crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE - } else { - 0x02 - }; - return Err(crate::error::Error::DiscRead { - sector: self.bad_sector as u64, - status: Some(status), - sense: None, - }); - } - let bytes = count as usize * 2048; - buf[..bytes].fill(0); - Ok(bytes) - } - - fn capacity_sectors(&self) -> u32 { - self.capacity - } - } - - // Regression (rc.5.2 audit #2): a transport failure on the 60s ECC - // RECOVERY read (not just the initial read) must ABORT even under - // skip_errors=true, not fall into skip/advance and march forever. - #[test] - fn transport_failure_on_recovery_read_aborts_even_with_skip_errors() { - const COUNT: u32 = 10; - let bad = 4u32; - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = RecoveryTransportFailReader { - capacity: COUNT, - bad_sector: bad, - log: log.clone(), - }; - let mut stream = DiscStream::new( - Box::new(reader), - synthetic_title(COUNT), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - stream.skip_errors = true; - - // Drive fill_extents across batches: the batch covering the bad sector - // shrinks to size 1, fails the fast read (0x02), then the bottom-out - // recovery read returns the transport failure (0xFF), which must abort. - let mut res = Ok(true); - for _ in 0..1000 { - res = stream.fill_extents(); - if !matches!(res, Ok(true)) { - break; - } - } - assert!( - res.is_err(), - "a transport failure on the recovery read must abort fill_extents, got {res:?}" - ); - assert_eq!( - stream.errors, 0, - "a recovery-read transport-failure abort must NOT count as a skip" - ); - assert_eq!( - stream.lost_bytes, 0, - "a transport-failure abort zero-fills nothing" - ); - // Prove the bottom-out recovery read was actually reached and aborted on. - let reads = log.lock().unwrap(); - assert!( - reads - .iter() - .any(|&(lba, count, rec)| rec && lba == bad && count == 1), - "expected a recovery=true read at the bad sector to have been attempted; got {reads:?}" - ); - } - - // Fast read fails at bad_sector (0x02); the recovery read returns `rec_err()`. - struct RecoveryErrReader { - bad_sector: u32, - rec_err: fn() -> crate::error::Error, - } - - impl crate::sector::SectorSource for RecoveryErrReader { - fn read_sectors( - &mut self, - lba: u32, - count: u16, - buf: &mut [u8], - recovery: bool, - ) -> crate::error::Result { - if self.bad_sector >= lba && self.bad_sector < lba + count as u32 { - if recovery { - return Err((self.rec_err)()); - } - return Err(crate::error::Error::DiscRead { - sector: self.bad_sector as u64, - status: Some(0x02), - sense: None, - }); - } - let bytes = count as usize * 2048; - buf[..bytes].fill(0); - Ok(bytes) - } - - fn capacity_sectors(&self) -> u32 { - 10 - } - } - - // A key stop or dead source on the RECOVERY read must abort even under - // skip_errors, never be zero-filled as a skipped sector. - #[test] - fn key_stop_or_dead_source_on_recovery_read_aborts_even_with_skip_errors() { - type MkErr = fn() -> crate::error::Error; - let cases: [(&str, MkErr); 3] = [ - ("NoDiscKey", || crate::error::Error::NoDiscKey { - disc_hash: String::new(), - }), - ("WholeDiscKeyMissing", || { - crate::error::Error::WholeDiscKeyMissing - }), - ("SourceTerminated", || crate::error::Error::SourceTerminated), - ]; - for (name, rec_err) in cases { - let mut stream = DiscStream::new( - Box::new(RecoveryErrReader { - bad_sector: 4, - rec_err, - }), - synthetic_title(10), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - stream.skip_errors = true; - let mut res = Ok(true); - for _ in 0..1000 { - res = stream.fill_extents(); - if !matches!(res, Ok(true)) { - break; - } - } - assert!( - res.is_err(), - "{name} on the recovery read must abort, got {res:?}" - ); - assert_eq!(stream.errors, 0, "{name}: must not count as a skip"); - assert_eq!(stream.lost_bytes, 0, "{name}: must not zero-fill"); - } - } - - // Regression: a USB-bridge transport crash (0xFF) during a single-pass - // disc://->mkv:// rip must ABORT immediately, even under skip_errors=true - // — mirrors the multipass sweep's short-circuit; exactly one read issued. - #[test] - fn transport_failure_aborts_single_pass_even_with_skip_errors() { - const COUNT: u32 = 10; - let bad = 4u32; - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = TransportFailReader { - capacity: COUNT, - bad_sector: bad, - log: log.clone(), - }; - let mut stream = DiscStream::new( - Box::new(reader), - synthetic_title(COUNT), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - stream.skip_errors = true; - - let res = stream.fill_extents(); - assert!( - res.is_err(), - "transport failure must abort fill_extents, not skip past it" - ); - assert_eq!( - stream.errors, 0, - "a transport-failure abort must NOT count as a skipped sector" - ); - let reads = log.lock().unwrap(); - assert_eq!( - reads.len(), - 1, - "transport failure must abort after the first failed read with no \ - shrink/retry/skip-ahead; got reads {reads:?}" - ); - } - - // REGRESSION (round-4 audit): an ordinary MEDIUM ERROR bad sector must keep its identity - // crossing the prefetch channel, not become a fabricated transport failure. - #[test] - fn bad_sector_keeps_its_identity_across_the_prefetch_channel() { - // Spawns the prefetch producer, a Drive holder. - let _serial = crate::sector::prefetched::holder_test_lock(); - const COUNT: u32 = 9; - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = RecordingReader { - capacity: COUNT, - bad_sector: 4, - log: log.clone(), - }; - let mut prefetched = crate::sector::PrefetchedSectorSource::new_with_events( - reader, - vec![crate::disc::Extent { - start_lba: 0, - sector_count: COUNT, - }], - 8, - 1, - None, - None, - ) - .expect("spawn producer"); - - let mut buf = vec![0u8; 8 * 2048]; - let err = crate::sector::SectorSource::read_sectors(&mut prefetched, 0, 8, &mut buf, false) - .expect_err("the batch covering the bad sector must fail"); - assert!( - !err.is_scsi_transport_failure(), - "a MEDIUM ERROR bad sector is not a dead bus; got {err:?}" - ); - assert!( - matches!( - err, - crate::error::Error::DiscRead { - sector: 4, - status: Some(0x02), - .. - } - ), - "the producer's typed error must survive the channel intact; got {err:?}" - ); - } - - // The prefetch producer dies permanently on its first read error; driving fill_extents to - // exhaustion after that must NOT look like a completed pass. - #[test] - fn dead_prefetch_producer_does_not_silently_zero_fill_the_title() { - // Spawns the prefetch producer, a Drive holder. - let _serial = crate::sector::prefetched::holder_test_lock(); - const COUNT: u32 = 30; - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = RecordingReader { - capacity: COUNT, - bad_sector: 4, - log: log.clone(), - }; - let prefetched = crate::sector::PrefetchedSectorSource::new_with_events( - reader, - vec![crate::disc::Extent { - start_lba: 0, - sector_count: COUNT, - }], - 8, - 1, - None, - None, - ) - .expect("spawn producer"); - let mut stream = DiscStream::new( - Box::new(prefetched), - synthetic_title(COUNT), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - stream.skip_errors = true; - - // Drive the whole title. Bounded so a regression cannot hang the suite. - let mut completed_clean = false; - let mut failure = None; - for _ in 0..(COUNT as usize * 4) { - match stream.fill_extents() { - Ok(true) => continue, - Ok(false) => { - completed_clean = true; - break; - } - Err(e) => { - failure = crate::error::error_code(&e); - break; - } - } - } - assert_eq!( - failure, - Some(crate::error::E_SOURCE_TERMINATED), - "a dead producer aborts with SourceTerminated, not another error" - ); - assert!( - !completed_clean, - "the producer died at sector 4, so sectors 4..{COUNT} were never \ - read — reporting the pass as complete zero-fills {} of {} bytes \ - and calls it success", - stream.lost_bytes, - COUNT as u64 * 2048 - ); - } - - // AACS unit-alignment skip: with unit_align=3 and skip_errors=true, a - // single bad mid-extent sector must not desync the title — every read - // starts on a 3-sector unit boundary and skips advance a whole unit. - #[test] - fn aacs_reads_stay_unit_aligned_and_skip_whole_units() { - const COUNT: u32 = 30; - const ALIGN: u32 = 3; - // Bad sector at offset 13 — inside unit 4 (offsets 12,13,14). The - // whole unit must be skipped, keeping the cursor unit-aligned. - let bad = 13u32; - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = RecordingReader { - capacity: COUNT, - bad_sector: bad, - log: log.clone(), - }; - let title = synthetic_title(COUNT); - let keys = crate::decrypt::DecryptKeys::Aacs { - unit_keys: vec![(0, [0u8; 16])], - format: crate::disc::ContentFormat::BdTs, - }; - let stream = DiscStream::new( - Box::new(reader), - title, - keys, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - // AACS decrypts only through a key map; without one every read is a decrypt refusal. - let map = crate::keys::test_key_map(vec![(0, COUNT, 0, crate::decrypt::Phase::All)]); - let mut stream = crate::keys::install_key_map(stream, map); - stream.skip_errors = true; - assert_eq!( - stream.unit_align, ALIGN as u16, - "AACS keys must set unit_align=3" - ); - - // Drive fill_extents to EOF (no PES demux needed — we observe the - // raw read pattern directly). - let ext_start = 0u32; - let mut guard = 0; - loop { - match stream.fill_extents() { - Ok(true) => {} - Ok(false) => break, - Err(e) => panic!("fill_extents errored unexpectedly: {e}"), - } - guard += 1; - assert!(guard < 1000, "fill_extents did not reach EOF"); - } - - let reads = log.lock().unwrap(); - assert!(!reads.is_empty(), "expected at least one read"); - for &(lba, count) in reads.iter() { - assert_eq!( - (lba - ext_start) % ALIGN, - 0, - "read at lba {lba} is not unit-aligned (offset {} % {ALIGN} != 0)", - lba - ext_start - ); - // Non-tail reads must be a whole number of units; the only permitted - // short read is a final partial unit. A mid-stream non-unit-multiple - // read would straddle AACS unit boundaries and decrypt wrong. - assert!( - (count as u32).is_multiple_of(ALIGN) || (count as u32) < ALIGN, - "read count {count} is neither a whole number of units nor a sub-unit tail" - ); - } - - // At least one error was skipped (the bad unit) and a SectorSkipped - // event was emitted; errors counter advanced by exactly the bad units. - assert!(stream.errors >= 1, "expected the bad unit to be skipped"); - - // Regression: `lost_bytes` must account for the WHOLE skipped unit - // (3 sectors = 6144 bytes), not one sector — `errors * 2048` would - // undercount AACS loss ~3x, the single-pass abort-gate bug this guards. - assert_eq!( - stream.lost_bytes, - stream.errors * ALIGN as u64 * 2048, - "AACS skip must record a whole unit (6144 B) per skip event, not 2048" - ); - assert!( - stream.lost_bytes > stream.errors * 2048, - "lost_bytes must exceed the errors*2048 undercount for AACS units" - ); - - // Crucial anti-desync assertion: the read that bottomed out and was - // skipped must have been a single 3-sector unit starting at offset 12 - // (the unit boundary at/below bad sector 13), NOT a 1-sector read at 13. - assert!( - reads - .iter() - .any(|&(lba, count)| lba == 12 && count == ALIGN as u16), - "expected a unit-aligned (lba=12,count=3) read over the bad unit; got {reads:?}" - ); - // And NO single-sector read at the bad sector itself (would be a desync). - assert!( - !reads.iter().any(|&(lba, count)| lba == bad && count == 1), - "a 1-sector read at the bad sector {bad} would desync the AACS unit stream" - ); - } - - /// `unit_align == 1` (DecryptKeys::None) variant: single-sector skips - /// still work (CSS/raw is self-synchronizing, so a 1-sector skip is - /// correct there — contrast with the AACS whole-unit skip above). - #[test] - fn unencrypted_single_sector_skip_works() { - const COUNT: u32 = 10; - let bad = 4u32; - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); - let reader = RecordingReader { - capacity: COUNT, - bad_sector: bad, - log: log.clone(), - }; - let mut stream = DiscStream::new( - Box::new(reader), - synthetic_title(COUNT), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - stream.skip_errors = true; - assert_eq!(stream.unit_align, 1, "None keys must leave unit_align=1"); - - let mut guard = 0; - loop { - match stream.fill_extents() { - Ok(true) => {} - Ok(false) => break, - Err(e) => panic!("fill_extents errored unexpectedly: {e}"), - } - guard += 1; - assert!(guard < 1000, "fill_extents did not reach EOF"); - } - - let reads = log.lock().unwrap(); - // The bad sector must have been retried down to a single sector and - // skipped at count==1 — the self-synchronizing per-sector path. - assert!( - reads.iter().any(|&(lba, count)| lba == bad && count == 1), - "align=1 must bottom out at a 1-sector read over the bad sector; got {reads:?}" - ); - assert!(stream.errors >= 1); - // align=1: a skip event covers exactly one sector, so lost_bytes - // and errors*2048 agree (the AACS undercount does not apply here). - assert_eq!( - stream.lost_bytes, - stream.errors * 2048, - "single-sector (align=1) skip must record exactly 2048 B per event" - ); - } - - #[test] - fn halt_via_with_halt_from_arc_observed_by_is_halted() { - let arc = Arc::new(AtomicBool::new(false)); - let stream = DiscStream::new( - Box::new(ZeroReader { capacity: 8 }), - synthetic_title(8), - crate::decrypt::DecryptKeys::None, - 8, - crate::disc::ContentFormat::BdTs, - false, - None, - ) - .unwrap() - .with_halt(Halt::from_arc(arc.clone())); - assert!(!stream.is_halted()); - arc.store(true, std::sync::atomic::Ordering::Relaxed); - assert!( - stream.is_halted(), - "with_halt(Halt::from_arc) must observe Arc-side flips" - ); - } - - /// Every read fails CSS-locked (`05/6F/03`) — a scrambled DVD whose title key - /// can't be cracked. Drives `resolve_dvd_title_key` to `ScrambledUncracked`. - struct LockedReader; - impl crate::sector::SectorSource for LockedReader { - fn read_sectors( - &mut self, - lba: u32, - _count: u16, - _buf: &mut [u8], - _recovery: bool, - ) -> crate::error::Result { - Err(crate::error::Error::DiscRead { - sector: lba as u64, - status: Some(2), - sense: Some(crate::scsi::ScsiSense { - sense_key: 0x05, - asc: 0x6F, - ascq: 0x03, - }), - }) - } - fn capacity_sectors(&self) -> u32 { - 64 - } - } - - fn mpegps_title(sector_count: u32) -> DiscTitle { - let mut t = synthetic_title(sector_count); - t.content_format = ContentFormat::MpegPs; - t - } - - // PARITY with build_iso_pipeline_dvd_none_keys_scrambled_hard_fails: the - // live-drive constructor must ALSO hard-fail on a None-keyed scrambled - // MPEG-PS DVD, not build a scrambled-passthrough stream. - #[test] - fn disc_stream_new_dvd_none_scrambled_hard_fails() { - let res = DiscStream::new( - Box::new(LockedReader), - mpegps_title(8), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::MpegPs, - false, - None, - ); - assert!( - res.is_err(), - "single-pass DiscStream must hard-fail on a scrambled, keyless CSS DVD" - ); - } - - /// `raw` must bypass the CSS crack at the DiscStream boundary too: the same - /// scrambled-uncrackable input that hard-fails above must CONSTRUCT in raw - /// mode (ciphertext passthrough), never hard-fail. - #[test] - fn disc_stream_new_raw_bypasses_css_crack() { - let res = DiscStream::new( - Box::new(LockedReader), - mpegps_title(8), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::MpegPs, - true, // raw - None, - ); - assert!( - res.is_ok(), - "raw single-pass must construct without cracking, even on scrambled-uncrackable input" - ); - } - - mod stream_surface_tests { - //! The `impl crate::pes::Stream for DiscStream` surface plus the adaptive - //! batch sizer. These are the accessors the CLI's abort gate and the - //! header-wait loop read; a constant in any of them hides read loss or - //! stalls the mux, so each is constrained against a value that is neither - //! the mutation constant nor the initial state. - use super::*; - use crate::pes::Stream as PesStream; - - // ── errors() / lost_bytes(): honest loss reporting ───────────────── - - // The ONLY channel a caller learns bytes went missing; two short - // reads drive both counters to distinct non-0/1 values to prove it. - #[test] - fn errors_and_lost_bytes_report_real_short_read_loss_through_the_trait() { - let mut s = short_read_stream(true); - - // Before any read the stream is clean — establishes the accessors are - // not simply echoing a preloaded value. - assert_eq!(PesStream::errors(&s), 0); - assert_eq!(PesStream::lost_bytes(&s), 0); - - for _ in 0..2 { - assert!( - s.fill_extents() - .expect("skip_errors absorbs the short read") - ); - } - - // Two 8-sector (16384 B) requests, 2048 B delivered each: two skip - // events, 2 * (16384 - 2048) = 28672 bytes lost. - assert_eq!( - PesStream::errors(&s), - 2, - "errors() must report BOTH short reads, not a constant" - ); - assert_eq!( - PesStream::lost_bytes(&s), - 28_672, - "lost_bytes() must report the byte total, not an event count or a constant" - ); - assert_ne!( - PesStream::errors(&s), - PesStream::lost_bytes(&s), - "the two accessors must read different fields" - ); - } - - // Frames the B1 resync gate discards must reach errors(), including after the gap - // resolves (ResyncGate::dropped zeroes on resync). - #[test] - fn errors_reports_frames_the_resync_gate_dropped_after_the_gap_resolves() { - let mut s = short_read_stream(true); - assert_eq!(PesStream::errors(&s), 0, "clean before anything happens"); - - // Drive a gate directly: a discontinuity, two dropped inter-coded - // frames, then a keyframe that resyncs and zeroes the per-run count - // (the short-read fixture's title has no streams, so give it a gate). - s.resync.push(crate::mux::resync::ResyncGate::new()); - let gate = s.resync.last_mut().expect("just pushed"); - assert!(!gate.admit(true, true, false)); - assert!(!gate.admit(true, false, false)); - assert!(gate.admit(true, false, true)); - assert_eq!( - gate.dropped_in_run(), - 0, - "the resync zeroed the per-run counter, which is the whole trap" - ); - - assert_eq!( - PesStream::errors(&s), - 2, - "the two discarded frames must still reach the caller after the \ - gap resolved; reading the per-run counter here would report 0" - ); - } - - // ── write(): DiscStream is read-only ────────────────────────────── - - // Ok(()) would make a caller muxing INTO it believe frames landed; - // it must refuse with the numeric code E_STREAM_READ_ONLY. - #[test] - fn write_refuses_with_the_read_only_code() { - let mut s = short_read_stream(false); - let frame = crate::pes::PesFrame { - discard_padding_ns: 0, - coding: None, - source: None, - track: 0, - pts: 0, - keyframe: true, - data: vec![0u8; 4], - duration_ns: None, - }; - let err = PesStream::write(&mut s, &frame) - .expect_err("a read-only stream must never accept a frame"); - assert_eq!(err.kind(), std::io::ErrorKind::Unsupported); - let code = format!("E{}", crate::error::Error::StreamReadOnly.code()); - assert!( - err.to_string().contains(&code), - "expected the read-only code {code}, got {err}" - ); - } - - // ── codec_private() / headers_ready() ───────────────────────────── - - /// MPEG-2 sequence header (ISO/IEC 13818-2 §6.2.2.1) — start code `0x000001B3` - /// followed by 12-bit horizontal_size, 12-bit vertical_size, then - /// aspect_ratio_information / frame_rate_code and the bit-rate/VBV tail. - fn seq_header(width: u16, height: u16) -> Vec { - let mut h = vec![0x00, 0x00, 0x01, 0xB3]; - h.push((width >> 4) as u8); - h.push((((width & 0x0F) as u8) << 4) | ((height >> 8) & 0x0F) as u8); - h.push((height & 0xFF) as u8); - h.push((3 << 4) | 4); // aspect_ratio_information=3, frame_rate_code=4 - h.extend_from_slice(&[0xFF, 0xFF, 0xFF, 0x00]); - h - } - - /// MPEG-2 picture header (ISO/IEC 13818-2 §6.2.3), coding type in bits 3..5 - /// of the sixth byte. Type 1 = I-picture. - fn picture_header(coding_type: u8) -> Vec { - vec![ - 0x00, - 0x00, - 0x01, - 0x00, - 0x00, - (coding_type & 0x07) << 3, - 0, - 0, - ] - } - - // One video (MPEG-2, track 1) behind one audio (AC-3, track 0). Video - // is deliberately NOT track 0 so a codec_private that mishandled the - // track arg or pid-map direction would answer with the audio parser. - fn audio_then_video_title() -> DiscTitle { - use crate::disc::{ - AudioChannels, AudioStream, Codec, ColorSpace, FrameRate, HdrFormat, LabelPurpose, - Resolution, SampleRate, VideoStream, - }; - let mut t = DiscTitle { - extents: vec![crate::disc::Extent { - start_lba: 0, - sector_count: 8, - }], - ..DiscTitle::empty() - }; - t.content_format = ContentFormat::MpegPs; - t.streams = vec![ - crate::disc::Stream::Audio(AudioStream { - pid: 0x00BD, - codec: Codec::Ac3, - channels: AudioChannels::Stereo, - language: "eng".to_string(), - sample_rate: SampleRate::S48, - secondary: false, - purpose: LabelPurpose::Normal, - label: String::new(), - }), - crate::disc::Stream::Video(VideoStream { - pid: 0x00E0, - codec: Codec::Mpeg2, - resolution: Resolution::R480i, - frame_rate: FrameRate::F29_97, - hdr: HdrFormat::Sdr, - color_space: ColorSpace::Smpte170m, - display_aspect: None, - secondary: false, - label: String::new(), - measured_cicp: None, - }), - ]; - t - } - - fn mixed_stream() -> DiscStream { - DiscStream::new( - Box::new(ZeroReader { capacity: 8 }), - audio_then_video_title(), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::MpegPs, - false, - None, - ) - .unwrap() - } - - // headers_ready() gates the CLI's "wait for codec private" loop: a - // constant true starts the mux with empty CodecPrivate (unplayable - // per RFC 9559 §5.1.4.1.5); constant false hangs forever. Both pinned. - #[test] - fn headers_ready_follows_the_video_codec_private_and_flips_both_ways() { - let mut s = mixed_stream(); - - assert!( - PesStream::codec_private(&s, 1).is_none(), - "no sequence header parsed yet" - ); - assert!( - !PesStream::headers_ready(&s), - "a non-secondary video track without codec private must NOT be reported ready" - ); - - // Feed the video parser a complete access unit: sequence header + - // I-picture, closed by a following picture so the AU boundary is hit. - let vpid = 0x00E0u16; - let (_, parser) = s - .parsers - .iter_mut() - .find(|(p, _)| *p == vpid) - .expect("video parser present"); - let mut au = seq_header(720, 480); - au.extend_from_slice(&picture_header(1)); - au.extend_from_slice(&[0xAA; 16]); - let pes = |data: Vec, pts: Option| crate::mux::ts::PesPacket { - source: None, - pid: vpid, - pts, - dts: None, - data, - discontinuity: false, - }; - let _ = parser.parse(&pes(au, Some(0))); - let mut next = picture_header(3); - next.extend_from_slice(&[0xBB; 16]); - let _ = parser.parse(&pes(next, None)); - - let cp = PesStream::codec_private(&s, 1) - .expect("codec_private must reach the VIDEO track's parser, not track 0's"); - assert_eq!( - &cp[..4], - &[0x00, 0x00, 0x01, 0xB3], - "codec private is the MPEG-2 sequence header" - ); - assert_eq!(&cp[4..7], &[0x2D, 0x01, 0xE0], "720x480 as authored"); - - assert!( - PesStream::headers_ready(&s), - "with the video's codec private present the mux may start" - ); - // The AC-3 track genuinely has none — so the Some() above is a real - // per-track lookup, not a fixed answer. - assert!(PesStream::codec_private(&s, 0).is_none()); - // And a track index past the end of the pid map has none either. - assert!(PesStream::codec_private(&s, 7).is_none()); - } - - // A resolved video config must not finalise headers while an AAC track - // still lacks its AudioSpecificConfig (frames are ADTS-stripped). - #[test] - fn headers_ready_also_waits_for_the_aac_config() { - use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate}; - let mut title = audio_then_video_title(); - title.content_format = ContentFormat::BdTs; - title.streams[1] = crate::disc::Stream::Video(match &title.streams[1] { - crate::disc::Stream::Video(v) => crate::disc::VideoStream { - pid: 0x1011, - ..v.clone() - }, - _ => unreachable!(), - }); - title.streams.push(crate::disc::Stream::Audio(AudioStream { - pid: 0x1100, - codec: Codec::Aac, - channels: AudioChannels::Stereo, - language: "eng".to_string(), - sample_rate: SampleRate::S44_1, - secondary: false, - purpose: LabelPurpose::Normal, - label: String::new(), - })); - let mut s = DiscStream::new( - Box::new(ZeroReader { capacity: 8 }), - title, - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - let pes = |pid: u16, data: Vec| crate::mux::ts::PesPacket { - source: None, - pid, - pts: Some(0), - dts: None, - data, - discontinuity: false, - }; - fn parser( - s: &mut DiscStream, - pid: u16, - ) -> &mut Box { - let i = s.parsers.iter().position(|(p, _)| *p == pid).unwrap(); - &mut s.parsers[i].1 - } - let mut au = seq_header(720, 480); - au.extend_from_slice(&picture_header(1)); - au.extend_from_slice(&[0xAA; 16]); - let _ = parser(&mut s, 0x1011).parse(&pes(0x1011, au)); - let _ = parser(&mut s, 0x1011).parse(&pes(0x1011, picture_header(3))); - assert!(PesStream::codec_private(&s, 1).is_some()); - assert!( - !PesStream::headers_ready(&s), - "AAC track 2 has no AudioSpecificConfig yet" - ); - - let mut adts = vec![0xFF, 0xF1, 0x50, 0x80, 0x02, 0x9F, 0xFC]; - adts.extend_from_slice(&[0u8; 13]); // frame_length = 20 - let _ = parser(&mut s, 0x1100).parse(&pes(0x1100, adts)); - assert_eq!(PesStream::codec_private(&s, 2), Some(vec![0x12, 0x10])); - assert!(PesStream::headers_ready(&s)); - } - - fn aac_only_stream() -> DiscStream { - use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate}; - let mut title = audio_then_video_title(); - title.content_format = ContentFormat::BdTs; - title.streams = vec![crate::disc::Stream::Audio(AudioStream { - pid: 0x1100, - codec: Codec::Aac, - channels: AudioChannels::Stereo, - language: "eng".to_string(), - sample_rate: SampleRate::S48, - secondary: false, - purpose: LabelPurpose::Normal, - label: String::new(), - })]; - DiscStream::new( - Box::new(ZeroReader { capacity: 8 }), - title, - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::BdTs, - false, - None, - ) - .unwrap() - } - - // read() must feed the header gate: EOF releases a silent AAC track. - #[test] - fn read_eof_releases_a_silent_aac_track() { - let mut s = aac_only_stream(); - assert!(!PesStream::headers_ready(&s)); - assert!( - PesStream::read(&mut s).unwrap().is_none(), - "all-zero source" - ); - assert!(PesStream::headers_ready(&s), "EOF expires the AAC wait"); - } - - // read() must feed the header gate every frame it returns. - #[test] - fn read_frames_advance_the_aac_wait() { - let mut s = aac_only_stream(); - for sec in 0..=5i64 { - s.pending_frames.push_back(crate::pes::PesFrame { - discard_padding_ns: 0, - coding: None, - source: None, - track: 0, - pts: sec * 1_000_000_000, - keyframe: true, - data: vec![0; 4], - duration_ns: None, - }); - } - for sec in 0..5 { - assert!(PesStream::read(&mut s).unwrap().is_some()); - assert!(!PesStream::headers_ready(&s), "{sec} s: still waiting"); - } - assert!(PesStream::read(&mut s).unwrap().is_some()); - assert!(PesStream::headers_ready(&s), "5 s of source time elapsed"); - } - - // ── on_event() / emit() ─────────────────────────────────────────── - - /// `on_event()` installs the sink and `emit()` feeds it. If either is a - /// no-op the CLI's progress bar never moves and skipped sectors are never - /// reported — the rip looks clean and stalled at 0 %. - #[test] - fn installed_event_sink_receives_skip_and_progress_events() { - let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::::new())); - let sink = log.clone(); - let mut s = short_read_stream(true); - s.on_event(move |e| { - let tag = match e.kind { - EventKind::SectorSkipped { sector } => format!("skip:{sector}"), - EventKind::BytesRead { bytes, total } => format!("bytes:{bytes}/{total}"), - other => format!("other:{other:?}"), - }; - sink.lock().unwrap().push(tag); - }); - - assert!(s.fill_extents().expect("short read absorbed")); - - let got = log.lock().unwrap().clone(); - assert!( - got.contains(&"skip:0".to_string()), - "the skipped unit at LBA 0 must reach the installed sink; got {got:?}" - ); - // 2048 B delivered out of a 64-sector (131072 B) title. - assert!( - got.contains(&"bytes:2048/131072".to_string()), - "progress must report DELIVERED bytes against the extent total; got {got:?}" - ); - } - - // ── set_raw() ───────────────────────────────────────────────────── - - // Must flip BOTH key holders — mirror and wrapped reader — or an - // AACS stream keeps decrypting when --raw asked for ciphertext. - #[test] - fn set_raw_clears_both_the_mirror_and_the_readers_keys() { - let mut s = DiscStream::new( - Box::new(ZeroReader { capacity: 8 }), - synthetic_title(8), - crate::decrypt::DecryptKeys::Aacs { - unit_keys: vec![(0, [0x11u8; 16])], - format: ContentFormat::BdTs, - }, - 3, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - assert!( - s.decrypt_keys.is_encrypted(), - "fixture must start encrypted or the test proves nothing" - ); - - // Behavioural witness for the WRAPPED reader's keys: while AACS keys - // are installed, a read not starting on a 6144-byte unit boundary - // hard-fails (it would mis-decrypt every following unit). - let mut buf = vec![0u8; 2048]; - s.reader.set_unit_base(0); // anchored: witnesses the keys, not the no-base gate - let before = s.reader.read_sectors(1, 1, &mut buf, false); - assert!( - matches!(before, Err(crate::error::Error::DecryptFailed)), - "the encrypted fixture must reject a mid-unit read; got {before:?}" - ); - - s.set_raw(); - - assert!( - !s.decrypt_keys.is_encrypted(), - "the metadata mirror must report the stream as raw" - ); - let after = s.reader.read_sectors(1, 1, &mut buf, false); - assert_eq!( - after.expect("raw mode must pass ciphertext straight through"), - 2048, - "the wrapped reader must stop decrypting, not just the mirror" - ); - } - - #[test] - fn disc_stream_stamps_source_provenance() { - use crate::disc::{Codec, ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream}; - use crate::pes::Stream; - - // Build 1 TS packet containing a complete PES for PID 0x1011 - // 4-byte BD prefix + 188-byte TS packet (PUSI=1, PID=0x1011) - let mut ts_pkt = vec![0x00, 0x00, 0x00, 0x00, 0x47, 0x50, 0x11, 0x10]; - // PES header: 00 00 01 E0, length 0x000E (14 bytes), flags 0x80 0x80 (PTS present), header_len 5, PTS 0 - ts_pkt.extend_from_slice(&[ - 0x00, 0x00, 0x01, 0xE0, 0x00, 0x0E, 0x80, 0x80, 0x05, 0x21, 0x00, 0x01, 0x00, 0x01, - ]); - // Annex B AU: SPS + PPS + IDR slice - ts_pkt.extend_from_slice(&[ - 0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0x00, 0x0A, // SPS - 0x00, 0x00, 0x00, 0x01, 0x68, 0xCE, 0x38, 0x80, // PPS - 0x00, 0x00, 0x00, 0x01, 0x65, 0x88, 0x80, // IDR slice - ]); - ts_pkt.resize(192, 0xFF); - - let mut sector = ts_pkt; - sector.resize(2048, 0xFF); - - struct SectorPayloadReader(Vec); - impl crate::sector::SectorSource for SectorPayloadReader { - fn read_sectors( - &mut self, - _lba: u32, - count: u16, - buf: &mut [u8], - _recovery: bool, - ) -> crate::error::Result { - let n = (count as usize * 2048).min(self.0.len()).min(buf.len()); - buf[..n].copy_from_slice(&self.0[..n]); - Ok(n) - } - fn capacity_sectors(&self) -> u32 { - 1 - } - } - - let mut title = synthetic_title(1); - title.streams = vec![crate::disc::Stream::Video(VideoStream { - pid: 0x1011, - codec: Codec::H264, - resolution: Resolution::R1080p, - frame_rate: FrameRate::F23_976, - hdr: HdrFormat::Sdr, - color_space: ColorSpace::Bt709, - display_aspect: None, - secondary: false, - label: String::new(), - measured_cicp: None, - })]; - title.content_format = ContentFormat::BdTs; - - let reader = SectorPayloadReader(sector); - let mut s = DiscStream::new( - Box::new(reader), - title, - crate::decrypt::DecryptKeys::None, - 1, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - - let frame = s.read().unwrap().expect("expected 1 frame"); - assert!( - frame.source.is_some(), - "DiscStream must stamp byte provenance (source) on demuxed frames" - ); - assert_eq!(frame.source.unwrap().byte, 0); - } - - // ── PS-path provenance ──────────────────────────────────────────── - - /// A 14-byte MPEG-2 PS pack header (`00 00 01 BA …`), no stuffing. - fn ps_pack_header() -> Vec { - let mut p = vec![0x00, 0x00, 0x01, 0xBAu8]; - p.extend_from_slice(&[0x44, 0x00, 0x04, 0x00, 0x04, 0x01, 0x01, 0x89, 0xC3]); - p.push(0xF8); // pack_stuffing_length = 0 (low 3 bits) - p - } - - /// One coded MPEG-2 picture: picture header (coding_type, temporal_ref) + - /// a picture-coding extension + slice padding, so the real `Mpeg2Parser` - /// treats it as a complete picture. - fn ps_pic(coding_type: u8, tr: u16) -> Vec { - let b4 = ((tr >> 2) & 0xFF) as u8; - let b5 = (((tr & 0x03) as u8) << 6) | ((coding_type & 0x07) << 3); - let mut au = vec![0x00, 0x00, 0x01, 0x00u8, b4, b5, 0x00, 0x00]; - au.extend_from_slice(&[0x00, 0x00, 0x01, 0xB5u8, 0x80, 0x00, 0x03, 0x80, 0x00]); - au.extend_from_slice(&[0xAA; 32]); - au - } - - /// One GOP's ES: seq header + GOP + I/P/B pictures. - fn ps_gop_es() -> Vec { - let mut es = seq_header(720, 480); - es.extend_from_slice(&[0x00, 0x00, 0x01, 0xB8u8, 0x00, 0x00, 0x00, 0x00]); // GOP header - es.extend_from_slice(&ps_pic(1, 0)); // I (keyframe) - es.extend_from_slice(&ps_pic(2, 2)); // P - es.extend_from_slice(&ps_pic(3, 1)); // B - es - } - - /// A bounded video PES (stream_id 0xE0) carrying `es` with a 90 kHz PTS. - fn ps_video_pes(es: &[u8], pts: u64) -> Vec { - let mut pes = vec![0x00, 0x00, 0x01, 0xE0u8]; - let mut body = vec![0x80u8, 0x80, 0x05]; // flags1, flags2 (PTS only), header_data_len - let p = pts & 0x1_FFFF_FFFF; - body.push(0x21 | (((p >> 30) & 0x07) << 1) as u8); - body.push(((p >> 22) & 0xFF) as u8); - body.push((0x01 | (((p >> 15) & 0x7F) << 1)) as u8); - body.push(((p >> 7) & 0xFF) as u8); - body.push((0x01 | ((p & 0x7F) << 1)) as u8); - body.extend_from_slice(es); - let len = body.len() as u16; - pes.extend_from_slice(&len.to_be_bytes()); - pes.extend_from_slice(&body); - pes - } - - /// Serves a fixed byte image sector-by-sector, mapping LBA to its slice. - struct ImageReader(Vec); - impl crate::sector::SectorSource for ImageReader { - fn read_sectors( - &mut self, - lba: u32, - count: u16, - buf: &mut [u8], - _recovery: bool, - ) -> crate::error::Result { - let start = lba as usize * 2048; - let want = count as usize * 2048; - for (i, b) in buf[..want].iter_mut().enumerate() { - *b = self.0.get(start + i).copied().unwrap_or(0); - } - Ok(want) - } - fn capacity_sectors(&self) -> u32 { - (self.0.len() / 2048) as u32 - } - } - - fn mpeg2_video_title(sector_count: u32) -> DiscTitle { - use crate::disc::{Codec, ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream}; - let mut t = synthetic_title(sector_count); - t.content_format = ContentFormat::MpegPs; - t.streams = vec![crate::disc::Stream::Video(VideoStream { - pid: 0xE0, // DVD video stream id - codec: Codec::Mpeg2, - resolution: Resolution::R480i, - frame_rate: FrameRate::F29_97, - hdr: HdrFormat::Sdr, - color_space: ColorSpace::Smpte170m, - display_aspect: Some((4, 3)), - secondary: false, - label: String::new(), - measured_cicp: None, - })]; - t - } - - // One frame per PES; a keyframe iff the ES starts with `K`. Carries the PES's - // discontinuity so the live stream's B1 gate is driven end to end. - struct KeyframeParser; - impl crate::mux::codec::CodecParser for KeyframeParser { - fn parse(&mut self, pes: &PesPacket) -> Vec { - vec![crate::mux::codec::Frame { - coding: None, - source: None, - pts_ns: pes.pts.unwrap_or(0), - keyframe: pes.data.first() == Some(&b'K'), - discontinuity: pes.discontinuity, - data: pes.data.clone(), - duration_ns: None, - }] - } - fn flush(&mut self) -> Vec { - Vec::new() - } - fn codec_private(&self) -> Option> { - None - } - } - - // A 192-byte BD-TS packet on `pid` with continuity counter `cc`, carrying one - // complete video PES (no PTS) whose ES is `es`; padded with stuffing. - fn ts_video_packet(pid: u16, cc: u8, es: &[u8]) -> Vec { - let mut pes = vec![0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x00, 0x00]; - pes.extend_from_slice(es); - let mut pkt = vec![0u8; 192]; - pkt[4] = 0x47; - pkt[5] = 0x40 | ((pid >> 8) as u8 & 0x1F); - pkt[6] = pid as u8; - let pad = 184 - pes.len(); - pkt[7] = 0x30 | (cc & 0x0F); // adaptation field + payload - pkt[8] = (pad - 1) as u8; - if pad > 1 { - pkt[9] = 0x00; - pkt[10..8 + pad].fill(0xFF); - } - pkt[8 + pad..].copy_from_slice(&pes); - pkt - } - - // The live stream runs the same B1 gate as the highway: after a TS continuity - // gap on a video track, inter frames are dropped until the next keyframe. - #[test] - fn live_ts_video_drops_to_keyframe_after_a_continuity_gap() { - use crate::disc::{Codec, ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream}; - let pid = 0x1011; - // CC 0,1 then a lost packet (2), then 3,4,5,6. - let es: [(&[u8], u8); 6] = [ - (b"K0", 0), - (b"P1", 1), - (b"P2", 3), - (b"P3", 4), - (b"K4", 5), - (b"P5", 6), - ]; - let mut image: Vec = es - .iter() - .flat_map(|(e, cc)| ts_video_packet(pid, *cc, e)) - .collect(); - let null = { - let mut p = vec![0u8; 192]; - p[4] = 0x47; - p[5] = 0x1F; - p[6] = 0xFF; - p[7] = 0x10; - p - }; - while image.len() < 6144 { - image.extend_from_slice(&null); - } - let mut title = synthetic_title(3); - title.streams = vec![crate::disc::Stream::Video(VideoStream { - pid, - codec: Codec::Hevc, - resolution: Resolution::R1080p, - frame_rate: FrameRate::F23_976, - hdr: HdrFormat::Sdr, - color_space: ColorSpace::Bt709, - display_aspect: None, - secondary: false, - label: String::new(), - measured_cicp: None, - })]; - let mut s = DiscStream::new( - Box::new(ImageReader(image)), - title, - crate::decrypt::DecryptKeys::None, - 3, - ContentFormat::BdTs, - false, - None, - ) - .unwrap(); - s.parsers = vec![(pid, Box::new(KeyframeParser))]; - let mut emitted = Vec::new(); - while let Some(f) = s.read().unwrap() { - emitted.push(f.data); - } - assert_eq!( - emitted, - vec![ - b"K0".to_vec(), - b"P1".to_vec(), - b"K4".to_vec(), - b"P5".to_vec() - ], - "post-gap inter frames dropped, the stream resumes at the keyframe" - ); - assert_eq!( - PesStream::errors(&s), - 2, - "the two dropped frames are counted" - ); - } - - /// The PS path (`ps_demuxer` + `Mpeg2Parser`) must forward `ps.source` - /// onto the `PesPacket` so demuxed frames carry byte provenance — the - /// second half of commit a0d4dd3 (the first half is the TS path above). - #[test] - fn ps_stream_stamps_source_provenance() { - use crate::pes::Stream; - - let mut sector = ps_pack_header(); - sector.extend_from_slice(&ps_video_pes(&ps_gop_es(), 0)); - sector.resize(2048, 0xFF); - - let mut s = DiscStream::new( - Box::new(ImageReader(sector)), - mpeg2_video_title(1), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::MpegPs, - false, - None, - ) - .unwrap(); - - let frame = s - .read() - .unwrap() - .expect("expected a frame from the PS path"); - assert!( - frame.source.is_some(), - "PS path must forward ps.source so demuxed frames carry provenance" - ); - } - - /// A DVD read with no declared extension track reports `0xD0|n` packets once at EOF. - #[test] - fn ps_stream_reports_undeclared_extension_packets_once() { - use crate::pes::Stream; - - let mut sector = ps_pack_header(); - for _ in 0..3 { - // MPEG-2 PES header without PTS, then an ext_frame's "ext_syncword". - sector.extend_from_slice(&[0x00, 0x00, 0x01, 0xD2, 0x00, 0x05, 0x81, 0x00, 0x00]); - sector.extend_from_slice(&[0x7F, 0xF0]); - } - sector.extend_from_slice(&ps_video_pes(&ps_gop_es(), 0)); - sector.resize(2048, 0xFF); - let mut s = DiscStream::new( - Box::new(ImageReader(sector)), - mpeg2_video_title(1), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::MpegPs, - false, - None, - ) - .unwrap(); - let ((), ev) = crate::testlog::capture(|| while s.read().unwrap().is_some() {}); - let warns: Vec<&str> = ev - .iter() - .filter(|e| e.message().contains("tag=mp2.extension")) - .map(|e| e.message()) - .collect(); - // Reported once, with the count, after the whole read (not per packet). - assert_eq!(warns.len(), 1, "{warns:?}"); - assert!(warns[0].contains("stream_id=0xd2 packets=3"), "{warns:?}"); - } - - /// Packets of an undeclared (or deselected) PS stream warn once, not per packet. - #[test] - fn ps_stream_warns_once_per_dropped_stream() { - use crate::pes::Stream; - - let mut sector = ps_pack_header(); - for _ in 0..30 { - // private_stream_1, no PTS, sub-stream 0x81 (AC-3 #2): no track declared. - sector.extend_from_slice(&[0x00, 0x00, 0x01, 0xBD, 0x00, 0x07, 0x81, 0x00, 0x00]); - sector.extend_from_slice(&[0x81, 0x01, 0x00, 0x01]); - } - sector.extend_from_slice(&ps_video_pes(&ps_gop_es(), 0)); - sector.resize(2048, 0xFF); - let mut s = DiscStream::new( - Box::new(ImageReader(sector)), - mpeg2_video_title(1), - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::MpegPs, - false, - None, - ) - .unwrap(); - let ((), ev) = crate::testlog::capture(|| while s.read().unwrap().is_some() {}); - let warns: Vec<&str> = ev - .iter() - .filter(|e| e.level == tracing::Level::WARN) - .map(|e| e.message()) - .collect(); - assert_eq!(warns.len(), 1, "{warns:?}"); - assert!( - ev.iter().any(|e| e.message().contains("packets=30")), - "the drop count is reported at EOF" - ); - } - - /// A PS video stream of `codec` whose AUs span two PES fragments must reach EOF - /// with every AU emitted exactly once (the final one drained at EOF). - fn ps_fragmented_au_count(codec: crate::disc::Codec, aus: &[Vec]) -> usize { - use crate::disc::{ColorSpace, FrameRate, HdrFormat, Resolution, VideoStream}; - use crate::pes::Stream; - - let mut sector = ps_pack_header(); - for (i, au) in aus.iter().enumerate() { - let (head, tail) = au.split_at(au.len() - 6); - sector.extend_from_slice(&ps_video_pes(head, 3003 * i as u64)); - // Continuation fragment carries no PTS: same AU. - let mut pes = vec![0x00, 0x00, 0x01, 0xE0u8]; - let body_len = (3 + tail.len()) as u16; - pes.extend_from_slice(&body_len.to_be_bytes()); - pes.extend_from_slice(&[0x80, 0x00, 0x00]); - pes.extend_from_slice(tail); - sector.extend_from_slice(&pes); - } - sector.resize(2048, 0xFF); - - let mut t = synthetic_title(1); - t.content_format = ContentFormat::MpegPs; - t.streams = vec![crate::disc::Stream::Video(VideoStream { - pid: 0xE0, - codec, - resolution: Resolution::R1080i, - frame_rate: FrameRate::F29_97, - hdr: HdrFormat::Sdr, - color_space: ColorSpace::Bt709, - display_aspect: None, - secondary: false, - label: String::new(), - measured_cicp: None, - })]; - let mut s = DiscStream::new( - Box::new(ImageReader(sector)), - t, - crate::decrypt::DecryptKeys::None, - 8, - ContentFormat::MpegPs, - false, - None, - ) - .unwrap(); - let mut n = 0; - while s.read().unwrap().is_some() { - n += 1; - } - n - } - - /// H.264 in a program stream (HD DVD EVO): AUs split across PES fragments - /// are reassembled and the last one is drained once at EOF (known HIGH - /// disc.rs:936). MPEG-2 tests cannot catch this: it is Passthrough. - #[test] - fn ps_stream_h264_fragmented_aus_emit_once_each_through_eof() { - let au = |first: bool, body: u8| { - let mut d = vec![0x00, 0x00, 0x01, 0x09, 0xF0]; // AUD - if first { - d.extend_from_slice(&[0x00, 0x00, 0x01, 0x67, 0x42, 0x00, 0x1E, 0x01]); - d.extend_from_slice(&[0x00, 0x00, 0x01, 0x68, 0xCE, 0x01]); - } - let nal = if first { 0x65 } else { 0x41 }; - d.extend_from_slice(&[0x00, 0x00, 0x01, nal, 0x88]); - d.extend_from_slice(&[body; 6]); - // Second slice of the same picture lands in the continuation fragment. - d.extend_from_slice(&[0x00, 0x00, 0x01, 0x41, 0x44, body]); - d - }; - let aus = vec![au(true, 0x11), au(false, 0x22), au(false, 0x33)]; - assert_eq!(ps_fragmented_au_count(crate::disc::Codec::H264, &aus), 3); - } - - /// Same for VC-1 (advanced profile BDUs) in a program stream. - #[test] - fn ps_stream_vc1_fragmented_aus_emit_once_each_through_eof() { - let au = |first: bool, body: u8| { - let mut d = Vec::new(); - if first { - d.extend_from_slice(&[0x00, 0x00, 0x01, 0x0F, 0xD0, 0x00, 0x00, 0x00, 0x80]); - } - d.extend_from_slice(&[0x00, 0x00, 0x01, 0x0D, 0x80]); - d.extend_from_slice(&[body; 12]); - d - }; - let aus = vec![au(true, 0x11), au(false, 0x22), au(false, 0x33)]; - assert_eq!(ps_fragmented_au_count(crate::disc::Codec::Vc1, &aus), 3); - } - - // fed_bytes accumulates ACROSS read buffers, not reset per read: with - // one GOP per sector (batch_sectors=1), a frame from sector 1 must - // carry source.byte >= 2048, proving the base carried over. - #[test] - fn ps_stream_carries_feed_base_across_reads() { - use crate::pes::Stream; - - let mut image = ps_pack_header(); - image.extend_from_slice(&ps_video_pes(&ps_gop_es(), 0)); - image.resize(2048, 0xFF); // sector 0 - let mut sector1 = ps_pack_header(); - sector1.extend_from_slice(&ps_video_pes(&ps_gop_es(), 3003)); - sector1.resize(2048, 0xFF); - image.extend_from_slice(§or1); // sector 1 - - let mut s = DiscStream::new( - Box::new(ImageReader(image)), - mpeg2_video_title(2), - crate::decrypt::DecryptKeys::None, - 1, // one sector per read → separate feed_at calls per sector - ContentFormat::MpegPs, - false, - None, - ) - .unwrap(); - - let mut max_byte = 0u64; - let mut min_byte = u64::MAX; - while let Some(frame) = s.read().unwrap() { - if let Some(src) = frame.source { - max_byte = max_byte.max(src.byte); - min_byte = min_byte.min(src.byte); - } - } - - assert!( - min_byte < 2048, - "a frame from the first sector must be stamped below one sector" - ); - assert!( - max_byte >= 2048, - "a frame from the second sector must carry the carried-over base \ - (source.byte >= 2048), not a per-read-reset offset" - ); - } - - // ── AdaptiveBatch ───────────────────────────────────────────────── - - // Batch sizes >= 6 must stay a multiple of 3 (AACS units) or the next - // read straddles a boundary; below 6 the ladder descends 3 -> 1. - #[test] - fn halve_batch_size_keeps_unit_alignment_and_bottoms_out_at_one() { - assert_eq!( - halve_batch_size(64), - 30, - "32 rounded down to a unit multiple" - ); - assert_eq!(halve_batch_size(30), 15); - assert_eq!(halve_batch_size(12), 6, "6 is already unit-aligned"); - assert_eq!(halve_batch_size(11), 5, "below 6: no alignment rounding"); - assert_eq!(halve_batch_size(6), 3); - assert_eq!(halve_batch_size(3), 1); - assert_eq!(halve_batch_size(2), 1); - assert_eq!( - halve_batch_size(1), - 1, - "must never reach 0 — a 0-sector read" - ); - for size in 1u16..=4096 { - let h = halve_batch_size(size); - assert!(h >= 1, "halve({size}) must never be 0"); - assert!(h <= size, "halve({size}) = {h} must not grow"); - assert!( - h < 6 || h.is_multiple_of(3), - "halve({size}) = {h} is unit-unaligned" - ); - } - } - - #[test] - fn double_batch_size_grows_toward_preferred_without_breaking_alignment() { - assert_eq!(double_batch_size(30, 64), 60); - assert_eq!( - double_batch_size(4, 64), - 6, - "8 rounded down to a unit multiple" - ); - assert_eq!( - double_batch_size(1, 64), - 2, - "below 6: no alignment rounding" - ); - assert_eq!( - double_batch_size(60, 64), - 63, - "clamped to preferred, then aligned" - ); - for size in 1u16..=2048 { - let d = double_batch_size(size, 4096); - assert!(d >= size, "double({size}) = {d} must not shrink"); - assert!( - d < 6 || d.is_multiple_of(3), - "double({size}) = {d} is unit-unaligned" - ); - } - } - - // The sizer must probe back up: after a failure drops the batch, a - // sustained clean run must return BatchSizeChanged{Probed} AND raise - // current — never probing locks the rip at reduced size forever. - #[test] - fn on_success_probes_up_after_a_sustained_clean_run_and_resets_the_streak() { - let mut b = AdaptiveBatch::new(64); - assert!( - matches!( - b.on_failure(), - Some(EventKind::BatchSizeChanged { - new_size: 30, - reason: BatchSizeReason::Shrunk - }) - ), - "a failure must shrink 64 -> 30" - ); - assert_eq!(b.current(), 30); - - // Just under the 51200-sector probe threshold: still silent. - let mut fed = 0u32; - while fed + 30 < PROBE_THRESHOLD_SECTORS { - assert!( - b.on_success(30).is_none(), - "no probe before {PROBE_THRESHOLD_SECTORS} clean sectors (at {fed})" - ); - fed += 30; - } - assert_eq!(b.current(), 30, "still at the reduced size"); - - // The read that crosses the threshold probes up. - let ev = b - .on_success(30) - .expect("a sustained clean run must probe the batch size back up"); - assert!( - matches!( - ev, - EventKind::BatchSizeChanged { - new_size: 60, - reason: BatchSizeReason::Probed - } - ), - "expected a Probed grow to 60, got {ev:?}" - ); - assert_eq!( - b.current(), - 60, - "the sizer must actually adopt the new size" - ); - assert_eq!( - b.streak_sectors, 0, - "the streak resets so the next probe needs a fresh clean run" - ); - - // A failure mid-streak must reset it: no probe right after a failure. - let mut b = AdaptiveBatch::new(64); - b.on_failure(); - b.on_success(30); - assert!(b.streak_sectors > 0); - b.on_failure(); - assert_eq!(b.streak_sectors, 0, "a failure must reset the clean streak"); - - // At the preferred size a clean run must NOT keep firing events. - let mut b = AdaptiveBatch::new(64); - for _ in 0..(PROBE_THRESHOLD_SECTORS / 64 + 2) { - assert!( - b.on_success(64).is_none(), - "no probe is possible when already at the preferred size" - ); - } - assert_eq!(b.current(), 64); - } - } -} diff --git a/src/mux/driver.rs b/src/mux/driver.rs index e0c5e634..8e5f553d 100644 --- a/src/mux/driver.rs +++ b/src/mux/driver.rs @@ -5,30 +5,33 @@ //! [`mux_with_keys`] DRIVES the existing pipeline via the same ISO pipeline builders //! and a [`WRITE_PIPELINE_DEPTH`]-deep write [`Pipeline`]; it does not replace either. -use std::path::Path; use std::sync::Arc; use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; use std::time::Duration; +use crate::ctx::Ctx; use crate::decrypt::DecryptKeys; use crate::disc::DiscTitle; use crate::error::Error; -use crate::event::{BatchSizeReason, Event, EventKind}; +use crate::event::Event; use crate::halt::Halt; use crate::io::FlushProgress; use crate::io::pipeline::{Flow, Pipeline, Sink, WRITE_PIPELINE_DEPTH}; -use crate::pes::{CountingStream, PesFrame, Stream}; +use crate::pes::{CountingStream, PesFrame, PesSink, PesSource}; use crate::sector::{FileSectorSource, SectorSource}; use crate::session::DiscSession; use super::resolve::{ - InputOptions, StreamUrl, build_iso_pipeline, input_with_halt, output, output_with, parse_url, + InputOptions, SinkCaps, StreamUrl, build_iso_pipeline, output, output_with, parse_url, }; +#[cfg(test)] +use super::source::ScannedTitle; +use super::source::{Origin, Source}; use super::videomap::{Medium, SourceInfo}; // The source medium a parsed input URL denotes, used only for provenance. Exhaustive, so a // new scheme must be placed; sink-only schemes never reach here and take the `File` default. -fn url_medium(parsed: &StreamUrl) -> Medium { +pub(crate) fn url_medium(parsed: &StreamUrl) -> Medium { match parsed { StreamUrl::Disc { .. } => Medium::Disc, // `dir://` is an image-level source: a UDF volume synthesized over the folder. @@ -59,10 +62,10 @@ pub(crate) const HEADER_BUFFER_CAP_BYTES: usize = 512 * 1024 * 1024; /// `Default` = keep-everything, no-skip, decrypt — the archival default. Added /// so callers set only the fields they care about (and so an additive field /// doesn't churn every constructor). -#[derive(Default)] +#[derive(Default, Clone)] pub struct MuxOptions { /// Skip past read errors (zero-fill + continue) on the live-drive path - /// instead of aborting. Wired onto `DiscStream::skip_errors`. + /// instead of aborting. Applied by the live drive's Read policy. pub skip_errors: bool, /// Read batch size in logical (2048-byte) sectors. pub batch_sectors: u16, @@ -73,56 +76,22 @@ pub struct MuxOptions { /// demux pipeline is built, so track headers, `codec_privates`, and frame /// routing all follow the pruned list. See [`crate::StreamSelection`]. /// - /// Applies to the `Iso`, `Session` and `Live` inputs. It does NOT apply to - /// [`MuxSource::Url`], which builds its demux inside `input()` — a Url-source - /// caller sets `InputOptions::selection` instead. Setting this field for a Url - /// input has no effect. + /// Applies to every [`Source`]. pub selection: crate::StreamSelection, + /// 0-based index of the title to mux in the source's layout (a container source has + /// one title, 0). Out of range is [`Error::MuxTrackRange`]. + pub title_index: usize, } -/// Progress / event callbacks the consumer implements (CLI `CliProgress`, -/// autorip's stream event handler). Every method has a no-op default so a -/// consumer overrides only what it renders. -/// -/// `Send + Sync + 'static` so [`mux_with_keys`] can clone the handle into the reader constructors' -/// `'static` `EventFn`. Progress is split into read-side and write-side callbacks (CLI renders -/// WRITE, autorip READ). -pub trait MuxEvents: Send + Sync + 'static { - /// Fired once, immediately after the output sink is created. The title lists - /// only streams the sink will write (tracks it refused are removed, so indices - /// compact); `MuxOutcome::undelivered_streams` keeps source-title indices. - fn on_output_opened(&self, _title: &DiscTitle) {} - /// Fired periodically from the reader side with the running read-byte count - /// and the source extents' total byte estimate. - fn on_read_progress(&self, _bytes_read: u64, _bytes_total: u64) {} - /// Fired periodically during the frame pump with the running written-byte - /// count and the title's total byte estimate. - fn on_write_progress(&self, _bytes_written: u64, _bytes_total: u64) {} - /// A bad sector was skipped (zero-filled) at `lba`. - fn on_sector_skipped(&self, _lba: u32) {} - /// The adaptive read batch size changed. - fn on_batch_size_changed(&self, _batch: u16, _reason: BatchSizeReason) {} - /// A read error occurred at `lba`. - fn on_read_error(&self, _lba: u32) {} - /// While the output is being flushed at the end (the driver waiting on a closing - /// consumer): more bytes became durable. One call per increase, at most 4 per second, - /// none while nothing moves, so silence means a stalled flush (stop design §4.5). - fn on_flush_progress(&self, _bytes_durable: u64, _bytes_total: u64) {} -} - -/// A [`MuxEvents`] that ignores everything — test-only (production callers -/// supply their own events sink). -#[cfg(test)] -pub(crate) struct NoopEvents; -#[cfg(test)] -impl MuxEvents for NoopEvents {} - /// The result of a [`mux_with_keys`] run. #[derive(Debug, Clone)] pub struct MuxOutcome { /// The mux drained to a natural EOF, finalised cleanly, and produced real /// output. `false` on interrupt (halt), or a wedged/failed finalise. pub completed: bool, + /// The run's halt stopped it: during the open or the pump alike (`completed` is + /// `false`). A wedged finalise with no Stop is `completed = false, halted = false`. + pub halted: bool, /// The output sink was created (`output()` succeeded). `false` if the mux /// bailed before opening the sink (header gate, halt during header read). pub output_opened: bool, @@ -145,425 +114,284 @@ pub struct MuxOutcome { pub undelivered_streams: Vec, } -// The mux of a source with no AACS key set (clear, CSS, `raw`, or a URL that scans): -// construct the source stream, open the `dest_url` sink and pump it (`drive_mux`). -fn mux_unkeyed( - input_src: MuxSource, - dest_url: &str, - opts: &MuxOptions, - halt: &Halt, - events: std::sync::Arc, -) -> std::io::Result { - // Construct the source stream (ISO/live/URL) — reader constructors need a - // `'static` EventFn, so we clone `Arc` into `reader_event_fn`. - // Each arm also derives SourceInfo since `output()` has no access to source. - let (stream, playlist_name, mut source): (Box, Option, SourceInfo) = - match input_src { - // The Url path builds its demux INSIDE `input()`, pruned via - // `InputOptions.selection`. `MuxOptions.selection` does NOT apply - // here — that's the File/Session arms' field. - MuxSource::Url { url, opts: in_opts } => { - // Provenance: source URL verbatim, scheme's medium, and the title - // `input()` will open. `playlist` fills in below from the opened - // stream's scanned title — not known until the scan runs. - let source = SourceInfo { - medium: url_medium(&parse_url(url)), - path: url.to_string(), - title: in_opts.title_index.unwrap_or(0), - ..SourceInfo::default() - }; - // A Stop while `network://` waits for its sender is a clean interrupt. - let stream = match input_with_halt(url, &in_opts, Some(halt)) { - Ok(s) => s, - Err(e) if crate::error::is_halt(&e) => { - return Ok(MuxOutcome { - completed: false, - output_opened: false, - bytes_written: 0, - errors: 0, - lost_bytes: 0, - streams: 0, - undelivered_streams: Vec::new(), - }); - } - Err(e) => return Err(e), - }; - let source = SourceInfo { - playlist: stream.info().playlist.clone(), - ..source - }; - (stream, None, source) - } - MuxSource::Iso { - path, - title, - format, - } => { - // Prune to selected audio/subtitle streams BEFORE the highway - // builds demux state (and before the DVD AC-3 channel probe). - // Video is always kept; no-op for default All/All. - let mut title = title; - opts.selection - .apply(&mut title) - .map_err(std::io::Error::from)?; - // Provenance: staged image as an `iso://` URL, plus playlist. Title - // INDEX is NOT reachable here — `MuxSource::Iso` carries an already- - // scanned `DiscTitle` with no index, so it stays 0 rather than guessed. - let source = SourceInfo { - medium: Medium::Iso, - path: format!("iso://{}", path.display()), - playlist: title.playlist.clone(), - ..SourceInfo::default() - }; - let reader = FileSectorSource::open(path)?; - let stream = build_iso_pipeline( - reader, - title, - DecryptKeys::None, - opts.batch_sectors, - format, - opts.raw, - Some(halt.clone()), - Some(reader_event_fn(events.clone())), - )?; - (Box::new(stream), None, source) - } - MuxSource::Session { - session, - title_index, - } => { - // Pull everything we need out of the disc as owned values so the - // immutable disc borrow is released before the mutable - // `take_reader` below. - let (mut title, format, keys, playlist, source) = { - let disc = session.disc().ok_or_else(|| Error::DeviceNotReady { - path: session.device_path().to_string(), - })?; - let title = - disc.titles - .get(title_index) - .cloned() - .ok_or(Error::MuxTrackRange { - track: title_index, - tracks: disc.titles.len(), - })?; - let playlist = disc - .meta_title - .clone() - .unwrap_or_else(|| disc.volume_id.clone()); - // Provenance: every member is reachable on this arm — the device - // the session is bound to, the title index the caller named, the - // title's own playlist, and the scanned volume id. - let source = SourceInfo { - medium: Medium::Disc, - path: format!("disc://{}", session.device_path()), - title: title_index, - playlist: title.playlist.clone(), - volume_id: disc.volume_id.clone(), - }; - // DVD CSS is per-VTS: resolve the per-title key via the pipeline - // (see `session_mux_keys`), never the whole-disc `decrypt_keys()`. - ( - title, - disc.content_format, - session_mux_keys(disc), - playlist, - source, - ) - }; - // Prune to selected streams before `DiscStream::new` builds demux tables. - opts.selection - .apply(&mut title) - .map_err(std::io::Error::from)?; - // A missing staged reader ("already consumed" / never staged) is a - // clean error, not a panic (contract Q2). - let reader = session.take_reader().ok_or_else(|| Error::DeviceNotReady { - path: session.device_path().to_string(), - })?; - let mut stream = crate::mux::DiscStream::new( - reader, - title, - keys, - opts.batch_sectors, - format, - opts.raw, - Some(halt.clone()), - )?; - if opts.raw { - stream.set_raw(); - } - stream.skip_errors = opts.skip_errors; - // Live path: the `DiscStream` emits the full reader-side vocabulary - // (`SectorSkipped` on skip-mode zero-fill, `BatchSizeChanged` on the - // adaptive sizer, `BytesRead` progress) — forward them all. - stream.on_event(reader_event_fn(events.clone())); - (Box::new(stream), Some(playlist), source) - } - MuxSource::Live { - reader, - title, - format, - } => { - // Prune to selected streams, exactly as the Iso/Session arms do — - // without this, selection was silently ignored on the live-drive - // path despite the field's doc saying it was applied. - let mut title = title; - opts.selection - .apply(&mut title) - .map_err(std::io::Error::from)?; - // Provenance: medium is certain, playlist is on hand. Device PATH - // and title INDEX are NOT reachable — `MuxSource::Live` hands an - // opaque reader/title with neither, so both stay empty/0. - let source = SourceInfo { - medium: Medium::Disc, - playlist: title.playlist.clone(), - ..SourceInfo::default() - }; - // INLINE `DiscStream`, same constructor as the `Session` arm, NOT - // `build_iso_pipeline` — the highway would bypass the adaptive - // batch-retry that lives in `DiscStream::fill_extents`. - let mut stream = crate::mux::DiscStream::new( - reader, - title, - DecryptKeys::None, - opts.batch_sectors, - format, - opts.raw, - Some(halt.clone()), - )?; - if opts.raw { - stream.set_raw(); - } - stream.skip_errors = opts.skip_errors; - // Same reader-side event vocabulary as the `Session` arm - // (`SectorSkipped` / `BatchSizeChanged` / `BytesRead`). - stream.on_event(reader_event_fn(events.clone())); - (Box::new(stream), None, source) - } - }; - - // The consumer-supplied disc name overrides the title's own playlist for the - // muxed title (see `drive_mux`); mirror that into the provenance so a - // `fvi://` header and its sibling MKV agree on what the playlist was called. - if let Some(name) = playlist_name.as_deref() { - source.playlist = name.to_string(); +impl MuxOutcome { + // A run that `ctx.halt` stopped before any frame reached a sink. + fn stopped_before_output(errors: u64, lost_bytes: u64) -> Self { + MuxOutcome { + completed: false, + halted: true, + output_opened: false, + bytes_written: 0, + errors, + lost_bytes, + streams: 0, + undelivered_streams: Vec::new(), + } } - - drive_mux( - stream, - dest_url, - halt, - events.as_ref(), - playlist_name.as_deref(), - Some(&source), - ) } -/// Where [`mux_with_keys`] reads its PES frames from (KU §3.1). No variant carries key -/// material: keys come only from the rip's [`ResolvedKeySet`](crate::keys::ResolvedKeySet). -pub enum MuxSource<'a> { - /// Live single-pass mux off an opened [`DiscSession`] (its reader staged). - Session { - session: &'a mut DiscSession, - /// Index into `session.disc().titles`. - title_index: usize, - }, - /// An image file and a title the caller ALREADY scanned (the drive scan, J14): the - /// image is never rescanned, so a sweep's unreadable UDF/MPLS/AACS sectors do not - /// matter. The set must cover the title and be for a sector-exact image of its disc. - Iso { - path: &'a Path, - title: DiscTitle, - format: crate::disc::ContentFormat, - }, - /// Live single-pass mux off a raw disc reader (the inline `DiscStream`). - Live { - reader: Box, - title: DiscTitle, - format: crate::disc::ContentFormat, - }, - /// Any URL-addressed source; the only variant that scans. `keys` goes into - /// [`InputOptions::keys`]. - Url { url: &'a str, opts: InputOptions }, -} - -/// Run the decrypt + mux pipeline end-to-end with the rip's up-front key set (KU §3.1): -/// every AACS read goes through the set's readers (its map, and the on-arrival proof for -/// pieces it left unproven), with no key lookup. `keys == None`, a non-AACS set, or -/// `opts.raw`: no AACS decryption (CSS and clear discs decrypt as before). E7013 when the -/// set is not for the disc or does not cover the title; E7026 when the title needs -/// forensic keys that are Pending. Unresolved codec headers are [`Error::MkvInvalid`], a -/// zero-output drain [`Error::NoStreams`]; a `halt` mid-run yields `completed = false`. +/// Mux `source` with the rip's up-front key set (KU §3.1): AACS reads go through the set's +/// map and on-arrival proof, with no key lookup. With no AACS set (or `opts.raw`) CSS and +/// clear content decrypt as before and a BD-TS disc refuses its first flagged unit (E7022). +/// E7013 when the set is not for the disc or title; E7026 for Pending forensic keys. +/// Unresolved codec headers are [`Error::MkvInvalid`], a zero-output drain +/// [`Error::NoStreams`]. A Stop of `ctx.halt`, during the open or the pump alike, yields +/// `completed = false, halted = true`, not an error. pub fn mux_with_keys( - source: MuxSource, - keys: Option<&crate::keys::ResolvedKeySet>, + source: Source, + keys: Option<&crate::keys::KeyRing>, dest_url: &str, opts: &MuxOptions, - halt: &Halt, - events: std::sync::Arc, + ctx: &Ctx, ) -> std::io::Result { - let set = keys.filter(|s| s.is_aacs() && !opts.raw); - // KU §3.1: "`keys` must be `Some` for AACS". A BD-TS Iso/Live mux with no AACS set reads - // through a keyless set: the first AACS-flagged unit is E7022, never muxed as content. - // MPEG-PS keeps the old path (a DVD cracks its own CSS key; `MuxSource` has no disc). - let keyless = match (&source, set) { - (MuxSource::Iso { title, format, .. } | MuxSource::Live { title, format, .. }, None) - if !opts.raw && *format == crate::disc::ContentFormat::BdTs => - { - Some(crate::keys::ResolvedKeySet::keyless_for(title, *format)) + match mux_routed(source, keys, dest_url, opts, ctx) { + Err(e) if crate::error::is_halt(&e) && ctx.halt.is_cancelled() => { + Ok(MuxOutcome::stopped_before_output(0, 0)) } - // A Session over an AACS disc with no set: BD-TS as above; HD DVD is never - // probed (KU §2.6), so it refuses up front. - ( - MuxSource::Session { - session, - title_index, - }, - None, - ) if !opts.raw => match session.disc() { - Some(d) if d.aacs.is_some() => { - if d.content_format != crate::disc::ContentFormat::BdTs { - return Err(Error::NoDiscKey { - disc_hash: d.aacs_disc_hash(), - } - .into()); - } - crate::keys::ResolvedKeySet::keyless_for_disc(d, *title_index) - } - _ => None, - }, - _ => None, - }; - let set = set.or(keyless.as_ref()); - match (source, set) { - (MuxSource::Url { url, opts: mut o }, _) => { - if let Some(k) = keys { - o.keys = Some(k.clone()); - } - mux_unkeyed( - MuxSource::Url { url, opts: o }, - dest_url, - opts, - halt, - events, - ) - } - (src, None) => mux_unkeyed(src, dest_url, opts, halt, events), - (src, Some(set)) => mux_keyed(src, set, dest_url, opts, halt, events), + r => r, } } -// The keyed arms of `mux_with_keys`: the set's readers, never a resolve. -fn mux_keyed( - source: MuxSource, - set: &crate::keys::ResolvedKeySet, +/// [`open_source`](super::source::open_source) then [`mux_with_keys`]: mux title +/// `opts.title_index` of the input `url`. A Stop during the open is a stopped outcome, as +/// during the pump. +pub fn mux_url( + url: &str, + keys: Option<&crate::keys::KeyRing>, dest_url: &str, opts: &MuxOptions, - halt: &Halt, - events: std::sync::Arc, + ctx: &Ctx, ) -> std::io::Result { - let (stream, playlist, source): (Box, Option, SourceInfo) = match source { - MuxSource::Iso { path, title, .. } => { + let source = match super::source::open_source(url, crate::disc::ScanOptions::default(), ctx) { + Ok(s) => s, + Err(Error::Halted) if ctx.halt.is_cancelled() => { + return Ok(MuxOutcome::stopped_before_output(0, 0)); + } + Err(e) => return Err(e.into()), + }; + mux_with_keys(source, keys, dest_url, opts, ctx) +} + +// `mux_with_keys` before its one halt rule. +fn mux_routed( + source: Source, + keys: Option<&crate::keys::KeyRing>, + dest_url: &str, + opts: &MuxOptions, + ctx: &Ctx, +) -> std::io::Result { + let name = source.title_name(); + let mut info = source.provenance(opts.title_index); + let from_stream = matches!(source.origin, Origin::Stream { .. }); + let stream = open_pes(source, keys, opts, ctx)?; + if from_stream { + info.playlist = stream.info().playlist.clone(); + } + if let Some(n) = &name { + info.playlist = n.clone(); + } + drive_mux(stream, dest_url, ctx, name.as_deref(), Some(&info)) +} + +// The PES stream of `opts.title_index` read from `source`: the one place that turns any +// opened input into frames (decrypting with `keys` unless `opts.raw`). +pub(crate) fn open_pes( + source: Source, + keys: Option<&crate::keys::KeyRing>, + opts: &MuxOptions, + ctx: &Ctx, +) -> std::io::Result> { + let set = keys.filter(|s| s.is_aacs() && !opts.raw); + let idx = opts.title_index; + let input_opts = || InputOptions { + title_index: Some(idx), + raw: opts.raw, + selection: opts.selection.clone(), + keys: keys.cloned(), + }; + match source.origin { + Origin::Stream { url } => super::resolve::open_stream_url(&url, &input_opts(), ctx), + Origin::Image { reader, disc, .. } => Ok(Box::new(super::resolve::image_input_scanned( + reader, + disc, + &input_opts(), + ctx, + )?)), + Origin::Prescanned { path, title: t } => { + let (title, format) = (t.title, t.format); + let keyless = keyless_ring(&title, format, set, opts); + let set = set.or(keyless.as_ref()); let mut title = title; opts.selection .apply(&mut title) .map_err(std::io::Error::from)?; - let source = SourceInfo { - medium: Medium::Iso, - path: format!("iso://{}", path.display()), - playlist: title.playlist.clone(), - ..SourceInfo::default() - }; - let reader = FileSectorSource::open(path)?; - let stream = super::resolve::build_iso_pipeline_keyed( - reader, - title, - set, - opts.batch_sectors, - Some(halt.clone()), - Some(reader_event_fn(events.clone())), - )?; - (Box::new(stream), None, source) + let reader = FileSectorSource::open(&path)?; + match set { + Some(set) => Ok(Box::new(super::resolve::build_iso_pipeline_keyed( + reader, + title, + set, + opts.batch_sectors, + ctx, + )?)), + None => Ok(Box::new(build_iso_pipeline( + reader, + title, + DecryptKeys::None, + opts.batch_sectors, + format, + opts.raw, + ctx, + )?)), + } } - MuxSource::Session { - session, - title_index, - } => { - let (title, format, playlist, source) = { - let disc = session.disc().ok_or_else(|| Error::DeviceNotReady { - path: session.device_path().to_string(), - })?; - let scope = crate::keys::KeyScope::Titles(vec![title_index]); - if !set.is_for(disc) || !set.covers(&scope) { - tracing::error!(target: "freemkv::keys", "key set is not for this session's title"); - return Err(Error::DecryptFailed.into()); + Origin::Live { reader, title: t } => { + let (title, format) = (t.title, t.format); + let keyless = keyless_ring(&title, format, set, opts); + match set.or(keyless.as_ref()) { + Some(set) => { + let title = live_keyed_title(&*reader, title, set, opts)?; + live_keyed(reader, title, format, set, opts, ctx) } - let title = disc - .titles - .get(title_index) - .cloned() - .ok_or(Error::MuxTrackRange { - track: title_index, - tracks: disc.titles.len(), - })?; - let source = SourceInfo { - medium: Medium::Disc, - path: format!("disc://{}", session.device_path()), - title: title_index, - playlist: title.playlist.clone(), - volume_id: disc.volume_id.clone(), - }; - let playlist = disc - .meta_title - .clone() - .unwrap_or_else(|| disc.volume_id.clone()); - (title, disc.content_format, playlist, source) - }; - // Refuse the selection / the set's gate BEFORE taking the reader, so a - // refused mux leaves the session retryable (as mux_unkeyed does). - let path = session.device_path().to_string(); - let not_ready = || Error::DeviceNotReady { path: path.clone() }; + None => { + let mut title = title; + opts.selection + .apply(&mut title) + .map_err(std::io::Error::from)?; + live_unkeyed(reader, title, format, DecryptKeys::None, opts, ctx) + } + } + } + Origin::Session(session) => session_pes(session, set, opts, ctx), + Origin::Drive(mut session) => session_pes(&mut session, set, opts, ctx), + } +} + +// Title `idx` of a scanned layout, and the layout's container format. +fn title_of( + disc: &crate::disc::Disc, + idx: usize, +) -> std::io::Result<(DiscTitle, crate::disc::ContentFormat)> { + let title = disc.titles.get(idx).cloned().ok_or(Error::MuxTrackRange { + track: idx, + tracks: disc.titles.len(), + })?; + Ok((title, disc.content_format)) +} + +// KU §3.1: "`keys` must be `Some` for AACS". A BD-TS read with no AACS set reads through a +// keyless set: the first AACS-flagged unit is E7022, never muxed as content. MPEG-PS keeps +// its own path (a DVD cracks its CSS key in the stream). +fn keyless_ring( + title: &DiscTitle, + format: crate::disc::ContentFormat, + set: Option<&crate::keys::KeyRing>, + opts: &MuxOptions, +) -> Option { + (set.is_none() && !opts.raw && format == crate::disc::ContentFormat::BdTs) + .then(|| crate::keys::KeyRing::keyless_for(title, format)) +} + +// A live mux off a drive session: the set's readers when keyed; else the session's own +// CSS/clear keys, and a keyless set for an AACS disc (HD DVD refuses up front). +fn session_pes( + session: &mut DiscSession, + set: Option<&crate::keys::KeyRing>, + opts: &MuxOptions, + ctx: &Ctx, +) -> std::io::Result> { + let idx = opts.title_index; + let path = session.device_path().to_string(); + let not_ready = || Error::DeviceNotReady { path: path.clone() }; + let disc = session.disc().ok_or_else(not_ready)?; + let (title, format) = title_of(disc, idx)?; + let keyless = match set { + None if !opts.raw && disc.aacs.is_some() => { + if disc.content_format != crate::disc::ContentFormat::BdTs { + return Err(Error::NoDiscKey { + disc_hash: disc.aacs_disc_hash(), + } + .into()); + } + crate::keys::KeyRing::keyless_for_disc(disc, idx) + } + _ => None, + }; + let keys = session_mux_keys(disc); + let set = set.or(keyless.as_ref()); + if let Some(set) = set { + let scope = crate::keys::KeyScope::Titles(vec![idx]); + if !set.is_for(&disc.media_id()) || !set.covers(&scope) { + tracing::error!(target: "freemkv::keys", "key set is not for this session's title"); + return Err(Error::DecryptFailed.into()); + } + } + let batch = match opts.batch_sectors { + 0 => crate::disc::detect_max_batch_sectors(&path), + n => n, + }; + let opts = &MuxOptions { + batch_sectors: batch, + ..opts.clone() + }; + if session.staged_reader().is_none() { + session.stage_drive_as_reader(); + } + match set { + Some(set) => { + // Refuse the selection / the set's gate BEFORE taking the reader, so a refused + // mux leaves the session retryable. let staged = session.staged_reader().ok_or_else(not_ready)?; let title = live_keyed_title(staged, title, set, opts)?; let reader = session.take_reader().ok_or_else(not_ready)?; - let stream = live_keyed(reader, title, format, set, opts, halt, &events)?; - (stream, Some(playlist), source) + live_keyed(reader, title, format, set, opts, ctx) } - MuxSource::Live { - reader, - title, - format, - } => { - let source = SourceInfo { - medium: Medium::Disc, - playlist: title.playlist.clone(), - ..SourceInfo::default() - }; - let title = live_keyed_title(&*reader, title, set, opts)?; - let stream = live_keyed(reader, title, format, set, opts, halt, &events)?; - (stream, None, source) + None => { + let mut title = title; + opts.selection + .apply(&mut title) + .map_err(std::io::Error::from)?; + let reader = session.take_reader().ok_or_else(not_ready)?; + live_unkeyed(reader, title, format, keys, opts, ctx) } - MuxSource::Url { .. } => unreachable!("mux_with_keys routes Url to mux_unkeyed"), - }; - let mut source = source; - if let Some(name) = playlist.as_deref() { - source.playlist = name.to_string(); } - drive_mux( - stream, - dest_url, - halt, - events.as_ref(), - playlist.as_deref(), - Some(&source), - ) +} + +// A live drive's title with no AACS set: `keys` (CSS cracks per title when `None` on a +// DVD), or ciphertext under `opts.raw`, read under the drive's Read policy. +fn live_unkeyed( + reader: Box, + title: DiscTitle, + format: crate::disc::ContentFormat, + keys: DecryptKeys, + opts: &MuxOptions, + ctx: &Ctx, +) -> std::io::Result> { + let keys = if opts.raw { DecryptKeys::None } else { keys }; + Ok(Box::new(super::resolve::build_sector_pipeline( + reader, + title, + keys, + live_policy(opts), + format, + opts.raw, + ctx, + )?)) +} + +// A live drive's Read policy: adaptive batches from `opts.batch_sectors`, then skip or fail. +fn live_policy(opts: &MuxOptions) -> crate::sector::read_stage::ReadPolicy { + crate::sector::read_stage::ReadPolicy::Live { + batch: opts.batch_sectors, + skip_errors: opts.skip_errors, + } } // The live title after the selection, once the set's gate admits it over `reader`. fn live_keyed_title( reader: &dyn SectorSource, mut title: DiscTitle, - set: &crate::keys::ResolvedKeySet, + set: &crate::keys::KeyRing, opts: &MuxOptions, ) -> std::io::Result { opts.selection @@ -578,37 +406,27 @@ fn live_keyed_title( Ok(title) } -// The inline live-drive `DiscStream` over the set (`title` from `live_keyed_title`): its -// keys, map and on-arrival proof. +// A live drive's title over the set (`title` from `live_keyed_title`): its keys, map and +// on-arrival proof, read under the drive's Read policy. fn live_keyed( reader: Box, title: DiscTitle, - format: crate::disc::ContentFormat, - set: &crate::keys::ResolvedKeySet, + _format: crate::disc::ContentFormat, + set: &crate::keys::KeyRing, opts: &MuxOptions, - halt: &Halt, - events: &std::sync::Arc, -) -> std::io::Result> { - let stream = crate::mux::DiscStream::new( + ctx: &Ctx, +) -> std::io::Result> { + Ok(Box::new(super::resolve::build_keyed_pipeline( reader, title, - set.decrypt_keys(), - opts.batch_sectors, - format, - false, - Some(halt.clone()), - )?; - let mut stream = crate::keys::install_key_map(stream, set.key_map()); - if let Some(a) = set.arrival(set.title_stop()) { - stream = stream.with_arrival(a); - } - stream.skip_errors = opts.skip_errors; - stream.on_event(reader_event_fn(events.clone())); - Ok(Box::new(stream)) + set, + live_policy(opts), + ctx, + )?)) } // Decrypt keys for the live `Session` mux of `disc`. A DVD is handed `DecryptKeys::None` so -// `DiscStream::new` cracks the CORRECT per-title CSS key rather than the whole-disc +// the title's pipeline cracks the CORRECT per-title CSS key rather than the whole-disc // (largest-title) VTS key. fn session_mux_keys(disc: &crate::disc::Disc) -> DecryptKeys { if matches!(disc.format, crate::disc::DiscFormat::Dvd) { @@ -618,19 +436,6 @@ fn session_mux_keys(disc: &crate::disc::Disc) -> DecryptKeys { } } -// Adapt reader events to mux progress callbacks with an owned, 'static closure. -fn reader_event_fn(events: Arc) -> crate::sector::prefetched::EventFn { - Box::new(move |e: Event| match e.kind { - EventKind::BytesRead { bytes, total } => events.on_read_progress(bytes, total), - EventKind::SectorSkipped { sector } => events.on_sector_skipped(sector as u32), - EventKind::BatchSizeChanged { new_size, reason } => { - events.on_batch_size_changed(new_size, reason) - } - EventKind::ReadError { sector, .. } => events.on_read_error(sector as u32), - _ => {} - }) -} - // Join the write consumer after the pump. A send that hit its deadline means the // consumer is wedged, so the join gets only the short grace, not JOIN_TIMEOUT. // While it waits, each increase of the output's durable bytes is forwarded (§4.5, LP20). @@ -639,11 +444,11 @@ fn finish_pumped( halt: &Halt, send_timed_out: bool, flush: &FlushProgress, - events: &dyn MuxEvents, + ctx: &Ctx, ) -> Result { let mut fwd = FlushForwarder { flush, - events, + ctx, last: flush.bytes_durable(), last_call: None, }; @@ -660,14 +465,14 @@ fn finish_pumped( joined } -// At most one `on_flush_progress` per this long (4 Hz, §4.5). +// At most one `BytesDurable` per this long (4 Hz, §4.5). const FLUSH_PROGRESS_EVERY: Duration = Duration::from_millis(250); -// Forwards increases of the output's durable bytes to `MuxEvents::on_flush_progress`: -// one call per increase, rate-limited, none while nothing moves. +// Forwards increases of the output's durable bytes as `Event::BytesDurable`: +// one event per increase, rate-limited, none while nothing moves. struct FlushForwarder<'a> { flush: &'a FlushProgress, - events: &'a dyn MuxEvents, + ctx: &'a Ctx, last: u64, last_call: Option, } @@ -680,8 +485,10 @@ impl FlushForwarder<'_> { .last_call .is_none_or(|t| t.elapsed() >= FLUSH_PROGRESS_EVERY); if done > self.last && (due || last_word) { - self.events - .on_flush_progress(done, self.flush.bytes_total()); + self.ctx.emit(Event::BytesDurable { + bytes: done, + total: self.flush.bytes_total(), + }); self.last = done; self.last_call = Some(std::time::Instant::now()); } @@ -698,13 +505,13 @@ fn mux_run_completed(interrupted: bool, finalize_failed: bool, halt_cancelled: b // Split out so it can be unit-tested against a synthetic Stream (the // injection seam), independent of which constructor built `stream`. fn drive_mux( - mut stream: Box, + mut stream: Box, dest_url: &str, - halt: &Halt, - events: &dyn MuxEvents, + ctx: &Ctx, playlist_name: Option<&str>, source: Option<&SourceInfo>, ) -> std::io::Result { + let halt = &ctx.halt; // Title assembled from the scanned metadata; the playlist name (disc name) // overrides `info().playlist` where the consumer supplied one. let mut out_title = stream.info().clone(); @@ -712,18 +519,33 @@ fn drive_mux( out_title.playlist = name.to_string(); } - // ── chapters:// / json:// short-circuit — BEFORE the header pump/gate ── - // These sinks write their whole file at `output()` time and consume no PES - // frames — running the header gate first could false-fail a metadata export. - if matches!( - parse_url(dest_url), - StreamUrl::Chapters { .. } | StreamUrl::Json { .. } - ) { + let caps = SinkCaps::of(&parse_url(dest_url)); + let mut buffered: Vec = Vec::new(); + let mut buffered_bytes: usize = 0; + + // ── Metadata sinks (`needs_frames = false`) — BEFORE the header pump/gate ── + // They write their whole file at `output()` and take no frames, so the header gate + // could false-fail them; the title's TrueHD labels are still completed first. + if !caps.needs_frames { + let done = complete_truehd( + &mut *stream, + &mut buffered, + &mut buffered_bytes, + &mut out_title.streams, + halt, + )?; + if !done { + return Ok(MuxOutcome::stopped_before_output( + stream.errors(), + stream.lost_bytes(), + )); + } let mut sink = CountingStream::new(output(dest_url, &out_title, source)?); - events.on_output_opened(&out_title); + ctx.emit(Event::OutputOpened { title: &out_title }); sink.finish()?; return Ok(MuxOutcome { completed: true, + halted: false, output_opened: true, bytes_written: sink.bytes_written(), errors: stream.errors(), @@ -736,19 +558,12 @@ fn drive_mux( // ── Header pump ── Buffer frames until every video (and AAC) track's // codec_private has resolved; MKV can't write a track header without codec init data. // The loop breaks on EOF/None too, so the gate below re-checks. - let mut buffered: Vec = Vec::new(); - let mut buffered_bytes: usize = 0; while !stream.headers_ready() { if halt.is_cancelled() { - return Ok(MuxOutcome { - completed: false, - output_opened: false, - bytes_written: 0, - errors: stream.errors(), - lost_bytes: stream.lost_bytes(), - streams: 0, - undelivered_streams: Vec::new(), - }); + return Ok(MuxOutcome::stopped_before_output( + stream.errors(), + stream.lost_bytes(), + )); } let read = match stream.read() { Ok(r) => r, @@ -756,15 +571,10 @@ fn drive_mux( // (reads dominate wall-clock, so a stop usually lands here). Not a // failure — yield `completed = false` so stop-preserves-staging runs. Err(e) if crate::error::is_halt(&e) => { - return Ok(MuxOutcome { - completed: false, - output_opened: false, - bytes_written: 0, - errors: stream.errors(), - lost_bytes: stream.lost_bytes(), - streams: 0, - undelivered_streams: Vec::new(), - }); + return Ok(MuxOutcome::stopped_before_output( + stream.errors(), + stream.lost_bytes(), + )); } Err(e) => return Err(e), }; @@ -798,15 +608,10 @@ fn drive_mux( // highway path a halt can end the stream as `Ok(None)`, and that EOF also // releases the in-band config wait, so a ready gate does not mean the pump finished. if halt.is_cancelled() { - return Ok(MuxOutcome { - completed: false, - output_opened: false, - bytes_written: 0, - errors: stream.errors(), - lost_bytes: stream.lost_bytes(), - streams: 0, - undelivered_streams: Vec::new(), - }); + return Ok(MuxOutcome::stopped_before_output( + stream.errors(), + stream.lost_bytes(), + )); } // The pump can break on EOF without headers resolving. Finalising then would // write a track header with no CODEC_PRIVATE — a structurally-invalid MKV the @@ -815,9 +620,23 @@ fn drive_mux( return Err(Error::MkvInvalid.into()); } - // Assemble the output title now that codec_privates have resolved. + // Assemble the output title now that codec_privates have resolved, its TrueHD + // labels completed from the stream (BUG-6). let info = stream.info().clone(); out_title.streams = info.streams.clone(); + let done = complete_truehd( + &mut *stream, + &mut buffered, + &mut buffered_bytes, + &mut out_title.streams, + halt, + )?; + if !done { + return Ok(MuxOutcome::stopped_before_output( + stream.errors(), + stream.lost_bytes(), + )); + } out_title.size_bytes = info.size_bytes; out_title.codec_privates = (0..info.streams.len()) .map(|i| stream.codec_private(i)) @@ -844,7 +663,7 @@ fn drive_mux( // ── Open the sink, wrap in a byte counter, hand it to the write pipeline ── // The output file's flush counters share the consumer's progress (§2.10 item 3). - let flush = FlushProgress::new(crate::halt::Progress::new()); + let flush = FlushProgress::new(crate::halt::Liveness::new()); let out_flush = super::resolve::OutputFlush { progress: &flush, halt, @@ -857,13 +676,13 @@ fn drive_mux( // mapping never writes an MPEG-2 extension track (reported lost only once its packets // arrive); the opened title lists only what will be written. let mut refused = output_stream.undelivered_streams(); - if drops_mp2_extensions(dest_url) { + if !caps.carries_mp2_extensions { refused.extend((out_title.streams.iter().enumerate()).filter_map(|(i, s)| { matches!(s, crate::disc::Stream::Audio(a) if a.is_mp2_extension()).then_some(i) })); } if refused.is_empty() { - events.on_output_opened(&out_title); + ctx.emit(Event::OutputOpened { title: &out_title }); } else { let mut opened = out_title.clone(); let keep = |i: &usize| !refused.contains(i); @@ -875,13 +694,13 @@ fn drive_mux( .filter(keep) .map(|i| out_title.codec_privates[i].clone()) .collect(); - events.on_output_opened(&opened); + ctx.emit(Event::OutputOpened { title: &opened }); } let output_stream = CountingStream::new(output_stream); // The write consumer runs on its own thread so the latency-bound sink write // overlaps the next `stream.read()`. `bytes` mirrors the consumer's running - // written-byte count out to the driving thread for `on_write_progress`. + // written-byte count out to the driving thread for `BytesWritten`. let bytes = Arc::new(AtomicU64::new(0)); let read_failed = Arc::new(AtomicBool::new(false)); let sink = WriteSink { @@ -916,9 +735,12 @@ fn drive_mux( break; } // Feed write-side progress during the drain exactly as the steady-state - // loop below does — the watchdog is fed only from `on_write_progress`, + // loop below does — the watchdog is fed only from `BytesWritten`, // and no `stream.read()` runs here to do it for us. - events.on_write_progress(bytes.load(Ordering::Relaxed), total_bytes); + ctx.emit(Event::BytesWritten { + bytes: bytes.load(Ordering::Relaxed), + total: total_bytes, + }); } // Then the remainder of the stream. @@ -939,7 +761,10 @@ fn drive_mux( send_timed_out = !halt.is_cancelled() && !pipe.consumer_failed(); break; } - events.on_write_progress(bytes.load(Ordering::Relaxed), total_bytes); + ctx.emit(Event::BytesWritten { + bytes: bytes.load(Ordering::Relaxed), + total: total_bytes, + }); } Ok(None) => break, // A halt landing mid-read is a clean operator stop, not a read @@ -991,7 +816,7 @@ fn drive_mux( // the container. On halt/wedge this returns an error variant, translated // to `completed = false` rather than a hard failure. let (bytes_written, undelivered_streams, finalize_failed) = - match finish_pumped(pipe, halt, send_timed_out, &flush, events) { + match finish_pumped(pipe, halt, send_timed_out, &flush, ctx) { Ok(c) => (c.bytes, c.undelivered, false), Err(Error::Halted | Error::PipelineJoinTimeout) => { (bytes.load(Ordering::Relaxed), Vec::new(), true) @@ -1019,6 +844,7 @@ fn drive_mux( if !mux_run_completed(interrupted, finalize_failed, halt.is_cancelled()) { return Ok(MuxOutcome { completed: false, + halted: halt.is_cancelled(), output_opened: true, bytes_written, errors: stream.errors(), @@ -1037,6 +863,7 @@ fn drive_mux( Ok(MuxOutcome { completed: true, + halted: false, output_opened: true, bytes_written, errors: stream.errors(), @@ -1054,7 +881,7 @@ fn lock_late(q: &LateConfigs) -> std::sync::MutexGuard<'_, Vec<(usize, Vec)> } // Move tracks whose codec_private has now resolved from `pending` to the queue. -fn collect_late_configs(stream: &dyn Stream, pending: &mut Vec, out: &LateConfigs) { +fn collect_late_configs(stream: &dyn PesSource, pending: &mut Vec, out: &LateConfigs) { pending.retain(|&track| match stream.codec_private(track) { Some(cp) => { lock_late(out).push((track, cp)); @@ -1064,17 +891,65 @@ fn collect_late_configs(stream: &dyn Stream, pending: &mut Vec, out: &Lat }); } -// Sinks that never write a DVD MPEG-2 multichannel extension track (no mapping for 13818-3 -// `ext_frame`s); the FMKV wire (network://, stdio://) carries it. -fn drops_mp2_extensions(dest_url: &str) -> bool { - matches!( - parse_url(dest_url), - StreamUrl::Mkv { .. } - | StreamUrl::Mp4 { .. } - | StreamUrl::M2ts { .. } - | StreamUrl::Demux { .. } - | StreamUrl::Audio { .. } - ) +// Bytes of frames read past the header gate while looking for TrueHD major syncs: about +// the 8 MiB of title start the playlist-era probe read. +const TRUEHD_PROBE_BYTES: usize = 8 * 1024 * 1024; +// Bytes of one TrueHD track kept for the major-sync parse. +const TRUEHD_PROBE_TRACK_BYTES: usize = 1024 * 1024; + +// Header completion (BUG-6): a playlist labels a 7.1/Atmos TrueHD track 5.1, so read on into +// `buffered` until each TrueHD track shows a major sync (at most `TRUEHD_PROBE_BYTES`) and +// label it from that. `Ok(false)` when a Stop ended the read. +fn complete_truehd( + stream: &mut dyn PesSource, + buffered: &mut Vec, + buffered_bytes: &mut usize, + streams: &mut [crate::disc::Stream], + halt: &Halt, +) -> std::io::Result { + use crate::disc::{Codec, Stream as Track}; + let mut pending: Vec = (0..streams.len()) + .filter(|&i| matches!(&streams[i], Track::Audio(a) if matches!(a.codec, Codec::TrueHd))) + .collect(); + let mut payload: std::collections::HashMap> = Default::default(); + let start = *buffered_bytes; + let mut seen = 0; + while !pending.is_empty() { + while seen < buffered.len() && !pending.is_empty() { + let f = &buffered[seen]; + seen += 1; + let Some(pos) = pending.iter().position(|&t| t == f.track) else { + continue; + }; + let buf = payload.entry(f.track).or_default(); + if buf.len() >= TRUEHD_PROBE_TRACK_BYTES { + continue; + } + buf.extend_from_slice(&f.data); + let has_sync = f.data.windows(4).any(|w| w == [0xF8, 0x72, 0x6F, 0xBA]); + if let (true, Track::Audio(a)) = (has_sync, &mut streams[f.track]) + && crate::disc::correct_truehd_stream(a, buf) + { + pending.remove(pos); + } + } + if pending.is_empty() || *buffered_bytes - start > TRUEHD_PROBE_BYTES { + break; + } + if halt.is_cancelled() { + return Ok(false); + } + match stream.read() { + Ok(Some(f)) => { + *buffered_bytes = buffered_bytes.saturating_add(f.data.len()); + buffered.push(f); + } + Ok(None) => break, + Err(e) if crate::error::is_halt(&e) => return Ok(false), + Err(e) => return Err(e), + } + } + Ok(true) } // The sink for `dest_url`; a test may substitute its own (thread-local seam). @@ -1083,7 +958,7 @@ fn open_output( title: &DiscTitle, source: Option<&SourceInfo>, flush: super::resolve::OutputFlush<'_>, -) -> std::io::Result> { +) -> std::io::Result> { #[cfg(test)] if let Some(sink) = tests::TEST_SINK.with(|s| s.borrow_mut().take()) { return Ok(sink); @@ -1168,7 +1043,7 @@ mod tests { thread_local! { // A sink `drive_mux` opens instead of its URL's, once (see `open_output`). - pub(super) static TEST_SINK: std::cell::RefCell>> = + pub(super) static TEST_SINK: std::cell::RefCell>> = const { std::cell::RefCell::new(None) }; } @@ -1181,10 +1056,17 @@ mod tests { log: Arc>>, } - impl Stream for EndSpy { + impl crate::pes::PesSource for EndSpy { fn read(&mut self) -> std::io::Result> { Ok(None) } + + fn info(&self) -> &DiscTitle { + &self.info + } + } + + impl crate::pes::PesSink for EndSpy { fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { self.writes += 1; if self.fail_at.is_some_and(|n| self.writes >= n) { @@ -1192,14 +1074,17 @@ mod tests { } Ok(()) } + fn finish(&mut self) -> std::io::Result<()> { self.log.lock().unwrap().push("finish"); Ok(()) } + fn finish_incomplete(&mut self) -> std::io::Result<()> { self.log.lock().unwrap().push("finish_incomplete"); Ok(()) } + fn info(&self) -> &DiscTitle { &self.info } @@ -1219,7 +1104,13 @@ mod tests { log: log.clone(), }; TEST_SINK.with(|s| *s.borrow_mut() = Some(Box::new(spy))); - let res = drive_mux(Box::new(stream), "null://", halt, &NoopEvents, None, None); + let res = drive_mux( + Box::new(stream), + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ); TEST_SINK.with(|s| s.borrow_mut().take()); let log = log.lock().unwrap().clone(); (res, log) @@ -1362,7 +1253,7 @@ mod tests { } } - impl Stream for FakeStream { + impl crate::pes::PesSource for FakeStream { fn read(&mut self) -> std::io::Result> { if let Some((halt, after)) = &self.cancel_halt && self.reads >= *after @@ -1392,37 +1283,121 @@ mod tests { } Ok(f) } - fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { - Ok(()) - } - fn finish(&mut self) -> std::io::Result<()> { - Ok(()) - } + fn info(&self) -> &DiscTitle { &self.info } + fn codec_private(&self, _track: usize) -> Option> { self.codec_private_ready.then(|| vec![1, 2, 3]) } + fn headers_ready(&self) -> bool { self.reads >= self.headers_ready_after || (self.ready_on_eof && self.eof_seen) } } - /// Records whether `on_output_opened` fired. + impl crate::pes::PesSink for FakeStream { + fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { + Ok(()) + } + + fn finish(&mut self) -> std::io::Result<()> { + Ok(()) + } + + fn info(&self) -> &DiscTitle { + &self.info + } + } + + /// Records whether `OutputOpened` fired. struct SpyEvents { opened: AtomicBool, } impl SpyEvents { - fn new() -> Self { - SpyEvents { + fn new() -> Arc { + Arc::new(SpyEvents { opened: AtomicBool::new(false), + }) + } + } + impl crate::event::Events for SpyEvents { + fn event(&self, e: &Event<'_>) { + if let Event::OutputOpened { .. } = e { + self.opened.store(true, Ordering::SeqCst); } } } - impl MuxEvents for SpyEvents { - fn on_output_opened(&self, _title: &DiscTitle) { - self.opened.store(true, Ordering::SeqCst); + + // BUG-6: a playlist labels a 7.1/Atmos TrueHD track 5.1 at 48 kHz. Every source's mux + // completes the label from the track's first major sync before the output opens, and a + // metadata sink (no frames) still gets the completed title. + #[test] + fn truehd_labels_complete_from_the_stream_for_every_sink() { + use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate, Stream}; + // format_info: top nibble 0x1 -> 96 kHz; low 13 bits 0x1F -> 7.1 (8ch); 4 substreams. + let mut es = vec![0u8; 24]; + es[0] = 0xAA; + es[1] = 0xBB; + es[2..6].copy_from_slice(&0xF872_6FBAu32.to_be_bytes()); + es[6..10].copy_from_slice(&((0x1u32 << 28) | 0x1F).to_be_bytes()); + es[2 + 16] = 4 << 4; + let opened_with = |dest: &str| { + let mut s = FakeStream::new(0); + s.info.streams = vec![Stream::Audio(AudioStream { + pid: 0x1100, + codec: Codec::TrueHd, + channels: AudioChannels::Surround51, + language: "eng".into(), + sample_rate: SampleRate::S48, + secondary: false, + purpose: LabelPurpose::Normal, + label: crate::labels::generate_audio_label( + &Codec::TrueHd, + &AudioChannels::Surround51, + false, + ), + })]; + for pts in 0..3 { + s.frames.push_back(PesFrame { + discard_padding_ns: 0, + track: 0, + pts, + keyframe: true, + data: es.clone(), + duration_ns: None, + source: None, + coding: None, + }); + } + let seen: Arc>> = Arc::default(); + let got = seen.clone(); + let ctx = Ctx::default().with_events(Arc::new(move |e: &Event<'_>| { + if let Event::OutputOpened { title } = e { + *got.lock().unwrap() = Some((*title).clone()); + } + })); + let _ = drive_mux(Box::new(s), dest, &ctx, None, None); + let t = seen.lock().unwrap().take().expect("output opened"); + let Stream::Audio(a) = &t.streams[0] else { + panic!("audio") + }; + (a.channels, a.sample_rate, a.label.clone()) + }; + let dir = tempfile::tempdir().unwrap(); + let json = format!("json://{}", dir.path().join("t.json").display()); + let atmos = crate::labels::generate_audio_label_atmos( + &Codec::TrueHd, + &AudioChannels::Surround71, + false, + ); + for dest in ["null://", json.as_str()] { + assert_eq!( + opened_with(dest), + (AudioChannels::Surround71, SampleRate::S96, atmos.clone()), + "{dest}" + ); } } @@ -1557,7 +1532,14 @@ mod tests { let stream = Box::new(FakeStream::new(1).with_frames(2)); let halt = Halt::new(); let spy = SpyEvents::new(); - drive_mux(stream, &url, &halt, &spy, None, Some(&source)).expect("fvi mux runs"); + drive_mux( + stream, + &url, + &crate::ctx::Ctx::new(halt.clone()).with_events(spy.clone()), + None, + Some(&source), + ) + .expect("fvi mux runs"); let text = std::fs::read_to_string(&dst).expect("index written"); let hdr: serde_json::Value = serde_json::from_str(text.lines().next().unwrap()).unwrap(); @@ -1597,8 +1579,14 @@ mod tests { let (_dir, url) = tmp("out.xml"); let halt = Halt::new(); let spy = SpyEvents::new(); - let out = - drive_mux(stream, &url, &halt, &spy, None, None).expect("chapters must short-circuit"); + let out = drive_mux( + stream, + &url, + &crate::ctx::Ctx::new(halt.clone()).with_events(spy.clone()), + None, + None, + ) + .expect("chapters must short-circuit"); assert!(out.completed, "metadata sink completes without headers"); assert!(out.output_opened); assert!(spy.opened.load(Ordering::SeqCst), "sink was opened"); @@ -1611,8 +1599,14 @@ mod tests { let url = format!("json://{}", dir.path().join("out.json").display()); let stream = Box::new(FakeStream::new(1).never_ready()); let halt = Halt::new(); - let out = drive_mux(stream, &url, &halt, &NoopEvents, None, None) - .expect("json must short-circuit"); + let out = drive_mux( + stream, + &url, + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect("json must short-circuit"); assert!(out.completed); assert!(out.output_opened); } @@ -1623,8 +1617,14 @@ mod tests { fn header_gate_rejects_unresolved_codec_private() { let stream = Box::new(FakeStream::new(1).with_frames(3).never_ready()); let halt = Halt::new(); - let err = drive_mux(stream, "null://", &halt, &NoopEvents, None, None) - .expect_err("unresolved headers must be refused"); + let err = drive_mux( + stream, + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect_err("unresolved headers must be refused"); // This gate is the GENUINE stub case: no video track's codec_private // resolved. `MkvInvalid` now means only this (bad `mkv://` input is // `MkvSourceInvalid`), and must stay skippable for all-titles rips. @@ -1641,11 +1641,45 @@ mod tests { fn zero_output_gate_refuses_empty_drain() { let stream = Box::new(FakeStream::new(1)); // headers ready, no frames let halt = Halt::new(); - let err = drive_mux(stream, "null://", &halt, &NoopEvents, None, None) - .expect_err("empty drain must be refused"); + let err = drive_mux( + stream, + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect_err("empty drain must be refused"); assert_eq!(err.to_string(), format!("E{}", crate::error::E_NO_STREAMS)); } + // HP2: a Stop landing during the open (here the CSS crack of a `Live` DVD title) is the + // same outcome as one in the pump: `completed = false, halted = true`, never an error. + #[test] + fn a_stop_during_the_open_is_a_halted_outcome_not_an_error() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); + let ctx = Ctx::default(); + ctx.halt.cancel(); + let title = DiscTitle { + extents: vec![crate::disc::Extent { + start_lba: 0, + sector_count: 16, + }], + ..DiscTitle::empty() + }; + let reader = crate::test_util::MemSource::new(vec![0u8; 16 * 2048]); + let src = Source::from_reader( + Box::new(reader), + ScannedTitle::new(title, crate::disc::ContentFormat::MpegPs), + ); + let opts = MuxOptions { + batch_sectors: 16, + ..Default::default() + }; + let out = mux_with_keys(src, None, "null://", &opts, &ctx).expect("a Stop is not an error"); + assert!(out.halted && !out.completed && !out.output_opened); + } + // ── halt mid-pump stops cleanly with completed=false, no panic. ── #[test] fn halt_mid_pump_stops_cleanly() { @@ -1656,8 +1690,14 @@ mod tests { .with_frames(1000) .cancels(halt.clone(), 2), ); - let out = drive_mux(stream, "null://", &halt, &NoopEvents, None, None) - .expect("halt is a clean stop, not an error"); + let out = drive_mux( + stream, + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect("halt is a clean stop, not an error"); assert!(!out.completed, "an interrupted mux is not complete"); assert!(out.output_opened, "the sink was opened before the halt"); } @@ -1671,8 +1711,14 @@ mod tests { fs.headers_ready_after = usize::MAX; fs.ready_on_eof = true; let events = SpyEvents::new(); - let out = drive_mux(Box::new(fs), "null://", &halt, &events, None, None) - .expect("halt is a clean stop"); + let out = drive_mux( + Box::new(fs), + "null://", + &crate::ctx::Ctx::new(halt.clone()).with_events(events.clone()), + None, + None, + ) + .expect("halt is a clean stop"); assert!(!out.completed); assert!(!out.output_opened, "no sink may be opened after a halt"); assert!(!events.opened.load(Ordering::SeqCst)); @@ -1686,8 +1732,14 @@ mod tests { let halt = Halt::new(); // Headers ready immediately; the 3rd read (in the frame pump) errors Halted. let stream = Box::new(FakeStream::new(1).with_frames(1000).halt_errs_at(2)); - let out = drive_mux(stream, "null://", &halt, &NoopEvents, None, None) - .expect("a halt mid frame-read is a clean stop, not an Err"); + let out = drive_mux( + stream, + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect("a halt mid frame-read is a clean stop, not an Err"); assert!(!out.completed, "interrupted mux is not complete"); assert!(out.output_opened, "sink opened before the mid-read halt"); } @@ -1702,8 +1754,14 @@ mod tests { .never_ready() .halt_errs_at(1), ); - let out = drive_mux(stream, "null://", &halt, &NoopEvents, None, None) - .expect("a halt mid header-read is a clean stop, not an Err"); + let out = drive_mux( + stream, + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect("a halt mid header-read is a clean stop, not an Err"); assert!(!out.completed, "interrupted mux is not complete"); assert!( !out.output_opened, @@ -1717,8 +1775,14 @@ mod tests { let stream = Box::new(FakeStream::new(2).with_frames(10)); let halt = Halt::new(); let spy = SpyEvents::new(); - let out = - drive_mux(stream, "null://", &halt, &spy, None, None).expect("normal mux completes"); + let out = drive_mux( + stream, + "null://", + &crate::ctx::Ctx::new(halt.clone()).with_events(spy.clone()), + None, + None, + ) + .expect("normal mux completes"); assert!(out.completed); assert!(out.output_opened); assert!(spy.opened.load(Ordering::SeqCst)); @@ -1728,21 +1792,16 @@ mod tests { // ── reader-side event forwarding through the Arc ──────────────────────── - /// A [`MuxEvents`] backed by atomics that records every callback so a test - /// can assert reader-side events actually flowed through the `Arc` clone. + /// An [`Events`](crate::event::Events) backed by atomics that records the events a test + /// asserts reached the run's context. struct CountingEvents { opened: AtomicBool, progress_calls: AtomicU64, - /// Set once `on_read_progress` is called with a `total` equal to the ISO - /// extents' byte total — the fingerprint of the *read-side* `BytesRead` - /// event (the write-side `on_write_progress` carries the title's - /// `size_bytes`, a different number), so it isolates the `EventFn` - /// translation. + /// Set once `BytesRead` carries a `total` equal to the ISO extents' byte total — the + /// fingerprint of the *read-side* event (the write-side `BytesWritten` carries the + /// title's `size_bytes`, a different number). saw_read_total: AtomicBool, read_total: u64, - skipped: AtomicU64, - batch_changed: AtomicU64, - read_errors: AtomicU64, } impl CountingEvents { fn new(read_total: u64) -> std::sync::Arc { @@ -1751,85 +1810,22 @@ mod tests { progress_calls: AtomicU64::new(0), saw_read_total: AtomicBool::new(false), read_total, - skipped: AtomicU64::new(0), - batch_changed: AtomicU64::new(0), - read_errors: AtomicU64::new(0), }) } } - impl MuxEvents for CountingEvents { - fn on_output_opened(&self, _title: &DiscTitle) { - self.opened.store(true, Ordering::SeqCst); - } - fn on_read_progress(&self, _bytes_read: u64, bytes_total: u64) { - self.progress_calls.fetch_add(1, Ordering::SeqCst); - if bytes_total == self.read_total { - self.saw_read_total.store(true, Ordering::SeqCst); + impl crate::event::Events for CountingEvents { + fn event(&self, e: &Event<'_>) { + match *e { + Event::OutputOpened { .. } => self.opened.store(true, Ordering::SeqCst), + Event::BytesRead { total, .. } => { + self.progress_calls.fetch_add(1, Ordering::SeqCst); + if total == self.read_total { + self.saw_read_total.store(true, Ordering::SeqCst); + } + } + _ => {} } } - fn on_sector_skipped(&self, _lba: u32) { - self.skipped.fetch_add(1, Ordering::SeqCst); - } - fn on_batch_size_changed(&self, _batch: u16, _reason: BatchSizeReason) { - self.batch_changed.fetch_add(1, Ordering::SeqCst); - } - fn on_read_error(&self, _lba: u32) { - self.read_errors.fetch_add(1, Ordering::SeqCst); - } - } - - // `reader_event_fn` maps every real `EventKind` onto the matching - // `MuxEvents` method. Mutation: dropping a match arm leaves that - // counter at 0 and fails here. - #[test] - fn reader_event_fn_translates_every_variant() { - let events = CountingEvents::new(6144); - let f = reader_event_fn(events.clone()); - f(Event { - kind: EventKind::BytesRead { - bytes: 4096, - total: 6144, - }, - }); - f(Event { - kind: EventKind::SectorSkipped { sector: 42 }, - }); - f(Event { - kind: EventKind::BatchSizeChanged { - new_size: 8, - reason: BatchSizeReason::Shrunk, - }, - }); - f(Event { - kind: EventKind::ReadError { - sector: 7, - error: Error::NoStreams, - }, - }); - assert_eq!( - events.progress_calls.load(Ordering::SeqCst), - 1, - "BytesRead → on_read_progress" - ); - assert!( - events.saw_read_total.load(Ordering::SeqCst), - "read-side total forwarded" - ); - assert_eq!( - events.skipped.load(Ordering::SeqCst), - 1, - "SectorSkipped → on_sector_skipped" - ); - assert_eq!( - events.batch_changed.load(Ordering::SeqCst), - 1, - "BatchSizeChanged → on_batch_size_changed" - ); - assert_eq!( - events.read_errors.load(Ordering::SeqCst), - 1, - "ReadError → on_read_error" - ); } /// A 192-byte BD-TS packet carrying `payload` as a payload-only TS packet on @@ -1875,10 +1871,9 @@ mod tests { } // End-to-end through `mux_unkeyed` on the ISO path: asserts reader-side - // progress AND `on_output_opened` reach the `Arc`. - // Mutation: dropping `reader_event_fn` leaves `saw_read_total` false. + // progress AND `OutputOpened` reach the run's events. #[test] - fn mux_iso_forwards_reader_progress_through_arc() { + fn mux_iso_reports_reader_progress_to_the_ctx() { // Spawns the prefetch producer, a Drive holder. let _serial = crate::sector::prefetched::holder_test_lock(); let es = [0xDE, 0xAD, 0xBE, 0xEF, 0x11, 0x22]; @@ -1902,33 +1897,33 @@ mod tests { batch_sectors: 8192, raw: false, selection: Default::default(), + title_index: 0, }; let halt = Halt::new(); - let out = mux_unkeyed( - MuxSource::Iso { - path: &iso_path, - title, - format: crate::disc::ContentFormat::BdTs, - }, + let out = mux_with_keys( + Source::from_image( + &iso_path, + ScannedTitle::new(title, crate::disc::ContentFormat::BdTs), + ), + None, "null://", &opts, - &halt, - events.clone(), + &crate::ctx::Ctx::new(halt.clone()).with_events(events.clone()), ) .expect("audio-only ISO muxes to null sink"); assert!(out.completed, "the clip drained and finalised"); assert!( events.saw_read_total.load(Ordering::SeqCst), - "the reader-side BytesRead (total=6144) reached the Arc via the EventFn" + "the reader-side BytesRead (total=6144) reached the ctx's events" ); assert!( events.opened.load(Ordering::SeqCst), - "on_output_opened fired through the Arc" + "OutputOpened reached the ctx's events" ); assert!( events.progress_calls.load(Ordering::SeqCst) > 0, - "at least one on_read_progress call observed" + "at least one BytesRead observed" ); } @@ -1949,7 +1944,7 @@ mod tests { let bytes = count as usize * 2048; buf[..bytes].fill(0); // The content unit lives at LBA 0..3; serve it whenever a read starts - // there (the inline `DiscStream` reads the [0,3) extent as one batch). + // there (the live Read stage reads the [0,3) extent as one batch). if lba == 0 && bytes >= self.unit.len() { buf[..self.unit.len()].copy_from_slice(&self.unit); } @@ -1978,7 +1973,7 @@ mod tests { /// A synthetic AACS `Disc` with one title whose sole extent is the encrypted unit at /// LBA 0..3 — the disc a `MuxSource::Session` mux scans off a live drive, minus the - /// hardware. It holds no key: AACS keys come only from a `ResolvedKeySet`. + /// hardware. It holds no key: AACS keys come only from a `KeyRing`. fn aacs_session_disc(title: DiscTitle) -> crate::disc::Disc { crate::disc::Disc { volume_id: "TEST".into(), @@ -2011,6 +2006,8 @@ mod tests { // `ok_or_else(|| Error::DeviceNotReady …)` against `.unwrap()` regression. #[test] fn mux_session_missing_reader_is_clean_error_not_panic() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); use crate::disc::Extent; use crate::session::DiscSession; @@ -2028,17 +2025,15 @@ mod tests { batch_sectors: 3, raw: false, selection: Default::default(), + title_index: 0, }; let halt = Halt::new(); - let err = mux_unkeyed( - MuxSource::Session { - session: &mut session, - title_index: 0, - }, + let err = mux_with_keys( + Source::from_session(&mut session), + None, "null://", &opts, - &halt, - Arc::new(NoopEvents), + &crate::ctx::Ctx::new(halt.clone()), ) .expect_err("a missing staged reader must be a clean error, not a panic"); // The device-name-carrying DeviceNotReady round-trips through io::Error. @@ -2080,6 +2075,8 @@ mod tests { // on the out-of-range `titles.get(idx)`. #[test] fn mux_session_out_of_range_title_is_clean_error_not_panic() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); use crate::disc::Extent; use crate::session::DiscSession; @@ -2102,17 +2099,15 @@ mod tests { batch_sectors: 3, raw: false, selection: Default::default(), + title_index: num_titles + 5, }; let halt = Halt::new(); - let err = mux_unkeyed( - MuxSource::Session { - session: &mut session, - title_index: num_titles + 5, // out of range - }, + let err = mux_with_keys( + Source::from_session(&mut session), + None, "null://", &opts, - &halt, - Arc::new(NoopEvents), + &crate::ctx::Ctx::new(halt.clone()), ) .expect_err("an out-of-range title index must be a clean error, not a panic"); // MuxTrackRange renders as "E9011: track/tracks". @@ -2187,21 +2182,17 @@ mod tests { } } - impl Stream for LateAacStream { + impl crate::pes::PesSource for LateAacStream { fn read(&mut self) -> std::io::Result> { let f = self.frames.pop_front(); self.aac_seen |= f.as_ref().is_some_and(|f| f.track == 1); Ok(f) } - fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { - Ok(()) - } - fn finish(&mut self) -> std::io::Result<()> { - Ok(()) - } + fn info(&self) -> &DiscTitle { &self.info } + fn codec_private(&self, track: usize) -> Option> { match track { 0 => Some(vec![1, 0x64, 0, 0x28, 0xFF, 0xE0, 0, 0]), @@ -2210,6 +2201,20 @@ mod tests { } } + impl crate::pes::PesSink for LateAacStream { + fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { + Ok(()) + } + + fn finish(&mut self) -> std::io::Result<()> { + Ok(()) + } + + fn info(&self) -> &DiscTitle { + &self.info + } + } + // A seekable MKV must end up with the late AAC track's CodecPrivate. #[test] fn late_aac_config_is_backpatched_into_a_seekable_mkv() { @@ -2219,8 +2224,7 @@ mod tests { let out = drive_mux( Box::new(LateAacStream::new()), &format!("mkv://{}", path.display()), - &halt, - &NoopEvents, + &crate::ctx::Ctx::new(halt.clone()), None, None, ) @@ -2339,8 +2343,7 @@ mod tests { let out = drive_mux( Box::new(LateAacStream::new()), "null://", - &halt, - &NoopEvents, + &crate::ctx::Ctx::new(halt.clone()), None, None, ) @@ -2355,19 +2358,29 @@ mod tests { info: DiscTitle, } - impl Stream for UndeliveringSink { + impl crate::pes::PesSource for UndeliveringSink { fn read(&mut self) -> std::io::Result> { Ok(None) } - fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { - Ok(()) - } - fn finish(&mut self) -> std::io::Result<()> { - Ok(()) - } + fn info(&self) -> &DiscTitle { &self.info } + } + + impl crate::pes::PesSink for UndeliveringSink { + fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { + Ok(()) + } + + fn finish(&mut self) -> std::io::Result<()> { + Ok(()) + } + + fn info(&self) -> &DiscTitle { + &self.info + } + fn undelivered_streams(&self) -> Vec { vec![1] } @@ -2405,7 +2418,7 @@ mod tests { } } } - impl Stream for LpcmSource { + impl crate::pes::PesSource for LpcmSource { fn read(&mut self) -> std::io::Result> { if self.frames == 0 { return Ok(None); @@ -2422,31 +2435,44 @@ mod tests { coding: None, })) } - fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { - Ok(()) - } - fn finish(&mut self) -> std::io::Result<()> { - Ok(()) - } + fn info(&self) -> &DiscTitle { &self.info } + fn codec_private(&self, _track: usize) -> Option> { self.layout.clone() } } - /// Keeps the title handed to `on_output_opened`. - #[derive(Default)] - struct TitleSpy(std::sync::Mutex>); - impl MuxEvents for TitleSpy { - fn on_output_opened(&self, title: &DiscTitle) { - *self.0.lock().unwrap() = Some(title.clone()); + impl crate::pes::PesSink for LpcmSource { + fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { + Ok(()) + } + + fn finish(&mut self) -> std::io::Result<()> { + Ok(()) + } + + fn info(&self) -> &DiscTitle { + &self.info } } - fn run(src: LpcmSource, url: &str, spy: &TitleSpy) -> MuxOutcome { - drive_mux(Box::new(src), url, &Halt::new(), spy, None, None).unwrap() + /// Keeps the title of `OutputOpened`. + #[derive(Default)] + struct TitleSpy(std::sync::Mutex>); + impl crate::event::Events for TitleSpy { + fn event(&self, e: &Event<'_>) { + if let Event::OutputOpened { title } = *e { + *self.0.lock().unwrap() = Some(title.clone()); + } + } + } + + fn run(src: LpcmSource, url: &str, spy: &Arc) -> MuxOutcome { + let ctx = crate::ctx::Ctx::default().with_events(spy.clone()); + drive_mux(Box::new(src), url, &ctx, None, None).unwrap() } // Every entry point (Url/Session/Iso/Live) funnels into drive_mux: the BD LPCM @@ -2458,7 +2484,7 @@ mod tests { let url = format!("mkv://{}", dir.path().join("o.mkv").display()); let layout = Some(b"BDLP\xB4\x18".to_vec()); let src = LpcmSource::new(AudioChannels::Surround51, SampleRate::S48, layout); - let spy = TitleSpy::default(); + let spy = Arc::new(TitleSpy::default()); run(src, &url, &spy); let t = spy.0.lock().unwrap().clone().unwrap(); let crate::disc::Stream::Audio(a) = &t.streams[0] else { @@ -2471,7 +2497,7 @@ mod tests { } /// BD source whose LPCM layout byte exists only once the first PES has been - /// parsed, gated by the real `HeaderGate` (as DiscStream/PipelinedPesStream are). + /// parsed, gated by the real `HeaderGate` (as PipelinedPesStream is). struct GatedLpcm { info: DiscTitle, parser: crate::mux::codec::lpcm::LpcmParser, @@ -2507,7 +2533,7 @@ mod tests { self.info.streams.len() - 1 } } - impl Stream for GatedLpcm { + impl crate::pes::PesSource for GatedLpcm { fn read(&mut self) -> std::io::Result> { use crate::mux::codec::CodecParser as _; let Some(data) = self.pes.pop_front() else { @@ -2536,15 +2562,11 @@ mod tests { self.gate.observe(&frame); Ok(Some(frame)) } - fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { - Ok(()) - } - fn finish(&mut self) -> std::io::Result<()> { - Ok(()) - } + fn info(&self) -> &DiscTitle { &self.info } + fn codec_private(&self, track: usize) -> Option> { use crate::mux::codec::CodecParser as _; if track == self.lpcm_track() { @@ -2553,11 +2575,26 @@ mod tests { Some(vec![1, 0x64, 0, 0x28, 0xFF, 0xE0, 0, 0]) // minimal avcC } } + fn headers_ready(&self) -> bool { self.gate.ready(&self.info, |i| self.codec_private(i)) } } + impl crate::pes::PesSink for GatedLpcm { + fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { + Ok(()) + } + + fn finish(&mut self) -> std::io::Result<()> { + Ok(()) + } + + fn info(&self) -> &DiscTitle { + &self.info + } + } + // The layout byte arrives only after the first read: the header gate must wait // for it, or a 7.1/96k track is declared as the playlist's 5.1/48k. #[test] @@ -2569,8 +2606,15 @@ mod tests { let dir = tempfile::tempdir().unwrap(); // m2ts: an MKV with a declared but frameless video track is refused. let url = format!("m2ts://{}", dir.path().join("o.m2ts").display()); - let spy = TitleSpy::default(); - drive_mux(Box::new(src), &url, &Halt::new(), &spy, None, None).unwrap(); + let spy = Arc::new(TitleSpy::default()); + drive_mux( + Box::new(src), + &url, + &crate::ctx::Ctx::default().with_events(spy.clone()), + None, + None, + ) + .unwrap(); let t = spy.0.lock().unwrap().clone().unwrap(); let crate::disc::Stream::Audio(a) = t.streams.last().unwrap() else { panic!("audio") @@ -2589,7 +2633,7 @@ mod tests { let mut src = LpcmSource::new(AudioChannels::Stereo, SampleRate::S48, None); let other = LpcmSource::new(AudioChannels::Stereo, SampleRate::S44_1, None); src.info.streams.extend(other.info.streams); - let spy = TitleSpy::default(); + let spy = Arc::new(TitleSpy::default()); let out = run(src, &url, &spy); assert_eq!(out.undelivered_streams, vec![1]); let opened = spy.0.lock().unwrap().clone().unwrap(); @@ -2614,7 +2658,7 @@ mod tests { purpose: LabelPurpose::Normal, label: crate::disc::MP2_EXTENSION_LABEL.into(), })); - let spy = TitleSpy::default(); + let spy = Arc::new(TitleSpy::default()); let (out, ev) = crate::testlog::capture(|| run(src, &url, &spy)); let opened = spy.0.lock().unwrap().clone().unwrap(); assert!( @@ -2680,8 +2724,14 @@ mod tests { }); } let halt = Halt::new(); - let err = drive_mux(Box::new(fs), "null://", &halt, &NoopEvents, None, None) - .expect_err("over-cap header buffer must fail fast, not OOM"); + let err = drive_mux( + Box::new(fs), + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect_err("over-cap header buffer must fail fast, not OOM"); // The cap overflow must carry its OWN code, not `MkvInvalid`: that code // means "skippable empty stub" and would silently drop a title that // had just produced 512 MiB of real frames. @@ -2706,7 +2756,7 @@ mod tests { // ── Regression B: watchdog fed during the buffered header drain ───────── // Headers resolve only after K frames buffer, then flush to the sink. - // Every flushed frame must fire `on_write_progress`, the sole watchdog feed. + // Every flushed frame must fire `BytesWritten`, the sole watchdog feed. #[test] fn write_progress_fed_during_header_drain() { const K: usize = 6; @@ -2717,27 +2767,35 @@ mod tests { struct WriteProgressCounter { writes: AtomicU64, } - impl MuxEvents for WriteProgressCounter { - fn on_write_progress(&self, _bytes_written: u64, _bytes_total: u64) { - self.writes.fetch_add(1, Ordering::SeqCst); + impl crate::event::Events for WriteProgressCounter { + fn event(&self, e: &Event<'_>) { + if let Event::BytesWritten { .. } = e { + self.writes.fetch_add(1, Ordering::SeqCst); + } } } - let events = WriteProgressCounter { + let events = Arc::new(WriteProgressCounter { writes: AtomicU64::new(0), - }; + }); let halt = Halt::new(); - let out = drive_mux(Box::new(fs), "null://", &halt, &events, None, None) - .expect("K-frame stream muxes cleanly"); + let out = drive_mux( + Box::new(fs), + "null://", + &crate::ctx::Ctx::new(halt.clone()).with_events(events.clone()), + None, + None, + ) + .expect("K-frame stream muxes cleanly"); assert!(out.completed); assert_eq!( events.writes.load(Ordering::SeqCst), K as u64, - "each of the K buffered header frames must feed on_write_progress on drain" + "each of the K buffered header frames must feed BytesWritten on drain" ); } // ── Regression C: Session key selection special-cases DVD ─────────────── - // A DVD must get `None` (so DiscStream cracks the correct per-title/VTS + // A DVD must get `None` (so the pipeline cracks the correct per-title/VTS // CSS key); a non-DVD passes `decrypt_keys()` through unconditionally. fn disc_with_css(format: crate::disc::DiscFormat) -> crate::disc::Disc { crate::disc::Disc { @@ -2772,7 +2830,7 @@ mod tests { ); assert!( matches!(session_mux_keys(&dvd), DecryptKeys::None), - "a DVD must be handed None so DiscStream cracks the per-title key" + "a DVD must be handed None so the pipeline cracks the per-title key" ); } @@ -2862,7 +2920,9 @@ mod tests { let (tx, rx) = std::sync::mpsc::channel(); std::thread::spawn(move || { let flush = FlushProgress::default(); - let _ = tx.send(finish_pumped(pipe, &halt, true, &flush, &NoopEvents).is_err()); + let _ = tx.send( + finish_pumped(pipe, &halt, true, &flush, &crate::ctx::Ctx::default()).is_err(), + ); }); let failed = rx .recv_timeout(Duration::from_secs(30)) @@ -2931,8 +2991,7 @@ mod tests { let out = drive_mux( stream, &format!("mkv://{}", dst.display()), - &Halt::new(), - &SpyEvents::new(), + &crate::ctx::Ctx::default().with_events(SpyEvents::new()), None, None, ) @@ -2942,13 +3001,15 @@ mod tests { assert_eq!(back.track_timing(0), timing); } - // Records every `on_flush_progress` call. + // Records every `BytesDurable` event. #[derive(Default)] struct FlushSpy(std::sync::Mutex>); - impl MuxEvents for FlushSpy { - fn on_flush_progress(&self, durable: u64, total: u64) { - self.0.lock().unwrap().push((durable, total)); + impl crate::event::Events for FlushSpy { + fn event(&self, e: &Event<'_>) { + if let Event::BytesDurable { bytes, total } = *e { + self.0.lock().unwrap().push((bytes, total)); + } } } @@ -2977,7 +3038,7 @@ mod tests { } fn finish_flushing(steps: u64, gap: Duration, tail: Duration) -> Vec<(u64, u64)> { - let flush = FlushProgress::new(crate::halt::Progress::new()); + let flush = FlushProgress::new(crate::halt::Liveness::new()); let sink = FlushingClose { flush: flush.clone(), steps, @@ -2986,13 +3047,16 @@ mod tests { }; let progress = flush.progress().clone(); let pipe = Pipeline::spawn_named_with_progress("t-flush", 4, sink, progress).unwrap(); - let spy = FlushSpy::default(); - finish_pumped(pipe, &Halt::new(), false, &flush, &spy).unwrap(); + let spy = Arc::new(FlushSpy::default()); + let ctx = crate::ctx::Ctx::default().with_events(spy.clone()); + finish_pumped(pipe, &Halt::new(), false, &flush, &ctx).unwrap(); + drop(ctx); + let spy = Arc::try_unwrap(spy).ok().expect("the only handle"); spy.0.into_inner().unwrap() } /// LP20 (§4.5): while the driver waits on a closing consumer, each increase of the - /// flusher's bytes produces one `on_flush_progress`, and none while it is static. + /// flusher's bytes produces one `BytesDurable`, and none while it is static. #[test] fn finish_with_halt_forwards_flush_progress() { let calls = finish_flushing(4, Duration::from_millis(300), Duration::from_millis(600)); @@ -3020,11 +3084,12 @@ mod tests { batch_sectors: 3, raw: false, selection: Default::default(), + title_index: 0, } } // One encrypted audio unit at LBA 0..3 under `key`, its title, and a set keying it. - fn keyed_live(key: [u8; 16]) -> (Box, DiscTitle, crate::keys::ResolvedKeySet) { + fn keyed_live(key: [u8; 16]) -> (Box, DiscTitle, crate::keys::KeyRing) { let reader = Box::new(AacsUnitReader { unit: encrypted_audio_unit(&key), capacity: 16, @@ -3036,7 +3101,7 @@ mod tests { }]; let mut disc = aacs_session_disc(title.clone()); disc.capacity_sectors = 16; - let set = crate::keys::ResolvedKeySet::keyed_for_test(&disc, key, &[(0, 3)]); + let set = crate::keys::KeyRing::keyed_for_test(&disc, key, &[(0, 3)]); (reader, title, set) } @@ -3053,20 +3118,20 @@ mod tests { /// resolution in the driver. #[test] fn mux_with_keys_live_decrypts_through_the_set() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); let (reader, title, set) = keyed_live([0x5A; 16]); let dir = tempfile::tempdir().unwrap(); let out_path = dir.path().join("live.mkv"); let out = mux_with_keys( - MuxSource::Live { + Source::from_reader( reader, - title, - format: crate::disc::ContentFormat::BdTs, - }, + ScannedTitle::new(title, crate::disc::ContentFormat::BdTs), + ), Some(&set), &format!("mkv://{}", out_path.display()), &keyed_opts(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .expect("the set's key opens the unit"); assert!(out.completed); @@ -3079,22 +3144,20 @@ mod tests { /// `MuxSource::Session` decrypts with the set: the disc holds no key. #[test] fn mux_with_keys_session_uses_the_set() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); let key = [0x5A; 16]; let (reader, title, _) = keyed_live(key); let mut disc = aacs_session_disc(title); disc.capacity_sectors = 16; - let set = crate::keys::ResolvedKeySet::keyed_for_test(&disc, key, &[(0, 3)]); + let set = crate::keys::KeyRing::keyed_for_test(&disc, key, &[(0, 3)]); let mut session = DiscSession::from_parts_for_test(Some(disc), Some(reader)); let out = mux_with_keys( - MuxSource::Session { - session: &mut session, - title_index: 0, - }, + Source::from_session(&mut session), Some(&set), "null://", &keyed_opts(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .expect("the set's key opens the unit"); assert!(out.completed && out.bytes_written > 0); @@ -3104,25 +3167,23 @@ mod tests { /// session muxes instead of failing DeviceNotReady. #[test] fn mux_with_keys_session_keeps_its_reader_when_the_selection_is_refused() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); let key = [0x5A; 16]; let (reader, title, _) = keyed_live(key); let mut disc = aacs_session_disc(title); disc.capacity_sectors = 16; - let set = crate::keys::ResolvedKeySet::keyed_for_test(&disc, key, &[(0, 3)]); + let set = crate::keys::KeyRing::keyed_for_test(&disc, key, &[(0, 3)]); let mut session = DiscSession::from_parts_for_test(Some(disc), Some(reader)); let mut bad = keyed_opts(); bad.selection.audio = crate::mux::select::PidFilter::Only(vec![0x0FFF]); let run = |session: &mut DiscSession, opts: &MuxOptions| { mux_with_keys( - MuxSource::Session { - session, - title_index: 0, - }, + Source::from_session(session), Some(&set), "null://", opts, - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) }; let err = run(&mut session, &bad).expect_err("unknown PID is refused"); @@ -3137,6 +3198,8 @@ mod tests { /// A key set for another disc is a typed E7013 on a session, never a debug-build panic. #[test] fn mux_with_keys_session_wrong_disc_set_is_e7013() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); let key = [0x5A; 16]; let (reader, title, _) = keyed_live(key); let mut other = aacs_session_disc(title.clone()); @@ -3145,18 +3208,14 @@ mod tests { if let Some(a) = other.aacs.as_mut() { a.disc_hash = "0xdef".into(); } - let set = crate::keys::ResolvedKeySet::keyed_for_test(&other, key, &[(0, 3)]); + let set = crate::keys::KeyRing::keyed_for_test(&other, key, &[(0, 3)]); let mut session = DiscSession::from_parts_for_test(Some(disc), Some(reader)); let err = mux_with_keys( - MuxSource::Session { - session: &mut session, - title_index: 0, - }, + Source::from_session(&mut session), Some(&set), "null://", &keyed_opts(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .expect_err("a set for another disc is refused"); assert!(err.to_string().contains("E7013"), "got: {err}"); @@ -3176,16 +3235,14 @@ mod tests { std::fs::write(&path, &image).unwrap(); let out_path = dir.path().join("iso.mkv"); let out = mux_with_keys( - MuxSource::Iso { - path: &path, - title, - format: crate::disc::ContentFormat::BdTs, - }, + Source::from_image( + &path, + ScannedTitle::new(title, crate::disc::ContentFormat::BdTs), + ), Some(&set), &format!("mkv://{}", out_path.display()), &keyed_opts(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .expect("no rescan: the scanned title is muxed as given"); assert!(out.completed); @@ -3194,16 +3251,12 @@ mod tests { "the muxed audio is the plaintext" ); let url = format!("iso://{}", path.display()); - let rescanned = mux_with_keys( - MuxSource::Url { - url: &url, - opts: InputOptions::default(), - }, + let rescanned = mux_url( + &url, Some(&set), "null://", &keyed_opts(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ); // The scan reads the UDF anchor (LBA 256) past this 16-sector image's end. let err = rescanned.expect_err("a URL source scans the image, which has no filesystem"); @@ -3221,8 +3274,8 @@ mod tests { fn mux_with_keys_without_an_aacs_set_refuses_aacs_content() { let _serial = crate::sector::prefetched::holder_test_lock(); let key = [0x5A; 16]; - let none = crate::keys::ResolvedKeySet::none(); - let live = |keys: Option<&crate::keys::ResolvedKeySet>, raw: bool, unit: Vec| { + let none = crate::keys::KeyRing::none(); + let live = |keys: Option<&crate::keys::KeyRing>, raw: bool, unit: Vec| { let (_, title, _) = keyed_live(key); let reader = Box::new(AacsUnitReader { unit, capacity: 16 }); let opts = MuxOptions { @@ -3230,16 +3283,14 @@ mod tests { ..keyed_opts() }; mux_with_keys( - MuxSource::Live { + Source::from_reader( reader, - title, - format: crate::disc::ContentFormat::BdTs, - }, + ScannedTitle::new(title, crate::disc::ContentFormat::BdTs), + ), keys, "null://", &opts, - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) }; let code = |r: std::io::Result| r.err().and_then(|e| crate::error_code(&e)); @@ -3266,16 +3317,14 @@ mod tests { image.resize(16 * 2048, 0); std::fs::write(&path, &image).unwrap(); let iso = mux_with_keys( - MuxSource::Iso { - path: &path, - title, - format: crate::disc::ContentFormat::BdTs, - }, + Source::from_image( + &path, + ScannedTitle::new(title, crate::disc::ContentFormat::BdTs), + ), None, "null://", &keyed_opts(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ); assert_eq!(code(iso), e7022); @@ -3295,15 +3344,11 @@ mod tests { }); let mut session = DiscSession::from_parts_for_test(Some(disc), Some(reader)); let r = mux_with_keys( - MuxSource::Session { - session: &mut session, - title_index: 0, - }, + Source::from_session(&mut session), None, "null://", &keyed_opts(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ); let err = r.expect_err("no key, no mux"); assert_eq!(crate::error_code(&err), e7022, "{format:?}"); @@ -3325,7 +3370,7 @@ mod tests { tempfile::TempDir, std::path::PathBuf, DiscTitle, - crate::keys::ResolvedKeySet, + crate::keys::KeyRing, ) { let (_, mut title, _) = keyed_live(key); let sectors = units * 3; @@ -3336,7 +3381,7 @@ mod tests { let mut disc = aacs_session_disc(title.clone()); disc.aacs.as_mut().unwrap().bus_encryption = true; disc.capacity_sectors = sectors + 16; - let set = crate::keys::ResolvedKeySet::keyed_for_test(&disc, key, &[(0, sectors)]); + let set = crate::keys::KeyRing::keyed_for_test(&disc, key, &[(0, sectors)]); let mut image = encrypted_audio_unit(&key).repeat(units as usize); damage(&mut image); image.resize((sectors + 16) as usize * 2048, 0); @@ -3349,23 +3394,21 @@ mod tests { fn mux_damaged_iso( path: &std::path::Path, title: DiscTitle, - set: &crate::keys::ResolvedKeySet, + set: &crate::keys::KeyRing, dest: &str, ) -> std::io::Result { mux_with_keys( - MuxSource::Iso { + Source::from_image( path, - title, - format: crate::disc::ContentFormat::BdTs, - }, + ScannedTitle::new(title, crate::disc::ContentFormat::BdTs), + ), Some(set), dest, &MuxOptions { batch_sectors: 64, ..keyed_opts() }, - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) } @@ -3482,28 +3525,24 @@ mod tests { ..keyed_opts() }; let iso = mux_with_keys( - MuxSource::Iso { - path: &path, - title: title.clone(), - format: crate::disc::ContentFormat::BdTs, - }, + Source::from_image( + &path, + ScannedTitle::new(title.clone(), crate::disc::ContentFormat::BdTs), + ), Some(&set), "null://", &opts, - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ); let live = mux_with_keys( - MuxSource::Live { - reader: Box::new(ImageReader(std::fs::read(&path).unwrap())), - title, - format: crate::disc::ContentFormat::BdTs, - }, + Source::from_reader( + Box::new(ImageReader(std::fs::read(&path).unwrap())), + ScannedTitle::new(title, crate::disc::ContentFormat::BdTs), + ), Some(&set), "null://", &opts, - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ); [iso, live] } @@ -3639,15 +3678,14 @@ mod tests { title } - fn clear_live(units: usize, bad: Option) -> MuxSource<'static> { - MuxSource::Live { - reader: Box::new(BadSectorImage { + fn clear_live(units: usize, bad: Option) -> Source<'static> { + Source::from_reader( + Box::new(BadSectorImage { data: clear_audio_image(units), bad, }), - title: clear_title(units), - format: crate::disc::ContentFormat::BdTs, - } + ScannedTitle::new(clear_title(units), crate::disc::ContentFormat::BdTs), + ) } // `raw` routes a clear Live source through mux_unkeyed; otherwise mux_keyed runs it @@ -3667,18 +3705,27 @@ mod tests { info: DiscTitle, halt: Halt, } - impl Stream for WedgedFinish { + impl crate::pes::PesSource for WedgedFinish { fn read(&mut self) -> std::io::Result> { Ok(None) } + + fn info(&self) -> &DiscTitle { + &self.info + } + } + + impl crate::pes::PesSink for WedgedFinish { fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { self.halt.cancel(); std::thread::sleep(Duration::from_secs(9)); Ok(()) } + fn finish(&mut self) -> std::io::Result<()> { Ok(()) } + fn info(&self) -> &DiscTitle { &self.info } @@ -3691,16 +3738,24 @@ mod tests { }; TEST_SINK.with(|s| *s.borrow_mut() = Some(Box::new(sink))); let t = std::time::Instant::now(); - let out = drive_mux(Box::new(fs), "null://", &halt, &NoopEvents, None, None) - .expect("a wedged finalise is an incomplete run, not an error"); + let out = drive_mux( + Box::new(fs), + "null://", + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect("a wedged finalise is an incomplete run, not an error"); TEST_SINK.with(|s| s.borrow_mut().take()); assert!(!out.completed && out.output_opened); assert!(t.elapsed() < Duration::from_secs(8), "bounded by the grace"); } - // skip_errors reaches the live DiscStream on every arm that builds one. + // skip_errors reaches the live Read stage on every arm that builds one. #[test] fn skip_errors_reaches_every_live_arm() { + // A live mux spawns the prefetch producer, a Drive holder. + let _serial = crate::sector::prefetched::holder_test_lock(); let session = |bad| { let mut disc = aacs_session_disc(clear_title(2)); disc.aacs = None; @@ -3717,26 +3772,13 @@ mod tests { let (src, raw) = match arm { "live-unkeyed" => (clear_live(2, Some(4)), true), "live-keyed" => (clear_live(2, Some(4)), false), - _ => ( - MuxSource::Session { - session: &mut s, - title_index: 0, - }, - false, - ), + _ => (Source::from_session(&mut s), false), }; let opts = MuxOptions { skip_errors: skip, ..clear_opts(raw) }; - let res = mux_with_keys( - src, - None, - "null://", - &opts, - &Halt::new(), - Arc::new(NoopEvents), - ); + let res = mux_with_keys(src, None, "null://", &opts, &crate::ctx::Ctx::default()); match skip { true => { let out = res.expect("the bad sector is skipped"); @@ -3783,19 +3825,12 @@ mod tests { let (src, raw) = match arm { "live-unkeyed" => (clear_live(2, None), true), "live-keyed" => (clear_live(2, None), false), - "session" => ( - MuxSource::Session { - session: &mut session, - title_index: 0, - }, - false, - ), + "session" => (Source::from_session(&mut session), false), _ => ( - MuxSource::Iso { - path: &iso, - title: clear_title(2), - format: crate::disc::ContentFormat::BdTs, - }, + Source::from_image( + &iso, + ScannedTitle::new(clear_title(2), crate::disc::ContentFormat::BdTs), + ), arm == "iso-unkeyed", ), }; @@ -3804,8 +3839,14 @@ mod tests { ..clear_opts(raw) }; let spy = Arc::new(TitleSpy(std::sync::Mutex::new(None))); - mux_with_keys(src, None, "null://", &opts, &Halt::new(), spy.clone()) - .unwrap_or_else(|e| panic!("{arm}: {e}")); + mux_with_keys( + src, + None, + "null://", + &opts, + &crate::ctx::Ctx::default().with_events(spy.clone()), + ) + .unwrap_or_else(|e| panic!("{arm}: {e}")); let opened = spy.0.lock().unwrap().take().expect("opened"); assert_eq!(opened.streams.len(), 1, "{arm}"); } diff --git a/src/mux/fit.rs b/src/mux/fit.rs index 64ef23e9..24d3e515 100644 --- a/src/mux/fit.rs +++ b/src/mux/fit.rs @@ -5,7 +5,7 @@ //! The plan is a PREDICTION made before any frame moves. Exclusions that only the //! mux can see (a DVD MPEG-2 multichannel extension track whose packets arrive, an //! mp4 track with no sample) are reported post-mux through -//! [`Stream::undelivered_streams`](crate::pes::Stream::undelivered_streams) and +//! [`Stream::undelivered_streams`](crate::pes::PesSink::undelivered_streams) and //! [`Mp4Sink::final_report`](super::mp4::Mp4Sink::final_report), never here: a //! declared-only extension track is never planned out (J23). diff --git a/src/mux/fmp4/mod.rs b/src/mux/fmp4/mod.rs deleted file mode 100644 index 26400844..00000000 --- a/src/mux/fmp4/mod.rs +++ /dev/null @@ -1,729 +0,0 @@ -//! Fragmented MP4 muxer — **STUB**: fragment emission is not implemented. -//! -//! Goal: ISO/IEC 14496-12 fragmented MP4 (`ftyp` + `moov` init segment, then a sequence of -//! `moof+mdat` media fragments) targeting a -//! [`SequentialSink`](crate::io::sink::SequentialSink). DASH-friendly, no Cues backpatch. Emits -//! the init segment via [`Fmp4Mux::write_init_segment`]; [`Fmp4Mux::write_video`] returns -//! [`Error::Fmp4Unimplemented`](crate::error::Error::Fmp4Unimplemented). - -use std::io::{self, Write}; - -// Box type literals — four-character codes per ISO/IEC 14496-12 §4.2. - -const FTYP: [u8; 4] = *b"ftyp"; -const MOOV: [u8; 4] = *b"moov"; -const MVHD: [u8; 4] = *b"mvhd"; -const TRAK: [u8; 4] = *b"trak"; -const TKHD: [u8; 4] = *b"tkhd"; -const MDIA: [u8; 4] = *b"mdia"; -const MDHD: [u8; 4] = *b"mdhd"; -const HDLR: [u8; 4] = *b"hdlr"; -const MINF: [u8; 4] = *b"minf"; -const VMHD: [u8; 4] = *b"vmhd"; -const DINF: [u8; 4] = *b"dinf"; -const DREF: [u8; 4] = *b"dref"; -const URL_: [u8; 4] = *b"url "; -const STBL: [u8; 4] = *b"stbl"; -const STSD: [u8; 4] = *b"stsd"; -const STTS: [u8; 4] = *b"stts"; -const STSC: [u8; 4] = *b"stsc"; -const STSZ: [u8; 4] = *b"stsz"; -const STCO: [u8; 4] = *b"stco"; -const MVEX: [u8; 4] = *b"mvex"; -const TREX: [u8; 4] = *b"trex"; - -/// Default movie timescale — 90 kHz lines up with MPEG-TS PTS and the -/// HEVC SPS `vui_time_scale` for film content, simplifying the math -/// when fragment emission lands. -const MOVIE_TIMESCALE: u32 = 90_000; -/// Video track ID. fMP4 init segments conventionally use track_ID=1 -/// for the primary video track; a single-track DASH representation has -/// no reason to deviate. -const VIDEO_TRACK_ID: u32 = 1; - -/// Fragmented MP4 muxer — stub. -/// -/// See the module-level doc comment for what is and isn't shipped in -/// this stub. Fragment emission is not implemented: -/// [`write_video`](Self::write_video) returns -/// [`Error::Fmp4Unimplemented`](crate::error::Error::Fmp4Unimplemented) -/// rather than discarding media. -pub struct Fmp4Mux { - writer: W, - header_written: bool, - /// hvcC bytes, if provided. Embedded in the `moov.…stsd.hvc1.hvcC` - /// box once the emission path lands. - #[allow(dead_code)] - codec_private: Option>, -} - -impl Fmp4Mux { - pub fn new(writer: W) -> Self { - Self { - writer, - header_written: false, - codec_private: None, - } - } - - /// Provide the `HEVCDecoderConfigurationRecord` for the video track. - /// The stub stores it but doesn't yet embed it in `moov` — that's - /// part of the unimplemented emission path. - pub fn set_video_codec_private(&mut self, hvcc: Vec) { - self.codec_private = Some(hvcc); - } - - /// Emit the init segment (`ftyp` + `moov`) once. Idempotent — a - /// second call is a no-op. Lets a consumer that just wants the - /// container shape obtain a valid (if sample-less) init segment. - pub fn write_init_segment(&mut self) -> io::Result<()> { - if self.header_written { - return Ok(()); - } - let ftyp = build_ftyp(); - let moov = build_moov(); - self.writer.write_all(&ftyp)?; - self.writer.write_all(&moov)?; - self.header_written = true; - Ok(()) - } - - /// Write one video PES frame. - /// - /// **Stub:** `moof`/`mdat` emission is not implemented. To avoid - /// silently dropping media (and avoid unbounded buffering), this - /// emits the init segment on the first call and then returns - /// [`Error::Fmp4Unimplemented`](crate::error::Error::Fmp4Unimplemented). - /// No frame bytes are buffered or written. - pub fn write_video(&mut self, _pts_ns: i64, _keyframe: bool, _data: &[u8]) -> io::Result<()> { - self.write_init_segment()?; - Err(crate::error::Error::Fmp4Unimplemented.into()) - } - - /// Flush the underlying writer. - pub fn finish(&mut self) -> io::Result<()> { - self.writer.flush() - } -} - -/// Build the `ftyp` box. `major_brand = "iso6"`, `minor_version = 1`, -/// compatible brands `iso6 dash msdh hvc1` — the same conservative set -/// shaka-packager uses for HEVC-in-fMP4 outputs. -fn build_ftyp() -> Vec { - let mut body = Vec::new(); - body.extend_from_slice(b"iso6"); - body.extend_from_slice(&1u32.to_be_bytes()); - body.extend_from_slice(b"iso6"); - body.extend_from_slice(b"dash"); - body.extend_from_slice(b"msdh"); - body.extend_from_slice(b"hvc1"); - wrap_box(&FTYP, &body) -} - -/// Build the `moov` box — minimal skeleton. Single video trak, no -/// hvcC inside stsd yet (stub: hvcC/hvc1 stsd entry not yet implemented). -fn build_moov() -> Vec { - let mvhd = build_mvhd(); - let trak = build_video_trak(); - let mvex = build_mvex(); - let mut body = Vec::new(); - body.extend_from_slice(&mvhd); - body.extend_from_slice(&trak); - body.extend_from_slice(&mvex); - wrap_box(&MOOV, &body) -} - -fn build_mvhd() -> Vec { - // Version 0, 100 bytes total body. Fields per ISO/IEC 14496-12 §8.2.2. - let mut body = Vec::new(); - body.extend_from_slice(&[0, 0, 0, 0]); // version + flags - body.extend_from_slice(&0u32.to_be_bytes()); // creation_time - body.extend_from_slice(&0u32.to_be_bytes()); // modification_time - body.extend_from_slice(&MOVIE_TIMESCALE.to_be_bytes()); - body.extend_from_slice(&0u32.to_be_bytes()); // duration = 0 (fragmented) - body.extend_from_slice(&0x0001_0000u32.to_be_bytes()); // rate 1.0 - body.extend_from_slice(&0x0100u16.to_be_bytes()); // volume 1.0 - body.extend_from_slice(&[0u8; 2]); // reserved - body.extend_from_slice(&[0u8; 8]); // reserved - // 3x3 identity transformation matrix in 16.16 fixed point. - for v in [0x1_0000u32, 0, 0, 0, 0x1_0000, 0, 0, 0, 0x4000_0000] { - body.extend_from_slice(&v.to_be_bytes()); - } - body.extend_from_slice(&[0u8; 24]); // pre_defined[6] - body.extend_from_slice(&2u32.to_be_bytes()); // next_track_ID (1 reserved for video) - wrap_box(&MVHD, &body) -} - -fn build_video_trak() -> Vec { - let tkhd = build_tkhd(); - let mdia = build_mdia(); - let mut body = Vec::new(); - body.extend_from_slice(&tkhd); - body.extend_from_slice(&mdia); - wrap_box(&TRAK, &body) -} - -fn build_tkhd() -> Vec { - let mut body = Vec::new(); - // version=0 | flags=0x000007 (track_enabled | in_movie | in_preview) - body.extend_from_slice(&[0, 0, 0, 7]); - body.extend_from_slice(&0u32.to_be_bytes()); // creation_time - body.extend_from_slice(&0u32.to_be_bytes()); // modification_time - body.extend_from_slice(&VIDEO_TRACK_ID.to_be_bytes()); - body.extend_from_slice(&[0u8; 4]); // reserved - body.extend_from_slice(&0u32.to_be_bytes()); // duration - body.extend_from_slice(&[0u8; 8]); // reserved - body.extend_from_slice(&0u16.to_be_bytes()); // layer - body.extend_from_slice(&0u16.to_be_bytes()); // alternate_group - body.extend_from_slice(&0u16.to_be_bytes()); // volume (video=0) - body.extend_from_slice(&[0u8; 2]); // reserved - // 3x3 identity matrix. - for v in [0x1_0000u32, 0, 0, 0, 0x1_0000, 0, 0, 0, 0x4000_0000] { - body.extend_from_slice(&v.to_be_bytes()); - } - // width / height in 16.16 fixed point — placeholder; replace with - // SPS-derived dimensions (and matching stsd visual width/height) when - // fragment emission lands. - body.extend_from_slice(&(1920u32 << 16).to_be_bytes()); - body.extend_from_slice(&(1080u32 << 16).to_be_bytes()); - wrap_box(&TKHD, &body) -} - -fn build_mdia() -> Vec { - let mdhd = build_mdhd(); - let hdlr = build_hdlr_vide(); - let minf = build_minf(); - let mut body = Vec::new(); - body.extend_from_slice(&mdhd); - body.extend_from_slice(&hdlr); - body.extend_from_slice(&minf); - wrap_box(&MDIA, &body) -} - -fn build_mdhd() -> Vec { - let mut body = Vec::new(); - body.extend_from_slice(&[0, 0, 0, 0]); // version + flags - body.extend_from_slice(&0u32.to_be_bytes()); // creation_time - body.extend_from_slice(&0u32.to_be_bytes()); // modification_time - body.extend_from_slice(&MOVIE_TIMESCALE.to_be_bytes()); - body.extend_from_slice(&0u32.to_be_bytes()); // duration - // language: 'und' in 5-bit-per-char ISO 639-2 packed (bit 15 = 0). - body.extend_from_slice(&[0x55, 0xC4]); - body.extend_from_slice(&0u16.to_be_bytes()); // pre_defined - wrap_box(&MDHD, &body) -} - -fn build_hdlr_vide() -> Vec { - let mut body = Vec::new(); - body.extend_from_slice(&[0, 0, 0, 0]); // version + flags - body.extend_from_slice(&0u32.to_be_bytes()); // pre_defined - body.extend_from_slice(b"vide"); - body.extend_from_slice(&[0u8; 12]); // reserved - body.extend_from_slice(b"VideoHandler\0"); - wrap_box(&HDLR, &body) -} - -fn build_minf() -> Vec { - let vmhd = build_vmhd(); - let dinf = build_dinf(); - let stbl = build_stbl(); - let mut body = Vec::new(); - body.extend_from_slice(&vmhd); - body.extend_from_slice(&dinf); - body.extend_from_slice(&stbl); - wrap_box(&MINF, &body) -} - -fn build_vmhd() -> Vec { - let mut body = Vec::new(); - body.extend_from_slice(&[0, 0, 0, 1]); // version + flags=1 - body.extend_from_slice(&0u16.to_be_bytes()); // graphicsmode - body.extend_from_slice(&[0u8; 6]); // opcolor - wrap_box(&VMHD, &body) -} - -fn build_dinf() -> Vec { - let mut dref_body = Vec::new(); - dref_body.extend_from_slice(&[0, 0, 0, 0]); - dref_body.extend_from_slice(&1u32.to_be_bytes()); // entry_count - // url with flags=1 (self-contained) and zero name. - let url_body = [0u8, 0, 0, 1]; - dref_body.extend_from_slice(&wrap_box(&URL_, &url_body)); - let dref = wrap_box(&DREF, &dref_body); - wrap_box(&DINF, &dref) -} - -fn build_stbl() -> Vec { - // Stub stsd: zero entries. Must land together with build_mvex's trex — - // once hvc1+hvcC is added here (entry_count=1), trex's - // default_sample_description_index=1 becomes valid. - let mut stsd_body = Vec::new(); - stsd_body.extend_from_slice(&[0, 0, 0, 0]); - stsd_body.extend_from_slice(&0u32.to_be_bytes()); // entry_count - let stsd = wrap_box(&STSD, &stsd_body); - - // Empty stts/stsc/stsz/stco — fragmented init has no samples here. - let stts = wrap_box(&STTS, &[0, 0, 0, 0, 0, 0, 0, 0]); // version+flags, count=0 - let stsc = wrap_box(&STSC, &[0, 0, 0, 0, 0, 0, 0, 0]); - let stsz = wrap_box( - &STSZ, - &[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], // version+flags, sample_size=0, count=0 - ); - let stco = wrap_box(&STCO, &[0, 0, 0, 0, 0, 0, 0, 0]); - - let mut body = Vec::new(); - body.extend_from_slice(&stsd); - body.extend_from_slice(&stts); - body.extend_from_slice(&stsc); - body.extend_from_slice(&stsz); - body.extend_from_slice(&stco); - wrap_box(&STBL, &body) -} - -fn build_mvex() -> Vec { - // trex: track_ID=1, default_sample_description_index=1, others=0. - // dsdi=1 is only valid once build_stbl's stsd has the matching hvc1 - // entry (entry_count=1) — keep the two in sync. - let mut trex_body = Vec::new(); - trex_body.extend_from_slice(&[0, 0, 0, 0]); // version + flags - trex_body.extend_from_slice(&VIDEO_TRACK_ID.to_be_bytes()); - trex_body.extend_from_slice(&1u32.to_be_bytes()); // default_sample_description_index - trex_body.extend_from_slice(&0u32.to_be_bytes()); // default_sample_duration - trex_body.extend_from_slice(&0u32.to_be_bytes()); // default_sample_size - trex_body.extend_from_slice(&0u32.to_be_bytes()); // default_sample_flags - let trex = wrap_box(&TREX, &trex_body); - wrap_box(&MVEX, &trex) -} - -// Wrap a box body in `[size:u32-BE][type:4]`; oversized boxes (>4 GiB) need the 64-bit -// large-size extension, not generated in the stub. -fn wrap_box(box_type: &[u8; 4], body: &[u8]) -> Vec { - let total = body.len() + 8; - debug_assert!(total <= u32::MAX as usize, "fMP4 box exceeds u32 size"); - let size = u32::try_from(total).unwrap_or(u32::MAX); - let mut out = Vec::with_capacity(total); - out.extend_from_slice(&size.to_be_bytes()); - out.extend_from_slice(box_type); - out.extend_from_slice(body); - out -} - -#[cfg(test)] -mod tests { - use super::*; - - /// Decode the first box's size + type from `buf`. - fn read_box_header(buf: &[u8]) -> (u32, [u8; 4]) { - let size = u32::from_be_bytes([buf[0], buf[1], buf[2], buf[3]]); - let bt = [buf[4], buf[5], buf[6], buf[7]]; - (size, bt) - } - - #[test] - fn init_segment_starts_with_ftyp_then_moov() { - let mut sink: Vec = Vec::new(); - let mut mux = Fmp4Mux::new(&mut sink); - mux.write_init_segment().unwrap(); - mux.finish().unwrap(); - drop(mux); - - let (ftyp_size, ftyp_type) = read_box_header(&sink); - assert_eq!(&ftyp_type, b"ftyp"); - assert!(ftyp_size >= 24, "ftyp too small: {ftyp_size}"); - - let (moov_size, moov_type) = read_box_header(&sink[ftyp_size as usize..]); - assert_eq!(&moov_type, b"moov"); - assert!(moov_size > 100, "moov skeleton too small: {moov_size}"); - - // Stub guarantee: no media bytes after the init segment. - let total = ftyp_size as usize + moov_size as usize; - assert_eq!(sink.len(), total, "stub leaked media bytes past moov"); - } - - #[test] - fn write_video_reports_unimplemented_and_buffers_nothing() { - // write_video must NOT silently accept-and-drop media: it emits the - // init segment, then signals that fragment emission is unimplemented. - let mut sink: Vec = Vec::new(); - let mut mux = Fmp4Mux::new(&mut sink); - let err = mux.write_video(0, true, &[0xDE; 4096]).unwrap_err(); - assert_eq!(err.kind(), io::ErrorKind::Unsupported); - mux.finish().unwrap(); - drop(mux); - // Only the init segment (ftyp + moov) was written — no media bytes. - let (ftyp_size, _) = read_box_header(&sink); - let (moov_size, _) = read_box_header(&sink[ftyp_size as usize..]); - assert_eq!(sink.len(), ftyp_size as usize + moov_size as usize); - } - - #[test] - fn moov_contains_trak_mvex() { - let mut buf: Vec = Vec::new(); - let mut mux2 = Fmp4Mux::new(&mut buf); - mux2.write_init_segment().unwrap(); - mux2.finish().unwrap(); - drop(mux2); - - // Find moov payload start. - let (ftyp_size, _) = read_box_header(&buf); - let moov_start = ftyp_size as usize; - let (moov_size, _) = read_box_header(&buf[moov_start..]); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size as usize]; - - // Scan for the trak and mvex four-CC anywhere in the moov payload. - let has_trak = moov_payload.windows(4).any(|w| w == b"trak"); - let has_mvex = moov_payload.windows(4).any(|w| w == b"mvex"); - assert!(has_trak, "moov missing trak"); - assert!(has_mvex, "moov missing mvex"); - } - - // ISO/IEC 14496-12 box-tree invariants: every box is [size:u32-BE][type:4] - // [body], size covers the header too. A wrong size silently desyncs every - // ISO BMFF/DASH parser, so these tests walk the tree byte-exactly. - - /// Walk a flat sequence of top-level boxes, returning - /// (type, box_start, box_total_size). Asserts each declared size lands - /// exactly on a box boundary (no overlap, no gap, no overrun). - fn walk_boxes(buf: &[u8]) -> Vec<([u8; 4], usize, usize)> { - let mut out = Vec::new(); - let mut pos = 0; - while pos + 8 <= buf.len() { - let (size, bt) = read_box_header(&buf[pos..]); - let size = size as usize; - assert!(size >= 8, "box {bt:?} size {size} < 8-byte header"); - assert!( - pos + size <= buf.len(), - "box {bt:?} at {pos} size {size} overruns buffer {}", - buf.len() - ); - out.push((bt, pos, size)); - pos += size; - } - assert_eq!(pos, buf.len(), "boxes did not tile the buffer exactly"); - out - } - - /// Find the immediate child box of the given type within a container's - /// payload (the bytes after the 8-byte header). Returns the child's full - /// box slice. Recurses one level only. - fn child<'a>(container_payload: &'a [u8], want: &[u8; 4]) -> Option<&'a [u8]> { - let mut pos = 0; - while pos + 8 <= container_payload.len() { - let (size, bt) = read_box_header(&container_payload[pos..]); - let size = size as usize; - if size < 8 || pos + size > container_payload.len() { - return None; - } - if &bt == want { - return Some(&container_payload[pos..pos + size]); - } - pos += size; - } - None - } - - fn init_segment() -> Vec { - let mut buf: Vec = Vec::new(); - let mut mux = Fmp4Mux::new(&mut buf); - mux.write_init_segment().unwrap(); - mux.finish().unwrap(); - drop(mux); - buf - } - - #[test] - fn init_segment_box_sizes_tile_exactly() { - // Top level must be exactly [ftyp][moov] with no slack. - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let types: Vec<[u8; 4]> = boxes.iter().map(|(t, _, _)| *t).collect(); - assert_eq!(types, vec![*b"ftyp", *b"moov"]); - } - - #[test] - fn ftyp_major_brand_and_compatible_brands() { - // ISO/IEC 14496-12 §4.3 ftyp layout: major_brand(4)+minor_version(4)+ - // compatible_brands[]. A dropped/reordered brand breaks DASH negotiation. - let buf = init_segment(); - let (ftyp_size, _) = read_box_header(&buf); - let body = &buf[8..ftyp_size as usize]; - assert_eq!(&body[0..4], b"iso6", "major_brand"); - assert_eq!( - u32::from_be_bytes([body[4], body[5], body[6], body[7]]), - 1, - "minor_version" - ); - // Remaining bytes are 4-byte compatible brands. - let brands = &body[8..]; - assert_eq!(brands.len() % 4, 0, "compatible_brands must be 4-byte each"); - let set: Vec<&[u8]> = brands.chunks(4).collect(); - assert!(set.contains(&&b"iso6"[..])); - assert!(set.contains(&&b"dash"[..])); - assert!(set.contains(&&b"msdh"[..])); - assert!(set.contains(&&b"hvc1"[..]), "HEVC brand required for hvc1"); - } - - #[test] - fn moov_child_order_is_mvhd_trak_mvex() { - // §8.1: moov contains mvhd then track(s) then mvex (for fragmented). - // Order matters for some strict parsers; assert the exact child - // sequence rather than mere presence. - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let (_, moov_start, moov_size) = boxes.iter().find(|(t, _, _)| t == b"moov").unwrap(); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size]; - let children = walk_boxes(moov_payload); - let types: Vec<[u8; 4]> = children.iter().map(|(t, _, _)| *t).collect(); - assert_eq!(types, vec![*b"mvhd", *b"trak", *b"mvex"]); - } - - #[test] - fn mvhd_timescale_and_next_track_id() { - // §8.2.2 mvhd v0: version+flags(4) creation(4) modification(4) - // timescale(4) duration(4) ... next_track_ID(4). Stub uses 90000 Hz - // and next_track_ID=2 (track 1 reserved for video). - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let (_, moov_start, moov_size) = boxes.iter().find(|(t, _, _)| t == b"moov").unwrap(); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size]; - let mvhd = child(moov_payload, b"mvhd").expect("mvhd present"); - let body = &mvhd[8..]; // skip box header - assert_eq!(&body[0..4], &[0, 0, 0, 0], "mvhd version 0, flags 0"); - // timescale is at body offset 12 (after version+flags, creation, mod). - let timescale = u32::from_be_bytes([body[12], body[13], body[14], body[15]]); - assert_eq!(timescale, MOVIE_TIMESCALE); - assert_eq!(timescale, 90_000, "spec-fixed default timescale"); - // next_track_ID is the last 4 bytes of the body. - let n = body.len(); - let next_id = u32::from_be_bytes([body[n - 4], body[n - 3], body[n - 2], body[n - 1]]); - assert_eq!(next_id, 2, "next_track_ID must exceed the sole track ID"); - } - - #[test] - fn trex_references_video_track_id() { - // §8.8.3 trex: track_ID must match the trak's track_ID (1) so the - // fragment defaults bind to the right track. A mismatch would make - // every future moof default-sample lookup target a non-existent track. - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let (_, moov_start, moov_size) = boxes.iter().find(|(t, _, _)| t == b"moov").unwrap(); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size]; - let mvex = child(moov_payload, b"mvex").expect("mvex"); - let trex = child(&mvex[8..], b"trex").expect("trex"); - let body = &trex[8..]; - // version+flags(4), then track_ID(4). - let track_id = u32::from_be_bytes([body[4], body[5], body[6], body[7]]); - assert_eq!(track_id, VIDEO_TRACK_ID); - assert_eq!(track_id, 1); - // default_sample_description_index(4) must be 1 (points at stsd entry 1). - let dsdi = u32::from_be_bytes([body[8], body[9], body[10], body[11]]); - assert_eq!(dsdi, 1); - } - - #[test] - fn tkhd_track_id_matches_trex() { - // §8.3.2 tkhd: the track_ID field (after version+flags, creation, - // modification) must equal VIDEO_TRACK_ID and the trex track_ID, or the - // fragment defaults never bind. tkhd is moov.trak.tkhd. - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let (_, moov_start, moov_size) = boxes.iter().find(|(t, _, _)| t == b"moov").unwrap(); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size]; - let trak = child(moov_payload, b"trak").expect("trak"); - let tkhd = child(&trak[8..], b"tkhd").expect("tkhd"); - let body = &tkhd[8..]; - // version(1)+flags(3), creation(4), modification(4), then track_ID(4). - let track_id = u32::from_be_bytes([body[12], body[13], body[14], body[15]]); - assert_eq!(track_id, VIDEO_TRACK_ID, "tkhd track_ID must match trex"); - // flags = 0x000007 (enabled | in_movie | in_preview), §8.3.1. - assert_eq!(&body[0..4], &[0, 0, 0, 7]); - } - - #[test] - fn stbl_present_with_empty_sample_tables() { - // Fragmented init segment carries no samples in moov: stsd entry_count - // is 0 and stts/stsc/stsz/stco are empty (current stub state). If stsd - // ever gains an hvc1 entry, trex's dsdi=1 becomes meaningful — see build_mvex. - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let (_, moov_start, moov_size) = boxes.iter().find(|(t, _, _)| t == b"moov").unwrap(); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size]; - let trak = child(moov_payload, b"trak").expect("trak"); - let mdia = child(&trak[8..], b"mdia").expect("mdia"); - let minf = child(&mdia[8..], b"minf").expect("minf"); - let stbl = child(&minf[8..], b"stbl").expect("stbl"); - let stsd = child(&stbl[8..], b"stsd").expect("stsd"); - let body = &stsd[8..]; - // version+flags(4), entry_count(4). - let entry_count = u32::from_be_bytes([body[4], body[5], body[6], body[7]]); - assert_eq!(entry_count, 0, "stub stsd has no sample entries yet"); - // All of stts/stsc/stsz/stco must be present children of stbl. - for fourcc in [b"stts", b"stsc", b"stsz", b"stco"] { - assert!( - child(&stbl[8..], fourcc).is_some(), - "stbl missing {:?}", - std::str::from_utf8(fourcc).unwrap() - ); - } - } - - #[test] - fn hdlr_declares_video_handler() { - // §8.4.3 hdlr: handler_type must be 'vide' for a video track, else - // players won't route the track to the video decoder. Path: - // moov.trak.mdia.hdlr; handler_type is at body offset 8. - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let (_, moov_start, moov_size) = boxes.iter().find(|(t, _, _)| t == b"moov").unwrap(); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size]; - let trak = child(moov_payload, b"trak").expect("trak"); - let mdia = child(&trak[8..], b"mdia").expect("mdia"); - let hdlr = child(&mdia[8..], b"hdlr").expect("hdlr"); - let body = &hdlr[8..]; - // version+flags(4), pre_defined(4), handler_type(4). - assert_eq!(&body[8..12], b"vide", "handler_type must be 'vide'"); - } - - // mdhd (§8.4.2), vmhd (§12.1.2), dinf/dref (§8.7.1-2) are mandatory - // in a video track's media tree; none is read back by this crate, - // so only the spec-fixed field values constrain them here. - #[test] - fn mdhd_vmhd_and_dinf_carry_the_values_the_spec_fixes() { - let buf = init_segment(); - let boxes = walk_boxes(&buf); - let (_, moov_start, moov_size) = boxes.iter().find(|(t, _, _)| t == b"moov").unwrap(); - let moov_payload = &buf[moov_start + 8..moov_start + moov_size]; - let trak = child(moov_payload, b"trak").expect("trak"); - let mdia = child(&trak[8..], b"mdia").expect("mdia"); - let minf = child(&mdia[8..], b"minf").expect("minf"); - - // ── mdhd. Version 0 layout: vflags(4) creation(4) modification(4) - // timescale(4) duration(4) language(2) pre_defined(2) = 24 bytes. - let mdhd = child(&mdia[8..], b"mdhd").expect("mdia must carry mdhd"); - let body = &mdhd[8..]; - assert_eq!(body[0], 0, "mdhd version 0 (32-bit times)"); - assert_eq!( - body.len(), - 24, - "version-0 mdhd body: vflags(4) creation(4) modification(4) timescale(4) duration(4) language(2) pre_defined(2)" - ); - let media_ts = u32::from_be_bytes([body[12], body[13], body[14], body[15]]); - assert_ne!( - media_ts, 0, - "every fragment's tfdt is expressed in this timescale; zero is a divide-by-zero" - ); - // The mvhd timescale is the movie clock; this single-track init has one - // clock, so the media timescale must agree with it rather than carry an - // independent copy that can drift. - let mvhd = child(moov_payload, b"mvhd").expect("mvhd"); - let mvhd_body = &mvhd[8..]; - let movie_ts = - u32::from_be_bytes([mvhd_body[12], mvhd_body[13], mvhd_body[14], mvhd_body[15]]); - assert_eq!(media_ts, movie_ts, "media clock must match the movie clock"); - - // language: ISO 639-2/T packed as three 5-bit values, bit 15 = 0 (§8.4.2). - let packed = u16::from_be_bytes([body[20], body[21]]); - assert_eq!(packed & 0x8000, 0, "language pad bit must be 0"); - let lang: String = (0..3) - .map(|i| (((packed >> (10 - 5 * i)) & 0x1F) as u8 + 0x60) as char) - .collect(); - assert_eq!(lang, "und", "unknown language is 'und', not empty or 'eng'"); - - // ── vmhd. §12.1.2 fixes flags to 1 and the stub picks the neutral - // compositing mode: graphicsmode 0 (copy) with a zero opcolor. - let vmhd = child(&minf[8..], b"vmhd").expect("minf must carry vmhd"); - let body = &vmhd[8..]; - assert_eq!(body[0], 0, "vmhd version 0"); - assert_eq!( - u32::from_be_bytes([0, body[1], body[2], body[3]]), - 1, - "vmhd flags must be 1" - ); - assert_eq!(body.len(), 12, "vmhd body: vflags(4) mode(2) opcolor(6)"); - assert_eq!( - u16::from_be_bytes([body[4], body[5]]), - 0, - "graphicsmode copy" - ); - assert_eq!(&body[6..12], &[0u8; 6], "opcolor is black"); - - // ── dinf > dref > 'url ' with flags 1: the media lives in this same - // file/segment, so no external location follows (§8.7.2). - let dinf = child(&minf[8..], b"dinf").expect("minf must carry dinf"); - let dref = child(&dinf[8..], b"dref").expect("dinf must carry dref"); - let body = &dref[8..]; - assert_eq!( - u32::from_be_bytes([body[4], body[5], body[6], body[7]]), - 1, - "dref entry_count" - ); - let url = child(&body[8..], b"url ").expect("dref must carry a 'url ' entry"); - assert_eq!( - u32::from_be_bytes([0, url[9], url[10], url[11]]), - 1, - "url flags=1 means self-contained" - ); - assert_eq!( - url.len(), - 12, - "a self-contained url is header(8) + vflags(4) and no location string" - ); - } - - #[test] - fn wrap_box_size_includes_header() { - // §4.2: a box's size field counts the full box including the 8-byte - // header. A body of N bytes yields size N+8 and the type at offset 4. - let body = [0xAAu8; 13]; - let boxed = wrap_box(b"test", &body); - assert_eq!(boxed.len(), 13 + 8); - let (size, bt) = read_box_header(&boxed); - assert_eq!(size as usize, 13 + 8, "size must include the 8-byte header"); - assert_eq!(&bt, b"test"); - assert_eq!(&boxed[8..], &body); - // Empty body → just the 8-byte header. - let empty = wrap_box(b"free", &[]); - assert_eq!(empty.len(), 8); - assert_eq!( - u32::from_be_bytes([empty[0], empty[1], empty[2], empty[3]]), - 8 - ); - } - - #[test] - fn write_init_segment_is_idempotent() { - // The doc contract says a second write_init_segment is a no-op. A - // regression that re-emitted ftyp+moov would produce two init segments - // and corrupt the stream. - let mut buf: Vec = Vec::new(); - let mut mux = Fmp4Mux::new(&mut buf); - mux.write_init_segment().unwrap(); - mux.write_init_segment().unwrap(); // second call must be a no-op - mux.finish().unwrap(); - drop(mux); - // Exactly one ftyp + one moov. - let boxes = walk_boxes(&buf); - let ftyp_count = boxes.iter().filter(|(t, _, _)| t == b"ftyp").count(); - let moov_count = boxes.iter().filter(|(t, _, _)| t == b"moov").count(); - assert_eq!(ftyp_count, 1, "second write_init_segment must be a no-op"); - assert_eq!(moov_count, 1); - } - - #[test] - fn write_video_after_init_still_unimplemented_and_no_media() { - // Even after init segment emission, write_video must keep returning - // Unimplemented and append no media bytes (no moof/mdat), so a caller - // can't be fooled into thinking the call succeeded. - let mut buf: Vec = Vec::new(); - let mut mux = Fmp4Mux::new(&mut buf); - mux.write_init_segment().unwrap(); - let err = mux.write_video(0, true, &[0u8; 8]).unwrap_err(); - assert_eq!(err.kind(), io::ErrorKind::Unsupported); - mux.finish().unwrap(); - drop(mux); - // Still only ftyp + moov. - let boxes = walk_boxes(&buf); - let types: Vec<[u8; 4]> = boxes.iter().map(|(t, _, _)| *t).collect(); - assert_eq!(types, vec![*b"ftyp", *b"moov"]); - } -} diff --git a/src/mux/fvi_pipeline_tests.rs b/src/mux/fvi_pipeline_tests.rs index 9e83b001..41f7446e 100644 --- a/src/mux/fvi_pipeline_tests.rs +++ b/src/mux/fvi_pipeline_tests.rs @@ -12,7 +12,7 @@ use crate::disc::{ VideoStream, }; use crate::mux::resolve::build_iso_pipeline; -use crate::pes::Stream as PesStream; +use crate::pes::PesSource; use crate::{Medium, SectorSource, SourceInfo, output}; use std::path::PathBuf; @@ -179,8 +179,7 @@ fn run_to_fvi(image: Vec, title: DiscTitle, path: &std::path::Path) { 3, // 3-sector (one AACS unit) batches → one source stamp per GOP region ContentFormat::MpegPs, false, - None, - None, + &crate::ctx::Ctx::default(), ) .expect("pipeline builds"); diff --git a/src/mux/fvi_sink.rs b/src/mux/fvi_sink.rs index 69911cfb..ad5d89f6 100644 --- a/src/mux/fvi_sink.rs +++ b/src/mux/fvi_sink.rs @@ -1,6 +1,6 @@ //! `fvi://` sink — freemkv's own native video-index output. //! -//! A write-only [`crate::pes::Stream`] that emits one machine-readable +//! A [`crate::pes::PesSink`] that emits one machine-readable //! *video-index* record per coded picture of the title's primary video //! track, instead of muxing frames into a container. //! @@ -12,7 +12,7 @@ use crate::mux::videomap::{ FVI_FORMAT, FVI_GENERATOR, FVI_SECTOR_SIZE, FVI_TIMESCALE, FVI_VERSION, MapHeader, PictureRecord, SourceInfo, field_order_label, is_random_access, type_label, }; -use crate::pes::{PesFrame, Stream}; +use crate::pes::{PesFrame, PesSink}; use std::fs::File; use std::io::{self, BufWriter, Write}; use std::path::Path; @@ -164,12 +164,7 @@ impl FviSink { } } -impl Stream for FviSink { - fn read(&mut self) -> io::Result> { - // Write-only sink, per the Stream trait contract. - Err(crate::error::Error::StreamWriteOnly.into()) - } - +impl PesSink for FviSink { fn write(&mut self, frame: &PesFrame) -> io::Result<()> { // Only index pictures of the primary video track. Audio / subtitle / // secondary-video frames carry no PictureInfo and are not part of the @@ -343,19 +338,6 @@ mod tests { assert_eq!(n, [0, 1]); } - #[test] - fn sink_is_write_only() { - let dir = tempfile::tempdir().unwrap(); - let mut sink = FviSink::create( - &dir.path().join("x.fvi"), - &mpeg2_title(), - SourceInfo::default(), - ) - .unwrap(); - let err = Stream::read(&mut sink).expect_err("read must error"); - assert_eq!(err.kind(), io::ErrorKind::Unsupported); - } - #[test] fn sink_writes_header_and_only_video_records() { let dir = tempfile::tempdir().unwrap(); diff --git a/src/mux/hevc/mod.rs b/src/mux/hevc/mod.rs index 4b620e0e..3763cc30 100644 --- a/src/mux/hevc/mod.rs +++ b/src/mux/hevc/mod.rs @@ -1,108 +1,10 @@ -//! HEVC (H.265) elementary stream muxer — Annex B byte stream. -//! -//! Consumes [`PesFrame`](crate::pes::PesFrame)s for a single video track and writes them as a -//! raw `.hevc` / `.h265` Annex B byte stream with no container framing. -//! -//! Sequential-only — no Cues, no backpatch. Target sink is any -//! [`SequentialSink`](crate::io::sink::SequentialSink): file, socket, -//! pipe, anything `Write + Send`. - -use std::io::{self, Write}; +//! H.264 / HEVC parameter-set helpers: decoder configuration records (`avcC` / `hvcC`) and +//! length-prefixed NAL units as an Annex B byte stream, for the elementary-stream sinks and +//! the TS muxer. /// Annex B 4-byte start code. pub(crate) const START_CODE: [u8; 4] = [0x00, 0x00, 0x00, 0x01]; -/// HEVC NAL unit type bits live in `(byte0 >> 1) & 0x3F` in Annex B. -/// We don't filter NAL types here — the muxer is format-only — but we -/// keep the constant as documentation of the field layout. -#[allow(dead_code)] -const HEVC_NAL_TYPE_MASK: u8 = 0x3F; - -/// Streaming HEVC Annex B muxer. -/// -/// One instance per output stream. Tracks whether parameter sets have -/// already been emitted so they're written exactly once at the head of -/// the stream, per the Annex B convention of ITU-T H.265 / ISO/IEC -/// 23008-2 (parameter sets precede the coded slices they govern). -pub struct HevcMux { - writer: W, - /// `HEVCDecoderConfigurationRecord` payload (hvcC). Parsed lazily - /// on the first `write_frame` so callers can set it after - /// construction but before the first frame. - codec_private: Option>, - /// Set once VPS/SPS/PPS have been written to the stream. Subsequent - /// frames write only their own NAL units. - params_written: bool, -} - -impl HevcMux { - /// Construct over `writer`. The muxer does not impose any extra - /// buffering of its own — the sink owns its write buffering policy - /// (see [`LocalFileSink`](crate::io::sink::LocalFileSink) and - /// [`SocketSink`](crate::io::sink::SocketSink)). - pub fn new(writer: W) -> Self { - Self { - writer, - codec_private: None, - params_written: false, - } - } - - /// Provide the `HEVCDecoderConfigurationRecord` (hvcC) so the muxer - /// can prepend VPS/SPS/PPS Annex B NALs at stream start. Optional — - /// if the PES frames already carry inline parameter sets (some - /// upstream demuxers do this), skipping this call is fine. - pub fn set_codec_private(&mut self, data: Vec) { - self.codec_private = Some(data); - } - - /// Write one PES frame (= one access unit) as Annex B NAL units. - /// - /// Input may be either length-prefixed (`[u32-BE len][NAL bytes]` repeated, the form - /// libfreemkv's HEVC parser emits) — converted to Annex B — or already Annex B (a buffer - /// beginning with a start code), passed through unchanged. - /// - /// `_pts_ns` is accepted for symmetry with other muxers but ignored - /// — Annex B has no timing layer. - pub fn write_frame(&mut self, _pts_ns: i64, data: &[u8]) -> io::Result<()> { - if !self.params_written { - // Mark written *before* the write: on a write error, a later - // re-entry must not re-emit VPS/SPS/PPS on top of bytes the sink - // already received. Callers discard the mux on any write error. - self.params_written = true; - if let Some(cp) = &self.codec_private { - match hvcc_to_annex_b(cp) { - Some(params) => self.writer.write_all(¶ms)?, - // A non-empty hvcC yielding no NAL is a caller contract - // violation: emitting without VPS/SPS/PPS is undecodable. - None if !cp.is_empty() => { - return Err(crate::error::Error::HevcParamParse.into()); - } - None => {} - } - } - } - // The hvcC declares the NAL length-prefix width (ISO/IEC 14496-15 - // §8.3.3.1.2 `lengthSizeMinusOne + 1`). Assuming 4 for a source that - // declares 1 or 2 emits the raw prefixed bytes with no start codes. - let length_size = nal_length_size(crate::disc::Codec::Hevc, self.codec_private.as_deref()); - let annex_b = if starts_with_start_code(data) { - data.to_vec() - } else { - let mut out = Vec::with_capacity(data.len() + (data.len() / 32)); - append_length_prefixed_as_annex_b_sized(&mut out, data, length_size); - out - }; - self.writer.write_all(&annex_b) - } - - /// Flush the underlying writer. No trailer NAL is needed — an Annex - /// B stream ends whenever the file/socket ends. - pub fn finish(&mut self) -> io::Result<()> { - self.writer.flush() - } -} - // Convert a HEVCDecoderConfigurationRecord (hvcC) into Annex B NAL units. Single source of // truth for hvcC -> Annex B across all muxers (HEVC ES, BD-TS, standard MPEG-TS) — do not // reimplement. @@ -164,6 +66,7 @@ fn hvcc_parse(hvcc: &[u8]) -> (Option>, bool) { // Convert length-prefixed NALs ([u32-BE len][NAL] repeated) to Annex B; already-Annex-B input // passes through unchanged. Truncation policy (shared across muxers): drop truncated trailing // NAL. +#[cfg(test)] pub(crate) fn length_prefixed_to_annex_b(data: &[u8]) -> Vec { // Probe for a leading Annex B start code before attempting to parse // length prefixes: `00 00 00 01` would otherwise parse as length 1. @@ -197,6 +100,7 @@ pub(crate) fn nal_length_size(codec: crate::disc::Codec, record: Option<&[u8]>) // Annex B form of length-prefixed `data`, written into caller-owned `out` to avoid an // allocation on hot per-frame paths. Non-length-prefixed input is appended unchanged (assumed // already Annex B). +#[cfg(test)] pub(crate) fn append_length_prefixed_as_annex_b(out: &mut Vec, data: &[u8]) { append_length_prefixed_as_annex_b_sized(out, data, DEFAULT_NAL_LENGTH_SIZE); } @@ -252,6 +156,7 @@ pub(crate) fn append_length_prefixed_as_annex_b_sized( /// Whether `data` begins with a 4-byte (`00 00 00 01`) or 3-byte /// (`00 00 01`) Annex B start code. +#[cfg(test)] fn starts_with_start_code(data: &[u8]) -> bool { data.starts_with(&START_CODE) || data.starts_with(&[0x00, 0x00, 0x01]) } @@ -437,17 +342,6 @@ mod tests { assert_eq!(&annex_b[..], &want[..]); } - #[test] - fn write_frame_errors_on_unparseable_non_empty_hvcc() { - // A non-empty hvcC that yields no NAL must surface an error - // instead of silently producing a parameter-set-less stream. - let mut sink: Vec = Vec::new(); - let mut mux = HevcMux::new(&mut sink); - mux.set_codec_private(vec![0xDE, 0xAD]); // too short to be valid hvcC - let err = mux.write_frame(0, &[]).unwrap_err(); - assert_eq!(err.kind(), io::ErrorKind::InvalidData); - } - #[test] fn zero_length_nal_is_skipped_not_bare_start_code() { // A zero-length prefix between two real NALs must be skipped, not @@ -744,151 +638,6 @@ mod tests { assert!(!starts_with_start_code(&[])); } - // --- HevcMux: params-once + error semantics --- - - #[test] - fn mux_empty_codec_private_emits_no_params() { - // An EMPTY (not absent) hvcC: hvcc_to_annex_b returns None, but cp is - // empty so it's NOT a contract violation → no error, just no params. - let mut sink: Vec = Vec::new(); - let mut mux = HevcMux::new(&mut sink); - mux.set_codec_private(Vec::new()); - let mut frame = 2u32.to_be_bytes().to_vec(); - frame.extend_from_slice(&[0xAA, 0xBB]); - mux.write_frame(0, &frame).unwrap(); - mux.finish().unwrap(); - // Only the frame NAL, no parameter sets. - let mut want = START_CODE.to_vec(); - want.extend_from_slice(&[0xAA, 0xBB]); - assert_eq!(sink, want); - } - - #[test] - fn mux_no_codec_private_writes_frames_only() { - // No hvcC set at all: frames pass through, no params, no error. - let mut sink: Vec = Vec::new(); - let mut mux = HevcMux::new(&mut sink); - let mut frame = 2u32.to_be_bytes().to_vec(); - frame.extend_from_slice(&[0xAA, 0xBB]); - mux.write_frame(0, &frame).unwrap(); - let mut want = START_CODE.to_vec(); - want.extend_from_slice(&[0xAA, 0xBB]); - assert_eq!(sink, want); - } - - #[test] - fn mux_params_not_re_emitted_after_unparseable_error() { - // params_written is set BEFORE the write, so after an error on the first - // frame, a retry must NOT re-emit params (the comment's invariant). - let mut sink: Vec = Vec::new(); - let mut mux = HevcMux::new(&mut sink); - mux.set_codec_private(vec![0xDE, 0xAD]); // unparseable, non-empty → error - assert!(mux.write_frame(0, &[]).is_err(), "first frame errors"); - // A second frame must not retry the (already-marked) params. - let mut frame = 2u32.to_be_bytes().to_vec(); - frame.extend_from_slice(&[0xAA, 0xBB]); - mux.write_frame(0, &frame).unwrap(); - // Sink holds only the frame NAL — no parameter bytes, no duplication. - let mut want = START_CODE.to_vec(); - want.extend_from_slice(&[0xAA, 0xBB]); - assert_eq!(sink, want, "params not re-emitted after the error"); - } - - #[test] - fn mux_annex_b_frame_passes_through() { - // A frame already in Annex B (leading start code) is written verbatim, - // not re-framed as length-prefixed. - let mut sink: Vec = Vec::new(); - let mut mux = HevcMux::new(&mut sink); - let frame = [0x00, 0x00, 0x00, 0x01, 0x26, 0x01, 0xDE]; - mux.write_frame(0, &frame).unwrap(); - assert_eq!(sink, frame); - } - - #[test] - fn mux_writes_params_then_frames() { - // Build hvcC with one SPS to verify params-once semantics. - let mut hvcc = vec![0u8; 22]; - hvcc.push(1); - hvcc.push(33); - hvcc.extend_from_slice(&1u16.to_be_bytes()); - hvcc.extend_from_slice(&3u16.to_be_bytes()); - hvcc.extend_from_slice(&[0x42, 0x01, 0x01]); - - let mut frame_data = Vec::new(); - frame_data.extend_from_slice(&2u32.to_be_bytes()); - frame_data.extend_from_slice(&[0xAA, 0xBB]); - - let mut sink: Vec = Vec::new(); - let mut mux = HevcMux::new(&mut sink); - mux.set_codec_private(hvcc); - mux.write_frame(0, &frame_data).unwrap(); - // Second frame — no SPS re-emission. - mux.write_frame(40_000_000, &frame_data).unwrap(); - mux.finish().unwrap(); - - // SPS NAL (7 bytes) + 2× frame NAL (6 bytes) = 19 bytes. - assert_eq!(sink.len(), 7 + 6 + 6); - // Start codes at offsets 0 (SPS), 7 (frame1), 13 (frame2). - assert_eq!(&sink[0..4], &START_CODE); - assert_eq!(&sink[7..11], &START_CODE); - assert_eq!(&sink[13..17], &START_CODE); - assert_eq!(sink[4], 0x42); - assert_eq!(sink[11], 0xAA); - assert_eq!(sink[17], 0xAA); - } - - // A sink that records only what actually reaches it, so "was flush called" - // is MEASURED rather than assumed. Its own `flush` is a no-op — the - // intermediate `BufWriter` must be told to hand its bytes over. - #[derive(Clone, Default)] - struct SharedSink(std::sync::Arc>>); - - impl SharedSink { - fn bytes(&self) -> Vec { - self.0.lock().unwrap().clone() - } - } - - impl Write for SharedSink { - fn write(&mut self, buf: &[u8]) -> io::Result { - self.0.lock().unwrap().extend_from_slice(buf); - Ok(buf.len()) - } - fn flush(&mut self) -> io::Result<()> { - Ok(()) - } - } - - // `finish()` is the ONLY thing that pushes a buffered sink's tail to the - // file: a `finish` that skipped the flush truncates every `.hevc` output - // by up to a whole sink buffer, producing a file with a missing last GOP. - #[test] - fn finish_flushes_the_buffered_sink_or_the_stream_tail_is_lost() { - let sink = SharedSink::default(); - let mut mux = HevcMux::new(io::BufWriter::new(sink.clone())); - // One length-prefixed NAL: [len:u32-BE][NAL bytes] → Annex B. - let nal = [0x26u8, 0x01, 0xAA, 0xBB]; // (0x26 >> 1) = 19 = IDR_W_RADL - let mut frame = (nal.len() as u32).to_be_bytes().to_vec(); - frame.extend_from_slice(&nal); - mux.write_frame(0, &frame).unwrap(); - - assert!( - sink.bytes().is_empty(), - "the fixture must actually buffer, or this test proves nothing" - ); - - mux.finish().unwrap(); - - let out = sink.bytes(); - let mut expected = START_CODE.to_vec(); - expected.extend_from_slice(&nal); - assert_eq!( - out, expected, - "finish must deliver the whole Annex B stream to the sink" - ); - } - #[test] fn avcc_truncated_pps_keeps_sps() { // numPPS=1 but the PPS body runs past the end: SPS is salvaged. diff --git a/src/mux/interop_tests.rs b/src/mux/interop_tests.rs index bb617dc4..a8b7a10c 100644 --- a/src/mux/interop_tests.rs +++ b/src/mux/interop_tests.rs @@ -2,12 +2,10 @@ //! QA explicitly runs ignored `ffmpeg_` tests with software codecs installed. //! Fixtures are generated locally and never downloaded from a media corpus. -use super::driver::{MuxOptions, MuxSource, NoopEvents, mux_with_keys}; -use crate::halt::Halt; +use super::driver::MuxOptions; use serde_json::Value; use std::path::{Path, PathBuf}; use std::process::{Command, Output}; -use std::sync::Arc; pub(crate) struct FixtureDir { path: PathBuf, @@ -192,16 +190,12 @@ fn generate_with( fn remux(source: &Path, output: &Path, transport: bool) { let scheme = if transport { "m2ts" } else { "mkv" }; let url = format!("{scheme}://{}", source.display()); - let result = mux_with_keys( - MuxSource::Url { - url: &url, - opts: Default::default(), - }, + let result = crate::mux::mux_url( + &url, None, &format!("mkv://{}", output.display()), &MuxOptions::default(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .unwrap(); assert!(result.completed && result.output_opened && result.bytes_written > 0); diff --git a/src/mux/m2ts.rs b/src/mux/m2ts.rs index 90abd289..bd76d31f 100644 --- a/src/mux/m2ts.rs +++ b/src/mux/m2ts.rs @@ -240,14 +240,7 @@ impl M2tsStream { } } -impl crate::pes::Stream for M2tsStream { - fn read(&mut self) -> io::Result> { - // Write-only sink. The m2ts:// read direction is served by - // `super::resolve::build_m2ts_pipeline` → `PipelinedPesStream`; routing - // reads through this type was removed when the highway became sole ingress. - Err(crate::error::Error::StreamWriteOnly.into()) - } - +impl crate::pes::PesSink for M2tsStream { fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> { if self.excluded.drop_frame(frame.track) { return Ok(()); @@ -306,13 +299,6 @@ impl crate::pes::Stream for M2tsStream { out.sort_unstable(); out } - - fn codec_private(&self, _track: usize) -> Option> { - // Write side doesn't have parsers; codec_private flows in - // via the title metadata at `create` time and gets baked - // into the FMKV header. Nothing to surface back here. - None - } } #[cfg(test)] @@ -322,7 +308,7 @@ mod tests { Codec, ColorSpace, ContentFormat, DiscTitle, FrameRate, HdrFormat, Resolution, Stream as DiscStream, VideoStream, }; - use crate::pes::{PesFrame, Stream as PesStreamTrait}; + use crate::pes::{PesFrame, PesSink as _}; const VIDEO_PID: u16 = 0x1011; diff --git a/src/mux/m2ts_mux/mod.rs b/src/mux/m2ts_mux/mod.rs deleted file mode 100644 index cce55c4f..00000000 --- a/src/mux/m2ts_mux/mod.rs +++ /dev/null @@ -1,1580 +0,0 @@ -//! Standard MPEG-TS (188-byte packets) muxer — sequential-only. -//! -//! Distinct from `super::tsmux::TsMuxer` (BD-TS with 192-byte packets). -//! Emits ITU-T H.222.0 / ISO/IEC 13818-1 wire format for `.ts`/`.m2ts` files -//! over a [`SequentialSink`](crate::io::sink::SequentialSink), or MPEG-TS-over-UDP -//! via [`UdpSocketSink`](crate::io::sink::UdpSocketSink). -//! -//! Every output packet is exactly 188 bytes: -//! -//! ```text -//! sync_byte:8 = 0x47 -//! transport_error_indicator:1 = 0 -//! payload_unit_start_indicator:1 -//! transport_priority:1 = 0 -//! PID:13 -//! transport_scrambling_control:2 = 0 -//! adaptation_field_control:2 -//! continuity_counter:4 -//! [adaptation field, if signalled] -//! [payload, if signalled] -//! ``` - -use std::io::{self, Write}; - -mod packet; - -use packet::{Packet, PacketWriter}; - -/// PID `0x0000` — PAT, mandated by spec. -const PID_PAT: u16 = 0x0000; -/// PID for the single program's PMT. `0x1000` is the conventional -/// choice; anything outside reserved/null ranges works. -const PID_PMT: u16 = 0x1000; -/// PID for the video elementary stream. -const PID_VIDEO: u16 = 0x0100; -/// PID for the (optional) first audio elementary stream. -const PID_AUDIO: u16 = 0x0101; -/// Null PID — reserved by spec; never used here. -#[allow(dead_code)] -const PID_NULL: u16 = 0x1FFF; - -/// Re-emit PAT/PMT every N TS packets. ~250 × 188 B = 47 KB; at -/// 80 Mb/s UHD that's ~5 ms — well under typical receiver lock budget. -const PSI_INTERVAL_PACKETS: u64 = 250; -/// Re-stamp PCR every N TS packets carrying the video PID. Spec MAX -/// is 100 ms; 40 packets × 188 B at moderate bitrate keeps us under. -const PCR_INTERVAL_PACKETS: u64 = 40; -/// PCR lead time — the PCR must precede the PTS of the first byte of -/// the picture it timestamps. 200 ms in 90 kHz ticks. -const PCR_LEAD_90KHZ: u64 = 90_000 / 5; - -// PMT stream-type codes. These are the same elementary-stream coding-type -// registry as the parse side; the single source of truth is `consts::coding_type`. -use crate::consts::coding_type; -/// HEVC stream-type code, ISO/IEC 13818-1 Table 2-34 (2015 amendment). -const STREAM_TYPE_HEVC: u8 = coding_type::HEVC; -/// AC-3 / E-AC-3. Not an ISO assignment — sits in the user-private -/// 0x80-0xFF range and is the Blu-ray Disc Association / ATSC A/52 -/// convention. -const STREAM_TYPE_AC3: u8 = coding_type::AC3; -/// Dolby TrueHD. Also a private/BD-conventional value in the -/// user-private 0x80-0xFF range, not an ISO assignment. -const STREAM_TYPE_TRUEHD: u8 = coding_type::TRUEHD; - -/// Audio codec hint for [`M2tsMux::new`] / [`M2tsMux::set_audio`]. The -/// muxer needs to know the codec to pick the right PMT `stream_type` -/// and the right PES stream_id; it doesn't decode the audio. -#[derive(Debug, Clone, Copy)] -pub enum AudioCodec { - /// AC-3 / E-AC-3, stream_type `0x81`, PES stream_id `0xBD`. - Ac3, - /// Dolby TrueHD, stream_type `0x83`, PES stream_id `0xBD`. - TrueHd, -} - -impl AudioCodec { - fn stream_type(self) -> u8 { - match self { - AudioCodec::Ac3 => STREAM_TYPE_AC3, - AudioCodec::TrueHd => STREAM_TYPE_TRUEHD, - } - } -} - -/// Sequential MPEG-TS muxer with one video and optional one audio track. -pub struct M2tsMux { - out: PacketWriter, - /// hvcC parameter set bytes to prepend to the first video PES. - /// Optional — if the upstream frames already carry inline params, - /// callers can omit this. - video_codec_private: Option>, - /// Set on the first video keyframe: has the parameter-set prepend been - /// ATTEMPTED? Latched even when it produced nothing (no codec_private, or - /// one that would not parse) — the same bytes cannot parse better on the - /// next keyframe, so retrying is pointless. - params_attempted: bool, - /// Whether VPS/SPS/PPS actually REACHED the stream. Distinct from - /// `params_attempted`: `false` after an attempt means the emitted TS carries - /// no parameter sets and the video will not decode. See - /// [`parameter_sets_emitted`](Self::parameter_sets_emitted). - params_emitted: bool, - /// Audio codec, if an audio track is configured. `None` = video-only. - audio: Option, - /// First seen PTS (90 kHz). All subsequent PTS / PCR values are - /// relative to this so output streams start near t=0 and don't - /// confuse downstream parsers that don't tolerate huge starting - /// timestamps. - base_pts_90k: Option, - /// Per-PID continuity counter, 4 bits, monotonically increasing. - cc_video: u8, - cc_audio: u8, - cc_pat: u8, - cc_pmt: u8, - /// Total packets written, used to gate PSI / PCR cadence. - packets_written: u64, - /// Video packets written since last PCR, used to gate PCR cadence. - video_packets_since_pcr: u64, - /// Set once the first video TS packet has been emitted. Forces a PCR - /// onto the very first video PES so a receiver tuning at stream start - /// has a clock reference (the PMT advertises the video PID as PCR_PID). - first_video_written: bool, - /// Last PCR stamped (90 kHz); the PCR is held monotonic against it. - last_pcr_90k: u64, -} - -impl M2tsMux { - /// Construct a muxer wrapping `writer`. By default audio is - /// disabled; call [`set_audio`](Self::set_audio) before the first - /// frame to enable. - pub fn new(writer: W) -> Self { - Self { - out: PacketWriter::new(writer), - video_codec_private: None, - params_attempted: false, - params_emitted: false, - audio: None, - base_pts_90k: None, - cc_video: 0, - cc_audio: 0, - cc_pat: 0, - cc_pmt: 0, - packets_written: 0, - video_packets_since_pcr: 0, - first_video_written: false, - last_pcr_90k: 0, - } - } - - /// Provide the `HEVCDecoderConfigurationRecord` for video so the - /// muxer prepends VPS/SPS/PPS Annex B NALs at stream start. - pub fn set_video_codec_private(&mut self, hvcc: Vec) { - self.video_codec_private = Some(hvcc); - } - - /// Whether VPS/SPS/PPS parameter sets actually reached the emitted stream. - /// - /// `false` after any video frame has been written means the TS carries no - /// parameter sets — because none were set, or because the `hvcC` record would - /// not parse — and its video will not decode unless the frames themselves - /// carry inline parameter sets. The prepend is attempted exactly once (on the - /// first keyframe) and never retried, so a caller wiring this muxer into a - /// pipeline must check this rather than assume `finish() == Ok` means a - /// decodable stream. - pub fn parameter_sets_emitted(&self) -> bool { - self.params_emitted - } - - /// Enable a single audio track. Must be called before - /// [`write_audio`](Self::write_audio). - pub fn set_audio(&mut self, codec: AudioCodec) { - self.audio = Some(codec); - } - - /// Write one video PES frame. `data` is either length-prefixed - /// NALUs (MKV-style) or already Annex B; both are accepted. `keyframe` - /// drives the random_access_indicator bit on the first packet of this - /// PES (and gates codec_private NAL prepending — those only attach to - /// the first keyframe). - /// - /// Frames carry no DTS, so the PCR leads the PTS by `PCR_LEAD_90KHZ` and is - /// held monotonic; reordering deeper than that lead can still let it pass a - /// B-frame's PTS. - pub fn write_video(&mut self, pts_ns: i64, keyframe: bool, data: &[u8]) -> io::Result<()> { - let pts_90k = self.base_relative_pts(pts_ns, /* may_seed_base */ true); - let pcr = pts_90k - .saturating_sub(PCR_LEAD_90KHZ) - .max(self.last_pcr_90k); - self.last_pcr_90k = pcr; - - // Annex-B-ify the frame and prepend VPS/SPS/PPS once, on the - // FIRST keyframe (not first frame — non-key frames before the - // first keyframe can't carry params usefully). - let mut es = Vec::with_capacity(data.len() + 64); - if keyframe && !self.params_attempted { - // Must not fail SILENT: if params don't parse, warn loudly rather than - // letting `finish()` return Ok on an undecodable TS. Mirrors the same - // fix in the BD-TS sibling (`tsmux.rs`). - match self.video_codec_private.as_deref() { - Some(cp) => match super::hevc::hvcc_to_annex_b(cp) { - Some(params) => { - es.extend_from_slice(¶ms); - self.params_emitted = true; - } - None => tracing::warn!( - target: "mux", - codec_private_len = cp.len(), - "m2ts: video codec_private did not parse as an hvcC record; no VPS/SPS/PPS \ - emitted and the video will not decode (parameter_sets_emitted() == false)" - ), - }, - None => tracing::warn!( - target: "mux", - "m2ts: no video codec_private was set before the first keyframe; no \ - VPS/SPS/PPS emitted and the video will not decode unless the frames carry \ - inline parameter sets (parameter_sets_emitted() == false)" - ), - } - self.params_attempted = true; - } - // The source's hvcC declares the NAL length-prefix width (ISO/IEC - // 14496-15 §8.3.3.1.2 `lengthSizeMinusOne + 1`); assuming 4 mangles a - // legal 1- or 2-octet-prefixed source into a start-code-free stream. - let length_size = super::hevc::nal_length_size( - crate::disc::Codec::Hevc, - self.video_codec_private.as_deref(), - ); - super::hevc::append_length_prefixed_as_annex_b_sized(&mut es, data, length_size); - - let pes = build_video_pes(pts_90k, &es); - self.write_pes(PID_VIDEO, &pes, Some(pcr), keyframe) - } - - /// Write one audio PES frame. Returns `Ok(())` and silently drops - /// the frame if no audio track was configured — the design assumes - /// the upstream picks tracks and won't ship audio to a video-only - /// muxer, but defending against it keeps the API a single shape. - pub fn write_audio(&mut self, pts_ns: i64, data: &[u8]) -> io::Result<()> { - if self.audio.is_none() { - return Ok(()); - } - // Audio seeds the shared origin too. Video-only seeding made a leading audio - // frame fall back to `base = unwrap_or(raw_90k)` (its OWN pts), collapsing all - // pre-video audio onto PTS 0; `get_or_insert` still lets the FIRST frame fix it. - let pts_90k = self.base_relative_pts(pts_ns, /* may_seed_base */ true); - let pes = build_audio_pes(pts_90k, data); - self.write_pes(PID_AUDIO, &pes, None, false) - } - - /// Drain the underlying writer. No TS-level trailer is mandatory — - /// receivers detect end-of-stream from socket close or file EOF. - pub fn finish(&mut self) -> io::Result<()> { - self.out.flush() - } - - // PTS (ns) -> 90 kHz ticks rebased on the stream origin, seeded by the FIRST frame of any - // kind (video or audio) when `may_seed_base`. - fn base_relative_pts(&mut self, pts_ns: i64, may_seed_base: bool) -> u64 { - let raw_90k = if pts_ns > 0 { - // Widen to u128 so adversarial timestamps can't overflow the - // intermediate multiply (pts_ns * 9 exceeds u64 above - // ~2.05e18 ns), then clamp to the 33-bit PTS range. - (((pts_ns as u128) * 9 / 100_000) as u64) & 0x1_FFFF_FFFF - } else { - 0 - }; - if may_seed_base { - // Seeded one headroom early (see tsmux) so a slightly earlier frame - // arriving second keeps its offset. - let headroom = (super::tsmux::ORIGIN_HEADROOM_NS as u64) * 9 / 100_000; - self.base_pts_90k - .get_or_insert(raw_90k.wrapping_sub(headroom) & 0x1_FFFF_FFFF); - } - let base = self.base_pts_90k.unwrap_or(raw_90k); - // Modular 33-bit subtraction: the PTS clock wraps every 2^33 ticks - // (~26.5 h), so plain `saturating_sub` would flat-line timing across a - // wrap. Wrap the delta into 33 bits and treat it as signed instead. - let delta = raw_90k.wrapping_sub(base) & 0x1_FFFF_FFFF; - // Signed 33-bit: the sign bit is bit 32 (value 2^32), so the entire - // upper half [2^32, 2^33) is negative (frame before base) and floors - // to 0. delta == 2^32 is the most-negative value (-2^32), hence `>=`. - if delta >= (1 << 32) { 0 } else { delta } - } - - // Emit one PES payload as a chain of TS packets on `pid`; `pcr`, if given, is stamped on - // the first packet's AF. - fn write_pes( - &mut self, - pid: u16, - pes: &[u8], - pcr: Option, - is_keyframe_video: bool, - ) -> io::Result<()> { - // Interleave fresh PAT+PMT every PSI_INTERVAL_PACKETS so long - // single-PES emissions (e.g. one 60 KB video frame) don't - // starve receivers tuning in mid-stream. - let mut offset = 0; - let mut first = true; - while offset < pes.len() { - self.maybe_emit_psi()?; - - // PCR cadence is per VIDEO TS packet, NOT per PES: a single UHD HEVC - // I-frame spans thousands of packets, so re-stamping only at PES - // start risks a multi-second T-STD clock-discontinuity violation. - let attach_pcr = (pid == PID_VIDEO) - && (pcr.is_some()) - && (!self.first_video_written - || self.video_packets_since_pcr >= PCR_INTERVAL_PACKETS); - - // RAI rides only the FIRST packet of a keyframe video PES. - let attach_rai = first && is_keyframe_video && pid == PID_VIDEO; - - let mut af_body: Vec = if attach_pcr { - build_pcr_adaptation(pcr.unwrap_or(0)) - } else { - Vec::new() - }; - if attach_rai { - if af_body.is_empty() { - af_body.push(0x40); // flags: RAI only - } else { - af_body[0] |= 0x40; // OR RAI into existing PCR flags - } - } - - let remaining = pes.len() - offset; - // Per ISO/IEC 13818-1 Table 2-6, an adaptation field's first body byte - // is the mandatory 8-bit flags byte, even when stuffing-only (all 0) — - // omitting it lets a decoder misread the 0xFF stuffing as flags. - let (af_present, payload_len, stuffing): (bool, usize, usize) = if !af_body.is_empty() { - // AF is mandatory (PCR, RAI, …). 1 byte length + body + - // stuffing + payload = 184. - let max_payload = 184 - 1 - af_body.len(); - let p = remaining.min(max_payload); - let s = max_payload - p; - (true, p, s) - } else if remaining >= 184 { - // Full payload packet — no AF at all. - (false, 184, 0) - } else { - // Last (small) packet — stuff via an AF whose body is a - // single zero-flags byte. 1 byte length + 1 flags byte + - // stuffing + payload = 184, so payload caps at 182. - let max_payload = 182; - let p = remaining.min(max_payload); - let s = max_payload - p; - af_body.push(0x00); // zero-flags byte - (true, p, s) - }; - - let cc = self.advance_cc(pid); - let mut packet = Packet::new(); - packet.set_header(pid, first, true, af_present, cc); - if af_present { - packet.append_adaptation(&af_body, stuffing)?; - } - packet.append_payload(&pes[offset..offset + payload_len])?; - self.out.write_packet(&packet)?; - - self.packets_written += 1; - if pid == PID_VIDEO { - self.first_video_written = true; - if attach_pcr { - self.video_packets_since_pcr = 0; - } else { - self.video_packets_since_pcr += 1; - } - } - offset += payload_len; - first = false; - } - Ok(()) - } - - fn advance_cc(&mut self, pid: u16) -> u8 { - let slot = match pid { - PID_VIDEO => &mut self.cc_video, - PID_AUDIO => &mut self.cc_audio, - PID_PAT => &mut self.cc_pat, - PID_PMT => &mut self.cc_pmt, - _ => return 0, - }; - let cc = *slot; - *slot = (*slot + 1) & 0x0F; - cc - } - - fn maybe_emit_psi(&mut self) -> io::Result<()> { - if self.packets_written == 0 || self.packets_written.is_multiple_of(PSI_INTERVAL_PACKETS) { - self.emit_pat()?; - self.emit_pmt()?; - } - Ok(()) - } - - fn emit_pat(&mut self) -> io::Result<()> { - let payload = build_pat(PID_PMT); - let cc = self.advance_cc(PID_PAT); - let mut packet = Packet::new(); - packet.set_header(PID_PAT, true, true, false, cc); - packet.append_payload(&payload)?; - packet.pad_to_188(); - self.out.write_packet(&packet)?; - self.packets_written += 1; - Ok(()) - } - - fn emit_pmt(&mut self) -> io::Result<()> { - let payload = build_pmt(self.audio); - let cc = self.advance_cc(PID_PMT); - let mut packet = Packet::new(); - packet.set_header(PID_PMT, true, true, false, cc); - packet.append_payload(&payload)?; - packet.pad_to_188(); - self.out.write_packet(&packet)?; - self.packets_written += 1; - Ok(()) - } -} - -/// Build a PES packet for a video access unit. -fn build_video_pes(pts_90k: u64, es: &[u8]) -> Vec { - build_pes_packet( - crate::consts::pes_stream_id::VIDEO, - pts_90k, - es, - /* length_in_header */ false, - ) -} - -/// Build a PES packet for an audio access unit. -fn build_audio_pes(pts_90k: u64, es: &[u8]) -> Vec { - // A bounded length is written whenever the PES fits a u16; larger units - // (rare — e.g. a large TrueHD frame) fall back to the unbounded (0x0000) - // form, which most demuxers tolerate for private_stream_1. - build_pes_packet( - crate::consts::pes_stream_id::PRIVATE_STREAM_1, - pts_90k, - es, - /* length_in_header */ true, - ) -} - -fn build_pes_packet(stream_id: u8, pts_90k: u64, es: &[u8], length_in_header: bool) -> Vec { - let mut out = Vec::with_capacity(es.len() + 14); - out.extend_from_slice(&[0x00, 0x00, 0x01, stream_id]); - // PES_packet_length: total bytes after this field. 3 flag bytes + 5 - // PTS bytes + es.len(). Zero means "unbounded" — used for video - // where PES can exceed u16. - let pes_len = 8 + es.len(); - if length_in_header && pes_len <= u16::MAX as usize { - out.extend_from_slice(&(pes_len as u16).to_be_bytes()); - } else { - out.extend_from_slice(&[0x00, 0x00]); - } - // Flags byte 1: 10 = MPEG-2 marker, then scrambling/priority/etc 0. - out.push(0x80); - // Flags byte 2: PTS flag (bit 7). - out.push(0x80); - // PES_header_data_length = 5 (just PTS). - out.push(5); - // PTS bytes — 33-bit timestamp split across 5 bytes with marker bits. - let pts = pts_90k & 0x1_FFFF_FFFF; - out.push(0x21 | (((pts >> 29) & 0x0E) as u8)); - out.push(((pts >> 22) & 0xFF) as u8); - out.push(0x01 | (((pts >> 14) & 0xFE) as u8)); - out.push(((pts >> 7) & 0xFF) as u8); - out.push(0x01 | (((pts << 1) & 0xFE) as u8)); - out.extend_from_slice(es); - out -} - -/// Build the PAT payload (section, with pointer_field). -fn build_pat(pmt_pid: u16) -> Vec { - let mut section = Vec::new(); - section.push(0x00); // table_id = PAT - // section_length covers from end of this field through CRC: body - // (transport_stream_id 2 + version/cni 1 + section/last_section 2 + - // program 4 = 9 bytes) plus CRC(4) = 13. - section.extend_from_slice(&[0xB0, 13]); - section.extend_from_slice(&[0x00, 0x01]); // transport_stream_id = 1 - section.push(0xC1); // reserved | version=0 | current_next=1 - section.push(0x00); // section_number - section.push(0x00); // last_section_number - section.extend_from_slice(&[0x00, 0x01]); // program_number = 1 - // reserved(3) | network_PID/program_map_PID(13) - let pid_bytes = (0xE000u16 | (pmt_pid & 0x1FFF)).to_be_bytes(); - section.extend_from_slice(&pid_bytes); - let crc = mpegts_crc32(§ion); - section.extend_from_slice(&crc.to_be_bytes()); - // Prepend pointer_field=0 (section starts immediately). - let mut payload = Vec::with_capacity(section.len() + 1); - payload.push(0x00); - payload.extend_from_slice(§ion); - payload -} - -/// Build the PMT payload (section, with pointer_field). -fn build_pmt(audio: Option) -> Vec { - let mut section = Vec::new(); - section.push(0x02); // table_id = PMT - // section_length filled in after we know the body size. - let len_placeholder = section.len(); - section.extend_from_slice(&[0xB0, 0x00]); - - section.extend_from_slice(&1u16.to_be_bytes()); // program_number - section.push(0xC1); // reserved | version=0 | current_next=1 - section.push(0x00); // section_number - section.push(0x00); // last_section_number - // reserved(3) | PCR_PID(13) - let pcr_pid = (0xE000u16 | (PID_VIDEO & 0x1FFF)).to_be_bytes(); - section.extend_from_slice(&pcr_pid); - // program_info_length = 0 - section.extend_from_slice(&[0xF0, 0x00]); - - // Video elementary stream entry. - section.push(STREAM_TYPE_HEVC); - let v_pid = (0xE000u16 | (PID_VIDEO & 0x1FFF)).to_be_bytes(); - section.extend_from_slice(&v_pid); - section.extend_from_slice(&[0xF0, 0x00]); // ES_info_length = 0 - - if let Some(codec) = audio { - section.push(codec.stream_type()); - let a_pid = (0xE000u16 | (PID_AUDIO & 0x1FFF)).to_be_bytes(); - section.extend_from_slice(&a_pid); - section.extend_from_slice(&[0xF0, 0x00]); - } - - // Now patch section_length: covers everything after the length field - // through the CRC, so (current body size - 3 bytes consumed by - // table_id + 2 length bytes) + 4 (CRC). - let section_len = section.len() - 3 + 4; - section[len_placeholder] = 0xB0 | ((section_len >> 8) as u8 & 0x0F); - section[len_placeholder + 1] = section_len as u8; - - let crc = mpegts_crc32(§ion); - section.extend_from_slice(&crc.to_be_bytes()); - - let mut payload = Vec::with_capacity(section.len() + 1); - payload.push(0x00); - payload.extend_from_slice(§ion); - payload -} - -/// Build the adaptation field carrying a PCR (no other flags). -fn build_pcr_adaptation(pcr_90k: u64) -> Vec { - // adaptation_field_length is set by `Packet::append_adaptation` — - // this function returns just the field body: flags byte (PCR_flag - // etc.) followed by the 48-bit PCR (33b base | 6b reserved | 9b ext). - let mut af = vec![0x10]; // PCR_flag=1; RAI is OR'd in by the caller when applicable - let pcr_base = pcr_90k & 0x1_FFFF_FFFF; // 33-bit - let pcr_ext: u16 = 0; // 9-bit, we keep it zero (no sub-tick precision) - // Encode PCR: 33b base | 6b reserved | 9b extension = 48b - af.push((pcr_base >> 25) as u8); - af.push((pcr_base >> 17) as u8); - af.push((pcr_base >> 9) as u8); - af.push((pcr_base >> 1) as u8); - af.push(((pcr_base << 7) as u8 & 0x80) | 0x7E | ((pcr_ext >> 8) as u8 & 0x01)); - af.push(pcr_ext as u8); - af -} - -/// MPEG-TS CRC-32 (poly 0x04C11DB7, init 0xFFFFFFFF, no reflection, no -/// final XOR). Implementation: bitwise so we don't need a table — -/// PSI sections are tiny, the cost is negligible. -fn mpegts_crc32(data: &[u8]) -> u32 { - let mut crc: u32 = 0xFFFF_FFFF; - for &b in data { - crc ^= (b as u32) << 24; - for _ in 0..8 { - if crc & 0x8000_0000 != 0 { - crc = (crc << 1) ^ 0x04C1_1DB7; - } else { - crc <<= 1; - } - } - } - crc -} - -#[cfg(test)] -mod tests { - use super::*; - - // The origin headroom in 90 kHz ticks: the seeding frame's encoded PTS. - const H: u64 = super::super::tsmux::ORIGIN_HEADROOM_NS as u64 * 9 / 100_000; - - // Golden vector for an audio PES header (ISO/IEC 13818-1 §2.4.3.7). - #[test] - fn audio_pes_header_is_byte_exact_and_bounded_unlike_video() { - let es = [0xDEu8, 0xAD, 0xBE, 0xEF]; - // PTS = 90000 ticks = exactly 1 s at 90 kHz. Encoded across 5 bytes as - // '0010' | PTS[32..30] | marker, PTS[29..15] | marker, PTS[14..0] | marker. - let pes = build_audio_pes(90_000, &es); - assert_eq!( - pes, - vec![ - 0x00, 0x00, 0x01, // packet_start_code_prefix - 0xBD, // stream_id = private_stream_1 (BD audio) - 0x00, 0x0C, // PES_packet_length = 3 flag bytes + 5 PTS + 4 ES - 0x80, // '10' MPEG-2 marker, no scrambling/priority - 0x80, // PTS_DTS_flags = '10' (PTS only) - 0x05, // PES_header_data_length = 5 - 0x21, 0x00, 0x05, 0xBF, 0x21, // PTS 90000 with marker bits - 0xDE, 0xAD, 0xBE, 0xEF, // the access unit, verbatim - ] - ); - - // The video builder over the SAME inputs differs in exactly the two - // fields above — proving neither is incidental. - let vid = build_video_pes(90_000, &es); - assert_eq!(vid[3], 0xE0, "video uses the video stream_id"); - assert_eq!( - &vid[4..6], - &[0x00, 0x00], - "video length is the unbounded form (a video PES can exceed u16)" - ); - assert_eq!(&vid[6..], &pes[6..], "everything after the length matches"); - - // A different PTS must move the timestamp bytes and nothing else. - let later = build_audio_pes(180_000, &es); - assert_eq!(&later[..9], &pes[..9], "header up to the PTS is unchanged"); - assert_ne!(&later[9..14], &pes[9..14], "the PTS bytes change"); - assert_eq!(&later[14..], &es, "the access unit is unchanged"); - - // An access unit too large for the u16 length field falls back to the - // unbounded form rather than writing a truncated (wrapped) length. - let big = vec![0x5Au8; u16::MAX as usize - 7]; - let big_pes = build_audio_pes(90_000, &big); - assert_eq!( - &big_pes[4..6], - &[0x00, 0x00], - "8 + es.len() overflows u16 → unbounded length, never a wrapped one" - ); - assert_eq!(&big_pes[14..], &big[..]); - } - - /// All emitted bytes must align to 188-byte packet boundaries and - /// every packet must start with `0x47`. - fn assert_ts_well_formed(buf: &[u8]) { - assert_eq!( - buf.len() % 188, - 0, - "stream not packet-aligned: {} bytes", - buf.len() - ); - for (i, chunk) in buf.chunks(188).enumerate() { - assert_eq!(chunk[0], 0x47, "packet {} missing sync byte", i); - } - } - - fn extract_pids(buf: &[u8]) -> Vec { - buf.chunks(188) - .map(|p| u16::from_be_bytes([p[1] & 0x1F, p[2]])) - .collect() - } - - #[test] - fn crc32_is_self_validating() { - // Rather than hardcoding sample bytes and a magic CRC residue, - // verify the algorithm behaviorally: distinct inputs produce - // distinct CRCs and the same input is deterministic. - let a = [ - 0u8, 0xB0, 0x0D, 0x00, 0x01, 0xC1, 0x00, 0x00, 0x00, 0x01, 0xE1, 0x00, - ]; - let mut b = a; - b[5] ^= 0x01; // flip one bit - let crc_a = mpegts_crc32(&a); - let crc_b = mpegts_crc32(&b); - assert_ne!(crc_a, crc_b); - assert_eq!(crc_a, mpegts_crc32(&a)); // deterministic - // Sanity check for all-zero input: don't assert an exact value - // (depends on poly choice), just that it differs from non-zero data. - let crc_zero = mpegts_crc32(&[0u8; 12]); - assert_ne!(crc_zero, crc_a); - } - - #[test] - fn video_only_mux_emits_pat_pmt_then_video() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - // One small video frame, no codec_private (so no params NAL inline). - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - drop(mux); - - assert_ts_well_formed(&sink); - let pids = extract_pids(&sink); - // First two packets: PAT, PMT. At least one video packet after. - assert_eq!(pids[0], PID_PAT); - assert_eq!(pids[1], PID_PMT); - assert!(pids.contains(&PID_VIDEO)); - } - - #[test] - fn first_video_pes_carries_pcr() { - // A receiver tuning at stream start needs the clock reference the - // PMT promises (video PID = PCR_PID). The very first video PES must - // therefore carry a PCR even though PAT+PMT precede it. - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - drop(mux); - - // First PUSI video packet must carry an AF with the PCR flag (0x10). - let pkt = sink - .chunks(188) - .find(|p| u16::from_be_bytes([p[1] & 0x1F, p[2]]) == PID_VIDEO && (p[1] & 0x40) != 0) - .expect("video PUSI packet exists"); - let af = af_body(pkt).expect("first video PES must carry an adaptation field"); - assert!(!af.is_empty(), "AF flags byte present"); - assert_eq!(af[0] & 0x10, 0x10, "PCR flag set on first video PES"); - } - - #[test] - fn audio_track_appears_in_pmt_and_stream() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - mux.set_audio(AudioCodec::Ac3); - // Video + audio frame pair. - let mut frame = Vec::new(); - frame.extend_from_slice(&3u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C]); - mux.write_video(0, true, &frame).unwrap(); - mux.write_audio(20_000_000, &[0x0B, 0x77, 0x12, 0x34]) - .unwrap(); - mux.finish().unwrap(); - drop(mux); - - assert_ts_well_formed(&sink); - let pids = extract_pids(&sink); - assert!(pids.contains(&PID_VIDEO)); - assert!(pids.contains(&PID_AUDIO)); - } - - #[test] - fn psi_re_emits_at_interval() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - // Build a frame large enough to span > PSI_INTERVAL_PACKETS TS packets. - // 184 B payload per packet ⇒ ~250 packets = ~46 KB elementary stream. - let big: Vec = (0..(60 * 1024)).map(|i| (i & 0xff) as u8).collect(); - let mut frame = Vec::new(); - frame.extend_from_slice(&(big.len() as u32).to_be_bytes()); - frame.extend_from_slice(&big); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - drop(mux); - - assert_ts_well_formed(&sink); - let pids = extract_pids(&sink); - // Count PAT/PMT pairs — must be at least 2 given the input size. - let pat_count = pids.iter().filter(|p| **p == PID_PAT).count(); - let pmt_count = pids.iter().filter(|p| **p == PID_PMT).count(); - assert!(pat_count >= 2, "expected ≥2 PAT, got {}", pat_count); - assert!(pmt_count >= 2, "expected ≥2 PMT, got {}", pmt_count); - } - - #[test] - fn continuity_counter_increments_per_pid() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - // Three small video frames to get a sequence of video TS packets. - for pts in [0i64, 40_000_000, 80_000_000] { - let mut frame = Vec::new(); - frame.extend_from_slice(&3u32.to_be_bytes()); - frame.extend_from_slice(&[0xAA, 0xBB, 0xCC]); - mux.write_video(pts, pts == 0, &frame).unwrap(); - } - mux.finish().unwrap(); - drop(mux); - - // Collect CCs for video packets in order. - let ccs: Vec = sink - .chunks(188) - .filter(|p| u16::from_be_bytes([p[1] & 0x1F, p[2]]) == PID_VIDEO) - .map(|p| p[3] & 0x0F) - .collect(); - for w in ccs.windows(2) { - assert_eq!(w[1], (w[0] + 1) & 0x0F); - } - } - - /// Return the adaptation field body (length byte stripped) for one - /// 188-byte TS packet, or None when the packet has no AF. - fn af_body(packet: &[u8]) -> Option> { - let afc = (packet[3] >> 4) & 0x03; - if afc & 0b10 == 0 { - return None; - } - let af_len = packet[4] as usize; - if af_len == 0 { - return Some(Vec::new()); - } - Some(packet[5..5 + af_len].to_vec()) - } - - #[test] - fn stuffing_only_tail_packet_is_spec_valid() { - // A short final PES packet must stuff via an AF whose first body byte is - // the mandatory zero-flags byte (ISO/IEC 13818-1 Table 2-6), never a bare - // 0xFF a decoder could misread as flags. Uses an AUDIO PES (no PCR/RAI). - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - mux.set_audio(AudioCodec::Ac3); - // Drive a keyframe first so the stream is well-formed, then the - // audio frame whose tail is short. - let mut vframe = Vec::new(); - vframe.extend_from_slice(&4u32.to_be_bytes()); - vframe.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(0, true, &vframe).unwrap(); - // 200-byte audio payload → PES > 184 → spills into a short tail. - let audio: Vec = (0..200u32).map(|i| (i & 0xFF) as u8).collect(); - mux.write_audio(20_000_000, &audio).unwrap(); - mux.finish().unwrap(); - drop(mux); - - assert_ts_well_formed(&sink); - - // Find every audio packet that carries an adaptation field; the - // short tail packet is one of them. Each such AF must have a - // length >= 1 and a zero-flags body byte (not 0xFF). - let mut saw_stuffing_af = false; - for pkt in sink.chunks(188) { - let pid = u16::from_be_bytes([pkt[1] & 0x1F, pkt[2]]); - if pid != PID_AUDIO { - continue; - } - let afc = (pkt[3] >> 4) & 0x03; - if afc & 0b10 == 0 { - continue; // no AF on this packet - } - let af_len = pkt[4] as usize; - assert!( - af_len >= 1, - "stuffing AF must include the mandatory flags byte" - ); - // First AF body byte is the flags byte — must be zero, never - // a 0xFF stuffing byte masquerading as flags. - assert_eq!( - pkt[5], 0x00, - "stuffing-only AF flags byte must be 0x00, not 0x{:02X}", - pkt[5] - ); - // adaptation_field_length + payload must fill exactly 184. - // (4 header + 1 AF-length + af_len + payload = 188.) - assert!(af_len <= 183, "AF length overflows the 184-byte body"); - saw_stuffing_af = true; - } - assert!( - saw_stuffing_af, - "expected at least one audio packet with a stuffing AF" - ); - } - - #[test] - fn rai_set_on_keyframe_pes_packet() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - drop(mux); - - // Find the first PUSI packet on PID_VIDEO. - let pkt = sink - .chunks(188) - .find(|p| u16::from_be_bytes([p[1] & 0x1F, p[2]]) == PID_VIDEO && (p[1] & 0x40) != 0) - .expect("video PUSI packet exists"); - let af = af_body(pkt).expect("AF present on first packet of keyframe video PES"); - assert!(!af.is_empty(), "AF flags byte present"); - assert_eq!(af[0] & 0x40, 0x40, "RAI bit set"); - } - - // Regression: a silent no-op parameter-set prepend must be observable via - // `parameter_sets_emitted()`, not just a warning. - #[test] - fn absent_or_unparseable_codec_private_is_reported_not_silent() { - // A VPS NAL the parser can find, and the hvcC that carries it. - let vps: [u8; 4] = [0x40, 0x01, 0x0C, 0x77]; - let mut hvcc = vec![0u8; 22]; - hvcc.push(1); // num_arrays - hvcc.push(0x20); // array_completeness | nal_type = VPS - hvcc.extend_from_slice(&1u16.to_be_bytes()); // num_nalus - hvcc.extend_from_slice(&(vps.len() as u16).to_be_bytes()); - hvcc.extend_from_slice(&vps); - - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x26, 0x01, 0xAF, 0x10]); // an IDR-ish slice NAL - - // ── Control: a parseable hvcC really does emit its parameter sets ── - let mut good_sink: Vec = Vec::new(); - let emitted_good = { - let mut mux = M2tsMux::new(&mut good_sink); - mux.set_video_codec_private(hvcc.clone()); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - mux.parameter_sets_emitted() - }; - assert!( - emitted_good, - "a parseable hvcC must emit its parameter sets" - ); - assert!( - good_sink.windows(vps.len()).any(|w| w == vps), - "the VPS must appear in the emitted TS" - ); - - // ── A truncated hvcC: hvcc_to_annex_b returns None ── - let mut bad_sink: Vec = Vec::new(); - let emitted_bad = { - let mut mux = M2tsMux::new(&mut bad_sink); - mux.set_video_codec_private(hvcc[..10].to_vec()); - mux.write_video(0, true, &frame).unwrap(); - // A LATER keyframe cannot recover it — the latch is already armed. - mux.write_video(40_000_000, true, &frame).unwrap(); - mux.finish().unwrap(); // Ok, despite undecodable video - mux.parameter_sets_emitted() - }; - assert!( - !emitted_bad, - "an unparseable hvcC emits nothing — the caller must be able to see that" - ); - assert!( - !bad_sink.windows(vps.len()).any(|w| w == vps), - "no parameter sets reached the stream" - ); - - // ── No codec_private at all: same undecodable outcome, same signal ── - let mut none_sink: Vec = Vec::new(); - let emitted_none = { - let mut mux = M2tsMux::new(&mut none_sink); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - mux.parameter_sets_emitted() - }; - assert!( - !emitted_none, - "no codec_private means no parameter sets, and that must be observable" - ); - } - - #[test] - fn pcr_packet_without_keyframe_has_rai_clear() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - // First frame is the keyframe (gates codec_private; also gets PCR). - let mut frame0 = Vec::new(); - frame0.extend_from_slice(&4u32.to_be_bytes()); - frame0.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(0, true, &frame0).unwrap(); - // Push enough non-key video frames to cross PCR_INTERVAL_PACKETS - // video packets so a later PCR-bearing packet exists. - // Each frame is ~50 KB → ~270 packets, well over 40. - let big: Vec = (0..(50 * 1024)).map(|i| (i & 0xff) as u8).collect(); - for i in 1..3 { - let mut frame = Vec::new(); - frame.extend_from_slice(&(big.len() as u32).to_be_bytes()); - frame.extend_from_slice(&big); - mux.write_video((i as i64) * 40_000_000, false, &frame) - .unwrap(); - } - mux.finish().unwrap(); - drop(mux); - - // RAI rides only the FIRST packet of a KEYFRAME PES; every other - // PCR-bearing video packet must have RAI clear. Skip that first - // (keyframe RAI carrier) packet and check the rest. - let video_pkts: Vec<&[u8]> = sink - .chunks(188) - .filter(|p| u16::from_be_bytes([p[1] & 0x1F, p[2]]) == PID_VIDEO) - .collect(); - assert!( - video_pkts.len() >= 2, - "expected ≥2 video packets, got {}", - video_pkts.len() - ); - // Find a later one with AF that carries PCR (flags & 0x10 set). - let later_pcr = video_pkts - .iter() - .skip(1) - .find_map(|p| { - let af = af_body(p)?; - if !af.is_empty() && (af[0] & 0x10) != 0 { - Some(af) - } else { - None - } - }) - .expect("a later PCR-bearing video packet exists"); - assert_eq!( - later_pcr[0] & 0x40, - 0, - "RAI must be clear on a non-keyframe-start PCR packet" - ); - } - - // Regression: PCR must be re-stamped MID-PES, not only at PES boundaries. - #[test] - fn pcr_restamped_mid_pes_within_interval() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - // ONE big frame → ONE PES spanning ~330 packets (≫ 40). - let big: Vec = (0..(60 * 1024)).map(|i| (i & 0xff) as u8).collect(); - let mut frame = Vec::new(); - frame.extend_from_slice(&(big.len() as u32).to_be_bytes()); - frame.extend_from_slice(&big); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - drop(mux); - - assert_ts_well_formed(&sink); - - // Walk every video TS packet; record which carry a PCR (AF present with - // PCR_flag 0x10). The video-packet index of consecutive PCR carriers must - // never advance by more than PCR_INTERVAL_PACKETS. - let mut video_idx = 0usize; - let mut pcr_indices: Vec = Vec::new(); - let mut total_video = 0usize; - for pkt in sink.chunks(188) { - let pid = u16::from_be_bytes([pkt[1] & 0x1F, pkt[2]]); - if pid != PID_VIDEO { - continue; - } - total_video += 1; - if let Some(af) = af_body(pkt) - && !af.is_empty() - && (af[0] & 0x10) != 0 - { - pcr_indices.push(video_idx); - } - video_idx += 1; - } - assert!( - total_video > PCR_INTERVAL_PACKETS as usize, - "test needs a PES spanning more than one PCR interval, got {total_video} video packets" - ); - // More than one PCR across the single PES (the whole point of the fix). - assert!( - pcr_indices.len() >= 2, - "PCR must be re-stamped mid-PES, but only {} PCR-bearing packet(s) \ - appeared across {} video packets of one PES", - pcr_indices.len(), - total_video - ); - // First PCR is on the first video packet. - assert_eq!(pcr_indices[0], 0, "first video packet must carry PCR"); - // Counter is post-incremented and PCR attaches on `>= PCR_INTERVAL_PACKETS`, - // so the max index gap is `PCR_INTERVAL_PACKETS + 1` — the spec's - // "every 40 packets" bound. - let max_gap = PCR_INTERVAL_PACKETS + 1; - for w in pcr_indices.windows(2) { - assert!( - (w[1] - w[0]) as u64 <= max_gap, - "PCR gap {} exceeds the {}-packet bound", - w[1] - w[0], - max_gap - ); - } - let tail = total_video - 1 - *pcr_indices.last().unwrap(); - assert!( - tail as u64 <= max_gap, - "trailing run after the last PCR ({tail}) exceeds the {max_gap}-packet bound" - ); - } - - #[test] - fn keyframe_video_with_pcr_combines_flags() { - // The FIRST video PES always carries both PCR (forced at stream start) - // and RAI, exercising the flag-OR path (0x10 | 0x40 = 0x50). Pinned on - // the first PES since PCR is now per-packet, not per-PES-boundary. - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - let mut small = Vec::new(); - small.extend_from_slice(&4u32.to_be_bytes()); - small.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(0, true, &small).unwrap(); - mux.finish().unwrap(); - drop(mux); - - let video_pusi: Vec<&[u8]> = sink - .chunks(188) - .filter(|p| u16::from_be_bytes([p[1] & 0x1F, p[2]]) == PID_VIDEO && (p[1] & 0x40) != 0) - .collect(); - assert!(!video_pusi.is_empty(), "a video PES start exists"); - let af = af_body(video_pusi[0]).expect("AF present on first keyframe PES"); - assert!(!af.is_empty(), "AF flags byte present"); - assert_eq!(af[0], 0x50, "flags == RAI | PCR on the first keyframe PES"); - } - - // ════════════════════════════════════════════════════════════════════ - // Added hardening tests - // ════════════════════════════════════════════════════════════════════ - - /// Find the first packet on `pid` (optionally requiring PUSI). - fn find_pkt(buf: &[u8], pid: u16, pusi: bool) -> Option<&[u8]> { - buf.chunks(188).find(|p| { - u16::from_be_bytes([p[1] & 0x1F, p[2]]) == pid && (!pusi || (p[1] & 0x40) != 0) - }) - } - - /// Every packet on `pid` (optionally requiring PUSI), in stream order. - fn find_all_pkts(buf: &[u8], pid: u16, pusi: bool) -> Vec<&[u8]> { - buf.chunks(188) - .filter(|p| { - u16::from_be_bytes([p[1] & 0x1F, p[2]]) == pid && (!pusi || (p[1] & 0x40) != 0) - }) - .collect() - } - - /// Extract a PSI section (after the pointer_field) from a PUSI PSI - /// packet: payload starts at byte 4 (no AF on PSI here), first payload - /// byte is pointer_field, section follows. - fn psi_section(pkt: &[u8]) -> &[u8] { - let pointer = pkt[4] as usize; - &pkt[5 + pointer..] - } - - // ── MPEG-TS CRC-32 (poly 0x04C11DB7) self-validation ────────────────── - - #[test] - fn crc32_residue_over_section_plus_crc_is_zero() { - // Defining property of the MPEG-TS CRC (ISO 13818-1 Annex B): for this - // poly/init (no final XOR), running it over [data || crc(data)] yields - // residue 0 — this pins the algorithm independent of any sample vector. - let data = [ - 0x00u8, 0xB0, 0x0D, 0x00, 0x01, 0xC1, 0x00, 0x00, 0x00, 0x01, 0xE1, 0x00, - ]; - let crc = mpegts_crc32(&data); - // Known-answer vector for CRC-32/MPEG-2 (poly 0x04C11DB7, init 0xFFFFFFFF, - // no reflection, no final XOR — ISO/IEC 13818-1 Annex B), independently - // computed. Pins the polynomial, not just internal consistency. - assert_eq!(crc, 0xE8F9_5E7D, "CRC-32/MPEG-2 known-answer vector"); - let mut with_crc = data.to_vec(); - with_crc.extend_from_slice(&crc.to_be_bytes()); - assert_eq!( - mpegts_crc32(&with_crc), - 0, - "CRC residue over message+CRC must be 0" - ); - } - - #[test] - fn emitted_pat_pmt_crc_is_valid() { - // The PAT and PMT the muxer emits must carry a correct CRC-32 over - // the section (table_id .. end of body). A receiver that validates - // CRC would otherwise drop the table. - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - mux.set_audio(AudioCodec::Ac3); - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - } - for pid in [PID_PAT, PID_PMT] { - let pkt = find_pkt(&sink, pid, true).expect("PSI packet present"); - let sec = psi_section(pkt); - // section_length covers bytes after the 2-byte length field, - // i.e. (table_id + 2 length bytes) + section_length = whole - // section incl. CRC. - let section_len = (((sec[1] & 0x0F) as usize) << 8) | sec[2] as usize; - let total = 3 + section_len; - assert!(sec.len() >= total, "section fits in payload"); - assert_eq!( - mpegts_crc32(&sec[..total]), - 0, - "PID {pid:#06x} section CRC must validate (residue 0)" - ); - } - } - - // ── PAT / PMT structure ─────────────────────────────────────────────── - - #[test] - fn pat_points_at_pmt_pid() { - // PAT program loop entry: program_number(2) + reserved(3)|PID(13). - // The single program must point at PID_PMT. - let pat = build_pat(PID_PMT); - let sec = &pat[1..]; // skip pointer_field - assert_eq!(sec[0], 0x00, "table_id = PAT"); - // Body: tsid(2)@3 cni(1)@5 sec#(1)@6 last(1)@7 program(4)@8..12. - let prog_num = u16::from_be_bytes([sec[8], sec[9]]); - let pmt_pid = u16::from_be_bytes([sec[10] & 0x1F, sec[11]]); - assert_eq!(prog_num, 1, "program_number 1"); - assert_eq!(pmt_pid, PID_PMT, "PAT points at PMT PID"); - } - - #[test] - fn pmt_advertises_video_and_audio_stream_types() { - // PMT must list HEVC video (stream_type 0x24) and, when audio is - // configured, the audio stream_type. Stream-type codes per ISO - // 13818-1 Table 2-34 / BD convention. - let pmt = build_pmt(Some(AudioCodec::Ac3)); - let sec = &pmt[1..]; // skip pointer_field - assert_eq!(sec[0], 0x02, "table_id = PMT"); - let section_len = (((sec[1] & 0x0F) as usize) << 8) | sec[2] as usize; - let prog_info_len = (((sec[10] & 0x0F) as usize) << 8) | sec[11] as usize; - let mut pos = 12 + prog_info_len; - let end = 3 + section_len - 4; // exclude CRC - let mut types = Vec::new(); - while pos + 5 <= end { - types.push(sec[pos]); - let es_info = (((sec[pos + 3] & 0x0F) as usize) << 8) | sec[pos + 4] as usize; - pos += 5 + es_info; - } - assert!(types.contains(&STREAM_TYPE_HEVC), "HEVC video in PMT"); - assert!(types.contains(&STREAM_TYPE_AC3), "AC-3 audio in PMT"); - } - - #[test] - fn pmt_video_only_omits_audio_entry() { - // Video-only PMT must list exactly one ES entry (video) — no audio. - let pmt = build_pmt(None); - let sec = &pmt[1..]; - let section_len = (((sec[1] & 0x0F) as usize) << 8) | sec[2] as usize; - let prog_info_len = (((sec[10] & 0x0F) as usize) << 8) | sec[11] as usize; - let mut pos = 12 + prog_info_len; - let end = 3 + section_len - 4; - let mut count = 0; - while pos + 5 <= end { - count += 1; - let es_info = (((sec[pos + 3] & 0x0F) as usize) << 8) | sec[pos + 4] as usize; - pos += 5 + es_info; - } - assert_eq!(count, 1, "video-only PMT has exactly one ES entry"); - } - - #[test] - fn truehd_audio_uses_stream_type_0x83() { - // TrueHD maps to stream_type 0x83 (BD convention). - let pmt = build_pmt(Some(AudioCodec::TrueHd)); - let sec = &pmt[1..]; - let section_len = (((sec[1] & 0x0F) as usize) << 8) | sec[2] as usize; - let end = 3 + section_len - 4; - let mut pos = 12; // prog_info_len is 0 in this muxer - let mut found = false; - while pos + 5 <= end { - if sec[pos] == STREAM_TYPE_TRUEHD { - found = true; - } - let es_info = (((sec[pos + 3] & 0x0F) as usize) << 8) | sec[pos + 4] as usize; - pos += 5 + es_info; - } - assert!(found, "TrueHD stream_type 0x83 must appear in PMT"); - } - - #[test] - fn pmt_pcr_pid_is_video_pid() { - // PMT PCR_PID field (reserved(3)|PCR_PID(13) at section bytes 8..10) - // must be the video PID — the PCR rides the video adaptation field. - let pmt = build_pmt(None); - let sec = &pmt[1..]; - let pcr_pid = u16::from_be_bytes([sec[8] & 0x1F, sec[9]]); - assert_eq!(pcr_pid, PID_VIDEO, "PCR_PID advertised as the video PID"); - } - - // ── PCR encoding ────────────────────────────────────────────────────── - - #[test] - fn pcr_base_round_trips_through_adaptation_field() { - // build_pcr_adaptation packs a 33-bit PCR base across 6 bytes: - // base[32:25],[24:17],[16:9],[8:1] then bit0 in top of byte 5. - // (ISO 13818-1 §2.4.3.5.) Decode it back and compare. - let pcr: u64 = 0x1_2345_6789 & ((1 << 33) - 1); - let af = build_pcr_adaptation(pcr); - assert_eq!(af[0], 0x10, "PCR_flag set, others clear"); - let base = ((af[1] as u64) << 25) - | ((af[2] as u64) << 17) - | ((af[3] as u64) << 9) - | ((af[4] as u64) << 1) - | ((af[5] as u64 >> 7) & 0x01); - assert_eq!(base, pcr, "PCR base must round-trip through the AF"); - } - - #[test] - fn first_video_pcr_leads_pts_by_lead_time() { - // The PCR on the first video PES = pts_90k - PCR_LEAD_90KHZ, clamped - // at 0. With pts_ns large enough not to clamp, decode the PCR and the - // PTS and verify the lead. PCR_LEAD_90KHZ = 18000 (200 ms). - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - // A single frame: it seeds the origin one headroom below itself. - mux.write_video(1_000_000_000, true, &frame).unwrap(); - mux.finish().unwrap(); - } - let pkt = find_pkt(&sink, PID_VIDEO, true).unwrap(); - let af = af_body(pkt).unwrap(); - // PCR present. - assert_eq!(af[0] & 0x10, 0x10); - let base = ((af[1] as u64) << 25) - | ((af[2] as u64) << 17) - | ((af[3] as u64) << 9) - | ((af[4] as u64) << 1) - | ((af[5] as u64 >> 7) & 0x01); - // The seeding frame sits one headroom past the origin; PCR leads it. - assert_eq!(base, H - PCR_LEAD_90KHZ, "first frame PCR leads its PTS"); - } - - // ── base_relative_pts overflow / saturation ─────────────────────────── - - #[test] - fn extreme_pts_does_not_overflow_and_clamps_to_33bit() { - // Two frames are required: a single frame always encodes the headroom, making - // the only assertion vacuous. Bit 32 is unreachable (the - // signed-33-bit delta rule floors the upper half to 0), so this pins encoding. - let decode_pts = |pkt: &[u8]| -> u64 { - // Payload after the AF: AF area = 1 (length byte) + af_len. - let af_len = pkt[4] as usize; - let pes = &pkt[4 + 1 + af_len..]; - // PES: 00 00 01 E0 00 00 80 80 05 PTS[5]. PTS at pes[9..14]. - ((((pes[9] >> 1) & 0x07) as u64) << 30) - | ((pes[10] as u64) << 22) - | (((pes[11] >> 1) as u64) << 15) - | ((pes[12] as u64) << 7) - | ((pes[13] >> 1) as u64) - }; - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - - // (a) A LARGE but forward delta, exact-value assertion. 477_218_477 x - // 100_000 ns divides exactly, giving 4_294_966_293 ticks — just under - // 2^32, the largest magnitude the signed-33-bit rule treats as forward. - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - mux.write_video(0, true, &frame).unwrap(); - // Less one headroom (10_000 x 100 us), which the origin already adds. - mux.write_video((477_218_477 - 10_000) * 100_000, true, &frame) - .unwrap(); - mux.finish().unwrap(); - } - assert_ts_well_formed(&sink); - let pkts = find_all_pkts(&sink, PID_VIDEO, true); - assert_eq!(pkts.len(), 2, "one PUSI packet per video frame"); - let pts = decode_pts(pkts[1]); - assert!(pts < (1u64 << 33), "PTS stays within the 33-bit field"); - assert_eq!( - pts, 4_294_966_293, - "the encoded PES PTS must be the exact tick value, not a truncated one" - ); - - // (b) The adversarial i64::MAX: no overflow/panic. Its tick count masks to - // 6_564_084_417, in the UPPER half of the 33-bit range, so the delta rule - // reads it as a frame BEFORE the base and floors it to 0 — pinned explicitly. - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - mux.write_video(0, true, &frame).unwrap(); - mux.write_video(i64::MAX, true, &frame).unwrap(); - mux.finish().unwrap(); - } - assert_ts_well_formed(&sink); - let masked = (((i64::MAX as u128) * 9 / 100_000) as u64) & 0x1_FFFF_FFFF; - assert!( - masked >= 1 << 32, - "i64::MAX masks into the upper (negative) half" - ); - let pkts = find_all_pkts(&sink, PID_VIDEO, true); - let pts = decode_pts(pkts[1]); - assert!(pts < (1u64 << 33), "PTS stays within the 33-bit field"); - assert_eq!( - pts, 0, - "an i64::MAX tick lands in the signed-33-bit upper half and floors to 0" - ); - } - - #[test] - fn base_relative_pts_wraps_across_33bit_clock_rollover() { - // Regression: a real 90 kHz clock wrap must NOT collapse to PTS 0. The OLD - // `saturating_sub` returned 0 here (flat-lining timing); modular subtraction - // must return the true small forward delta instead. - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - // Force the base to 2^33 - 100 directly (a value reachable only after - // ~26.5 h of stream; set it rather than ripping that long). - mux.base_pts_90k = Some((1u64 << 33) - 100); - // pts_ns = 1ms → raw_90k = 1_000_000 * 9 / 100_000 = 90 ticks (wrapped - // past 0, far below the near-max base). - let pts_ns = 1_000_000i64; - let rel = mux.base_relative_pts(pts_ns, /* may_seed_base */ false); - // 90 - (2^33 - 100) mod 2^33 = 190 ticks forward across the wrap (NOT 0). - assert_eq!( - rel, 190, - "a 33-bit clock wrap must produce the true forward delta, not 0" - ); - - // And a frame genuinely a little BEFORE the base still floors to 0 - // (documented pre-base behavior — e.g. leading audio). base = 200 ticks, - // frame at 90 ticks (< base) → backward step → floor to 0. - mux.base_pts_90k = Some(200); - let rel0 = mux.base_relative_pts(1_000_000i64, false); // raw_90k = 90 < 200 - assert_eq!(rel0, 0, "a frame before the base must still floor to 0"); - } - - #[test] - fn negative_pts_ns_encodes_zero() { - // base_relative_pts treats pts_ns <= 0 as raw 0. A negative input - // must encode as t=0 (the headroom), not a wrapped value. - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - let mut frame = Vec::new(); - frame.extend_from_slice(&4u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C, 0x01]); - mux.write_video(-5, true, &frame).unwrap(); - mux.finish().unwrap(); - } - let pkt = find_pkt(&sink, PID_VIDEO, true).unwrap(); - let af_len = pkt[4] as usize; - let pes = &pkt[4 + 1 + af_len..]; - let pts = ((((pes[9] >> 1) & 0x07) as u64) << 30) - | ((pes[10] as u64) << 22) - | (((pes[11] >> 1) as u64) << 15) - | ((pes[12] as u64) << 7) - | ((pes[13] >> 1) as u64); - assert_eq!(pts, H, "negative pts_ns is treated as 0, the seeding frame"); - } - - #[test] - fn pre_video_audio_frames_keep_their_spacing() { - // Two audio frames before any video. The FIRST (audio) seeds the shared origin, - // so the second keeps its 1s offset. OLD code seeded only on video → both fell - // back to `base = unwrap_or(raw_90k)` (own pts), collapsing onto PTS 0, losing 1s. - let decode_audio_pts = |pkt: &[u8]| -> u64 { - let afc = (pkt[3] >> 4) & 0x03; - let pes_start = if afc & 0b10 != 0 { - 4 + 1 + pkt[4] as usize // adaptation field present (stuffing) - } else { - 4 - }; - let pes = &pkt[pes_start..]; - assert_eq!(&pes[0..3], &[0x00, 0x00, 0x01], "PES start code"); - assert!(pes[7] & 0x80 != 0, "the PES header carries a PTS"); - ((((pes[9] >> 1) & 0x07) as u64) << 30) - | ((pes[10] as u64) << 22) - | (((pes[11] >> 1) as u64) << 15) - | ((pes[12] as u64) << 7) - | ((pes[13] >> 1) as u64) - }; - - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - mux.set_audio(AudioCodec::Ac3); - mux.write_audio(0, &[0x0B, 0x77, 0x00, 0x00]).unwrap(); - mux.write_audio(1_000_000_000, &[0x0B, 0x77, 0x00, 0x01]) - .unwrap(); - mux.finish().unwrap(); - } - let pkts = find_all_pkts(&sink, PID_AUDIO, true); - assert_eq!(pkts.len(), 2, "one PUSI packet per small audio frame"); - assert_eq!( - decode_audio_pts(pkts[0]), - H, - "the first (audio) frame seeds the origin one headroom below it" - ); - assert_eq!( - decode_audio_pts(pkts[1]), - H + 90_000, - "the second audio frame (1s later) must keep its spacing, not collapse to 0" - ); - } - - // ── audio without configured track ──────────────────────────────────── - - #[test] - fn write_audio_without_track_is_silently_dropped() { - // write_audio on a video-only muxer must drop the frame (no audio - // PID configured) without error — and emit no audio PID packets. - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - let mut frame = Vec::new(); - frame.extend_from_slice(&3u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C]); - mux.write_video(0, true, &frame).unwrap(); - mux.write_audio(0, &[0x0B, 0x77]).unwrap(); // no track → dropped - mux.finish().unwrap(); - } - let pids = extract_pids(&sink); - assert!( - !pids.contains(&PID_AUDIO), - "no audio track configured → no audio PID emitted" - ); - } - - // ── empty stream ────────────────────────────────────────────────────── - - #[test] - fn finish_without_frames_emits_nothing() { - // A muxer with no frames written emits no packets (PSI is gated on - // write paths). finish() must be a clean no-op. - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - mux.finish().unwrap(); - drop(mux); - assert!(sink.is_empty(), "no frames → no output"); - } - - #[test] - fn pat_always_on_pid_zero() { - // ISO 13818-1 mandates the PAT on PID 0x0000. - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - let mut frame = Vec::new(); - frame.extend_from_slice(&3u32.to_be_bytes()); - frame.extend_from_slice(&[0x40, 0x01, 0x0C]); - mux.write_video(0, true, &frame).unwrap(); - mux.finish().unwrap(); - } - assert_eq!( - extract_pids(&sink)[0], - 0x0000, - "first packet is PAT on PID 0" - ); - } - - // Same for the 188-byte muxer: a frame shortly before the seeding one keeps - // its offset instead of flooring to the origin. - #[test] - fn a_frame_shortly_before_the_seeding_frame_keeps_its_offset() { - let mut sink: Vec = Vec::new(); - let mut mux = M2tsMux::new(&mut sink); - let audio = mux.base_relative_pts(100_000_000, true); - let video = mux.base_relative_pts(50_000_000, true); - assert_eq!(audio - video, 4_500); - } - - fn all_pcrs(buf: &[u8]) -> Vec { - find_all_pkts(buf, PID_VIDEO, false) - .into_iter() - .filter_map(af_body) - .filter(|af| af.first().is_some_and(|f| f & 0x10 != 0)) - .map(|af| { - ((af[1] as u64) << 25) - | ((af[2] as u64) << 17) - | ((af[3] as u64) << 9) - | ((af[4] as u64) << 1) - | ((af[5] as u64 >> 7) & 0x01) - }) - .collect() - } - - // B-frames arrive with PTS below an earlier P-frame's: the PCR must still - // never step backwards (ISO/IEC 13818-1 §2.4.2.2). - #[test] - fn pcr_never_goes_backwards_with_b_frames() { - let mut frame = Vec::new(); - frame.extend_from_slice(&8000u32.to_be_bytes()); - frame.extend(std::iter::repeat_n(0x41u8, 8000)); - let mut sink: Vec = Vec::new(); - { - let mut mux = M2tsMux::new(&mut sink); - for (i, ms) in [80i64, 200, 120, 160].into_iter().enumerate() { - mux.write_video(ms * 1_000_000, i == 0, &frame).unwrap(); - } - } - let pcrs = all_pcrs(&sink); - assert!(pcrs.len() >= 4, "{pcrs:?}"); - assert!(pcrs.windows(2).all(|w| w[1] >= w[0]), "{pcrs:?}"); - } -} diff --git a/src/mux/m2ts_mux/packet.rs b/src/mux/m2ts_mux/packet.rs deleted file mode 100644 index 06578745..00000000 --- a/src/mux/m2ts_mux/packet.rs +++ /dev/null @@ -1,378 +0,0 @@ -//! 188-byte MPEG-TS packet builder + writer. -//! -//! Internal helper for `super::M2tsMux`. Not public API — exposes raw -//! byte layout so the parent module can compose PSI / PCR / PES bytes -//! without each caller re-implementing the 188-byte boundary math. - -use crate::error::Error; -use std::io::{self, Write}; - -use crate::consts::TS_PACKET_BYTES; -/// Header is 4 bytes, leaving 184 bytes for the adaptation field area -/// plus payload. With a 1-byte `adaptation_field_length` prefix the -/// field body + stuffing can be at most 183 bytes. -const MAX_AF_LEN: usize = TS_PACKET_BYTES - 4 - 1; -const SYNC_BYTE: u8 = 0x47; -const STUFF_BYTE: u8 = 0xFF; - -// One TS packet under construction, backed by a fixed 188-byte array with -// a write cursor (no per-packet heap allocation). pad_to_188() fills to -// 188 bytes; otherwise the caller must fill it exactly. -pub(super) struct Packet { - buf: [u8; TS_PACKET_BYTES], - len: usize, -} - -impl Packet { - pub(super) fn new() -> Self { - Self { - buf: [0u8; TS_PACKET_BYTES], - len: 0, - } - } - - /// Push a byte, saturating at the packet boundary. The boundary is - /// never reached by the sole caller (mod.rs sizes every field to sum - /// to 188); the bound prevents a future caller from corrupting memory. - fn push(&mut self, b: u8) { - if self.len < TS_PACKET_BYTES { - self.buf[self.len] = b; - self.len += 1; - } - } - - fn extend(&mut self, bytes: &[u8]) { - let n = bytes.len().min(TS_PACKET_BYTES - self.len); - self.buf[self.len..self.len + n].copy_from_slice(&bytes[..n]); - self.len += n; - } - - // Write the 4-byte TS packet header. - pub(super) fn set_header( - &mut self, - pid: u16, - payload_unit_start: bool, - has_payload: bool, - has_adaptation: bool, - cc: u8, - ) { - self.len = 0; - self.push(SYNC_BYTE); - let pus_bit = if payload_unit_start { 0x40 } else { 0 }; - // transport_error_indicator(1)=0 | payload_unit_start(1) | transport_priority(1)=0 | PID(5 high) - self.push(pus_bit | ((pid >> 8) as u8 & 0x1F)); - self.push(pid as u8); - // transport_scrambling_control(2)=0 | adaptation_field_control(2) | continuity_counter(4) - let afc = match (has_adaptation, has_payload) { - (false, false) => 0b00, // reserved — should not happen - (false, true) => 0b01, // payload only - (true, false) => 0b10, // adaptation only - (true, true) => 0b11, // both - }; - self.push((afc << 4) | (cc & 0x0F)); - } - - // Append the adaptation field after the header: `body` is the field body (flags + optional - // PCR etc.), `stuffing` is the trailing 0xFF count. - pub(super) fn append_adaptation(&mut self, body: &[u8], stuffing: usize) -> io::Result<()> { - let af_len = body.len() + stuffing; - if af_len > MAX_AF_LEN { - return Err(Error::M2tsPacketMalformed.into()); - } - self.push(af_len as u8); - self.extend(body); - for _ in 0..stuffing { - self.push(STUFF_BYTE); - } - Ok(()) - } - - // Append payload bytes. Errors with M2tsPacketMalformed rather than - // silently truncating if this would push the packet past 188 bytes — - // overflow is a muxer invariant break. - pub(super) fn append_payload(&mut self, payload: &[u8]) -> io::Result<()> { - if self.len + payload.len() > TS_PACKET_BYTES { - return Err(Error::M2tsPacketMalformed.into()); - } - self.extend(payload); - Ok(()) - } - - // Pad the packet to exactly 188 bytes with 0xFF — for PSI emit paths - // where the section is much smaller than 184 bytes. Payload-carrying - // packets reserve stuffing room via append_adaptation instead. - pub(super) fn pad_to_188(&mut self) { - while self.len < TS_PACKET_BYTES { - self.push(STUFF_BYTE); - } - } - - pub(super) fn bytes(&self) -> &[u8] { - &self.buf[..self.len] - } - - pub(super) fn len(&self) -> usize { - self.len - } -} - -// Writer for assembled TS packets. Owns the sink and writes each -// 188-byte packet straight through with no buffering of its own — wrap -// the sink in a BufWriter if buffering is needed. -pub(super) struct PacketWriter { - inner: W, -} - -impl PacketWriter { - pub(super) fn new(inner: W) -> Self { - Self { inner } - } - - pub(super) fn write_packet(&mut self, packet: &Packet) -> io::Result<()> { - let bytes = packet.bytes(); - // Hard check, not a debug_assert: a non-188-byte packet would - // corrupt the transport stream, so refuse to write it in any - // build rather than emitting a short/long packet silently. - if bytes.len() != TS_PACKET_BYTES { - return Err(Error::M2tsPacketMalformed.into()); - } - self.inner.write_all(bytes) - } - - pub(super) fn flush(&mut self) -> io::Result<()> { - self.inner.flush() - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn pad_fills_to_188() { - let mut p = Packet::new(); - p.set_header(0x100, true, true, false, 0); - p.append_payload(&[1, 2, 3]).unwrap(); - p.pad_to_188(); - assert_eq!(p.bytes().len(), 188); - assert_eq!(p.bytes()[0], SYNC_BYTE); - // After 4-byte header + 3 payload, byte 7 starts stuffing. - assert_eq!(p.bytes()[7], STUFF_BYTE); - } - - #[test] - fn append_adaptation_rejects_overflow() { - let mut p = Packet::new(); - p.set_header(0x100, true, true, true, 0); - // body(1) + stuffing(MAX_AF_LEN) = MAX_AF_LEN + 1 > MAX_AF_LEN. - let err = p.append_adaptation(&[0x00], MAX_AF_LEN).unwrap_err(); - assert_eq!(err.kind(), io::ErrorKind::InvalidData); - } - - #[test] - fn append_payload_rejects_overflow() { - let mut p = Packet::new(); - p.set_header(0x100, true, true, false, 0); - // 4-byte header + 185 payload = 189 > 188. - let err = p.append_payload(&[0u8; 185]).unwrap_err(); - assert_eq!(err.kind(), io::ErrorKind::InvalidData); - } - - #[test] - fn write_packet_rejects_short_packet() { - let mut p = Packet::new(); - p.set_header(0x100, true, true, false, 0); - p.append_payload(&[1, 2, 3]).unwrap(); // only 7 bytes, not padded - let mut sink: Vec = Vec::new(); - let mut w = PacketWriter::new(&mut sink); - let err = w.write_packet(&p).unwrap_err(); - assert_eq!(err.kind(), io::ErrorKind::InvalidData); - assert!(sink.is_empty(), "short packet must not be written"); - } - - #[test] - fn header_pid_round_trips() { - let mut p = Packet::new(); - p.set_header(0x1ABC, false, true, false, 0xA); - let pid = u16::from_be_bytes([p.bytes()[1] & 0x1F, p.bytes()[2]]); - assert_eq!(pid, 0x1ABC); - assert_eq!(p.bytes()[3] & 0x0F, 0xA); - } - - // ════════════════════════════════════════════════════════════════════ - // Added hardening tests - // ════════════════════════════════════════════════════════════════════ - - #[test] - fn header_sync_byte_and_pusi_bit() { - // ISO 13818-1: sync_byte 0x47 at byte 0; PUSI is bit 6 of byte 1. - let mut p = Packet::new(); - p.set_header(0x0100, true, true, false, 0); - assert_eq!(p.bytes()[0], SYNC_BYTE); - assert_eq!(p.bytes()[1] & 0x40, 0x40, "PUSI set"); - // transport_error_indicator (bit 7) and priority (bit 5) clear. - assert_eq!(p.bytes()[1] & 0x80, 0, "TEI clear"); - assert_eq!(p.bytes()[1] & 0x20, 0, "transport_priority clear"); - - let mut p2 = Packet::new(); - p2.set_header(0x0100, false, true, false, 0); - assert_eq!(p2.bytes()[1] & 0x40, 0, "PUSI clear when not a unit start"); - } - - #[test] - fn header_afc_bits_per_combination() { - // adaptation_field_control (bits 5:4 of byte 3), ISO 13818-1 - // Table 2-5: 01 payload only, 10 AF only, 11 both, 00 reserved. - let cases = [ - (false, true, 0b01u8), - (true, false, 0b10), - (true, true, 0b11), - (false, false, 0b00), - ]; - for (af, pl, want) in cases { - let mut p = Packet::new(); - p.set_header(0x0100, true, pl, af, 0); - assert_eq!((p.bytes()[3] >> 4) & 0x03, want, "AFC for af={af} pl={pl}"); - } - } - - #[test] - fn append_adaptation_length_byte_matches_written_bytes() { - // The adaptation_field_length byte must equal body+stuffing — the - // written length and declared length must agree or a decoder - // misframes the payload. - let mut p = Packet::new(); - p.set_header(0x0100, true, true, true, 0); - p.append_adaptation(&[0x10, 0xAA, 0xBB], 4).unwrap(); // 3 body + 4 stuffing - // byte 4 is the length byte. - assert_eq!(p.bytes()[4], 3 + 4, "length byte = body+stuffing"); - // body bytes follow. - assert_eq!(&p.bytes()[5..8], &[0x10, 0xAA, 0xBB]); - // stuffing bytes are 0xFF. - assert_eq!(&p.bytes()[8..12], &[0xFF; 4]); - } - - #[test] - fn append_adaptation_at_exact_max_succeeds() { - // MAX_AF_LEN (183) is the largest legal adaptation field body+stuff. - // Exactly MAX_AF_LEN must succeed; the boundary itself is valid. - let mut p = Packet::new(); - p.set_header(0x0100, true, true, true, 0); - assert!(p.append_adaptation(&[0x00], MAX_AF_LEN - 1).is_ok()); - assert_eq!(p.bytes()[4] as usize, MAX_AF_LEN); - } - - #[test] - fn append_payload_at_exact_boundary_fills_188() { - // 4-byte header + 184 payload = exactly 188 (no AF). The boundary - // must be accepted, not rejected. - let mut p = Packet::new(); - p.set_header(0x0100, true, true, false, 0); - assert!(p.append_payload(&[0xAB; 184]).is_ok()); - assert_eq!(p.len(), 188); - } - - #[test] - fn pad_to_188_is_idempotent_when_already_full() { - // Padding a packet that already reached 188 bytes must not grow it - // past 188 (the push() bound prevents overflow). - let mut p = Packet::new(); - p.set_header(0x0100, true, true, false, 0); - p.append_payload(&[0xAB; 184]).unwrap(); - assert_eq!(p.len(), 188); - p.pad_to_188(); - assert_eq!(p.len(), 188, "no growth past 188"); - } - - #[test] - fn write_packet_rejects_long_packet() { - // A packet whose len somehow exceeds 188 must be refused (the writer - // checks exact equality). We can't push past 188 (push saturates), - // so test the under-188 rejection path which the writer guards. - let mut p = Packet::new(); - p.set_header(0x0100, true, true, false, 0); - p.append_payload(&[1, 2, 3, 4, 5]).unwrap(); // 9 bytes, not 188 - let mut sink: Vec = Vec::new(); - let mut w = PacketWriter::new(&mut sink); - assert!(w.write_packet(&p).is_err()); - assert!(sink.is_empty()); - } - - #[test] - fn write_packet_accepts_exactly_188() { - // A correctly-sized 188-byte packet must be written through verbatim. - let mut p = Packet::new(); - p.set_header(0x0100, true, true, false, 0); - p.append_payload(&[0x5A; 184]).unwrap(); - let mut sink: Vec = Vec::new(); - { - let mut w = PacketWriter::new(&mut sink); - w.write_packet(&p).unwrap(); - } - assert_eq!(sink.len(), 188); - assert_eq!(sink[0], SYNC_BYTE); - } - - #[test] - fn pid_high_bits_masked_to_13_bits() { - // PID is 13 bits. Bits above 0x1FFF must not leak into the - // transport_priority / PUSI / TEI bits of byte 1. - let mut p = Packet::new(); - // 0xE100 has bits set above the 13-bit PID range. - p.set_header(0xE100, false, true, false, 0); - assert_eq!( - p.bytes()[1] & 0xE0, - 0, - "top 3 bits of byte1 are flags, not PID" - ); - let pid = u16::from_be_bytes([p.bytes()[1] & 0x1F, p.bytes()[2]]); - assert_eq!(pid, 0xE100 & 0x1FFF, "PID masked to 13 bits"); - } - - // Records only what actually reaches it, so "was flush called" is - // MEASURED not assumed. Its own flush is a no-op — the point is that - // the intermediate BufWriter must be told to hand bytes over. - #[derive(Clone, Default)] - struct SharedSink(std::sync::Arc>>); - - impl SharedSink { - fn bytes(&self) -> Vec { - self.0.lock().unwrap().clone() - } - } - - impl Write for SharedSink { - fn write(&mut self, buf: &[u8]) -> io::Result { - self.0.lock().unwrap().extend_from_slice(buf); - Ok(buf.len()) - } - fn flush(&mut self) -> io::Result<()> { - Ok(()) - } - } - - // PacketWriter buffers nothing itself, so flush() is the only thing that empties the sink's - // buffer — skipping it truncates the stream mid-packet. - #[test] - fn flush_delivers_the_buffered_packets_to_the_sink() { - let sink = SharedSink::default(); - let mut w = PacketWriter::new(io::BufWriter::new(sink.clone())); - let mut p = Packet::new(); - p.set_header(0x1011, true, true, false, 3); - p.append_payload(&[0x11, 0x22, 0x33]).unwrap(); - p.pad_to_188(); - w.write_packet(&p).unwrap(); - - assert!( - sink.bytes().is_empty(), - "the fixture must actually buffer, or this test proves nothing" - ); - - w.flush().unwrap(); - - let out = sink.bytes(); - assert_eq!(out.len(), TS_PACKET_BYTES, "one whole 188-byte TS packet"); - assert_eq!(out[0], SYNC_BYTE, "ISO/IEC 13818-1 sync_byte 0x47"); - assert_eq!(out, p.bytes(), "the bytes written are the packet's own"); - } -} diff --git a/src/mux/meta_sink.rs b/src/mux/meta_sink.rs index 9b7ead08..253c53cc 100644 --- a/src/mux/meta_sink.rs +++ b/src/mux/meta_sink.rs @@ -7,7 +7,7 @@ //! ISO/disc scan that builds the title is all they need. use crate::disc::{Chapter, DiscTitle, TitleProfile}; -use crate::pes::{PesFrame, Stream}; +use crate::pes::{PesFrame, PesSink}; use std::fs::File; use std::io::{self, Write}; use std::path::Path; @@ -103,16 +103,15 @@ impl ChaptersSink { } } -impl Stream for ChaptersSink { - fn read(&mut self) -> io::Result> { - Err(crate::error::Error::StreamWriteOnly.into()) - } +impl PesSink for ChaptersSink { fn write(&mut self, _frame: &PesFrame) -> io::Result<()> { Ok(()) // whole file written at create() } + fn finish(&mut self) -> io::Result<()> { Ok(()) } + fn info(&self) -> &DiscTitle { &self.title } @@ -287,16 +286,15 @@ impl JsonSink { } } -impl Stream for JsonSink { - fn read(&mut self) -> io::Result> { - Err(crate::error::Error::StreamWriteOnly.into()) - } +impl PesSink for JsonSink { fn write(&mut self, _frame: &PesFrame) -> io::Result<()> { Ok(()) } + fn finish(&mut self) -> io::Result<()> { Ok(()) } + fn info(&self) -> &DiscTitle { &self.title } @@ -547,50 +545,6 @@ mod tests { assert_eq!(s["descriptive_service"], true); } - fn temp_path(name: &str) -> std::path::PathBuf { - use std::sync::atomic::{AtomicU64, Ordering}; - static N: AtomicU64 = AtomicU64::new(0); - let n = N.fetch_add(1, Ordering::Relaxed); - std::env::temp_dir().join(format!("fmkv_meta_sink_{}_{n}_{name}", std::process::id())) - } - - fn sink_title() -> crate::disc::DiscTitle { - let mut t = crate::disc::DiscTitle::empty(); - t.playlist = "MAIN".into(); - t.chapters = chaps(); - t - } - - // Write-only sinks must refuse read() with E_STREAM_WRITE_ONLY, not Ok(None). - #[test] - fn metadata_sinks_refuse_to_be_read_from() { - let code = format!("E{}", crate::error::Error::StreamWriteOnly.code()); - - let cpath = temp_path("chapters.xml"); - let mut c = ChaptersSink::create(&cpath, &sink_title()).unwrap(); - let err = c - .read() - .expect_err("chapters:// is write-only; read must not report a clean EOF"); - assert_eq!(err.kind(), io::ErrorKind::Unsupported); - assert!( - err.to_string().contains(&code), - "expected {code}, got {err}" - ); - let _ = std::fs::remove_file(&cpath); - - let jpath = temp_path("meta.json"); - let mut j = JsonSink::create(&jpath, &sink_title()).unwrap(); - let err = j - .read() - .expect_err("json:// is write-only; read must not report a clean EOF"); - assert_eq!(err.kind(), io::ErrorKind::Unsupported); - assert!( - err.to_string().contains(&code), - "expected {code}, got {err}" - ); - let _ = std::fs::remove_file(&jpath); - } - // A document missing the per-kind arrays must not panic the enrichment. #[test] fn enriching_a_document_without_stream_arrays_does_not_panic() { diff --git a/src/mux/mkv.rs b/src/mux/mkv.rs index 18abc428..dff006f2 100644 --- a/src/mux/mkv.rs +++ b/src/mux/mkv.rs @@ -2375,17 +2375,17 @@ mod tests { } let mut s = crate::mux::mkvstream::MkvStream::open(Cursor::new(data)).unwrap(); - let f0 = crate::pes::Stream::read(&mut s).unwrap().unwrap(); + let f0 = crate::pes::PesSource::read(&mut s).unwrap().unwrap(); assert!( f0.keyframe, "BlockGroup I-frame must read back keyframe=true" ); - let f1 = crate::pes::Stream::read(&mut s).unwrap().unwrap(); + let f1 = crate::pes::PesSource::read(&mut s).unwrap().unwrap(); assert!( !f1.keyframe, "BlockGroup P-frame must read back keyframe=false" ); - let f2 = crate::pes::Stream::read(&mut s).unwrap().unwrap(); + let f2 = crate::pes::PesSource::read(&mut s).unwrap().unwrap(); assert!( !f2.keyframe, "BlockGroup B-frame must read back keyframe=false" @@ -2419,12 +2419,12 @@ mod tests { let data = muxer.writer.into_inner(); let mut s = crate::mux::mkvstream::MkvStream::open(Cursor::new(data)).unwrap(); - let f0 = crate::pes::Stream::read(&mut s).unwrap().unwrap(); + let f0 = crate::pes::PesSource::read(&mut s).unwrap().unwrap(); assert!( f0.keyframe, "SimpleBlock keyframe must read back keyframe=true" ); - let f1 = crate::pes::Stream::read(&mut s).unwrap().unwrap(); + let f1 = crate::pes::PesSource::read(&mut s).unwrap().unwrap(); assert!( !f1.keyframe, "SimpleBlock non-keyframe must read back keyframe=false" @@ -2482,7 +2482,7 @@ mod tests { data: vec![0xCC; 768], duration_ns: None, }; - crate::pes::Stream::write(&mut s, &f).unwrap(); + crate::pes::PesSink::write(&mut s, &f).unwrap(); } // Then the true video keyframe — duration-bearing, as MPEG-2 always is, // so it is written as a BlockGroup (where the keyframe bit is reserved). @@ -2496,20 +2496,20 @@ mod tests { data: vec![0xAA; 188_459], duration_ns: Some(33_366_667), }; - crate::pes::Stream::write(&mut s, &vf).unwrap(); - crate::pes::Stream::finish(&mut s).unwrap(); + crate::pes::PesSink::write(&mut s, &vf).unwrap(); + crate::pes::PesSink::finish(&mut s).unwrap(); drop(s); let f = std::fs::File::open(&path).unwrap(); let mut r = crate::mux::mkvstream::MkvStream::open(f).unwrap(); - let f0 = crate::pes::Stream::read(&mut r).unwrap().unwrap(); + let f0 = crate::pes::PesSource::read(&mut r).unwrap().unwrap(); assert_eq!(f0.track, 0, "first readback frame must be track 0 (video)"); assert!( f0.keyframe, "video keyframe written with 5 audio frames ahead of it must read back keyframe=true" ); for i in 0..5 { - let audio = crate::pes::Stream::read(&mut r) + let audio = crate::pes::PesSource::read(&mut r) .unwrap() .expect("buffered audio must survive"); assert_eq!(audio.track, 1); @@ -2517,7 +2517,7 @@ mod tests { assert_eq!(audio.data, vec![0xCC; 768]); } assert!( - crate::pes::Stream::read(&mut r).unwrap().is_none(), + crate::pes::PesSource::read(&mut r).unwrap().is_none(), "trigger frame must not be duplicated" ); std::fs::remove_dir_all(&dir).ok(); @@ -5652,7 +5652,7 @@ mod tests { #[test] fn muxed_frames_round_trip_through_reader() { - use crate::pes::Stream as _; + use crate::pes::PesSource as _; let tracks = [make_video_track(), make_audio_track()]; // Two video keyframes + interleaved audio, all within one cluster. let frames = vec![ @@ -7422,7 +7422,7 @@ mod tests { // declared A_AC3 / S_HDMV/PGS / V_MPEG2; the mappable tracks keep their frames. #[test] fn mkv_stream_leaves_out_unmappable_codecs() { - use crate::pes::Stream as _; + use crate::pes::{PesSink as _, PesSource as _}; let audio = |codec| { crate::disc::Stream::Audio(AudioStream { pid: 0x1100, diff --git a/src/mux/mkvstream.rs b/src/mux/mkvstream.rs index 7fb149fc..17014fb6 100644 --- a/src/mux/mkvstream.rs +++ b/src/mux/mkvstream.rs @@ -1362,11 +1362,69 @@ impl MkvStream { } } -impl crate::pes::Stream for MkvStream { +impl MkvStream { + /// The title being read or written (both directions open this type). + pub fn info(&self) -> &crate::disc::DiscTitle { + &self.disc_title + } +} + +impl crate::pes::PesSource for MkvStream { fn read(&mut self) -> io::Result> { self.read_parsed() } + fn info(&self) -> &crate::disc::DiscTitle { + &self.disc_title + } + + fn track_timing(&self, track: usize) -> crate::pes::TrackTiming { + match &self.mode { + Mode::Read(rs) => rs.tracks.timings.get(track).copied().unwrap_or_default(), + _ => Default::default(), + } + } + + fn codec_private(&self, track: usize) -> Option> { + if let Mode::Read(ref rs) = self.mode { + // `track` is a stream index but `codec_privates` is keyed by Matroska + // TrackNumber (RFC 9559 §5.1.4.1.1 requires only non-zero) — translate + // through the real map instead of assuming `track + 1`. + let track_num = rs.tracks.num_of(track)?; + rs.codec_privates + .iter() + .find(|(tn, _)| *tn == track_num) + .map(|(_, data)| data.clone()) + } else { + None + } + } + + fn headers_ready(&self) -> bool { + true // MKV has all headers upfront in the EBML header + } + + // Units dropped on read-back: `BlockAdditions` (e.g. a 3D MVC dependent view), Block-less + // BlockGroups, malformed blocks and blocks of undecodable tracks. Reported like a disc-read + // skip: `0` for the write side / sources with none. + fn errors(&self) -> u64 { + match self.mode { + Mode::Read(ref rs) => rs.additions_dropped, + _ => 0, + } + } + + // Cumulative bytes of the units counted by `errors()`; counts the whole + // skipped subtree (payload + EBML framing), an upper bound on the payload. + fn lost_bytes(&self) -> u64 { + match self.mode { + Mode::Read(ref rs) => rs.additions_dropped_bytes, + _ => 0, + } + } +} + +impl crate::pes::PesSink for MkvStream { fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> { if matches!(self.mode, Mode::Read(_)) { return Err(crate::error::Error::StreamReadOnly.into()); @@ -1452,13 +1510,6 @@ impl crate::pes::Stream for MkvStream { self.excluded.seen() } - fn track_timing(&self, track: usize) -> crate::pes::TrackTiming { - match &self.mode { - Mode::Read(rs) => rs.tracks.timings.get(track).copied().unwrap_or_default(), - _ => Default::default(), - } - } - fn set_track_timing( &mut self, track: usize, @@ -1520,44 +1571,6 @@ impl crate::pes::Stream for MkvStream { _ => Ok(false), } } - - fn codec_private(&self, track: usize) -> Option> { - if let Mode::Read(ref rs) = self.mode { - // `track` is a stream index but `codec_privates` is keyed by Matroska - // TrackNumber (RFC 9559 §5.1.4.1.1 requires only non-zero) — translate - // through the real map instead of assuming `track + 1`. - let track_num = rs.tracks.num_of(track)?; - rs.codec_privates - .iter() - .find(|(tn, _)| *tn == track_num) - .map(|(_, data)| data.clone()) - } else { - None - } - } - - fn headers_ready(&self) -> bool { - true // MKV has all headers upfront in the EBML header - } - - // Units dropped on read-back: `BlockAdditions` (e.g. a 3D MVC dependent view), Block-less - // BlockGroups, malformed blocks and blocks of undecodable tracks. Reported like a disc-read - // skip: `0` for the write side / sources with none. - fn errors(&self) -> u64 { - match self.mode { - Mode::Read(ref rs) => rs.additions_dropped, - _ => 0, - } - } - - // Cumulative bytes of the units counted by `errors()`; counts the whole - // skipped subtree (payload + EBML framing), an upper bound on the payload. - fn lost_bytes(&self) -> u64 { - match self.mode { - Mode::Read(ref rs) => rs.additions_dropped_bytes, - _ => 0, - } - } } // ── MKV header parsing (read side) ──────────────────────────── @@ -2922,7 +2935,7 @@ mod tests { assert_eq!(laced, vec![&[1u8, 2][..], &[3, 4][..], &[5, 6][..]]); } use super::*; - use crate::pes::Stream as _; + use crate::pes::{PesSink as _, PesSource as _}; use std::io::Cursor; /// Length-prefix (4-byte big-endian) each NAL, as the H.264 parser emits. @@ -7280,7 +7293,7 @@ mod tests { let stream = MkvStream::open(Cursor::new(synthetic_mkv("V_MPEG2", &chapters, &[]))).unwrap(); - let got: Vec<(f64, String)> = crate::pes::Stream::info(&stream) + let got: Vec<(f64, String)> = crate::pes::PesSource::info(&stream) .chapters .iter() .map(|c| (c.time_secs, c.name.clone())) @@ -7310,7 +7323,7 @@ mod tests { fn chapter_names(chapters: &[u8]) -> Vec { let s = MkvStream::open(Cursor::new(synthetic_mkv("V_MPEG2", chapters, &[]))).unwrap(); - crate::pes::Stream::info(&s) + crate::pes::PesSource::info(&s) .chapters .iter() .map(|c| c.name.clone()) @@ -7362,7 +7375,7 @@ mod tests { assert!(chapter_names(&c).is_empty()); assert!(probe_mkv(Cursor::new(bytes.clone())).is_ok()); let mut s = MkvStream::open(Cursor::new(bytes)).unwrap(); - assert!(crate::pes::Stream::read(&mut s).unwrap().is_none()); + assert!(crate::pes::PesSource::read(&mut s).unwrap().is_none()); } } @@ -7379,14 +7392,14 @@ mod tests { bytes.extend_from_slice(&tags); assert!(probe_mkv(Cursor::new(bytes.clone())).is_ok()); let s = MkvStream::open(Cursor::new(bytes)).unwrap(); - assert_eq!(crate::pes::Stream::info(&s).streams.len(), 1); + assert_eq!(crate::pes::PesSource::info(&s).streams.len(), 1); } #[test] fn read_back_maps_mpeg1_and_av1_codec_ids() { for (id, want) in [("V_MPEG1", Codec::Mpeg1), ("V_AV1", Codec::Av1)] { let stream = MkvStream::open(Cursor::new(synthetic_mkv(id, &[], &[]))).unwrap(); - match &crate::pes::Stream::info(&stream).streams[0] { + match &crate::pes::PesSource::info(&stream).streams[0] { crate::disc::Stream::Video(v) => assert_eq!(v.codec, want, "{id}"), _ => panic!("expected a video stream"), } @@ -7403,9 +7416,9 @@ mod tests { cluster.extend_from_slice(&body); let mut stream = MkvStream::open(Cursor::new(synthetic_mkv("V_MPEG2", &[], &cluster))).unwrap(); - assert!(crate::pes::Stream::read(&mut stream).unwrap().is_none()); - assert_eq!(crate::pes::Stream::errors(&stream), 1); - assert!(crate::pes::Stream::lost_bytes(&stream) > 0); + assert!(crate::pes::PesSource::read(&mut stream).unwrap().is_none()); + assert_eq!(crate::pes::PesSource::errors(&stream), 1); + assert!(crate::pes::PesSource::lost_bytes(&stream) > 0); } } @@ -7413,7 +7426,7 @@ mod tests { #[cfg(test)] mod readback_tests { use super::*; - use crate::pes::Stream as _; + use crate::pes::PesSource as _; use std::io::Cursor; const DISPLAY_UNIT: u32 = 0x54B2; diff --git a/src/mux/mod.rs b/src/mux/mod.rs index 744c8292..f46c8ecf 100644 --- a/src/mux/mod.rs +++ b/src/mux/mod.rs @@ -1,9 +1,6 @@ -//! Stream-based I/O pipeline. Two muxer families live here: -//! -//! 1. **Bidirectional PES streams** (`disc`, `mkv`, `m2ts`, `network`, `stdio`, `null`) implement the [`crate::pes::Stream`] interface: read a format → PES frames, or write PES frames → a format. -//! 2. **Write-only sequential-sink muxers** (`fmp4`, `hevc`, `m2ts_mux`) consume PES frames and write a container to a `SequentialSink`; they do not implement the read loop below. -//! -//! The bidirectional family is driven like this: +//! The PES half of the pipeline: inputs implement [`crate::pes::PesSource`] (read a format → +//! PES frames), outputs [`crate::pes::PesSink`] (write PES frames → a format). Every input +//! scheme opens through [`open_source`] / [`input`], every output through [`output`]. //! //! ```text //! let mut input = input("iso://Disc.iso", &opts)?; @@ -15,14 +12,15 @@ //! output.finish()?; //! ``` //! -//! For disc→ISO (raw sector copy), use `freemkv_engine::recovery::copy` instead. +//! A whole-disc copy (`iso://`, `dir://`) is a block chain: see [`crate::io::open_block_sink`]. // Public modules — types here are intentionally part of the consumable API. -pub mod disc; pub mod driver; pub mod pipelined_stream; pub mod resolve; pub mod select; +mod selected; +pub mod source; // Internal-only modules (referenced only via `crate::mux::…`; not public API). // `#[allow(dead_code)]`: narrowing `pub`→`pub(crate)` surfaces helpers only @@ -50,16 +48,8 @@ pub(crate) mod m2ts; pub mod meta; pub(crate) mod meta_sink; -// ── Sequential-sink muxers ── write-only PES → `SequentialSink`. `pub(crate)`+ -// `allow(dead_code)`: `fmp4` is a STUB (pub would lock a half-built type into -// v1.0); `m2ts_mux`/`hevc` are sink-split scaffolding (production: `tsmux`/`mkv`). pub(crate) mod fit; -#[allow(dead_code)] -pub(crate) mod fmp4; -#[allow(dead_code)] pub(crate) mod hevc; -#[allow(dead_code)] -pub(crate) mod m2ts_mux; pub(crate) mod mkv; pub(crate) mod mkvstream; pub(crate) mod mp4; @@ -83,14 +73,14 @@ pub(crate) mod videomap; // `demux://`/`fvi://` sinks are built internally by `output()`; not public API. // The provenance types ARE public: `output()` takes a `SourceInfo` so an `fvi://` // destination records the INPUT it was built from (§6.2), not the file written. -pub use disc::DiscStream; -pub use driver::{MuxEvents, MuxOptions, MuxOutcome, MuxSource, mux_with_keys}; +pub use driver::{MuxOptions, MuxOutcome, mux_url, mux_with_keys}; pub use fit::{FitReport, SkipReason, fit_report}; pub use m2ts::M2tsStream; pub use mkvstream::{ MkvProbe, MkvProbeTrack, MkvStream, MkvTrackKind, parse_freemkv_version, probe_mkv, probe_mkv_with_cues, }; +pub use source::{ScannedTitle, Source, open_source}; pub use videomap::{Medium, SourceInfo}; // `Mp4Sink` is public so a caller driving the sink can ask `final_report()` what // the finished file contains — the pre-mux `mp4_fit_report` is only a prediction, @@ -100,7 +90,7 @@ pub use mpg::MpgSink; pub use network::{NetworkStream, is_blocked_ip}; pub use null::NullStream; pub use pipelined_stream::PipelinedPesStream; -pub use resolve::{InputOptions, StreamUrl, disc_root_of, input, output, parse_url}; +pub use resolve::{InputOptions, SinkCaps, StreamUrl, disc_root_of, input, output, parse_url}; pub use stdio::StdioStream; use std::io::{Seek, Write}; diff --git a/src/mux/mp4/mod.rs b/src/mux/mp4/mod.rs index 4f88a023..47485280 100644 --- a/src/mux/mp4/mod.rs +++ b/src/mux/mp4/mod.rs @@ -9,7 +9,7 @@ //! can't carry are **excluded, never silently dropped** — [`fit_report`] says why. use crate::disc::{Codec, DiscTitle, Stream as DiscStream}; -use crate::pes::{PesFrame, Stream}; +use crate::pes::{PesFrame, PesSink}; use std::io::{self, Seek, SeekFrom, Write}; mod audio; @@ -435,11 +435,7 @@ impl Mp4Sink { } } -impl Stream for Mp4Sink { - fn read(&mut self) -> io::Result> { - Err(crate::error::Error::StreamWriteOnly.into()) - } - +impl PesSink for Mp4Sink { fn write(&mut self, frame: &PesFrame) -> io::Result<()> { if self.finished { return Err(crate::error::Error::StreamClosed.into()); @@ -2291,22 +2287,6 @@ mod tests { // ── Stream::read is write-only ────────────────────────────────────────── - #[test] - fn mp4_sink_read_is_unsupported() { - let t = title( - vec![hevc_video(), audio(Codec::Ac3, "eng")], - vec![Some(vec![1, 2, 3]), None], - ); - let mut s = Mp4Sink::create(std::io::Cursor::new(Vec::new()), &t).unwrap(); - let err = s.read().unwrap_err(); - assert!( - err.to_string() - .starts_with(&format!("E{}", crate::error::E_STREAM_WRITE_ONLY)), - "an Mp4Sink is write-only; read() must report that, not silently return Ok(None); \ - got {err}" - ); - } - #[test] fn write_after_finish_is_rejected() { let t = title(vec![hevc_video()], vec![Some(vec![1, 2, 3])]); diff --git a/src/mux/mp4/read.rs b/src/mux/mp4/read.rs index 351ced20..45d89087 100644 --- a/src/mux/mp4/read.rs +++ b/src/mux/mp4/read.rs @@ -14,7 +14,7 @@ use crate::disc::{ Resolution, SampleRate, Stream as DiscStream, VideoStream, }; use crate::labels::LabelPurpose; -use crate::pes::{PesFrame, Stream}; +use crate::pes::{PesFrame, PesSource}; use std::io::{self, Read, Seek, SeekFrom}; const NS: i128 = 1_000_000_000; @@ -328,7 +328,7 @@ impl Mp4Reader { } } -impl Stream for Mp4Reader { +impl PesSource for Mp4Reader { fn read(&mut self) -> io::Result> { let Some(s) = self.samples.get(self.cursor) else { return Ok(None); @@ -356,14 +356,6 @@ impl Stream for Mp4Reader { })) } - fn write(&mut self, _frame: &PesFrame) -> io::Result<()> { - Err(crate::error::Error::StreamReadOnly.into()) - } - - fn finish(&mut self) -> io::Result<()> { - Ok(()) - } - fn info(&self) -> &DiscTitle { &self.title } @@ -1216,6 +1208,7 @@ fn parse_stss(b: &[u8]) -> std::collections::HashSet { #[cfg(test)] mod tests { use super::*; + use crate::pes::PesSink as _; // `Read + Seek` backed by a small crafted prefix + endless zeros, reporting `len` bytes on // `seek(End)` — exercises MAX_ALLOC_BYTES-scale reads without a real multi-GiB source @@ -1363,7 +1356,7 @@ mod tests { Codec, DiscTitle, FrameRate, HdrFormat, Resolution, Stream as DiscStreamE, VideoStream, }; use crate::mux::mp4::Mp4Sink; - use crate::pes::{PesFrame, Stream as _}; + use crate::pes::PesFrame; use std::io::Cursor; // The MP4 writer requires a primary video track, so build an HEVC one. @@ -1415,7 +1408,6 @@ mod tests { #[test] fn stream_read_allows_a_sample_of_exactly_max_alloc_bytes() { use crate::disc::DiscTitle; - use crate::pes::Stream as _; let mut rd = Mp4Reader { file: FakeBigReader { @@ -1447,7 +1439,6 @@ mod tests { #[test] fn stream_read_rejects_a_sample_one_byte_over_max_alloc_bytes() { use crate::disc::DiscTitle; - use crate::pes::Stream as _; let mut rd = Mp4Reader { file: FakeBigReader { @@ -1476,38 +1467,6 @@ mod tests { ); } - /// `Mp4Reader` is a read-only source — `mp4://` is never a mux destination - /// — so `write` must always fail, never silently succeed and drop the - /// frame on the floor. - #[test] - fn stream_write_is_always_rejected() { - use crate::disc::DiscTitle; - use crate::pes::Stream as _; - use std::io::Cursor; - - let mut rd = Mp4Reader { - file: Cursor::new(Vec::::new()), - file_len: 0, - title: DiscTitle::empty(), - samples: Vec::new(), - cursor: 0, - }; - let frame = PesFrame { - discard_padding_ns: 0, - track: 0, - pts: 0, - keyframe: true, - data: vec![1, 2, 3], - duration_ns: None, - source: None, - coding: None, - }; - assert!( - rd.write(&frame).is_err(), - "Mp4Reader::write must always return an error" - ); - } - #[test] // Underscores mark BITFIELD boundaries in the bitstream header being built // (e.g. 5-bit then 3-bit field), not thousands-style digit groups. @@ -1520,7 +1479,7 @@ mod tests { SampleRate, Stream as DiscStreamE, VideoStream, }; use crate::mux::mp4::Mp4Sink; - use crate::pes::{PesFrame, Stream as _}; + use crate::pes::PesFrame; use std::io::Cursor; let mut t = DiscTitle::empty(); @@ -1625,7 +1584,9 @@ mod tests { // `mp4://` reads the same file through the decryption stage: a clear container // (Opaque) is handed through untouched, seeks included. let url = format!("mp4://{}", path.display()); - let mut staged = crate::mux::resolve::input(&url, &Default::default()).unwrap(); + let mut staged = + crate::mux::resolve::input(&url, &Default::default(), &crate::ctx::Ctx::default()) + .unwrap(); let mut again = Vec::new(); while let Some(f) = staged.read().unwrap() { again.push((f.track, f.data.len(), f.keyframe, f.data)); @@ -1709,7 +1670,7 @@ mod tests { Codec, DiscTitle, FrameRate, HdrFormat, Resolution, Stream as DiscStreamE, VideoStream, }; use crate::mux::mp4::Mp4Sink; - use crate::pes::{PesFrame, Stream as _}; + use crate::pes::PesFrame; use std::io::Cursor; const FRAME_NS: i64 = 40_000_000; // 25 fps, exact @@ -1796,7 +1757,7 @@ mod tests { Codec, DiscTitle, FrameRate, HdrFormat, Resolution, Stream as DiscStreamE, VideoStream, }; use crate::mux::mp4::Mp4Sink; - use crate::pes::{PesFrame, Stream as _}; + use crate::pes::PesFrame; use std::io::Cursor; const FRAME_NS: i64 = 40_000_000; // 25 fps, exact @@ -1883,7 +1844,7 @@ mod tests { Resolution, SampleRate, Stream as DiscStreamE, VideoStream, }; use crate::mux::mp4::Mp4Sink; - use crate::pes::{PesFrame, Stream as _}; + use crate::pes::PesFrame; use std::io::Cursor; let mut t = DiscTitle::empty(); @@ -2610,7 +2571,7 @@ mod tests { fn a_96k_dts_hd_track_round_trips_its_rate() { use crate::disc::{AudioChannels, AudioStream, Codec, DiscTitle, LabelPurpose}; use crate::mux::mp4::Mp4Sink; - use crate::pes::{PesFrame, Stream as _}; + use crate::pes::PesFrame; use std::io::Cursor; let mut t = DiscTitle::empty(); diff --git a/src/mux/mpg/mod.rs b/src/mux/mpg/mod.rs index 9fc51520..78bad33a 100644 --- a/src/mux/mpg/mod.rs +++ b/src/mux/mpg/mod.rs @@ -1,5 +1,5 @@ //! `mpg://`: an ISO/IEC 13818-1 program stream written from the PES IR (mpg-output-design -//! v5 §2, L2), one more `pes::Stream` sink. +//! v5 §2, L2), one more `PesSink`. //! //! The pack layer is regenerated: packs, SCR, mux rate, system header, PSM, PES headers //! and DTS. ES bytes are kept (DVD LPCM is re-packed losslessly, G8); PTS is kept exactly, @@ -24,7 +24,7 @@ use pstd::{Au, BufferSpec, Mux, Payload, PstdCounters, StreamSpec}; use crate::disc::{Codec, DiscTitle, Stream as DiscStream}; use crate::mux::codec::ns_to_ticks; use crate::mux::decode_ts::{DtsCounters, DtsDeriver}; -use crate::pes::{PesFrame, Stream}; +use crate::pes::{PesFrame, PesSink}; use std::collections::VecDeque; use std::io::{self, Write}; @@ -653,11 +653,7 @@ impl MpgSink { } } -impl Stream for MpgSink { - fn read(&mut self) -> io::Result> { - Err(crate::error::Error::StreamWriteOnly.into()) - } - +impl PesSink for MpgSink { fn write(&mut self, frame: &PesFrame) -> io::Result<()> { if self.finished { return Err(crate::error::Error::StreamClosed.into()); diff --git a/src/mux/mpg/tests.rs b/src/mux/mpg/tests.rs index 5e06db45..92ec8c19 100644 --- a/src/mux/mpg/tests.rs +++ b/src/mux/mpg/tests.rs @@ -1237,7 +1237,8 @@ fn read_all(path: &std::path::Path) -> std::io::Result<(DiscTitle, Vec // The sector pipeline spawns a prefetcher (a Drive holder). let _g = crate::sector::prefetched::holder_test_lock(); let url = format!("mpg://{}", path.display()); - let mut input = crate::mux::resolve::input(&url, &Default::default())?; + let mut input = + crate::mux::resolve::input(&url, &Default::default(), &crate::ctx::Ctx::default())?; let mut frames = Vec::new(); while let Some(f) = input.read()? { frames.push(f); @@ -1462,16 +1463,21 @@ fn the_crack_honours_halt_and_runs_under_raw() { let _g = crate::sector::prefetched::holder_test_lock(); let halt = crate::halt::Halt::new(); halt.cancel(); - let e = crate::mux::resolve::input_with_halt(&url, &Default::default(), Some(&halt)) - .err() - .expect("a stopped crack is Halted"); + let e = crate::mux::resolve::input( + &url, + &Default::default(), + &crate::ctx::Ctx::new(halt.clone()), + ) + .err() + .expect("a stopped crack is Halted"); assert_eq!(crate::error::error_code(&e), Some(crate::error::E_HALTED)); let raw = crate::mux::resolve::InputOptions { raw: true, ..Default::default() }; let lead_url = format!("mpg://{}", lead_path.display()); - let input = crate::mux::resolve::input(&lead_url, &raw).expect("raw input"); + let input = crate::mux::resolve::input(&lead_url, &raw, &crate::ctx::Ctx::default()) + .expect("raw input"); assert!( matches!(&input.info().streams[0], DiscStream::Video(v) if v.resolution == Resolution::R1080p), "the head scan read the descrambled sequence header: {:?}", @@ -2053,7 +2059,7 @@ fn a_map_with_a_bad_crc_is_ignored() { // offset); the extension is the mkv sink's to exclude (M1, J23). #[test] fn an_mpg_source_remuxes_to_mkv() { - use crate::mux::driver::{MuxOptions, MuxSource, NoopEvents, mux_with_keys}; + use crate::mux::driver::MuxOptions; let fx = fixture(&Opts { spu_tracks: 0, ..Opts::default() @@ -2068,19 +2074,16 @@ fn an_mpg_source_remuxes_to_mkv() { batch_sectors: 8192, raw: false, selection: Default::default(), + title_index: 0, }; let out = { let _g = crate::sector::prefetched::holder_test_lock(); - mux_with_keys( - MuxSource::Url { - url: &url, - opts: Default::default(), - }, + crate::mux::mux_url( + &url, None, &format!("mkv://{}", mkv.display()), &opts, - &crate::halt::Halt::new(), - std::sync::Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .unwrap() }; @@ -2090,9 +2093,12 @@ fn an_mpg_source_remuxes_to_mkv() { vec![2], "the extension, once seen (J23)" ); - let mut input = - crate::mux::resolve::input(&format!("mkv://{}", mkv.display()), &Default::default()) - .unwrap(); + let mut input = crate::mux::resolve::input( + &format!("mkv://{}", mkv.display()), + &Default::default(), + &crate::ctx::Ctx::default(), + ) + .unwrap(); let mut from_mkv = Vec::new(); while let Some(f) = input.read().unwrap() { from_mkv.push(f); diff --git a/src/mux/network.rs b/src/mux/network.rs index 5eb135bd..5341a233 100644 --- a/src/mux/network.rs +++ b/src/mux/network.rs @@ -342,7 +342,14 @@ fn ensure_header_written( Ok(()) } -impl crate::pes::Stream for NetworkStream { +impl NetworkStream { + /// The title being read or written (both directions open this type). + pub fn info(&self) -> &crate::disc::DiscTitle { + &self.disc_title + } +} + +impl crate::pes::PesSource for NetworkStream { fn read(&mut self) -> io::Result> { match &mut self.mode { Mode::Read { reader, meta } => { @@ -352,6 +359,25 @@ impl crate::pes::Stream for NetworkStream { _ => Err(crate::error::Error::StreamWriteOnly.into()), } } + + fn info(&self) -> &DiscTitle { + &self.disc_title + } + + fn track_timing(&self, track: usize) -> crate::pes::TrackTiming { + match &self.mode { + Mode::Read { meta, .. } => meta.timing(track), + Mode::Write { .. } => Default::default(), + } + } + + // The sender's codec privates travel in the FMKV header (as for stdio). + fn codec_private(&self, track: usize) -> Option> { + self.disc_title.codec_privates.get(track).cloned().flatten() + } +} + +impl crate::pes::PesSink for NetworkStream { fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> { match &mut self.mode { Mode::Write { @@ -367,6 +393,7 @@ impl crate::pes::Stream for NetworkStream { _ => Err(crate::error::Error::StreamReadOnly.into()), } } + fn finish(&mut self) -> io::Result<()> { if let Mode::Write { writer, @@ -386,6 +413,7 @@ impl crate::pes::Stream for NetworkStream { } Ok(()) } + // A failed or stopped title: reset instead of the clean end, so the receiver // reports an error rather than a complete (truncated) title. fn finish_incomplete(&mut self) -> io::Result<()> { @@ -394,15 +422,11 @@ impl crate::pes::Stream for NetworkStream { } Ok(()) } + fn info(&self) -> &DiscTitle { &self.disc_title } - fn track_timing(&self, track: usize) -> crate::pes::TrackTiming { - match &self.mode { - Mode::Read { meta, .. } => meta.timing(track), - Mode::Write { .. } => Default::default(), - } - } + fn set_track_timing( &mut self, track: usize, @@ -419,10 +443,6 @@ impl crate::pes::Stream for NetworkStream { Mode::Read { .. } => Err(crate::error::Error::StreamReadOnly.into()), } } - // The sender's codec privates travel in the FMKV header (as for stdio). - fn codec_private(&self, track: usize) -> Option> { - self.disc_title.codec_privates.get(track).cloned().flatten() - } } // Record `timing` for stream `track` of a `tracks`-stream title. @@ -632,9 +652,9 @@ mod tests { let handle = std::thread::spawn(move || { addr_tx.send(addr).unwrap(); let mut ns = NetworkStream::accept_from(listener).unwrap(); - let info = pes::Stream::info(&ns).clone(); + let info = pes::PesSource::info(&ns).clone(); let mut frames = Vec::new(); - while let Ok(Some(f)) = pes::Stream::read(&mut ns) { + while let Ok(Some(f)) = pes::PesSource::read(&mut ns) { frames.push(f); } (info, frames) @@ -655,8 +675,8 @@ mod tests { data: vec![0x47; 192], duration_ns: None, }; - pes::Stream::write(&mut writer, &frame).unwrap(); - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::write(&mut writer, &frame).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let (info, frames) = handle.join().unwrap(); assert_eq!(info.playlist, "NetworkTest"); @@ -677,7 +697,7 @@ mod tests { let mut ns = NetworkStream::accept_from(listener).unwrap(); let mut n = 0; loop { - match crate::pes::Stream::read(&mut ns) { + match crate::pes::PesSource::read(&mut ns) { Ok(Some(_)) => n += 1, end => return (n, end), } @@ -703,7 +723,7 @@ mod tests { // reach the receiver as an error, never as the clean end of a short title. #[test] fn a_sender_that_fails_mid_title_is_an_error_at_the_receiver() { - use crate::pes::Stream as _; + use crate::pes::PesSink as _; for incomplete in [false, true] { let (addr, handle) = spawn_ending_reader(); let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) @@ -763,7 +783,7 @@ mod tests { addr_tx.send(addr).unwrap(); // listen()'s read_header must succeed (header present), not error. let ns = NetworkStream::accept_from(listener).unwrap(); - pes::Stream::info(&ns).playlist.clone() + pes::PesSource::info(&ns).playlist.clone() }); let addr = addr_rx.recv().unwrap(); @@ -772,7 +792,7 @@ mod tests { .unwrap() .meta(&dt); // No write() at all — straight to finish(). - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let playlist = handle.join().unwrap(); assert_eq!( @@ -806,9 +826,9 @@ mod tests { let addr = listener.local_addr().unwrap(); let handle = std::thread::spawn(move || { let mut ns = NetworkStream::accept_from(listener).unwrap(); - let info = pes::Stream::info(&ns).clone(); + let info = pes::PesSource::info(&ns).clone(); let mut frames = Vec::new(); - while let Ok(Some(f)) = pes::Stream::read(&mut ns) { + while let Ok(Some(f)) = pes::PesSource::read(&mut ns) { frames.push(f); } (info, frames) @@ -830,7 +850,7 @@ mod tests { let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) .unwrap() .meta(&dt); - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let (_info, _frames) = handle.join().unwrap(); // Now build a fresh read-side stream and confirm its write() errors. @@ -850,14 +870,14 @@ mod tests { duration_ns: None, }; // Read side: writing must be a typed StreamReadOnly error. - let err = pes::Stream::write(&mut ns, &frame).expect_err("read side write must error"); + let err = pes::PesSink::write(&mut ns, &frame).expect_err("read side write must error"); err.kind() }); // Drive the accept: connect + send header so accept_from completes. let mut w2 = NetworkStream::connect_vetted(&addr2.to_string(), false) .unwrap() .meta(&dt); - pes::Stream::finish(&mut w2).unwrap(); + pes::PesSink::finish(&mut w2).unwrap(); let kind = h.join().unwrap(); // E_STREAM_READ_ONLY (9000) maps to Unsupported. assert_eq!(kind, io::ErrorKind::Unsupported); @@ -874,10 +894,10 @@ mod tests { let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) .unwrap() .meta(&dt); - let err = pes::Stream::read(&mut writer).expect_err("write side read must error"); + let err = pes::PesSource::read(&mut writer).expect_err("write side read must error"); // E_STREAM_WRITE_ONLY (9001) maps to Unsupported. assert_eq!(err.kind(), io::ErrorKind::Unsupported); - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let _ = handle.join().unwrap(); } @@ -903,9 +923,9 @@ mod tests { data: vec![i; 100 + i as usize], duration_ns: None, }; - pes::Stream::write(&mut writer, &frame).unwrap(); + pes::PesSink::write(&mut writer, &frame).unwrap(); } - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let (info, frames) = handle.join().unwrap(); // Title parsed once and intact. assert_eq!(info.streams.len(), 2); @@ -935,7 +955,7 @@ mod tests { let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) .unwrap() .meta(&dt); - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let (info, _frames) = handle.join().unwrap(); // The receiver default title is empty (playlist ""); it must have // been replaced by the sender's header-carried title. @@ -1004,7 +1024,7 @@ mod tests { std::thread::spawn(move || { let r = NetworkStream::accept_from_with_halt(listener, Some(h)); let _ = ready_tx.send(()); - let _ = tx.send(r.and_then(|mut ns| pes::Stream::read(&mut ns).map(|_| ()))); + let _ = tx.send(r.and_then(|mut ns| pes::PesSource::read(&mut ns).map(|_| ()))); }); let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) .unwrap() @@ -1104,7 +1124,7 @@ mod tests { let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) .unwrap() .meta(&sample_title()); - crate::pes::Stream::finish(&mut writer).unwrap(); + crate::pes::PesSink::finish(&mut writer).unwrap(); assert_eq!(handle.join().unwrap(), None); } @@ -1191,7 +1211,7 @@ mod tests { }; ( keepalive, - pes::Stream::read(&mut ns).map(|f| f.map(|f| f.data)), + pes::PesSource::read(&mut ns).map(|f| f.map(|f| f.data)), ) }); let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) @@ -1218,8 +1238,8 @@ mod tests { data: vec![7; 4], duration_ns: None, }; - pes::Stream::write(&mut writer, &frame).unwrap(); - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::write(&mut writer, &frame).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let (keepalive, read) = handle.join().unwrap(); assert!(keepalive, "keepalive must be armed"); assert_eq!(read.unwrap(), Some(vec![7; 4])); @@ -1236,7 +1256,7 @@ mod tests { let mut writer = NetworkStream::connect_vetted(&addr.to_string(), false) .unwrap() .meta(&title); - pes::Stream::finish(&mut writer).unwrap(); + pes::PesSink::finish(&mut writer).unwrap(); let cps = handle.join().unwrap(); assert_eq!(cps, title.codec_privates); } @@ -1245,7 +1265,7 @@ mod tests { std::net::SocketAddr, std::thread::JoinHandle>>>, ) { - use crate::pes::Stream as _; + use crate::pes::PesSource as _; let listener = TcpListener::bind("127.0.0.1:0").unwrap(); let addr = listener.local_addr().unwrap(); let handle = std::thread::spawn(move || { @@ -1259,7 +1279,7 @@ mod tests { // survive the PES wire hop. #[test] fn opus_track_timing_and_padding_survive_network_to_mkv() { - use crate::pes::{self, Stream as _}; + use crate::pes::{self, PesSink as _, PesSource as _}; let mut title = sample_title(); title.streams.truncate(1); title.streams[0] = Stream::Audio(AudioStream { @@ -1323,15 +1343,18 @@ mod tests { // frames a receiver reads back (public `connect` refuses loopback, so this seam). #[test] fn parity_network_fmkv_wire() { - use crate::pes::Stream as _; + use crate::pes::{PesSink as _, PesSource as _}; use std::io::Read as _; let clip = crate::test_util::synthetic_bd_clip(6); let dir = tempfile::tempdir().unwrap(); let path = dir.path().join("c.m2ts"); std::fs::write(&path, &clip).unwrap(); - let mut src = - crate::mux::resolve::input(&format!("m2ts://{}", path.display()), &Default::default()) - .unwrap(); + let mut src = crate::mux::resolve::input( + &format!("m2ts://{}", path.display()), + &Default::default(), + &crate::ctx::Ctx::default(), + ) + .unwrap(); let title = src.info().clone(); let mut frames = Vec::new(); while let Some(f) = src.read().unwrap() { diff --git a/src/mux/null.rs b/src/mux/null.rs index 58b6e8f3..b1c633f9 100644 --- a/src/mux/null.rs +++ b/src/mux/null.rs @@ -16,19 +16,15 @@ impl NullStream { } } -impl crate::pes::Stream for NullStream { - fn read(&mut self) -> io::Result> { - // Write-only sink: per the Stream trait contract, read() on a - // write-opened stream returns StreamWriteOnly. Returning Ok(None) - // would be misread as a legitimate empty stream. - Err(crate::error::Error::StreamWriteOnly.into()) - } +impl crate::pes::PesSink for NullStream { fn write(&mut self, _: &crate::pes::PesFrame) -> io::Result<()> { Ok(()) } + fn finish(&mut self) -> io::Result<()> { Ok(()) } + fn info(&self) -> &DiscTitle { &self.disc_title } @@ -37,13 +33,13 @@ impl crate::pes::Stream for NullStream { #[cfg(test)] mod tests { use super::*; - use crate::pes::Stream; + use crate::pes::PesSink; /// Verify NullStream routes through the `Stream` trait object cleanly. #[test] fn stream_via_dyn_object_writes_and_finishes() { let title = DiscTitle::empty(); - let mut sink: Box = Box::new(NullStream::new(&title)); + let mut sink: Box = Box::new(NullStream::new(&title)); let frame = crate::pes::PesFrame { discard_padding_ns: 0, @@ -60,16 +56,6 @@ mod tests { sink.finish().unwrap(); } - /// read() on the write-only NullStream must return StreamWriteOnly, - /// not Ok(None) (which a caller would misread as an empty stream). - #[test] - fn read_returns_write_only_error() { - let title = DiscTitle::empty(); - let mut sink = NullStream::new(&title); - let err = Stream::read(&mut sink).expect_err("read on a sink must error"); - assert_eq!(err.kind(), io::ErrorKind::Unsupported); - } - /// finish() must be idempotent and safe to call repeatedly — a benchmark /// driver may finish more than once. Each must be Ok(()), and writes /// after finish must still succeed (NullStream has no terminal state). diff --git a/src/mux/parity_tests.rs b/src/mux/parity_tests.rs index d8d4573f..fa7deb0c 100644 --- a/src/mux/parity_tests.rs +++ b/src/mux/parity_tests.rs @@ -2,16 +2,14 @@ //! every sink driven through `mux_with_keys`, over the synthetic BD clip. Pinned as output //! hashes, loss counters and refusal codes (see [`Golden`]); cells are `parity_*`. -use super::driver::{MuxOptions, MuxOutcome, MuxSource, NoopEvents, mux_with_keys}; +use super::driver::{MuxOptions, MuxOutcome}; use super::resolve::{InputOptions, input}; use super::select::{PidFilter, StreamSelection}; use crate::aacs::content::{ALIGNED_UNIT_LEN, encrypt_unit}; use crate::consts::BD_SOURCE_PACKET_BYTES as PKT; -use crate::halt::Halt; -use crate::keys::ResolvedKeySet; +use crate::keys::KeyRing; use crate::test_util::{CLIP_AUDIO_PIDS, Golden, synthetic_bd_clip}; use std::path::Path; -use std::sync::Arc; // A synthetic test key, not key material from any disc. const KEY: [u8; 16] = [0x5A; 16]; @@ -121,16 +119,18 @@ fn sinks( "null" => "null://".to_string(), s => format!("{s}://{}", out.join(dest_name(s)).display()), }; - let r = mux_with_keys( - MuxSource::Url { - url: &url, - opts: iopts.clone(), - }, - None, + let m = MuxOptions { + title_index: iopts.title_index.unwrap_or(0), + raw: iopts.raw, + selection: iopts.selection.clone(), + ..mopts.clone() + }; + let r = crate::mux::mux_url( + &url, + iopts.keys.as_ref(), &dest, - mopts, - &Halt::new(), - Arc::new(NoopEvents), + &m, + &crate::ctx::Ctx::default(), ); record_run(g, sink, &r); record_tree(g, sink, &out); @@ -215,16 +215,12 @@ fn parity_sinks_from_mkv_mpg_and_fmkv_sources() { std::fs::write(&src, &clip).unwrap(); for (scheme, name) in [("mkv", "a.mkv"), ("mpg", "a.mpg"), ("m2ts", "a.fmkv.m2ts")] { let p = dir.path().join(name); - mux_with_keys( - MuxSource::Url { - url: &format!("m2ts://{}", src.display()), - opts: Default::default(), - }, + crate::mux::mux_url( + &format!("m2ts://{}", src.display()), None, &format!("{scheme}://{}", p.display()), &MuxOptions::default(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .unwrap(); let bytes = std::fs::read(&p).unwrap(); @@ -237,7 +233,12 @@ fn parity_sinks_from_mkv_mpg_and_fmkv_sources() { &Default::default(), ); let tag = format!("{scheme}-src"); - let mut s = input(&format!("{scheme}://{}", p.display()), &Default::default()).unwrap(); + let mut s = input( + &format!("{scheme}://{}", p.display()), + &Default::default(), + &crate::ctx::Ctx::default(), + ) + .unwrap(); record_frames(&mut g, &tag, s.as_mut()); } g.check(); @@ -245,7 +246,7 @@ fn parity_sinks_from_mkv_mpg_and_fmkv_sources() { // The frames a source yields (count, a digest over track/pts/keyframe/data), its stream // list, and the stream's loss counters; or the code that refused it first. -fn record_frames(g: &mut Golden, tag: &str, s: &mut dyn crate::pes::Stream) { +fn record_frames(g: &mut Golden, tag: &str, s: &mut dyn crate::pes::PesSource) { let streams: Vec = s .info() .streams @@ -293,7 +294,11 @@ fn record_source(g: &mut Golden, tag: &str, scheme: &str, bytes: &[u8], opts: &I // Keep the temp file until the stream is drained (the sector pipeline reads lazily). let p = std::env::temp_dir().join(format!("fmkv-parity-{tag}-{}", std::process::id())); std::fs::write(&p, bytes).unwrap(); - let r = input(&format!("{scheme}://{}", p.display()), opts); + let r = input( + &format!("{scheme}://{}", p.display()), + opts, + &crate::ctx::Ctx::default(), + ); match r { Ok(mut s) => { let mut digest = Vec::new(); @@ -329,7 +334,7 @@ fn record_source(g: &mut Golden, tag: &str, scheme: &str, bytes: &[u8], opts: &I fn keyed(keys: &[[u8; 16]]) -> InputOptions { InputOptions { - keys: Some(ResolvedKeySet::held_for_test(keys)), + keys: Some(KeyRing::held_for_test(keys)), ..Default::default() } } @@ -391,16 +396,12 @@ pub(crate) fn clear_mpg() -> Vec { let src = dir.path().join("c.m2ts"); std::fs::write(&src, &clip).unwrap(); let dst = dir.path().join("c.mpg"); - mux_with_keys( - MuxSource::Url { - url: &format!("m2ts://{}", src.display()), - opts: Default::default(), - }, + crate::mux::mux_url( + &format!("m2ts://{}", src.display()), None, &format!("mpg://{}", dst.display()), &MuxOptions::default(), - &Halt::new(), - Arc::new(NoopEvents), + &crate::ctx::Ctx::default(), ) .unwrap(); std::fs::read(&dst).unwrap() diff --git a/src/mux/pipelined_stream.rs b/src/mux/pipelined_stream.rs index e67e07e6..e51294fc 100644 --- a/src/mux/pipelined_stream.rs +++ b/src/mux/pipelined_stream.rs @@ -1,11 +1,11 @@ //! `PipelinedPesStream` — the read-side of the freemkv mux highway. //! //! Given a [`crate::mux::demux_thread::DemuxThread`] and a set of codec -//! parsers, this struct implements [`crate::pes::Stream`] by running codec +//! parsers, this struct implements [`crate::pes::PesSource`] by running codec //! parse on the caller's thread and emitting `PesFrame`s one at a time. //! //! ```text -//! Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher) +//! Thread A: read + decrypt (PrefetchedSectorSource) //! Thread B: M2TS demux (DemuxThread) //! Thread C: codec parse (this struct, on the caller's thread) //! ``` @@ -14,7 +14,7 @@ use super::codec::CodecParser; use super::demux_thread::{DemuxBatch, DemuxThread}; use super::ts::PesPacket; use crate::disc::DiscTitle; -use crate::pes::{PesFrame, Stream}; +use crate::pes::{PesFrame, PesSource}; use crossbeam_channel::Receiver; use std::io; @@ -37,10 +37,7 @@ pub struct PipelinedPesStream { eof: bool, // The terminal read error (kind, rendered code), repeated by every later read. failed: Option<(io::ErrorKind, String)>, - /// Cached `FREEMKV_SKIP_PARSE` profiling flag. Read once in `new()` - /// — the env var cannot change for the life of the stream, and - /// `std::env::var_os` takes a process-wide lock, so the per-batch / - /// per-poll reads it replaces were needless hot-path overhead. + /// `ctx.diag.skip_parse`: bypass the codec parsers (profiling). skip_parse: bool, /// Count of dropped DVD navigation packets (private_stream_2, 0xBF). These /// are expected on every disc; instead of a per-packet WARN they're tallied @@ -77,6 +74,8 @@ pub struct PipelinedPesStream { video_stream_id: Option, /// Packets of a video `stream_id` other than `video_stream_id`, dropped. other_video_packets: u64, + /// Read errors the Read stage skipped past (zero-filled), when it skips. + read_loss: Option>, } /// The `Codec` of a stream, for configuring its [`AuAssembler`](crate::mux::au_assembly::AuAssembler). @@ -130,7 +129,7 @@ impl PipelinedPesStream { pending_frames: std::collections::VecDeque::new(), eof: false, failed: None, - skip_parse: std::env::var_os("FREEMKV_SKIP_PARSE").is_some(), + skip_parse: false, dropped_nav_packets: 0, mpeg_extension_packets: [0; 8], dropped_ps: Default::default(), @@ -141,6 +140,7 @@ impl PipelinedPesStream { header_gate: super::header_gate::HeaderGate::default(), halt: crate::halt::Halt::new(), blanked: std::sync::Arc::default(), + read_loss: None, video_stream_id: None, other_video_packets: 0, } @@ -161,9 +161,23 @@ impl PipelinedPesStream { self } - // The op's stop token: a cancel ends a blocked `read` with `Halted` (LP11). - pub(crate) fn with_halt(mut self, halt: Option) -> Self { - self.halt = halt.unwrap_or_default(); + // The read loss the Read stage counts (a live drive's skipped units). + pub(crate) fn with_read_loss( + mut self, + loss: std::sync::Arc, + ) -> Self { + self.read_loss = Some(loss); + self + } + + // The run: its halt ends a blocked `read` with `Halted` (LP11), its stats count + // resync drops, its diagnostics apply. + pub(crate) fn with_ctx(mut self, ctx: &crate::ctx::Ctx) -> Self { + self.halt = ctx.halt.clone(); + self.skip_parse = ctx.diag.skip_parse; + for gate in &mut self.resync { + *gate = super::resync::ResyncGate::counted(ctx.stats.clone()); + } self } @@ -441,7 +455,7 @@ impl PipelinedPesStream { } } -impl Stream for PipelinedPesStream { +impl PesSource for PipelinedPesStream { fn read(&mut self) -> io::Result> { if let Some((kind, code)) = &self.failed { return Err(io::Error::new(*kind, code.clone())); @@ -455,28 +469,23 @@ impl Stream for PipelinedPesStream { read } - fn write(&mut self, _: &PesFrame) -> io::Result<()> { - Err(crate::error::Error::StreamReadOnly.into()) - } - - fn finish(&mut self) -> io::Result<()> { - Ok(()) - } - fn info(&self) -> &DiscTitle { &self.title } fn errors(&self) -> u64 { - // Blanked units plus frames the B1 gates dropped, as `DiscStream::errors` counts. + // Read errors skipped past, blanked units, and frames the B1 gates dropped. let dropped: u64 = self.resync.iter().map(|g| g.dropped_total()).sum(); - self.blanked.load(std::sync::atomic::Ordering::Relaxed) + dropped + let skips = self.read_loss.as_ref().map_or(0, |l| l.skips()); + skips + self.blanked.load(std::sync::atomic::Ordering::Relaxed) + dropped } fn lost_bytes(&self) -> u64 { - // Each blanked unit is one whole aligned unit of zeros (KS-2: 6144 bytes). + // Zero-filled read errors, plus each blanked unit: one whole aligned unit of zeros + // (KS-2: 6144 bytes). let blanked = self.blanked.load(std::sync::atomic::Ordering::Relaxed); - blanked * crate::aacs::content::ALIGNED_UNIT_LEN as u64 + let skipped = self.read_loss.as_ref().map_or(0, |l| l.bytes()); + skipped + blanked * crate::aacs::content::ALIGNED_UNIT_LEN as u64 } fn config_changes(&self) -> Vec<(usize, u64)> { @@ -492,7 +501,7 @@ impl Stream for PipelinedPesStream { fn headers_ready(&self) -> bool { // Match DiscStream semantics: video tracks need codec_private before the - // consumer can write the container header. FREEMKV_SKIP_PARSE forces ready + // consumer can write the container header. `Diag::skip_parse` forces ready // (no parser populates codec_private in that mode). if self.skip_parse { return true; @@ -514,10 +523,6 @@ impl Stream for PipelinedPesStream { // FLAC/Opus ES carry no init data: keep what the source header supplied. .or_else(|| self.title.codec_privates.get(track).cloned().flatten()) } - - // `lost_bytes` uses the trait default (0): the file-backed highway has no - // read-error zero-fill term (resolve/mapfile tracks physical read loss - // separately) and the decrypt path no longer reports a decrypt-loss term. } #[cfg(test)] @@ -539,8 +544,15 @@ mod tests { let (_pf_tx, pf_rx) = bounded::>>(1); let (rec_tx, _rec_rx) = bounded::>(2); // No TS/PS demuxer; the worker just drains (nothing) and exits Eof. - let (dt, _own_rx) = - DemuxThread::spawn_zero_copy(pf_rx, rec_tx, (), None, None, None).expect("spawn"); + let (dt, _own_rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rec_tx, + (), + &crate::ctx::Ctx::default(), + None, + None, + ) + .expect("spawn"); dt } @@ -614,7 +626,7 @@ mod tests { fn pipelined_stream_cancel_unblocks_reader() { let (stream, tx) = make_stream(DiscTitle::empty(), vec![], vec![]); let halt = crate::halt::Halt::new(); - let mut stream = stream.with_halt(Some(halt.clone())); + let mut stream = stream.with_ctx(&crate::ctx::Ctx::new(halt.clone())); let (done_tx, done_rx) = std::sync::mpsc::channel(); let (started_tx, started_rx) = std::sync::mpsc::channel(); std::thread::spawn(move || { @@ -641,11 +653,17 @@ mod tests { let halt = crate::halt::Halt::new(); let (pf_tx, pf_rx) = bounded::>>(4); let (rec_tx, _rec_rx) = bounded::>(8); - let (dt, rx) = - DemuxThread::spawn_zero_copy(pf_rx, rec_tx, (), Some(halt.clone()), None, None) - .expect("spawn"); + let (dt, rx) = DemuxThread::spawn_zero_copy( + pf_rx, + rec_tx, + (), + &crate::ctx::Ctx::new(halt.clone()), + None, + None, + ) + .expect("spawn"); let mut stream = PipelinedPesStream::new(dt, rx, DiscTitle::empty(), vec![], vec![]) - .with_halt(Some(halt.clone())); + .with_ctx(&crate::ctx::Ctx::new(halt.clone())); pf_tx.send(Ok(vec![0u8; 188])).unwrap(); halt.cancel(); // The halted prefetcher closes its channel. @@ -666,7 +684,7 @@ mod tests { let (mut stream, tx) = make_stream(DiscTitle::empty(), vec![], vec![]); let halt = crate::halt::Halt::new(); - stream = stream.with_halt(Some(halt.clone())); + stream = stream.with_ctx(&crate::ctx::Ctx::new(halt.clone())); halt.cancel(); tx.send(DemuxBatch::Eof).unwrap(); let e = stream.read().expect_err("an Eof after a Stop is not clean"); @@ -1084,7 +1102,7 @@ mod tests { // are not lost read bytes: lost_bytes stays the blanked-unit total. #[test] fn errors_include_frames_the_resync_gate_dropped() { - use crate::pes::Stream as _; + use crate::pes::PesSource as _; let (mut stream, _tx) = make_stream(DiscTitle::empty(), Vec::new(), Vec::new()); stream.resync.push(super::super::resync::ResyncGate::new()); let gate = stream.resync.last_mut().unwrap(); @@ -1343,25 +1361,6 @@ mod tests { assert_eq!(f.data, vec![0x55], "did not stop on the empty first batch"); } - /// write() on the read-only pipeline must return StreamReadOnly - /// (E9000 → Unsupported) — the highway is input-only. - #[test] - fn write_is_read_only_error() { - let (mut stream, _tx) = make_stream(DiscTitle::empty(), vec![], vec![]); - let frame = PesFrame { - discard_padding_ns: 0, - coding: None, - source: None, - track: 0, - pts: 0, - keyframe: false, - data: vec![1], - duration_ns: None, - }; - let err = stream.write(&frame).expect_err("write must error"); - assert_eq!(err.kind(), std::io::ErrorKind::Unsupported); - } - fn video_title(secondary: bool) -> DiscTitle { let mut t = DiscTitle::empty(); t.streams.push(crate::disc::Stream::Video(VideoStream { @@ -1491,14 +1490,6 @@ mod tests { assert!(stream.headers_ready(), "no video → always ready"); } - /// finish() on the read-only pipeline is a no-op that returns Ok — the - /// consumer drives termination via read() returning None. - #[test] - fn finish_is_ok_noop() { - let (mut stream, _tx) = make_stream(DiscTitle::empty(), vec![], vec![]); - assert!(stream.finish().is_ok()); - } - // --- AAC AudioSpecificConfig must exist before headers are finalised --- fn aac_audio(pid: u16) -> crate::disc::Stream { diff --git a/src/mux/resolve.rs b/src/mux/resolve.rs index 79ce778b..6d37a3a4 100644 --- a/src/mux/resolve.rs +++ b/src/mux/resolve.rs @@ -2,12 +2,8 @@ //! //! Format: `scheme://path`. //! -//! For disc→ISO (raw sector copy), use `freemkv_engine::recovery::copy` instead. -//! -//! Note: `disc://` cannot be opened through [`input`]; it returns -//! [`crate::error::Error::DiscUrlNotDirect`]. Live-disc input must go -//! through `Drive::open()` + `Disc::scan()` + `DiscStream::new()`, not -//! the URL resolver. +//! Every input scheme, `disc://` included, opens through [`super::source::open_source`]; +//! [`input`] is that plus the PES stream of one title. use super::network::NetworkStream; use super::null::NullStream; @@ -366,7 +362,7 @@ pub struct InputOptions { pub selection: crate::StreamSelection, /// The rip's up-front key set (KU §3.1). An AACS image is read through the set's /// reader, with no lookup here; `None` refuses an AACS image before any output. - pub keys: Option, + pub keys: Option, } // Hand-rolled so `InputOptions` stays printable without dumping key material. @@ -381,72 +377,69 @@ impl std::fmt::Debug for InputOptions { } } -/// Open a PES input stream (produces PES frames). -pub fn input(url: &str, opts: &InputOptions) -> io::Result> { - input_with_halt(url, opts, None) -} - -// `input` whose blocking network receive (`network://`) a halt can interrupt. -pub(crate) fn input_with_halt( +/// Open a PES input stream (produces PES frames) for the run `ctx`: [`open_source`] then the +/// PES stream of `opts.title_index`. Every input scheme opens here, `disc://` included (the +/// drive is brought up and scanned with default [`crate::disc::ScanOptions`]). Its halt +/// reaches the scan, crack, reads and demux and the network receive (`stdio://`, `mkv://`, +/// `mp4://` read inline); image and PS reads report `BytesRead`; stats count loss. +/// +/// [`open_source`]: super::source::open_source +pub fn input( url: &str, opts: &InputOptions, - halt: Option<&crate::halt::Halt>, -) -> io::Result> { + ctx: &crate::ctx::Ctx, +) -> io::Result> { + let source = super::source::open_source(url, crate::disc::ScanOptions::default(), ctx)?; + let mux = super::driver::MuxOptions { + raw: opts.raw, + selection: opts.selection.clone(), + title_index: opts.title_index.unwrap_or(0), + ..Default::default() + }; + super::driver::open_pes(source, opts.keys.as_ref(), &mux, ctx) +} + +pub(crate) fn validate_path(path: &Path, scheme: &str) -> crate::error::Result<()> { + validate_file_path(path, scheme).map_err(crate::error::Error::from) +} + +// The container and stream inputs (`m2ts://`, `mkv://`, `mp4://`, `mpg://`, `network://`, +// `stdio://`), each through the one decryption stage. +pub(crate) fn open_stream_url( + url: &str, + opts: &InputOptions, + ctx: &crate::ctx::Ctx, +) -> io::Result> { + let source = open_container(url, opts, ctx)?; + if matches!(parse_url(url), StreamUrl::Mpg { .. }) { + // A program stream is demuxed here: its title was pruned before the demux. + return Ok(source); + } + // A container's tracks arrive framed: the selection keeps a subset of them (DM5). + super::selected::SelectedSource::wrap(source, &opts.selection).map_err(io::Error::from) +} + +// The container or stream input `url`, all of its tracks. +fn open_container( + url: &str, + opts: &InputOptions, + ctx: &crate::ctx::Ctx, +) -> io::Result> { let parsed = parse_url(url); match parsed { - StreamUrl::Disc { .. } => { - // Disc sources require live SCSI state — caller must use - // `Drive::open() + Disc::scan() + DiscStream::new()` directly. - // Typed error only; the CLI/UI explains the entry point. - Err(crate::error::Error::DiscUrlNotDirect.into()) - } - StreamUrl::Iso { ref path } => { - validate_file_path(path, "iso")?; - // Sole file-backed sector source: carries the platform SEQUENTIAL - // fadvise hint (widens kernel readahead) plus periodic DONTNEED - // eviction bounding memory pressure during same-disk mux output. - let reader = crate::io::file_sector_source::FileSectorSource::open(path)?; - let probe_path = path.clone(); - let stream = image_input( - reader, - opts, - move || { - crate::io::file_sector_source::FileSectorSource::open(&probe_path) - .map_err(|e| -> io::Error { e.into() }) - }, - false, - )?; - Ok(Box::new(stream)) - } - // `dir://` as a SOURCE: an extracted disc folder presented as a - // synthetic UDF image (`crate::dirimage`), reaching the exact same - // body as `iso://` since `DirImage` is just another `SectorSource`. - StreamUrl::Dir { ref path } => { - validate_file_path(path, "dir")?; - let reader = crate::dirimage::DirImage::open(path)?; - let probe_path = path.clone(); - let stream = image_input( - reader, - opts, - move || { - crate::dirimage::DirImage::open(&probe_path) - .map_err(|e| -> io::Error { e.into() }) - }, - true, - )?; - Ok(Box::new(stream)) - } StreamUrl::M2ts { ref path } => { validate_file_path(path, "m2ts")?; let len = std::fs::metadata(path)?.len(); - let stage = crate::sector::DecryptingSectorSource::detecting( + let stage = crate::sector::DecryptingSectorSource::new( Box::new(crate::io::file_sector_source::FileSectorSource::open_padded(path)?) as Box, - stage_options(opts, halt, opts.raw), + crate::sector::Keying::detect(stage_options(opts, ctx, opts.raw)), ); let blanked = stage.blanked_counter(); let reader = crate::sector::stage::SectorBytes::new(stage, len); - Ok(Box::new(build_m2ts_pipeline(reader)?.with_blanked(blanked))) + Ok(Box::new( + build_m2ts_pipeline(reader, len, ctx)?.with_blanked(blanked), + )) } StreamUrl::Mkv { ref path } => { validate_file_path(path, "mkv")?; @@ -458,7 +451,7 @@ pub(crate) fn input_with_halt( validate_network_addr(addr)?; Ok(Box::new(NetworkStream::listen_staged( addr, - halt.cloned(), + Some(ctx.halt.clone()), opts.raw, )?)) } @@ -477,7 +470,7 @@ pub(crate) fn input_with_halt( } StreamUrl::Mpg { ref path } => { validate_file_path(path, "mpg")?; - Ok(Box::new(build_ps_pipeline(path, opts, halt)?)) + Ok(Box::new(build_ps_pipeline(path, opts, ctx)?)) } // `demux://` is an output-only sink (per-track ES files); never a source. StreamUrl::Demux { .. } @@ -488,38 +481,31 @@ pub(crate) fn input_with_halt( | StreamUrl::Json { .. } => Err(crate::error::Error::StreamWriteOnly.into()), // `fvi://` is an output-only sink (per-picture video index); never a source. StreamUrl::Fvi { .. } => Err(crate::error::Error::StreamWriteOnly.into()), + StreamUrl::Disc { .. } | StreamUrl::Iso { .. } | StreamUrl::Dir { .. } => { + Err(crate::error::Error::StreamUrlInvalid { + url: url.to_string(), + } + .into()) + } StreamUrl::Unknown { ref raw } => { Err(crate::error::Error::StreamUrlInvalid { url: raw.clone() }.into()) } } } -// Shared body of every IMAGE-level PES source (`iso://`/`dir://`). -fn image_input( - mut reader: S, +// Shared body of every IMAGE-level PES source (`iso://`/`dir://`) over the probe's reader +// and layout (`open_source` scanned it, and judged a folder's AACS verdict from content). +pub(crate) fn image_input_scanned( + reader: S, + mut disc: crate::disc::Disc, opts: &InputOptions, - reopen: F, - // A FOLDER's `encrypted` flag comes from tree shape and can be wrong; an - // image's cannot. See `session::apply_folder_encryption_verdict`. - is_folder: bool, + ctx: &crate::ctx::Ctx, ) -> io::Result where S: SectorSource + Send + 'static, - F: FnOnce() -> io::Result, { - let capacity = reader.capacity_sectors(); - let mut disc = - crate::disc::Disc::scan_image(&mut reader, capacity, &crate::disc::ScanOptions::default()) - .map_err(|e| -> io::Error { e.into() })?; - // Without this a folder reached here with a tree-shape verdict while - // `session::scan_dir` reached the opposite one from its CONTENT, so the - // same folder ripped through one door and failed through the other. - if is_folder { - crate::session::apply_folder_encryption_verdict(&mut reader, &mut disc) - .map_err(|e| -> io::Error { e.into() })?; - } if let Some(set) = opts.keys.as_ref().filter(|s| s.is_aacs() && !opts.raw) { - return keyed_image_input(reader, disc, opts, set, reopen); + return keyed_image_input(reader, disc, opts, set, ctx); } // Pre-flight decrypt gate: fails fast on a scrambled-but-uncracked CSS disc or an // AACS disc with no key set (else muxes garbage). Per-title CSS is checked below. @@ -535,9 +521,8 @@ where } .into()); } - // Prune to selected audio/subtitle streams now, pre-`probe_and_remap`, - // so all downstream consumers (TrueHD probe, title clone, - // `build_iso_pipeline`) see the pruned list. Video always kept. + // Prune to selected audio/subtitle streams now, so the title clone and + // `build_iso_pipeline` see the pruned list. Video always kept. opts.selection .apply(&mut disc.titles[idx]) .map_err(|e| -> io::Error { e.into() })?; @@ -549,26 +534,6 @@ where } else { disc.decrypt_keys() }; - // TrueHD channel counts (MPLS understates 7.1/Atmos as 5.1) are corrected by - // probing decrypted units via a fresh reader; skipped in --raw (no point). - if !opts.raw { - match reopen() { - Ok(probe) => { - let mut dec = crate::sector::DecryptingSectorSource::new(probe, keys.clone()); - crate::disc::correct_truehd_channels(&mut dec, &mut disc.titles[idx]); - } - Err(e) => { - // Non-fatal: a failed re-open leaves MPLS 7.1/Atmos counts - // uncorrected (as 5.1). Log the actual error so it's - // diagnosable, not swallowed by a bare `.ok()` -> None. - tracing::warn!( - target: "mux", - error = %e, - "TrueHD channel-correction probe re-open failed" - ); - } - } - } let title = disc.titles[idx].clone(); let format = disc.content_format; // Pass `DecryptKeys::None` to the decrypt decorator when --raw is set — @@ -586,24 +551,22 @@ where ISO_MUX_BATCH_SECTORS, format, opts.raw, - None, - None, + ctx, )?; Ok(stream) } // `image_input` over the rip's key set (KU §3.1, §3.5): the gate is `check_decryptable`, -// and the TrueHD probe and the mux read through the set's reader. No lookup, no banking. -fn keyed_image_input( +// and the mux reads through the set's reader. No lookup, no banking. +fn keyed_image_input( reader: S, mut disc: crate::disc::Disc, opts: &InputOptions, - set: &crate::keys::ResolvedKeySet, - reopen: F, + set: &crate::keys::KeyRing, + ctx: &crate::ctx::Ctx, ) -> io::Result where S: SectorSource + Send + 'static, - F: FnOnce() -> io::Result, { if disc.titles.is_empty() { return Err(crate::error::Error::NoStreams.into()); @@ -622,17 +585,8 @@ where opts.selection .apply(&mut disc.titles[idx]) .map_err(|e| -> io::Error { e.into() })?; - match reopen() { - Ok(probe) => match set.title_reader(&disc, idx, probe) { - Ok(mut dec) => crate::disc::correct_truehd_channels(&mut dec, &mut disc.titles[idx]), - Err(e) => tracing::warn!(target: "mux", error = %e, "TrueHD probe reader refused"), - }, - Err(e) => { - tracing::warn!(target: "mux", error = %e, "TrueHD channel-correction probe re-open failed") - } - } let title = disc.titles[idx].clone(); - build_iso_pipeline_keyed(reader, title, set, ISO_MUX_BATCH_SECTORS, None, None) + build_iso_pipeline_keyed(reader, title, set, ISO_MUX_BATCH_SECTORS, ctx) } // A `DemuxSink` over `dir`, its base name seeded from the playlist name. @@ -640,7 +594,7 @@ fn demux_sink_output( dir: &std::path::Path, title: &DiscTitle, mut opts: super::demux_sink::DemuxOptions, -) -> io::Result> { +) -> io::Result> { if !title.playlist.is_empty() { opts.base = title.playlist.clone(); } @@ -654,7 +608,7 @@ fn track_sink_output( dir: &std::path::Path, title: &DiscTitle, kind: super::demux_sink::TrackKind, -) -> io::Result> { +) -> io::Result> { let opts = super::demux_sink::DemuxOptions { kind_filter: Some(kind), export_chapters: false, @@ -663,6 +617,53 @@ fn track_sink_output( demux_sink_output(dir, title, opts) } +/// What an output scheme can take (pipeline design §2.2 sink caps): the one table the mux +/// consults, instead of matching on schemes in the pump. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SinkCaps { + /// The sink consumes PES frames. `false` for `chapters://` and `json://`, which write + /// their whole file from the (header-completed) title when opened. + pub needs_frames: bool, + /// The sink writes a DVD MPEG-2 multichannel extension track (13818-3 `ext_frame`s). + /// `mkv://`, `mp4://`, `m2ts://`, `demux://` and `audio://` have no mapping for it. + pub carries_mp2_extensions: bool, +} + +impl SinkCaps { + /// The caps of the output scheme `url`. + pub fn of(url: &StreamUrl) -> SinkCaps { + let full = SinkCaps { + needs_frames: true, + carries_mp2_extensions: true, + }; + match url { + StreamUrl::Chapters { .. } | StreamUrl::Json { .. } => SinkCaps { + needs_frames: false, + ..full + }, + StreamUrl::Mkv { .. } + | StreamUrl::Mp4 { .. } + | StreamUrl::M2ts { .. } + | StreamUrl::Demux { .. } + | StreamUrl::Audio { .. } => SinkCaps { + carries_mp2_extensions: false, + ..full + }, + StreamUrl::Mpg { .. } + | StreamUrl::Network { .. } + | StreamUrl::Stdio + | StreamUrl::Null + | StreamUrl::Video { .. } + | StreamUrl::Sub { .. } + | StreamUrl::Fvi { .. } + | StreamUrl::Disc { .. } + | StreamUrl::Iso { .. } + | StreamUrl::Dir { .. } + | StreamUrl::Unknown { .. } => full, + } + } +} + /// Open a PES output stream (consumes PES frames). /// /// `source` is the provenance of the material being written — the INPUT the caller is muxing @@ -674,7 +675,7 @@ pub fn output( url: &str, title: &crate::disc::DiscTitle, source: Option<&super::videomap::SourceInfo>, -) -> io::Result> { +) -> io::Result> { output_with(url, title, source, None) } @@ -705,7 +706,7 @@ pub(crate) fn output_with( title: &crate::disc::DiscTitle, source: Option<&super::videomap::SourceInfo>, flush: Option, -) -> io::Result> { +) -> io::Result> { let flush = flush.as_ref(); let parsed = parse_url(url); match parsed { @@ -758,11 +759,11 @@ pub(crate) fn output_with( StreamUrl::Stdio => Ok(Box::new(StdioStream::output(title))), StreamUrl::Null => Ok(Box::new(NullStream::new(title))), StreamUrl::Disc { .. } => Err(crate::error::Error::StreamReadOnly.into()), - StreamUrl::Iso { .. } => Err(crate::error::Error::StreamReadOnly.into()), - // `dir://` is NOT a PES sink — it writes raw decrypted files, not - // muxed frames. A stray `dir://` here fails loudly, like `iso://`. - // The CLI routes a `dir://` dest to `Disc::extract_tree` first. - StreamUrl::Dir { .. } => Err(crate::error::Error::StreamReadOnly.into()), + // `iso://` and `dir://` are block outputs, not PES sinks: a whole-disc copy writes + // them (`crate::io::open_block_sink`, `Disc::extract_tree`). + StreamUrl::Iso { .. } | StreamUrl::Dir { .. } => { + Err(crate::error::Error::StreamReadOnly.into()) + } // `demux://` with default options. The CLI constructs `DemuxSink` // directly (with parsed flags) before reaching here; this arm // covers the bare `output()` call with the default option set. @@ -871,22 +872,39 @@ pub(crate) fn build_demux_state(title: &DiscTitle, format: ContentFormat) -> Dem /// Assemble the ISO mux pipeline (read+decrypt → demux → parse) for a title with no /// AACS key set (clear, CSS, or `raw`); an AACS title goes through /// [`build_iso_pipeline_keyed`]. `keys` is `DecryptKeys::None` for raw/unencrypted -/// reads; `raw` skips the per-title CSS crack. `halt` is a cooperative cancel token. +/// reads; `raw` skips the per-title CSS crack. Every stage runs under `ctx`. #[allow(clippy::too_many_arguments)] pub(crate) fn build_iso_pipeline( - mut reader: S, + reader: S, title: DiscTitle, - mut keys: crate::decrypt::DecryptKeys, + keys: crate::decrypt::DecryptKeys, batch_sectors: u16, format: ContentFormat, raw: bool, - halt: Option, - event_fn: Option, + ctx: &crate::ctx::Ctx, ) -> io::Result { + let policy = crate::sector::read_stage::ReadPolicy::Image { + batch: batch_sectors, + }; + build_sector_pipeline(reader, title, keys, policy, format, raw, ctx) +} + +/// A sector source's title mux with no AACS key set, under the Read stage's `policy` +/// (an image's fixed batches, or a live drive's adaptive, recovering reads): the title's +/// CSS key cracked (unless `raw`), then decrypt → prefetch → demux → parse. +pub(crate) fn build_sector_pipeline( + mut reader: S, + title: DiscTitle, + mut keys: crate::decrypt::DecryptKeys, + policy: crate::sector::read_stage::ReadPolicy, + format: ContentFormat, + raw: bool, + ctx: &crate::ctx::Ctx, +) -> io::Result { + let batch_sectors = policy.batch(); let extents = title.extents.clone(); // CSS (DVD) key resolution — shared per-title step. A `None`/MPEG-PS - // title cracks its own key; `raw` skips it. `halt` lets /api/stop - // interrupt the crack scan. + // title cracks its own key; `raw` skips it. A Stop interrupts the crack scan. crate::css::resolve_dvd_title_key( &mut reader, &extents, @@ -894,7 +912,7 @@ pub(crate) fn build_iso_pipeline( batch_sectors, format, raw, - halt.as_ref(), + Some(&ctx.halt), )?; // Unit alignment is an AACS concept (whole 3-sector units); forcing it // on CSS/unencrypted (per-2048-byte-sector) would reject any extent @@ -909,11 +927,11 @@ pub(crate) fn build_iso_pipeline( let plan = IsoPlan { extents, full_extents, - batch_sectors, + policy, unit_align, format, }; - iso_pipeline_tail(decrypting, plan, title, halt, event_fn) + iso_pipeline_tail(decrypting, plan, title, ctx) } // The read plan of an ISO title mux: the extents read (FMTS alternate phase dropped), the @@ -921,22 +939,21 @@ pub(crate) fn build_iso_pipeline( struct IsoPlan { extents: Vec, full_extents: Vec, - batch_sectors: u16, + policy: crate::sector::read_stage::ReadPolicy, unit_align: u16, format: ContentFormat, } -/// The ISO title mux over a [`ResolvedKeySet`](crate::keys::ResolvedKeySet) (KU §3.1): the +/// The ISO title mux over a [`KeyRing`](crate::keys::KeyRing) (KU §3.1): the /// set's decrypting reader (its map and on-arrival proof) under the prefetcher, with no /// resolution here. `title` is an already-scanned title; E7013 if the set does not cover /// its extents or the image is not a sector-exact copy of the set's disc. pub(crate) fn build_iso_pipeline_keyed( reader: S, title: DiscTitle, - set: &crate::keys::ResolvedKeySet, + set: &crate::keys::KeyRing, batch_sectors: u16, - halt: Option, - event_fn: Option, + ctx: &crate::ctx::Ctx, ) -> io::Result { let cap = reader.capacity_sectors(); if (cap != 0 && set.capacity() != 0 && cap != set.capacity()) @@ -945,6 +962,21 @@ pub(crate) fn build_iso_pipeline_keyed( tracing::error!(target: "freemkv::keys", "key set does not cover this image title"); return Err(crate::error::Error::DecryptFailed.into()); } + let policy = crate::sector::read_stage::ReadPolicy::Image { + batch: batch_sectors, + }; + build_keyed_pipeline(reader, title, set, policy, ctx) +} + +/// A sector source's title mux over a key set (its map and on-arrival proof, KU §3.1), +/// under the Read stage's `policy`. The caller checked the set is for this source. +pub(crate) fn build_keyed_pipeline( + reader: S, + title: DiscTitle, + set: &crate::keys::KeyRing, + policy: crate::sector::read_stage::ReadPolicy, + ctx: &crate::ctx::Ctx, +) -> io::Result { let full_extents = title.extents.clone(); let ranges: Vec<(u32, u32)> = full_extents .iter() @@ -961,11 +993,11 @@ pub(crate) fn build_iso_pipeline_keyed( let plan = IsoPlan { extents: set.key_map().read_plan(&full_extents, 3), full_extents, - batch_sectors, + policy, unit_align: 3, format: set.content_format(), }; - iso_pipeline_tail(decrypting, plan, title, halt, event_fn) + iso_pipeline_tail(decrypting, plan, title, ctx) } // The shared tail of the ISO title mux: decrypting reader → prefetcher → demux → parse. @@ -973,13 +1005,12 @@ fn iso_pipeline_tail( mut decrypting: crate::sector::DecryptingSectorSource>, plan: IsoPlan, title: DiscTitle, - halt: Option, - event_fn: Option, + ctx: &crate::ctx::Ctx, ) -> io::Result { let IsoPlan { extents, full_extents, - batch_sectors, + policy, unit_align, format, } = plan; @@ -988,8 +1019,8 @@ fn iso_pipeline_tail( // undetectable by the tile check. Fall back to timestamps if mismatched. let feed_matches_spans = extents == full_extents; // The mux does not tally decrypt-quality misses (`lost_bytes()` is read-loss - // only); a missing key is an up-front resolve failure. Wrong-substream fix - // below: re-route declared AC-3 audio onto correct `0x8x` sub-streams. + // only); a missing key is an up-front resolve failure. The DVD AC-3 probe below is + // diagnostics only (routing is the scanner's PGC AST_CTL map). let mut title = title; if !feed_matches_spans { tracing::info!( @@ -1004,46 +1035,37 @@ fn iso_pipeline_tail( } crate::disc::dvd_audio_probe::probe_and_remap(&mut decrypting, &mut title); decrypting.clear_unit_base(); + decrypting.observe(ctx); let blanked = decrypting.blanked_counter(); - let prefetched = crate::sector::PrefetchedSectorSource::new_with_events( - decrypting, - extents, - batch_sectors, - unit_align, - halt.clone(), - event_fn, + let (prefetched, read_loss) = crate::sector::PrefetchedSectorSource::with_policy( + decrypting, extents, policy, unit_align, ctx, ) .map_err(|e| -> io::Error { e.into() })?; let (rx, recycle_tx, shell) = prefetched.into_channels(); let (parsers, pid_to_track, ts, ps) = build_demux_state(&title, format); - let (demux_thread, demux_rx) = super::demux_thread::DemuxThread::spawn_zero_copy( - rx, - recycle_tx, - shell, - halt.clone(), - ts, - ps, - ) - .map_err(|e| -> io::Error { e.into() })?; + let (demux_thread, demux_rx) = + super::demux_thread::DemuxThread::spawn_zero_copy(rx, recycle_tx, shell, ctx, ts, ps) + .map_err(|e| -> io::Error { e.into() })?; Ok( PipelinedPesStream::new(demux_thread, demux_rx, title, parsers, pid_to_track) - .with_halt(halt) - .with_blanked(blanked), + .with_ctx(ctx) + .with_blanked(blanked) + .with_read_loss(read_loss), ) } // The decryption stage's options for a content-detected input. fn stage_options( opts: &InputOptions, - halt: Option<&crate::halt::Halt>, + ctx: &crate::ctx::Ctx, raw: bool, ) -> crate::sector::decrypting::StageOptions { crate::sector::decrypting::StageOptions { raw, keys: opts.keys.clone(), - halt: halt.cloned(), + ctx: ctx.clone(), } } @@ -1071,15 +1093,15 @@ fn read_head(src: &mut dyn SectorSource, sectors: u32, batch: u16) -> io::Result fn build_ps_pipeline( path: &Path, opts: &InputOptions, - halt: Option<&crate::halt::Halt>, + ctx: &crate::ctx::Ctx, ) -> io::Result { const PS_MUX_BATCH_SECTORS: u16 = 8192; const HEAD_SECTORS: u32 = 2048; // 4 MiB let open = |raw: bool| -> io::Result<_> { let file = crate::io::file_sector_source::FileSectorSource::open_padded(path)?; - Ok(crate::sector::DecryptingSectorSource::detecting( + Ok(crate::sector::DecryptingSectorSource::new( Box::new(file) as Box, - stage_options(opts, halt, raw), + crate::sector::Keying::detect(stage_options(opts, ctx, raw)), )) }; let mut stage = open(opts.raw)?; @@ -1129,19 +1151,27 @@ fn build_ps_pipeline( let plan = IsoPlan { extents: vec![extent], full_extents: vec![extent], - batch_sectors: PS_MUX_BATCH_SECTORS, + policy: crate::sector::read_stage::ReadPolicy::Image { + batch: PS_MUX_BATCH_SECTORS, + }, unit_align: 1, format: ContentFormat::MpegPs, }; - iso_pipeline_tail(stage, plan, title, halt.cloned(), None) - .map(|p| p.with_video_stream_id(scan.video_id)) + iso_pipeline_tail(stage, plan, title, ctx).map(|p| p.with_video_stream_id(scan.video_id)) } -// Assemble the M2TS file mux pipeline (read -> demux -> parse). Scans the head -// for FMKV header or PMT/PAT, then wraps a chained reader (head + remainder) -// in a BytePrefetcher feeding the demux + parse threads. -fn build_m2ts_pipeline( - mut reader: R, +// Sectors per m2ts read: whole AACS units, ~1 MiB (the stage's byte-view refill). +const M2TS_READ_SECTORS: u16 = 510; + +// Assemble the M2TS file mux pipeline (read -> demux -> parse). Scans the head for an FMKV +// header or PMT/PAT through the stage's byte view, then hands the stage to the one Read +// stage and prefetcher, whose chunks follow the head's unconsumed bytes into the demux. +fn build_m2ts_pipeline( + mut reader: crate::sector::stage::SectorBytes< + crate::sector::DecryptingSectorSource>, + >, + len: u64, + ctx: &crate::ctx::Ctx, ) -> io::Result { use super::meta; use std::io::Read; @@ -1187,29 +1217,40 @@ fn build_m2ts_pipeline( } }; - // Chain: any un-consumed head bytes + the remainder of the - // reader. The demuxer sees a contiguous M2TS byte stream. - let remaining_head = head[head_consumed..].to_vec(); - let chained: Box = Box::new(io::Cursor::new(remaining_head).chain(reader)); - - let prefetcher = crate::io::byte_prefetcher::BytePrefetcher::new( - chained, - crate::io::byte_prefetcher::DEFAULT_CHUNK_BYTES, - None, - )?; - let (rx, recycle_tx, shell) = prefetcher.into_channels(); + // The demuxer sees one contiguous M2TS byte stream: the head's unconsumed bytes and what + // the byte view already holds, then the Read stage's chunks from the next sector on. + let (stage, rest, next) = reader.into_rest(); + let mut prefix = head[head_consumed..].to_vec(); + prefix.extend_from_slice(&rest); + let cap = stage.capacity_sectors(); + let extents = match cap > next { + true => vec![crate::disc::Extent { + start_lba: next, + sector_count: cap - next, + }], + false => Vec::new(), + }; + let view = crate::sector::prefetched::ByteView { + prefix, + len: len.saturating_sub(u64::from(next) * crate::consts::SECTOR_BYTES_U64), + }; + let policy = crate::sector::read_stage::ReadPolicy::Image { + batch: M2TS_READ_SECTORS, + }; + let (prefetched, read_loss) = + crate::sector::PrefetchedSectorSource::file_bytes(stage, extents, policy, ctx, view) + .map_err(|e| -> io::Error { e.into() })?; + let (rx, recycle_tx, shell) = prefetched.into_channels(); let (parsers, pid_to_track, ts, ps) = build_demux_state(&title, ContentFormat::BdTs); let (demux_thread, demux_rx) = - super::demux_thread::DemuxThread::spawn_zero_copy(rx, recycle_tx, shell, None, ts, ps) + super::demux_thread::DemuxThread::spawn_zero_copy(rx, recycle_tx, shell, ctx, ts, ps) .map_err(|e| -> io::Error { e.into() })?; - Ok(PipelinedPesStream::new( - demux_thread, - demux_rx, - title, - parsers, - pid_to_track, - )) + Ok( + PipelinedPesStream::new(demux_thread, demux_rx, title, parsers, pid_to_track) + .with_ctx(ctx) + .with_read_loss(read_loss), + ) } #[cfg(test)] @@ -1221,7 +1262,7 @@ mod tests { use super::{build_demux_state, build_iso_pipeline, input, output}; use crate::decrypt::DecryptKeys; use crate::disc::{ContentFormat, DiscTitle, Extent}; - use crate::pes::Stream as _; + use crate::pes::PesSource as _; use crate::sector::SectorSource; use std::path::PathBuf; @@ -1321,7 +1362,7 @@ mod tests { // Box isn't Debug, so unwrap_err() won't compile — these // helpers extract io::ErrorKind from the Err arm (panic on Ok) instead. fn input_err_kind(url: &str) -> std::io::ErrorKind { - match input(url, &Default::default()) { + match input(url, &Default::default(), &crate::ctx::Ctx::default()) { Ok(_) => panic!("expected input({url}) to error"), Err(e) => e.kind(), } @@ -1448,12 +1489,18 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } - /// The resolver doc table marks disc:// as input-only via the - /// `Drive::open` path — input("disc://") must surface DiscUrlNotDirect - /// (E9009 → Unsupported), never attempt to open a stream. + /// `disc://` is an ordinary input: `input()` opens the named drive (a missing device + /// fails as the drive open does), never refusing the scheme. #[test] - fn input_disc_url_is_not_direct() { - assert_eq!(input_err_kind("disc://"), std::io::ErrorKind::Unsupported); + fn input_disc_url_opens_the_drive() { + let err = super::input( + "disc:///nonexistent/freemkv-test-drive", + &super::InputOptions::default(), + &crate::ctx::Ctx::default(), + ) + .err() + .expect("no such drive"); + assert_ne!(err.kind(), std::io::ErrorKind::Unsupported, "got {err}"); } /// null:// is write-only per the table — input() must reject it with @@ -1715,8 +1762,7 @@ mod tests { 8192, ContentFormat::BdTs, false, - None, - None, + &crate::ctx::Ctx::default(), ) .expect("pipeline builds"); let first = stream.read().expect("read must not error on clean EOF"); @@ -1756,8 +1802,7 @@ mod tests { 8192, ContentFormat::BdTs, false, - None, - None, + &crate::ctx::Ctx::default(), ) .expect("pipeline builds"); @@ -1848,8 +1893,7 @@ mod tests { 8192, ContentFormat::BdTs, false, - None, - None, + &crate::ctx::Ctx::default(), ) .expect("pipeline builds"); @@ -1888,8 +1932,7 @@ mod tests { 0, ContentFormat::BdTs, false, - None, - None, + &crate::ctx::Ctx::default(), ); assert!(res.is_err(), "zero batch_sectors must be rejected"); } @@ -1929,8 +1972,7 @@ mod tests { 8192, ContentFormat::MpegPs, false, - None, - None, + &crate::ctx::Ctx::default(), ); assert!( res.is_err(), @@ -1938,3 +1980,110 @@ mod tests { ); } } + +#[cfg(test)] +mod stop_tests { + use super::{InputOptions, PipelinedPesStream, image_input_scanned}; + + // `open_source`'s image probe, then the image stream, over a test reader. + fn image_input( + mut reader: S, + opts: &InputOptions, + folder: bool, + ctx: &crate::ctx::Ctx, + ) -> std::io::Result + where + S: SectorSource + Send + 'static, + { + let cap = reader.capacity_sectors(); + let scan = crate::disc::ScanOptions { + halt: Some(ctx.halt.clone()), + ..Default::default() + }; + let mut disc = crate::disc::Disc::scan_image(&mut reader, cap, &scan)?; + if folder { + crate::session::apply_folder_encryption_verdict(&mut reader, &mut disc)?; + } + image_input_scanned(reader, disc, opts, ctx) + } + use crate::pes::PesSource as _; + use crate::sector::SectorSource; + use std::sync::mpsc; + use std::time::Duration; + + // A folder image whose prefetch producer stalls in its first read until released: the + // shape of a drive grinding on a bad sector under an `iso://`/`dir://` mux. + struct StallsInPrefetch { + inner: crate::DirImage, + entered: mpsc::Sender<()>, + release: std::sync::Mutex>, + } + + impl StallsInPrefetch { + fn new(dir: &std::path::Path) -> (Self, mpsc::Receiver<()>, mpsc::Sender<()>) { + let (entered, entered_rx) = mpsc::channel(); + let (release_tx, release_rx) = mpsc::channel(); + let s = Self { + inner: crate::DirImage::open(dir).unwrap(), + entered, + release: std::sync::Mutex::new(release_rx), + }; + (s, entered_rx, release_tx) + } + } + + impl SectorSource for StallsInPrefetch { + fn capacity_sectors(&self) -> u32 { + self.inner.capacity_sectors() + } + fn read_sectors( + &mut self, + lba: u32, + count: u16, + buf: &mut [u8], + recovery: bool, + ) -> crate::error::Result { + if std::thread::current().name() == Some("freemkv-prefetch") { + let _ = self.entered.send(()); + // Blocks until the test drops the sender, then fails the read. + let _ = self.release.lock().unwrap().recv(); + return Err(crate::error::Error::SourceTerminated); + } + self.inner.read_sectors(lba, count, buf, recovery) + } + } + + // BUG-4: Stop reaches an image pipeline blocked inside a read; the read ends Halted + // within a wait slice instead of waiting on the stalled source. + #[test] + fn stop_reaches_a_dir_pipeline_blocked_in_a_read() { + let _serial = crate::sector::prefetched::holder_test_lock(); + let s = crate::dirimage::tests::playable_bdmv("stopblocked", false); + let (src, entered_rx, release_tx) = StallsInPrefetch::new(s.path()); + let ctx = crate::ctx::Ctx::default(); + let mut stream = + image_input(src, &InputOptions::default(), true, &ctx).expect("the folder opens"); + entered_rx + .recv_timeout(Duration::from_secs(10)) + .expect("the producer reached its read"); + let (done_tx, done_rx) = mpsc::channel(); + let reader = std::thread::spawn(move || { + let r = stream.read().map(|f| f.is_some()); + let _ = done_tx.send(()); + (r, stream) + }); + ctx.halt.cancel(); + let stopped = done_rx.recv_timeout(Duration::from_secs(5)).is_ok(); + drop(release_tx); + let (r, stream) = reader.join().unwrap(); + drop(stream); + assert!( + stopped, + "a Stop must end the blocked read without the source" + ); + assert!( + crate::error::is_halt(&r.expect_err("a stopped read is not a frame")), + "the blocked read ends Halted" + ); + } +} diff --git a/src/mux/resync.rs b/src/mux/resync.rs index ad8cfa19..52e0c730 100644 --- a/src/mux/resync.rs +++ b/src/mux/resync.rs @@ -25,14 +25,20 @@ pub(crate) struct ResyncGate { /// trace at all. That is the common case — most gaps do resolve — which /// made concealed video loss invisible to every consumer. dropped_total: u64, + /// The run's loss counters, bumped per dropped frame. + stats: Option>, } impl ResyncGate { pub(crate) fn new() -> Self { + Self::default() + } + + // A gate that also counts its drops into the run's `stats`. + pub(crate) fn counted(stats: std::sync::Arc) -> Self { Self { - armed: false, - dropped: 0, - dropped_total: 0, + stats: Some(stats), + ..Self::default() } } @@ -53,6 +59,9 @@ impl ResyncGate { } else { self.dropped += 1; self.dropped_total += 1; + if let Some(s) = &self.stats { + s.add_resync_dropped(1); + } false } } else { diff --git a/src/mux/select.rs b/src/mux/select.rs index 7a95381e..90e4cd51 100644 --- a/src/mux/select.rs +++ b/src/mux/select.rs @@ -50,35 +50,7 @@ impl StreamSelection { if self.is_all() { return Ok(()); } - - // Validate every listed PID before mutating (unknown PID → no partial prune), - // PER CLASS: scanning both classes let a PID in the WRONG filter pass, then - // `keeps` matched it only against its own class, silently dropping the track. - let check = |filter: &PidFilter, is_class: fn(&Stream) -> bool| -> Result<()> { - if let PidFilter::Only(pids) = filter { - for &pid in pids { - let present = title - .streams - .iter() - .any(|s| is_class(s) && stream_pid(s) == Some(pid)); - if !present { - return Err(Error::SelectionPidUnknown { pid }); - } - } - } - Ok(()) - }; - check(&self.audio, |s| matches!(s, Stream::Audio(_)))?; - check(&self.subtitle, |s| matches!(s, Stream::Subtitle(_)))?; - - let effective = self.with_mp2_extension_bases(title); - // Retain by index so we can prune the parallel codec_privates in lockstep. - let keep: Vec = title - .streams - .iter() - .map(|s| effective.keeps(s)) - .collect::>(); - + let keep = self.keeps_by_index(title)?; let mut i = 0; title.streams.retain(|_| { let k = keep[i]; @@ -107,6 +79,37 @@ impl StreamSelection { Ok(()) } + /// Whether each of `title.streams` is kept, by index; the same validation as + /// [`apply`](Self::apply) ([`Error::SelectionPidUnknown`] for a PID the title lacks). + pub(crate) fn keeps_by_index(&self, title: &DiscTitle) -> Result> { + if self.is_all() { + return Ok(vec![true; title.streams.len()]); + } + + // Validate every listed PID before mutating (unknown PID → no partial prune), + // PER CLASS: scanning both classes let a PID in the WRONG filter pass, then + // `keeps` matched it only against its own class, silently dropping the track. + let check = |filter: &PidFilter, is_class: fn(&Stream) -> bool| -> Result<()> { + if let PidFilter::Only(pids) = filter { + for &pid in pids { + let present = title + .streams + .iter() + .any(|s| is_class(s) && stream_pid(s) == Some(pid)); + if !present { + return Err(Error::SelectionPidUnknown { pid }); + } + } + } + Ok(()) + }; + check(&self.audio, |s| matches!(s, Stream::Audio(_)))?; + check(&self.subtitle, |s| matches!(s, Stream::Subtitle(_)))?; + + let effective = self.with_mp2_extension_bases(title); + Ok(title.streams.iter().map(|s| effective.keeps(s)).collect()) + } + // An extension PID listed without its base pulls the base in (13818-3 2nd ed. §2.5.2.13: // the extension holds "a remainder of the multichannel ... information"). fn with_mp2_extension_bases(&self, title: &DiscTitle) -> StreamSelection { diff --git a/src/mux/selected.rs b/src/mux/selected.rs new file mode 100644 index 00000000..5917cf58 --- /dev/null +++ b/src/mux/selected.rs @@ -0,0 +1,176 @@ +//! Track selection over an already-demuxed source (pipeline design §2.4, DM5): a container +//! input (`mkv://`, `mp4://`, `m2ts://`, `network://`, `stdio://`) arrives as framed tracks, +//! so the selection keeps a subset of its tracks and renumbers them, exactly as a disc +//! title pruned before its demux is built. Video is always kept. + +use crate::disc::DiscTitle; +use crate::pes::{PesFrame, PesSource, TrackTiming}; + +pub(crate) struct SelectedSource { + inner: Box, + // Inner track index of each kept track, in order. + kept: Vec, + // Inner track index → kept index. + map: Vec>, + info: DiscTitle, +} + +impl SelectedSource { + // `inner` restricted to what `selection` keeps; an All selection returns `inner`. + pub(crate) fn wrap( + inner: Box, + selection: &crate::StreamSelection, + ) -> crate::error::Result> { + if selection.is_all() { + return Ok(inner); + } + let keep = selection.keeps_by_index(inner.info())?; + let mut info = inner.info().clone(); + selection.apply(&mut info)?; + let kept: Vec = (0..keep.len()).filter(|&i| keep[i]).collect(); + let mut map = vec![None; keep.len()]; + for (k, &i) in kept.iter().enumerate() { + map[i] = Some(k); + } + Ok(Box::new(SelectedSource { + inner, + kept, + map, + info, + })) + } +} + +impl PesSource for SelectedSource { + fn read(&mut self) -> std::io::Result> { + loop { + let Some(mut f) = self.inner.read()? else { + return Ok(None); + }; + if let Some(Some(k)) = self.map.get(f.track) { + f.track = *k; + return Ok(Some(f)); + } + } + } + + fn info(&self) -> &DiscTitle { + &self.info + } + + fn track_timing(&self, track: usize) -> TrackTiming { + self.kept + .get(track) + .map(|&i| self.inner.track_timing(i)) + .unwrap_or_default() + } + + fn codec_private(&self, track: usize) -> Option> { + self.kept + .get(track) + .and_then(|&i| self.inner.codec_private(i)) + } + + fn headers_ready(&self) -> bool { + self.inner.headers_ready() + } + + fn config_changes(&self) -> Vec<(usize, u64)> { + self.inner + .config_changes() + .into_iter() + .filter_map(|(i, n)| self.map.get(i).copied().flatten().map(|k| (k, n))) + .collect() + } + + fn errors(&self) -> u64 { + self.inner.errors() + } + + fn lost_bytes(&self) -> u64 { + self.inner.lost_bytes() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::disc::{AudioChannels, AudioStream, Codec, LabelPurpose, SampleRate, Stream}; + use crate::mux::select::{PidFilter, StreamSelection}; + + struct Frames(DiscTitle, std::collections::VecDeque); + + impl PesSource for Frames { + fn read(&mut self) -> std::io::Result> { + Ok(self.1.pop_front()) + } + fn info(&self) -> &DiscTitle { + &self.0 + } + fn codec_private(&self, track: usize) -> Option> { + Some(vec![track as u8]) + } + } + + fn audio(pid: u16) -> Stream { + Stream::Audio(AudioStream { + pid, + codec: Codec::Ac3, + channels: AudioChannels::Stereo, + language: "eng".into(), + sample_rate: SampleRate::S48, + secondary: false, + purpose: LabelPurpose::Normal, + label: String::new(), + }) + } + + fn frame(track: usize) -> PesFrame { + PesFrame { + discard_padding_ns: 0, + track, + pts: 0, + keyframe: true, + data: vec![track as u8], + duration_ns: None, + source: None, + coding: None, + } + } + + // A container source keeps only the selected tracks, renumbered in order. + #[test] + fn a_container_keeps_the_selected_tracks_renumbered() { + let mut title = DiscTitle::empty(); + title.streams = vec![audio(1), audio(2), audio(3)]; + let src = Frames(title, (0..3).map(frame).collect()); + let sel = StreamSelection { + audio: PidFilter::Only(vec![3]), + subtitle: PidFilter::All, + }; + let mut s = SelectedSource::wrap(Box::new(src), &sel).unwrap(); + assert_eq!(s.info().streams.len(), 1); + assert_eq!(s.codec_private(0), Some(vec![2])); + let f = s.read().unwrap().expect("the kept track's frame"); + assert_eq!((f.track, f.data.as_slice()), (0, &[2u8][..])); + assert!(s.read().unwrap().is_none()); + } + + // An unknown PID refuses at open, as on a disc title. + #[test] + fn an_unknown_pid_refuses() { + let mut title = DiscTitle::empty(); + title.streams = vec![audio(1)]; + let sel = StreamSelection { + audio: PidFilter::Only(vec![9]), + subtitle: PidFilter::All, + }; + let err = SelectedSource::wrap(Box::new(Frames(title, Default::default())), &sel) + .err() + .expect("unknown pid"); + assert!(matches!( + err, + crate::error::Error::SelectionPidUnknown { pid: 9 } + )); + } +} diff --git a/src/mux/source.rs b/src/mux/source.rs new file mode 100644 index 00000000..66b6b216 --- /dev/null +++ b/src/mux/source.rs @@ -0,0 +1,307 @@ +//! One way to open any input (pipeline design §2.2, slice 4): [`open_source`] takes a URL +//! for every input scheme, `disc://` included, and probes it once. The [`Source`] carries +//! that probe's [`Disc`] layout (for `disc://`, `iso://` and `dir://`), so listing titles, +//! acquiring keys and muxing read the same scan. A caller that already holds a scanned +//! layout (a drive scan reused over a staged image, J14) or an opened session builds the +//! same `Source` with [`Source::from_image`], [`Source::from_reader`] or +//! [`Source::from_session`]; [`crate::mux_with_keys`] and [`crate::input`] take it. + +use std::path::{Path, PathBuf}; + +use crate::ctx::Ctx; +use crate::disc::{Disc, ScanOptions}; +use crate::error::{Error, Result}; +use crate::sector::SectorSource; +use crate::session::{DeviceTarget, DiscSession, KeySpec}; + +use super::resolve::{StreamUrl, parse_url}; +use super::videomap::{Medium, SourceInfo}; + +/// One title of a disc the caller already scanned, for [`Source::from_image`] and +/// [`Source::from_reader`]: the title, its container format, and the disc's name. +#[derive(Debug, Clone)] +pub struct ScannedTitle { + pub title: crate::disc::DiscTitle, + pub format: crate::disc::ContentFormat, + /// The disc's name (meta title, else volume id), which names the output title; `None` + /// keeps the title's own playlist name. + pub disc_name: Option, + pub volume_id: String, +} + +impl ScannedTitle { + /// Title `idx` of `disc`, named by the disc. `None` when out of range. + pub fn of(disc: &Disc, idx: usize) -> Option { + Some(ScannedTitle { + title: disc.titles.get(idx)?.clone(), + format: disc.content_format, + disc_name: Some(disc_name(disc)), + volume_id: disc.volume_id.clone(), + }) + } + + /// `title` alone: no disc name or volume id. + pub fn new(title: crate::disc::DiscTitle, format: crate::disc::ContentFormat) -> Self { + ScannedTitle { + title, + format, + disc_name: None, + volume_id: String::new(), + } + } +} + +// A disc's name: its meta title, else its volume id. +fn disc_name(d: &Disc) -> String { + d.meta_title.clone().unwrap_or_else(|| d.volume_id.clone()) +} + +/// An opened input: where PES frames come from, and the layout probed from it. +pub struct Source<'a> { + pub(crate) origin: Origin<'a>, +} + +pub(crate) enum Origin<'a> { + // A live drive session the caller keeps (tray lock, eject, finish stay with it). + Session(&'a mut DiscSession), + // `disc://`: a session opened and scanned here. + Drive(Box), + // An image (`iso://`, `dir://`) and the probe's reader. + Image { + path: PathBuf, + folder: bool, + reader: Box, + disc: Disc, + }, + // An image file read with a title the caller scanned elsewhere (J14): opened at read + // time, never rescanned. + Prescanned { + path: PathBuf, + title: ScannedTitle, + }, + // A raw live reader the caller already holds, with the title scanned off it. + Live { + reader: Box, + title: ScannedTitle, + }, + // A container or stream input, opened when it is read. + Stream { + url: String, + }, +} + +/// Open the input `url` and probe it: a drive is brought up and scanned (`disc://`, with +/// `probe`'s credentials and key sources for the handshake), an image or folder is scanned +/// (`iso://`, `dir://`; a folder's AACS verdict is judged from its content). Container and +/// stream inputs (`m2ts://`, `mkv://`, `mp4://`, `mpg://`, `network://`, `stdio://`) open +/// when read. `ctx.halt` stops the bring-up and the scan. +pub fn open_source(url: &str, mut probe: ScanOptions, ctx: &Ctx) -> Result> { + if probe.halt.is_none() { + probe.halt = Some(ctx.halt.clone()); + } + let origin = match parse_url(url) { + StreamUrl::Disc { device } => { + let target = device.map_or(DeviceTarget::Autodetect, DeviceTarget::Path); + let spec = KeySpec { + credentials: probe.credentials.take(), + key_sources: std::mem::take(&mut probe.key_sources), + ..KeySpec::default() + }; + let mut session = DiscSession::open_with(target, spec, &ctx.halt)?; + session.scan_with(probe)?; + Origin::Drive(Box::new(session)) + } + StreamUrl::Iso { path } => { + super::resolve::validate_path(&path, "iso")?; + let (disc, reader) = probe_image(&path, false, &probe)?; + Origin::Image { + path, + folder: false, + reader, + disc, + } + } + StreamUrl::Dir { path } => { + super::resolve::validate_path(&path, "dir")?; + let (disc, reader) = probe_image(&path, true, &probe)?; + Origin::Image { + path, + folder: true, + reader, + disc, + } + } + StreamUrl::M2ts { .. } + | StreamUrl::Mkv { .. } + | StreamUrl::Mp4 { .. } + | StreamUrl::Mpg { .. } + | StreamUrl::Network { .. } + | StreamUrl::Stdio => Origin::Stream { + url: url.to_string(), + }, + StreamUrl::Null + | StreamUrl::Demux { .. } + | StreamUrl::Video { .. } + | StreamUrl::Audio { .. } + | StreamUrl::Sub { .. } + | StreamUrl::Fvi { .. } + | StreamUrl::Chapters { .. } + | StreamUrl::Json { .. } => return Err(Error::StreamWriteOnly), + StreamUrl::Unknown { raw } => return Err(Error::StreamUrlInvalid { url: raw }), + }; + Ok(Source { origin }) +} + +/// The one image probe (`iso://` file or `dir://` folder): scan the image, and judge a +/// folder's AACS verdict from its content (tree shape alone can be wrong for a decrypted +/// folder that kept `AACS/`). Returns the layout and the probe's reader. +pub(crate) fn probe_image( + path: &Path, + folder: bool, + probe: &ScanOptions, +) -> Result<(Disc, Box)> { + let mut reader: Box = if folder { + Box::new(crate::dirimage::DirImage::open(path)?) + } else { + Box::new(crate::io::file_sector_source::FileSectorSource::open(path)?) + }; + let cap = reader.capacity_sectors(); + let mut disc = Disc::scan_image(&mut reader, cap, probe)?; + if folder { + crate::session::apply_folder_encryption_verdict(&mut reader, &mut disc)?; + } + Ok((disc, reader)) +} + +impl<'a> Source<'a> { + /// A live drive the caller opened and scanned; the session stays the caller's, so it + /// still locks, ejects and finishes the tray. Its drive is staged as the reader here. + pub fn from_session(session: &'a mut DiscSession) -> Source<'a> { + Source { + origin: Origin::Session(session), + } + } + + /// The image file at `path` read with a title the caller already scanned (the drive's, + /// J14): it is never rescanned, so a sweep's unread filesystem sectors do not matter. + /// The mux reads this one title whatever `MuxOptions::title_index` says. + pub fn from_image(path: &Path, title: ScannedTitle) -> Source<'static> { + Source { + origin: Origin::Prescanned { + path: path.to_path_buf(), + title, + }, + } + } + + /// A raw live reader the caller holds, with the title scanned off it (read like + /// [`Self::from_image`]'s, whatever `MuxOptions::title_index` says). + pub fn from_reader(reader: Box, title: ScannedTitle) -> Source<'static> { + Source { + origin: Origin::Live { reader, title }, + } + } + + /// The probed layout: the scanned disc for `disc://`, `iso://`, `dir://` and the + /// `from_*` forms; `None` for a container or stream input. + pub fn layout(&self) -> Option<&Disc> { + match &self.origin { + Origin::Session(s) => s.disc(), + Origin::Drive(s) => s.disc(), + Origin::Image { disc, .. } => Some(disc), + Origin::Prescanned { .. } | Origin::Live { .. } | Origin::Stream { .. } => None, + } + } + + /// The opened drive session, for `disc://` and [`Self::from_session`]. + pub fn session_mut(&mut self) -> Option<&mut DiscSession> { + match &mut self.origin { + Origin::Session(s) => Some(s), + Origin::Drive(s) => Some(s), + _ => None, + } + } + + /// The drive session `open_source("disc://…")` opened, so the caller can finish it + /// (unlock, eject). `None` for every other input. + pub fn into_session(self) -> Option { + match self.origin { + Origin::Drive(s) => Some(*s), + _ => None, + } + } + + /// The probe's layout and raw reader, for a caller that keys or copies the image itself + /// (`iso://`, `dir://`). `None` for every other input. + pub fn into_image(self) -> Option<(Disc, Box)> { + match self.origin { + Origin::Image { reader, disc, .. } => Some((disc, reader)), + _ => None, + } + } + + /// The raw reader the layout was probed through, for key acquisition before a mux. + pub fn reader_mut(&mut self) -> Option<&mut dyn SectorSource> { + match &mut self.origin { + Origin::Session(s) => s.source_mut(), + Origin::Drive(s) => s.source_mut(), + Origin::Image { reader: r, .. } | Origin::Live { reader: r, .. } => Some(r.as_mut()), + _ => None, + } + } + + // The provenance of title `title` read from this source (one place, SO15). + pub(crate) fn provenance(&self, title: usize) -> SourceInfo { + let (medium, path) = match &self.origin { + Origin::Session(s) => (Medium::Disc, format!("disc://{}", s.device_path())), + Origin::Drive(s) => (Medium::Disc, format!("disc://{}", s.device_path())), + Origin::Image { path, folder, .. } => ( + Medium::Iso, + format!( + "{}://{}", + if *folder { "dir" } else { "iso" }, + path.display() + ), + ), + Origin::Prescanned { path, .. } => (Medium::Iso, format!("iso://{}", path.display())), + Origin::Live { .. } => (Medium::Disc, String::new()), + Origin::Stream { url } => (super::driver::url_medium(&parse_url(url)), url.clone()), + }; + let (playlist, volume_id) = match (&self.origin, self.layout()) { + (Origin::Prescanned { title: t, .. } | Origin::Live { title: t, .. }, _) => { + (t.title.playlist.clone(), t.volume_id.clone()) + } + (_, Some(d)) => ( + d.titles + .get(title) + .map(|t| t.playlist.clone()) + .unwrap_or_default(), + d.volume_id.clone(), + ), + (_, None) => (String::new(), String::new()), + }; + // A pre-scanned title carries no index; it is reported as 0. + let title = match &self.origin { + Origin::Prescanned { .. } | Origin::Live { .. } => 0, + _ => title, + }; + SourceInfo { + medium, + path, + title, + playlist, + volume_id, + } + } + + // The output title's name (SO14): every disc layout names it by the disc (its meta + // title, else its volume id); a container keeps its own. + pub(crate) fn title_name(&self) -> Option { + match &self.origin { + Origin::Prescanned { title, .. } | Origin::Live { title, .. } => { + title.disc_name.clone() + } + _ => self.layout().map(disc_name), + } + } +} diff --git a/src/mux/stage_tests.rs b/src/mux/stage_tests.rs index abddce69..00e7caa4 100644 --- a/src/mux/stage_tests.rs +++ b/src/mux/stage_tests.rs @@ -4,7 +4,7 @@ use crate::aacs::content::{ALIGNED_UNIT_LEN, encrypt_unit}; use crate::consts::BD_SOURCE_PACKET_BYTES as PKT; use crate::error::{E_CSS_KEY_MISSING, E_MP4_INVALID, E_NO_DISC_KEY, E_NO_STREAMS, error_code}; -use crate::keys::ResolvedKeySet; +use crate::keys::KeyRing; use crate::mux::resolve::{InputOptions, input}; use crate::pes::PesFrame; @@ -90,7 +90,11 @@ fn read(scheme: &str, bytes: &[u8], opts: &InputOptions) -> std::io::Result<(Vec let p = path(scheme); std::fs::write(&p, bytes).unwrap(); let got = (|| { - let mut s = input(&format!("{scheme}://{}", p.display()), opts)?; + let mut s = input( + &format!("{scheme}://{}", p.display()), + opts, + &crate::ctx::Ctx::default(), + )?; let mut frames = Vec::new(); while let Some(f) = s.read()? { frames.push(f); @@ -103,7 +107,7 @@ fn read(scheme: &str, bytes: &[u8], opts: &InputOptions) -> std::io::Result<(Vec fn keyed() -> InputOptions { InputOptions { - keys: Some(ResolvedKeySet::held_for_test(&[[0x11; 16], KEY])), + keys: Some(KeyRing::held_for_test(&[[0x11; 16], KEY])), ..Default::default() } } @@ -135,7 +139,11 @@ fn an_aacs_m2ts_is_decrypted_with_the_held_keys() { fn an_aacs_m2ts_with_no_keys_is_refused_before_any_frame() { let p = path("nokeys"); std::fs::write(&p, flagged(&clear_clip(), Some(&KEY))).unwrap(); - let opened = input(&format!("m2ts://{}", p.display()), &Default::default()); + let opened = input( + &format!("m2ts://{}", p.display()), + &Default::default(), + &crate::ctx::Ctx::default(), + ); let _ = std::fs::remove_file(&p); assert_eq!( opened.err().and_then(|e| error_code(&e)), @@ -147,7 +155,7 @@ fn an_aacs_m2ts_with_no_keys_is_refused_before_any_frame() { fn an_aacs_m2ts_no_held_key_opens_is_refused() { let enc = flagged(&clear_clip(), Some(&KEY)); let opts = InputOptions { - keys: Some(ResolvedKeySet::held_for_test(&[[0x22; 16]])), + keys: Some(KeyRing::held_for_test(&[[0x22; 16]])), ..Default::default() }; let e = read("m2ts", &enc, &opts).unwrap_err(); diff --git a/src/mux/stdio.rs b/src/mux/stdio.rs index a0620a6f..bb14cb62 100644 --- a/src/mux/stdio.rs +++ b/src/mux/stdio.rs @@ -131,7 +131,14 @@ impl Read for Counted<'_, R> { } } -impl crate::pes::Stream for StdioStream { +impl StdioStream { + /// The title being read or written (both directions open this type). + pub fn info(&self) -> &crate::disc::DiscTitle { + &self.disc_title + } +} + +impl crate::pes::PesSource for StdioStream { fn read(&mut self) -> io::Result> { self.ensure_header_read()?; match &mut self.reader { @@ -139,26 +146,7 @@ impl crate::pes::Stream for StdioStream { None => Err(crate::error::Error::StreamWriteOnly.into()), } } - fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> { - if self.writer.is_none() { - return Err(crate::error::Error::StreamReadOnly.into()); - } - self.ensure_header_written()?; - match &mut self.writer { - Some(w) => frame.serialize_ext(w, self.padded), - None => Err(crate::error::Error::StreamReadOnly.into()), - } - } - fn finish(&mut self) -> io::Result<()> { - // Emit the header even when write() was never called, so a zero-frame - // title still produces the FMKV magic + metadata header on stdout - // (symmetric with the read side's read_header()). - self.ensure_header_written()?; - if let Some(w) = &mut self.writer { - w.flush()?; - } - Ok(()) - } + fn info(&self) -> &DiscTitle { &self.disc_title } @@ -168,21 +156,6 @@ impl crate::pes::Stream for StdioStream { self.timings.get(track).copied().unwrap_or_default() } - fn set_track_timing( - &mut self, - track: usize, - timing: crate::pes::TrackTiming, - ) -> io::Result<()> { - if self.writer.is_none() { - return Err(crate::error::Error::StreamReadOnly.into()); - } - if self.header_written { - return Err(crate::error::Error::StreamHeaderWritten.into()); - } - let tracks = self.disc_title.streams.len(); - super::network::set_timing(&mut self.timings, track, timing, tracks) - } - fn codec_private(&self, track: usize) -> Option> { // Single source of truth: the title's own codec_privates. (The // previous `stored_codec_privates` field was a redundant clone @@ -201,10 +174,53 @@ impl crate::pes::Stream for StdioStream { } } +impl crate::pes::PesSink for StdioStream { + fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> { + if self.writer.is_none() { + return Err(crate::error::Error::StreamReadOnly.into()); + } + self.ensure_header_written()?; + match &mut self.writer { + Some(w) => frame.serialize_ext(w, self.padded), + None => Err(crate::error::Error::StreamReadOnly.into()), + } + } + + fn finish(&mut self) -> io::Result<()> { + // Emit the header even when write() was never called, so a zero-frame + // title still produces the FMKV magic + metadata header on stdout + // (symmetric with the read side's read_header()). + self.ensure_header_written()?; + if let Some(w) = &mut self.writer { + w.flush()?; + } + Ok(()) + } + + fn info(&self) -> &DiscTitle { + &self.disc_title + } + + fn set_track_timing( + &mut self, + track: usize, + timing: crate::pes::TrackTiming, + ) -> io::Result<()> { + if self.writer.is_none() { + return Err(crate::error::Error::StreamReadOnly.into()); + } + if self.header_written { + return Err(crate::error::Error::StreamHeaderWritten.into()); + } + let tracks = self.disc_title.streams.len(); + super::network::set_timing(&mut self.timings, track, timing, tracks) + } +} + #[cfg(test)] mod tests { use super::*; - use crate::pes::Stream as _; + use crate::pes::{PesSink as _, PesSource as _}; // The pub type keeps the auto traits it had when it held Stdin/Stdout. #[test] diff --git a/src/mux/tsmux.rs b/src/mux/tsmux.rs index 98b70706..f2cd7ef6 100644 --- a/src/mux/tsmux.rs +++ b/src/mux/tsmux.rs @@ -32,6 +32,7 @@ pub(crate) fn is_video_pid(pid: u16) -> bool { // Headroom below the first frame's PTS for frames presented before it but emitted // after it. ISO/IEC 13818-1 §2.4.2.6 caps T-STD buffer delay at 1 s. +#[cfg(test)] pub(crate) const ORIGIN_HEADROOM_NS: i64 = 1_000_000_000; // The same headroom in 90 kHz ticks: 1 s is exactly 90 000 ticks. const ORIGIN_HEADROOM_TICKS: i64 = 90_000; @@ -88,7 +89,7 @@ pub struct TsMuxer { video_codec: Vec, /// Global PTS origin in integer 90 kHz ticks, seeded by the FIRST frame of any /// kind (video or audio) so a single fixed origin rebases every frame and the - /// audio/video offset is preserved. It lies [`ORIGIN_HEADROOM_NS`] (90 000 ticks) + /// audio/video offset is preserved. It lies `ORIGIN_HEADROOM_TICKS` (1 s, 90 000 ticks) /// before the seeding frame's tick; only a frame earlier than that saturates to 0. origin_ticks: Option, /// Count of PES frames actually emitted (a frame dropped as non-key diff --git a/src/pes.rs b/src/pes.rs index 87c7d4a6..a0252c9a 100644 --- a/src/pes.rs +++ b/src/pes.rs @@ -235,36 +235,16 @@ pub struct TrackTiming { pub seek_preroll_ns: u64, } -/// A PES frame stream. One trait per format — same type opens for read -/// (`open()` / `listen()` / `input()`) or write (`create()` / `connect()` / -/// `output()`). Calling the wrong-direction method returns a typed -/// `StreamReadOnly` / `StreamWriteOnly` error. +/// Where PES frames come from (pipeline design §2.2): an opened input's title (its +/// [`DiscTitle`](crate::disc::DiscTitle) track table) and its frames. Built by +/// [`crate::input`] / [`crate::open_source`] for every input scheme. /// -/// `Send` is required so streams can move across the producer/consumer -/// threads in autorip's mux pipeline. -pub trait Stream: Send { - /// Read the next frame, or `Ok(None)` at end of stream. Returns - /// `StreamWriteOnly` on a stream opened for writing. +/// `Send` so a source can move to the mux's producer thread. +pub trait PesSource: Send { + /// Read the next frame, or `Ok(None)` at end of stream. fn read(&mut self) -> std::io::Result>; - /// Write a frame to the sink. Returns `StreamReadOnly` on a stream - /// opened for reading. - fn write(&mut self, frame: &PesFrame) -> std::io::Result<()>; - - /// Finalize the stream: flush buffered frames, write any container - /// index (MKV `Cues`), close the underlying file/socket. Idempotent - /// for read-only streams (no-op). - fn finish(&mut self) -> std::io::Result<()>; - - /// End a stream whose producer failed or was stopped before the end of the - /// title. Default: [`finish`](Self::finish) (a file keeps its partial output); - /// a wire sink overrides it so its receiver sees a failure, not a clean end. - fn finish_incomplete(&mut self) -> std::io::Result<()> { - self.finish() - } - - /// Stream metadata. Stable across reads — implementors must return a - /// consistent reference for the lifetime of the stream. + /// The title being read: its tracks, codec setup, chapters. Stable across reads. fn info(&self) -> &crate::disc::DiscTitle; /// Container timing metadata, indexed in the same order as `info().streams`. @@ -272,13 +252,6 @@ pub trait Stream: Send { TrackTiming::default() } - /// Set timing before the first frame is written. Carried by `mkv://` and the - /// FMKV `network://` / `stdio://` wire; `m2ts://` has no field for it and - /// `mp4://` carries no Opus track, so those keep the no-op default. - fn set_track_timing(&mut self, _track: usize, _timing: TrackTiming) -> std::io::Result<()> { - Ok(()) - } - /// Codec initialization data for a track (SPS/PPS, AC-3 fscod, etc.). /// `None` for tracks that don't need codec_private (raw passthrough). fn codec_private(&self, _track: usize) -> Option> { @@ -300,47 +273,66 @@ pub trait Stream: Send { Vec::new() } + /// Cumulative count of read errors the source skipped past (zero-filled bad sectors, + /// blanked damaged units, frames the resync gate dropped). + fn errors(&self) -> u64 { + 0 + } + + /// Cumulative bytes actually skipped (zero-filled) past read errors. Distinct from + /// [`errors`](Self::errors), which counts skip *events*: one AACS skip covers a whole + /// 6144-byte unit, so consumers estimating lost time scale by this byte count. + fn lost_bytes(&self) -> u64 { + 0 + } +} + +/// Where PES frames go (pipeline design §2.2): a container or wire writer, opened by +/// [`crate::output`] for every output scheme with the title it writes. +/// +/// `Send` so a sink can move to the mux's write consumer thread. +pub trait PesSink: Send { + /// Write a frame. + fn write(&mut self, frame: &PesFrame) -> std::io::Result<()>; + + /// Finalize: flush buffered frames, write any container index (MKV `Cues`), close the + /// underlying file/socket. + fn finish(&mut self) -> std::io::Result<()>; + + /// End a sink whose producer failed or was stopped before the end of the title. + /// Default: [`finish`](Self::finish) (a file keeps its partial output); a wire sink + /// overrides it so its receiver sees a failure, not a clean end. + fn finish_incomplete(&mut self) -> std::io::Result<()> { + self.finish() + } + + /// The title this sink was opened with. + fn info(&self) -> &crate::disc::DiscTitle; + + /// Set timing before the first frame is written. Carried by `mkv://` and the + /// FMKV `network://` / `stdio://` wire; `m2ts://` has no field for it and + /// `mp4://` carries no Opus track, so those keep the no-op default. + fn set_track_timing(&mut self, _track: usize, _timing: TrackTiming) -> std::io::Result<()> { + Ok(()) + } + /// Supply a codec_private that resolved after the header was written (late /// AAC config). `Ok(true)` when the sink recorded it; default `Ok(false)`. fn set_codec_private(&mut self, _track: usize, _data: &[u8]) -> std::io::Result { Ok(false) } - /// Cumulative count of read errors the stream skipped past (e.g. - /// zero-filled bad sectors on a live drive). Default `0` for - /// streams that don't have a notion of skip-on-error (file ISO, - /// network, stdio, the pipeline highway, etc.); concrete impls - /// with adaptive retry (`DiscStream` on the drive single-pass - /// path) override. - fn errors(&self) -> u64 { - 0 - } - - /// Cumulative bytes actually skipped (zero-filled) past read errors. - /// Distinct from [`errors`](Self::errors), which counts skip *events*: - /// a single AACS skip event covers a whole 6144-byte unit, so - /// `errors * 2048` understates real loss. Consumers estimating lost - /// video time must scale by this byte count, not the event count. - /// Default `0` for streams with no skip-on-error notion; `DiscStream` - /// overrides. - fn lost_bytes(&self) -> u64 { - 0 - } - - /// Sink side: `info().streams` indices this sink PLANNED to carry (and - /// accepted frames for) but could not put in the finished container, valid - /// after [`finish`](Self::finish). Empty for every sink that writes everything - /// it accepted, except `mp4://` (audio with no parseable sample entry, known at - /// finish) and `m2ts://` (LPCM BD LPCM can't carry, known from creation). + /// `info().streams` indices this sink PLANNED to carry (and accepted frames for) but + /// could not put in the finished container, valid after [`finish`](Self::finish). + /// Empty for every sink that writes everything it accepted, except `mp4://` (audio + /// with no parseable sample entry, known at finish) and `m2ts://` (LPCM BD LPCM can't + /// carry, known from creation). fn undelivered_streams(&self) -> Vec { - // Without this, such a drop would silently contradict the pre-mux - // plan the crate publishes (`mp4_fit_report`); the driver folds - // this into `MuxOutcome::undelivered_streams`. Vec::new() } } -/// Wraps any output stream and counts bytes written. +/// Wraps any output sink and counts bytes written. /// /// Progress tracking is a CLI concern — streams don't know their size. /// Wrap the output with `CountingStream`, then query `bytes_written()`. @@ -353,12 +345,12 @@ pub trait Stream: Send { /// } /// ``` pub struct CountingStream { - inner: Box, + inner: Box, written: u64, } impl CountingStream { - pub fn new(inner: Box) -> Self { + pub fn new(inner: Box) -> Self { Self { inner, written: 0 } } @@ -368,11 +360,7 @@ impl CountingStream { } } -impl Stream for CountingStream { - fn read(&mut self) -> std::io::Result> { - self.inner.read() - } - +impl crate::pes::PesSink for CountingStream { fn write(&mut self, frame: &PesFrame) -> std::io::Result<()> { // Only count bytes that actually made it to the inner sink, so a // failed write doesn't permanently inflate bytes_written(). @@ -393,40 +381,17 @@ impl Stream for CountingStream { self.inner.info() } - fn track_timing(&self, track: usize) -> TrackTiming { - self.inner.track_timing(track) - } fn set_track_timing(&mut self, track: usize, timing: TrackTiming) -> std::io::Result<()> { self.inner.set_track_timing(track, timing) } - fn codec_private(&self, track: usize) -> Option> { - self.inner.codec_private(track) - } - - fn headers_ready(&self) -> bool { - self.inner.headers_ready() - } - - fn config_changes(&self) -> Vec<(usize, u64)> { - self.inner.config_changes() - } - fn set_codec_private(&mut self, track: usize, data: &[u8]) -> std::io::Result { self.inner.set_codec_private(track, data) } - fn errors(&self) -> u64 { - self.inner.errors() - } - fn undelivered_streams(&self) -> Vec { self.inner.undelivered_streams() } - - fn lost_bytes(&self) -> u64 { - self.inner.lost_bytes() - } } #[cfg(test)] @@ -465,11 +430,17 @@ mod tests { } } - impl Stream for MockStream { + impl crate::pes::PesSource for MockStream { fn read(&mut self) -> std::io::Result> { Ok(self.read_queue.next()) } + fn info(&self) -> &DiscTitle { + &self.title + } + } + + impl crate::pes::PesSink for MockStream { fn write(&mut self, frame: &PesFrame) -> std::io::Result<()> { self.written.push(frame.clone()); Ok(()) @@ -499,20 +470,17 @@ mod tests { assert!(s.undelivered_streams().is_empty()); } - // CountingStream forwards reads/finish and sums bytes over several writes. + // CountingStream forwards finish and sums bytes over several writes. #[test] - fn counting_stream_forwards_reads_and_sums_writes() { + fn counting_stream_sums_writes() { let frames = vec![make_frame(0, 0), make_frame(1, 1_000)]; let mut cs = CountingStream::new(Box::new(MockStream::new(frames.clone()))); - assert_eq!(cs.read().unwrap().expect("frame 0").pts, 0); - assert_eq!(cs.read().unwrap().expect("frame 1").pts, 1_000); - assert!(cs.read().unwrap().is_none()); for f in &frames { cs.write(f).unwrap(); } assert_eq!(cs.bytes_written(), 6); cs.finish().unwrap(); - let _ = cs.info(); + let _ = PesSink::info(&cs); } // The 256 MiB frame ceiling applies on write and on read (before any allocation). @@ -585,16 +553,25 @@ mod tests { title: DiscTitle, } - impl Stream for FailingWriteStream { + impl crate::pes::PesSource for FailingWriteStream { fn read(&mut self) -> std::io::Result> { Ok(None) } + + fn info(&self) -> &DiscTitle { + &self.title + } + } + + impl crate::pes::PesSink for FailingWriteStream { fn write(&mut self, _frame: &PesFrame) -> std::io::Result<()> { Err(std::io::Error::from(std::io::ErrorKind::BrokenPipe)) } + fn finish(&mut self) -> std::io::Result<()> { Ok(()) } + fn info(&self) -> &DiscTitle { &self.title } diff --git a/src/progress.rs b/src/progress.rs index e8168ba9..a223e241 100644 --- a/src/progress.rs +++ b/src/progress.rs @@ -1,10 +1,10 @@ //! Pipeline-progress reporting for the rip pipeline. //! //! Architecture rule: ONE progress signal type. Every long-running -//! pipeline operation (`freemkv_engine::recovery::{copy, patch}`) emits the -//! same [`PassProgress`] shape via the [`Progress`] trait. Consumers (autorip) -//! compute their own single derived view from these fields and never reach -//! into per-pass internals. +//! pipeline operation (`freemkv_engine::recovery::{copy, patch}`, extract) emits the +//! same [`PassProgress`] shape as [`Event::Pass`](crate::Event::Pass) through the run's +//! [`Ctx`](crate::Ctx). Consumers compute their own single derived view from these fields +//! and never reach into per-pass internals. /// Identifies which pipeline phase the progress event belongs to. /// @@ -27,6 +27,8 @@ pub enum PassKind { Mux, /// Sector verification — reads every sector and classifies health. Verify, + /// A disc's decrypted file tree written out (`dir://`). + Extract, } /// One located bad/not-yet-good range, annotated with the chapter and movie @@ -235,26 +237,6 @@ impl Heartbeat { } } -/// A consumer of pipeline progress events. Library code calls -/// `Progress::report` once per inner-loop iteration (throttling is the -/// consumer's job — `report` is cheap; the library doesn't gate it). -/// -/// Returns `true` to continue, `false` to request early stop. -/// -/// No `Send`/`Sync` bound — `report` is always called from the same thread -/// running the pipeline, so closures with non-`Sync` captures (e.g. -/// `RefCell`) work directly. Blanket impl below lets -/// callers pass closures without explicit struct types. -pub trait Progress { - fn report(&self, p: &PassProgress) -> bool; -} - -impl bool> Progress for F { - fn report(&self, p: &PassProgress) -> bool { - (self)(p) - } -} - #[cfg(test)] mod heartbeat_tests { use super::Heartbeat; @@ -496,49 +478,4 @@ mod pass_progress_tests { "pending_pct must read bytes_pending_total" ); } - - // A closure is a `Progress` via the blanket impl; its return value is the - // cancellation signal. A constant-returning blanket body would make every - // closure-based consumer uncancellable. - #[test] - fn the_closure_blanket_impl_returns_the_closures_own_verdict() { - fn ask(p: &P, s: &PassProgress) -> bool { - p.report(s) - } - - let keep_going = |_: &PassProgress| true; - let cancel = |_: &PassProgress| false; - - assert!(ask(&keep_going, &sample()), "true must survive the forward"); - assert!( - !ask(&cancel, &sample()), - "a closure returning false is a CANCEL and must not be reported as \ - keep-going; a constant-true blanket impl makes cancellation a no-op" - ); - } - - /// The blanket impl must hand the closure the caller's sample, not a - /// fabricated one — a consumer decides whether to cancel FROM the numbers. - #[test] - fn the_closure_blanket_impl_passes_the_sample_through() { - use std::sync::{Arc, Mutex}; - fn ask(p: &P, s: &PassProgress) -> bool { - p.report(s) - } - - let seen = Arc::new(Mutex::new(Vec::new())); - let sink = seen.clone(); - let recorder = move |p: &PassProgress| { - sink.lock().unwrap().push((p.work_done, p.work_total)); - true - }; - - let s = PassProgress { - work_done: 7, - work_total: 9, - ..sample() - }; - assert!(ask(&recorder, &s)); - assert_eq!(*seen.lock().unwrap(), vec![(7, 9)]); - } } diff --git a/src/scsi/fd_handoff.rs b/src/scsi/fd_handoff.rs index a2665dc4..de498a4d 100644 --- a/src/scsi/fd_handoff.rs +++ b/src/scsi/fd_handoff.rs @@ -24,9 +24,14 @@ pub(crate) use std::sync::atomic::{AtomicBool, AtomicI32, Ordering}; pub(crate) fn reserve_recovery_slot(counter: &std::sync::atomic::AtomicUsize, max: usize) -> bool { use std::sync::atomic::Ordering::{AcqRel, Acquire}; // CAS loop: a refused reserver never bumps the counter, so it can't starve a fitting one. - counter - .fetch_update(AcqRel, Acquire, |n| (n < max).then_some(n + 1)) - .is_ok() + let mut n = counter.load(Acquire); + while n < max { + match counter.compare_exchange_weak(n, n + 1, AcqRel, Acquire) { + Ok(_) => return true, + Err(now) => n = now, + } + } + false } /// Give back a slot taken by [`reserve_recovery_slot`]. diff --git a/src/sector/decrypting.rs b/src/sector/decrypting.rs index 654b0d16..487ec322 100644 --- a/src/sector/decrypting.rs +++ b/src/sector/decrypting.rs @@ -47,10 +47,12 @@ pub struct DecryptingSectorSource { /// is a bug and fails loud on the first unit (AACS decrypts only via the map). key_map: Option>, /// The key set's on-arrival proof for pieces `resolve` could not prove up front - /// (KU §2.4). `None` for every reader not built by a `ResolvedKeySet`. + /// (KU §2.4). `None` for every reader not built by a `KeyRing`. arrival: Option>, /// Damaged AACS units blanked so far (see [`blanked_units`](Self::blanked_units)). blanked: BlankTally, + /// The run that also hears of each blanked unit (`UnitBlanked`, `Stats`). + ctx: Option, /// Content-detected stage: `Some` until the first read reaches the verdict. detect: Option>, /// Loose BD-TS: a CPI-flagged unit that is clean TS is clear (its flag is stale). @@ -67,8 +69,9 @@ pub(crate) struct StageOptions { /// Pass ciphertext through: never crack, decrypt or refuse. pub(crate) raw: bool, /// Held AACS keys for a loose BD-TS file. - pub(crate) keys: Option, - pub(crate) halt: Option, + pub(crate) keys: Option, + /// The run: its halt ends the crack scan, its stats count blanked units. + pub(crate) ctx: crate::ctx::Ctx, } // Sectors per CSS crack-scan read (the mpg:// batch). @@ -96,35 +99,94 @@ fn span_touches_content(content: Option<&[(u32, u32)]>, lba: u32, count: u16) -> content.is_none_or(|r| crate::decrypt::span_in_content_ranges(lba, count as u32, r)) } +/// What a [`DecryptingSectorSource`] decrypts with: the one input of its one constructor. +/// +/// Built from [`DecryptKeys`] (CSS title keys, or none: clear or raw), from a +/// [`KeyRing`](crate::keys::KeyRing)'s view (its per-unit map and on-arrival proof), or as +/// the content-detected stage that reaches its verdict on the first read. +pub struct Keying(KeyingKind); + +enum KeyingKind { + Keys(DecryptKeys), + Ring { + keys: DecryptKeys, + map: Arc, + arrival: Option, + }, + Detect(Box), +} + +impl From for Keying { + fn from(keys: DecryptKeys) -> Self { + Keying(KeyingKind::Keys(keys)) + } +} + +impl Keying { + /// A key ring's view: `keys` decrypts each unit with its `map` key, and `arrival` + /// proves the units of pieces the ring could not prove up front (KU §2.4). + pub(crate) fn ring( + keys: DecryptKeys, + map: Arc, + arrival: Option, + ) -> Self { + Keying(KeyingKind::Ring { keys, map, arrival }) + } + + /// The content-detected stage (every input but a disc, image or folder): the first read + /// classifies the head and resolves once (D3). A PS is cracked, a BD-TS read through + /// [`KeyRing::loose_file`](crate::keys::KeyRing::loose_file), else clear. + pub(crate) fn detect(opts: StageOptions) -> Self { + Keying(KeyingKind::Detect(Box::new(opts))) + } +} + impl DecryptingSectorSource { - /// Wrap `inner` with the given keys. AACS decrypts only through a key map, which - /// only a [`ResolvedKeySet`](crate::keys::ResolvedKeySet) reader installs; an AACS - /// source built here fails loud on its first encrypted unit. - pub fn new(inner: S, keys: DecryptKeys) -> Self { - Self { + /// Wrap `inner` as the decryption stage `keying` describes. AACS decrypts only through a + /// key ring's map; an AACS source built from bare [`DecryptKeys`] fails loud on its + /// first encrypted unit. + pub fn new(inner: S, keying: impl Into) -> Self { + let mut s = Self { inner, - keys, + keys: DecryptKeys::None, unit_base: None, content_ranges: None, key_map: None, arrival: None, blanked: BlankTally(Arc::default()), + ctx: None, detect: None, stale_cpi: false, watch_css: false, refused: false, + }; + match keying.into().0 { + KeyingKind::Keys(keys) => s.keys = keys, + KeyingKind::Ring { keys, map, arrival } => { + s.keys = keys; + s.key_map = Some(map); + s.arrival = arrival.map(Box::new); + } + KeyingKind::Detect(opts) => { + #[cfg(test)] + super::stage::STAGES.with(|n| n.set(n.get() + 1)); + s.observe(&opts.ctx); + s.detect = Some(opts); + } } + s } - /// The content-detected stage (every input but a disc, image or folder): the first read - /// classifies the head and resolves once (D3). A PS is cracked, a BD-TS read through - /// [`ResolvedKeySet::loose_file`](crate::keys::ResolvedKeySet::loose_file), else clear. - pub(crate) fn detecting(inner: S, opts: StageOptions) -> Self { - #[cfg(test)] - super::stage::STAGES.with(|n| n.set(n.get() + 1)); - let mut s = Self::new(inner, DecryptKeys::None); - s.detect = Some(Box::new(opts)); - s + // A test's own map on an already-built reader (the tests outside `keys`). + #[cfg(test)] + pub(crate) fn keyed_for_test(mut self, map: Arc) -> Self { + self.key_map = Some(map); + self + } + + /// Report each blanked unit to `ctx` too: an `UnitBlanked` event and its loss counter. + pub(crate) fn observe(&mut self, ctx: &crate::ctx::Ctx) { + self.ctx = Some(ctx.clone()); } // Reach the verdict and install what it needs. On `Err` the stage stays undecided. @@ -143,7 +205,7 @@ impl DecryptingSectorSource { start_lba: 0, sector_count: cap, }]; - let halt = opts.halt.as_ref(); + let halt = Some(&opts.ctx.halt); let cracked = crate::css::crack_title_key(&mut self.inner, &whole, CRACK_BATCH, halt); self.keys = match cracked { @@ -160,7 +222,7 @@ impl DecryptingSectorSource { self.watch_css = !self.keys.is_encrypted(); } Kind::BdTs if !opts.raw => { - let set = crate::keys::ResolvedKeySet::loose_file(opts.keys.as_ref(), cap); + let set = crate::keys::KeyRing::loose_file(opts.keys.as_ref(), cap); self.keys = set.decrypt_keys(); self.key_map = Some(set.key_map()); self.arrival = set.arrival(set.title_stop()).map(Box::new); @@ -174,35 +236,6 @@ impl DecryptingSectorSource { Ok(()) } - /// Install the key set's on-arrival proof (KU §2.4): an encrypted unit of a piece - /// `resolve` left unproven is decrypted with a held key proven on it here. - pub(crate) fn with_arrival(mut self, arrival: crate::keys::Arrival) -> Self { - self.arrival = Some(Box::new(arrival)); - self - } - - /// `&mut` counterpart of [`with_arrival`](Self::with_arrival). - pub(crate) fn set_arrival(&mut self, arrival: crate::keys::Arrival) { - self.arrival = Some(Box::new(arrival)); - } - - /// Install a proactive [`AacsKeyMap`](crate::decrypt::AacsKeyMap): the caller - /// resolved one key per CPS unit / segment up front, so every aligned unit is - /// decrypted with its MAPPED key and trusted — no per-unit `is_clean` check. - /// AACS-only; a CSS / clear disc ignores it. - pub(crate) fn with_key_map(mut self, map: Arc) -> Self { - self.key_map = Some(map); - self - } - - /// `&mut` counterpart of [`with_key_map`](Self::with_key_map): install the - /// proactive map on an already-constructed source (the inline live-drive - /// [`DiscStream`](crate::mux::DiscStream) builds the decorator first, then - /// installs the map via its own `with_key_map`). - pub(crate) fn set_key_map(&mut self, map: Arc) { - self.key_map = Some(map); - } - /// Restrict decrypt to every stream file's extents (e.g. /// [`Disc::stream_content_ranges`](crate::Disc::stream_content_ranges), /// not the title-only [`Disc::encrypted_content_ranges`](crate::Disc::encrypted_content_ranges)). @@ -346,7 +379,7 @@ impl SectorSource for DecryptingSectorSource { (self.arrival.as_deref(), &self.keys) { let blanked = arrival.process(&mut self.inner, lba, &mut buf[..n], unit_keys)?; - self.count_blanked(blanked); + self.count_blanked(lba, blanked); } // Proactive map path (storm-free mux): keys were resolved per unit up front, @@ -360,7 +393,7 @@ impl SectorSource for DecryptingSectorSource { &map, content_ref, )?; - self.count_blanked(blanked); + self.count_blanked(lba, blanked); return Ok(n); } @@ -462,13 +495,20 @@ impl DecryptingSectorSource { let end = lba as u64 + buf.len().div_ceil(crate::consts::SECTOR_BYTES) as u64; let at_end = cap != 0 && end >= cap as u64; let n = crate::decrypt::blank_damaged_units(buf, lba, format, &covered, at_end); - self.count_blanked(n); + self.count_blanked(lba, n); } - fn count_blanked(&self, n: usize) { + fn count_blanked(&self, lba: u32, n: usize) { self.blanked .0 .fetch_add(n as u64, std::sync::atomic::Ordering::Relaxed); + if let (Some(ctx), 1..) = (&self.ctx, n) { + ctx.stats.add_blanked(n as u64); + ctx.emit(crate::event::Event::UnitBlanked { + lba: u64::from(lba), + units: n as u64, + }); + } } } @@ -525,9 +565,9 @@ mod tests { let opts = StageOptions { raw: false, keys: None, - halt: None, + ctx: Default::default(), }; - let mut stage = DecryptingSectorSource::detecting(src, opts); + let mut stage = DecryptingSectorSource::new(src, Keying::detect(opts)); let scans = crate::css::CRACK_SCANS.with(|n| n.get()); let mut buf = vec![0u8; 2048]; for _ in 0..2 { @@ -837,7 +877,9 @@ mod tests { let inner = ArgRecorder { calls: Arc::new(Mutex::new(Vec::new())), }; - let pf = crate::sector::PrefetchedSectorSource::new(inner, ext, 3, None).unwrap(); + let pf = + crate::sector::PrefetchedSectorSource::new(inner, ext, 3, &crate::ctx::Ctx::default()) + .unwrap(); let d = DecryptingSectorSource::new(pf, DecryptKeys::None); assert!(!d.random_access()); let d = DecryptingSectorSource::new(FuaProbe { fua: vec![] }, DecryptKeys::None); @@ -1068,7 +1110,7 @@ mod tests { u32::MAX, 0, )])); - let mut dec = DecryptingSectorSource::new(src, keys).with_key_map(map); + let mut dec = DecryptingSectorSource::new(src, Keying::ring(keys, map, None)); dec.set_unit_base(0); let mut buf = vec![0u8; crate::aacs::content::ALIGNED_UNIT_LEN]; let n = dec.read_sectors(0, 3, &mut buf, false).unwrap(); @@ -1095,7 +1137,7 @@ mod tests { unit_keys: vec![(0, key)], format: crate::disc::ContentFormat::BdTs, }; - let mut dec = DecryptingSectorSource::new(src, keys); // no with_key_map + let mut dec = DecryptingSectorSource::new(src, keys); // no key map dec.set_unit_base(0); let mut buf = vec![0u8; crate::aacs::content::ALIGNED_UNIT_LEN]; let err = dec @@ -1124,8 +1166,7 @@ mod tests { )])); // Content covers a DIFFERENT extent (LBA 300..303); LBA 0 is outside it. let ranges: Arc<[(u32, u32)]> = Arc::from(vec![(300u32, 3u32)].into_boxed_slice()); - let mut dec = DecryptingSectorSource::new(src, keys) - .with_key_map(map) + let mut dec = DecryptingSectorSource::new(src, Keying::ring(keys, map, None)) .with_content_ranges(ranges); let mut buf = vec![0u8; crate::aacs::content::ALIGNED_UNIT_LEN]; let n = dec.read_sectors(0, 3, &mut buf, false).unwrap(); @@ -1148,8 +1189,7 @@ mod tests { }; let map = std::sync::Arc::new(crate::decrypt::AacsKeyMap::from_ranges(vec![(300, 303, 0)])); let ranges: Arc<[(u32, u32)]> = Arc::from(vec![(300u32, 3u32)].into_boxed_slice()); - let mut dec = DecryptingSectorSource::new(src, keys) - .with_key_map(map) + let mut dec = DecryptingSectorSource::new(src, Keying::ring(keys, map, None)) .with_content_ranges(ranges); let mut buf = vec![0u8; 2048]; // LBA 1 is misaligned AND outside content → passes through, no DecryptFailed. @@ -1188,8 +1228,10 @@ mod tests { u32::MAX, 0, )])); - let mut dec = - DecryptingSectorSource::new(CountingSource(reads.clone()), keys).with_key_map(map); + let mut dec = DecryptingSectorSource::new( + CountingSource(reads.clone()), + Keying::ring(keys, map, None), + ); dec.set_unit_base(0); let mut buf = vec![0u8; crate::aacs::content::ALIGNED_UNIT_LEN]; let err = dec @@ -1242,8 +1284,7 @@ mod tests { FILE_LBA + 6, 0, )])); - DecryptingSectorSource::new(MisalignedFile(data), keys) - .with_key_map(map) + DecryptingSectorSource::new(MisalignedFile(data), Keying::ring(keys, map, None)) .with_content_ranges(Arc::from(vec![(FILE_LBA, 6u32)])) } @@ -1315,7 +1356,8 @@ mod tests { let map = Arc::new(crate::decrypt::AacsKeyMap::from_ranges(vec![( FILE_LBA, end, 0, )])); - let mut dec = DecryptingSectorSource::new(MisalignedFile(data), keys).with_key_map(map); + let mut dec = + DecryptingSectorSource::new(MisalignedFile(data), Keying::ring(keys, map, None)); dec.set_unit_base(FILE_LBA); dec } @@ -1432,8 +1474,10 @@ mod tests { 0, crate::decrypt::Phase::Even, )]); - let mut dec = - DecryptingSectorSource::new(MisalignedFile(data), keys).with_key_map(Arc::new(map)); + let mut dec = DecryptingSectorSource::new( + MisalignedFile(data), + Keying::ring(keys, Arc::new(map), None), + ); dec.set_unit_base(FILE_LBA); dec } diff --git a/src/sector/mod.rs b/src/sector/mod.rs index 0da7a05a..7228edaa 100644 --- a/src/sector/mod.rs +++ b/src/sector/mod.rs @@ -11,6 +11,7 @@ pub mod bus_removal; pub mod decrypting; pub mod prefetched; +pub mod read_stage; pub(crate) mod stage; use crate::error::Result; @@ -190,7 +191,7 @@ impl SectorSource for &mut (dyn SectorSource + '_) { } pub use crate::io::file_sector_source::FileSectorSource; -pub use decrypting::DecryptingSectorSource; +pub use decrypting::{DecryptingSectorSource, Keying}; pub use prefetched::PrefetchedSectorSource; #[cfg(test)] diff --git a/src/sector/prefetched.rs b/src/sector/prefetched.rs index 5631a97f..0ca4ee15 100644 --- a/src/sector/prefetched.rs +++ b/src/sector/prefetched.rs @@ -7,19 +7,13 @@ //! sending batches into a bounded channel; sender drop reads as end-of-stream. `read_sectors` //! ignores its `lba`/`count` args. +use crate::ctx::Ctx; use crate::error::Result; -use crate::event::{Event, EventKind}; use crate::halt::{DriveHolder, Halt, Recv, SendOutcome}; use crate::sector::SectorSource; use crossbeam_channel::{Receiver, Sender, bounded}; use std::time::Duration; -/// Producer-thread event callback. Fires `BytesRead` after every -/// successful batch read so the consumer side can update a UI -/// progress indicator without polling. `Send + 'static` because the -/// callback runs on the producer thread. -pub type EventFn = Box; - const PREFETCH_CHANNEL_DEPTH: usize = 2; /// Smallest sector source the producer will issue per read. AACS @@ -43,8 +37,8 @@ pub struct PrefetchedSectorSource { /// cross-thread alloc/free was the dominant cost in the demux /// thread before this). recycle_tx: Sender>, - /// The producer, a Drive holder (§2.5): joined on drop. - producer: Option>, + /// The producer (a Drive holder for a disc or image, §2.5): joined on drop. + producer: Option, /// Total sector count across all extents, computed once at /// construction (the sum of each extent's `sector_count`) and /// returned by [`capacity_sectors`]. Never updated by reads. @@ -60,7 +54,31 @@ pub struct PrefetchedSectorSource { /// The reader's unmapped stream files, snapshotted before it moved to the producer. unmapped: Vec, /// The op's token: once cancelled, the closed channel is a stop, not EOF (L096). - halt: Option, + halt: Halt, +} + +// The producer thread: a Drive holder (§2.5) for a disc or image read, a plain thread for a +// file's byte view (`m2ts://`), which holds no Drive. +enum Producer { + Drive(DriveHolder<()>), + File(std::thread::JoinHandle<()>), +} + +impl Producer { + fn join(self) { + match self { + Producer::Drive(h) => drop(h.join()), + Producer::File(h) => drop(h.join()), + } + } +} + +/// A file's byte view through the Read stage (`m2ts://`): `prefix` (the bytes already read +/// past the head scan) goes out first, then the chunks, ending after `len` chunk bytes (the +/// file's real length, short of its zero-padded last sector). +pub(crate) struct ByteView { + pub(crate) prefix: Vec, + pub(crate) len: u64, } // Hand `item` to the consumer, waiting on a full channel; `false` once the consumer is @@ -81,65 +99,93 @@ impl PrefetchedSectorSource { /// else [`Error::ExtentNotUnitAligned`] surfaces through the channel. /// /// [`Error::ExtentNotUnitAligned`]: crate::error::Error::ExtentNotUnitAligned + /// + /// `ctx.halt` stops the producer; a `BytesRead` event goes to `ctx.events` after every + /// batch, from the producer thread. pub fn new( reader: S, extents: Vec, batch_sectors: u16, - halt: Option, + ctx: &Ctx, ) -> Result where S: SectorSource + Send + 'static, { - Self::new_with_events(reader, extents, batch_sectors, SECTOR_ALIGNMENT, halt, None) + Self::with_alignment(reader, extents, batch_sectors, SECTOR_ALIGNMENT, ctx) } - /// Same as [`Self::new`] but with a callback fired from the producer - /// thread after each successful batch — used by autorip's mux - /// path to surface `BytesRead` progress to the UI without the - /// consumer thread having to poll. - pub fn new_with_events( - mut reader: S, + /// [`Self::new`] reading in whole `unit_align`-sector units (1 for CSS/clear). + pub(crate) fn with_alignment( + reader: S, extents: Vec, batch_sectors: u16, unit_align: u16, - halt: Option, - event_fn: Option, + ctx: &Ctx, ) -> Result where S: SectorSource + Send + 'static, { - // A zero batch loops forever (sectors = remaining.min(0) = 0, offset - // never advances). All production callers pass nonzero; a 0 here is - // a caller bug, so reject rather than spin a thread with no progress. - if batch_sectors == 0 { - return Err(crate::error::Error::MuxBatchSectorsZero); - } - // A zero alignment is worse: the producer hits `remaining % unit_align` - // and panics divide-by-zero, surfacing as a misleading DemuxThreadPanicked - // from a constructor that returned `Ok`. Reject it here too. - if unit_align == 0 { - return Err(crate::error::Error::IoError { - source: std::io::Error::from(std::io::ErrorKind::InvalidInput), - }); - } - // Accumulate in u64 then clamp: extents come from untrusted nav/MPLS/UDF - // data, so a naive u32 sum could panic/wrap. Only affects the advisory - // capacity_sectors figure; the producer walks each extent independently. - let total_sectors: u32 = extents - .iter() - .map(|e| e.sector_count as u64) - .sum::() - .min(u32::MAX as u64) as u32; - let bytes_total_extents: u64 = extents - .iter() - .map(|e| e.sector_count as u64 * crate::consts::SECTOR_BYTES_U64) - .sum(); + let policy = crate::sector::read_stage::ReadPolicy::Image { + batch: batch_sectors, + }; + Self::with_policy(reader, extents, policy, unit_align, ctx).map(|(s, _)| s) + } + + /// The producer running the Read stage over `extents` under `policy` (an image's fixed + /// batches, or a live drive's adaptive, recovering reads), in `unit_align`-sector + /// units. Also returns the read loss the stage counts. + pub(crate) fn with_policy( + reader: S, + extents: Vec, + policy: crate::sector::read_stage::ReadPolicy, + unit_align: u16, + ctx: &Ctx, + ) -> Result<(Self, std::sync::Arc)> + where + S: SectorSource + Send + 'static, + { + Self::spawn(reader, extents, policy, unit_align, ctx, None) + } + + /// The producer over a file's byte view (`m2ts://`): the same Read stage under `policy` + /// in single-sector units, its chunks preceded by `view.prefix` and clipped to `view.len`. + pub(crate) fn file_bytes( + reader: S, + extents: Vec, + policy: crate::sector::read_stage::ReadPolicy, + ctx: &Ctx, + view: ByteView, + ) -> Result<(Self, std::sync::Arc)> + where + S: SectorSource + Send + 'static, + { + Self::spawn(reader, extents, policy, 1, ctx, Some(view)) + } + + fn spawn( + mut reader: S, + extents: Vec, + policy: crate::sector::read_stage::ReadPolicy, + unit_align: u16, + ctx: &Ctx, + view: Option, + ) -> Result<(Self, std::sync::Arc)> + where + S: SectorSource + Send + 'static, + { + let mut walk = + crate::sector::read_stage::ExtentWalk::new(extents, policy, unit_align, ctx)?; + let loss = walk.loss(); + let total_sectors = walk.total_sectors(); + let batch_sectors = match policy { + crate::sector::read_stage::ReadPolicy::Image { batch } + | crate::sector::read_stage::ReadPolicy::Live { batch, .. } => batch, + }; let unmapped = reader.unmapped_stream_files().to_vec(); let (tx, rx) = bounded::(PREFETCH_CHANNEL_DEPTH); let (recycle_tx, recycle_rx) = bounded::>(PREFETCH_CHANNEL_DEPTH + 1); let batch_bytes = batch_sectors as usize * crate::consts::SECTOR_BYTES; - // A never-cancelled stand-in keeps one halt-aware code path without a token. - let wait = halt.clone().unwrap_or_default(); + let wait = ctx.halt.clone(); // Seed the recycle pool so the producer has a buffer on the first // iteration; otherwise the first recycle_rx.recv() blocks forever. @@ -149,50 +195,27 @@ impl PrefetchedSectorSource { #[cfg(test)] assert!( - crate::halt::DRIVE_HOLDER_TEST_LOCK.try_lock().is_err(), + view.is_some() || crate::halt::DRIVE_HOLDER_TEST_LOCK.try_lock().is_err(), "a test that spawns a prefetcher (a Drive holder) must hold DRIVE_HOLDER_TEST_LOCK" ); - let producer = crate::halt::spawn_drive_holder("prefetch", move || { - // catch_unwind so a panic (decrypt source, read path, event_fn) isn't + let file_view = view.is_some(); + let body = move || { + // A byte view: what is still to go out after the prefix. + let (prefix, mut left) = match view { + Some(v) => (v.prefix, Some(v.len)), + None => (Vec::new(), None), + }; + // catch_unwind so a panic (decrypt source, read path, events) isn't // mistaken for clean EOF: a dropped `tx` alone would finalize a // TRUNCATED mux as success. Locals are thread-local, so this is sound. let body = std::panic::AssertUnwindSafe(|| { - let mut ext_idx = 0usize; - let mut offset: u32 = 0; - let mut bytes_read_total: u64 = 0; - while ext_idx < extents.len() { + if !prefix.is_empty() && !send_or_stop(&wait, &tx, Ok(prefix)) { + return; + } + loop { if wait.is_cancelled() { return; } - let extent = &extents[ext_idx]; - // AACS aligned units anchor at THIS extent's start LBA, so gate - // relative to it, not absolute disc LBA 0. No-op for - // non-decrypting / CSS / None sources. - reader.set_unit_base(extent.start_lba); - let remaining = extent.sector_count.saturating_sub(offset); - if remaining == 0 { - ext_idx += 1; - offset = 0; - continue; - } - // AACS units are SECTOR_ALIGNMENT (3) sectors; decrypt processes only - // full units, so a tail below one unit can't decrypt — error, don't emit - // encrypted bytes. (`remaining > 0` above, so `< align` IS the short tail; `!is_multiple_of` was redundant.) - if remaining < unit_align as u32 { - let e = crate::error::Error::ExtentNotUnitAligned.into(); - send_or_stop(&wait, &tx, Err(e)); - return; - } - let mut sectors = remaining.min(batch_sectors as u32) as u16; - // Trim to a whole number of units. A trim to 0 here means the - // batch window landed on a sub-unit boundary (not the - // trailing-tail case, rejected above); clamp to one unit. - if sectors >= unit_align { - sectors -= sectors % unit_align; - } else { - sectors = unit_align; - } - let bytes = sectors as usize * crate::consts::SECTOR_BYTES; // Halt-aware: a cancel does not disconnect the channel, so a // plain recv() would never re-reach the check. Disconnected = // the consumer dropped both channels. @@ -200,55 +223,29 @@ impl PrefetchedSectorSource { else { return; }; - // Sound resize (was `unsafe set_len` guarded only by capacity): - // public `into_channels` lets a caller recycle a cap-only Vec, so - // set_len could expose uninit memory (UB) — GHSA-j8ww-f5fg-9pmh. - buf.resize(bytes, 0); - // `start_lba + offset` derives from untrusted extent - // data — saturate rather than wrap/panic on a - // hostile start_lba near u32::MAX. - let lba = extent.start_lba.saturating_add(offset); - match reader.read_sectors(lba, sectors, &mut buf[..bytes], false) { - Ok(n) => { - // A short read must not desync the stream: advance by - // sectors actually read, and reject a non-whole-sector - // count (belt-and-braces; FileSectorSource read_exact's). - if n % crate::consts::SECTOR_BYTES != 0 { - let e = crate::error::Error::ExtentNotUnitAligned.into(); - send_or_stop(&wait, &tx, Err(e)); - return; + match walk.next(&mut reader, &mut buf) { + Ok(true) => { + // A byte view ends at the file's real length. + let last = left.is_some_and(|l| buf.len() as u64 >= l); + if let Some(l) = left.as_mut() { + buf.truncate((*l).min(buf.len() as u64) as usize); + *l -= buf.len() as u64; } - let sectors_read = (n / crate::consts::SECTOR_BYTES) as u32; - // A zero-byte read isn't EOF (extents still have - // `remaining`) and would spin forever; send a terminal - // sentinel instead of a clean EOF that reports success. - if sectors_read == 0 { - let e = crate::error::Error::SourceTerminated.into(); - send_or_stop(&wait, &tx, Err(e)); - return; - } - buf.truncate(n); - bytes_read_total = bytes_read_total.saturating_add(n as u64); - if let Some(ref f) = event_fn { - f(Event { - kind: EventKind::BytesRead { - bytes: bytes_read_total, - total: bytes_total_extents, - }, - }); - } - if !send_or_stop(&wait, &tx, Ok(buf)) { + if !buf.is_empty() && !send_or_stop(&wait, &tx, Ok(buf)) { return; // consumer dropped, or stopped } - offset = offset.saturating_add(sectors_read); + if last { + return; + } } + // Drop tx — the consumer sees RecvError → EOF. + Ok(false) => return, Err(e) => { send_or_stop(&wait, &tx, Err(e.into())); return; } } } - // Drop tx implicitly — consumer sees RecvError → EOF. }); if std::panic::catch_unwind(body).is_err() { // Panicked mid-stream — surface a typed error so the demux @@ -257,18 +254,28 @@ impl PrefetchedSectorSource { let e = crate::error::Error::DemuxThreadPanicked.into(); send_or_stop(&wait, &tx, Err(e)); } - }) + }; + let producer = match file_view { + false => crate::halt::spawn_drive_holder("prefetch", body).map(Producer::Drive), + true => std::thread::Builder::new() + .name("freemkv-prefetch".into()) + .spawn(body) + .map(Producer::File), + } .map_err(|e| crate::error::Error::IoError { source: e })?; - Ok(Self { - rx, - recycle_tx, - producer: Some(producer), - total_sectors, - producer_failed: false, - unmapped, - halt, - }) + Ok(( + Self { + rx, + recycle_tx, + producer: Some(producer), + total_sectors, + producer_failed: false, + unmapped, + halt: ctx.halt.clone(), + }, + loss, + )) } /// Peel off the receivers for zero-copy pipeline mode: the caller pulls buffers from `rx` @@ -295,7 +302,7 @@ impl PrefetchedSectorSource { /// producer thread join handle so dropping the shell joins the /// producer, even though the channels have been peeled off. pub struct PrefetchShell { - producer: Option>, + producer: Option, } // Every test that spawns a producer holds this (the halt module's Drive-holder test lock). @@ -309,7 +316,7 @@ pub(crate) fn holder_test_lock() -> std::sync::MutexGuard<'static, ()> { impl Drop for PrefetchShell { fn drop(&mut self) { if let Some(h) = self.producer.take() { - let _ = h.join(); + h.join(); } } } @@ -329,7 +336,7 @@ impl Drop for PrefetchedSectorSource { // joining gives a deterministic shutdown — no detached thread can outlive // the source. if let Some(h) = self.producer.take() { - let _ = h.join(); + h.join(); } } } @@ -392,9 +399,7 @@ impl SectorSource for PrefetchedSectorSource { } // Channel closed. A stop first (L096, LP17): a halted producer returns // silently, and `Ok(0)` would read as a short, complete source. - Err(_) if self.halt.as_ref().is_some_and(Halt::is_cancelled) => { - Err(crate::error::Error::Halted) - } + Err(_) if self.halt.is_cancelled() => Err(crate::error::Error::Halted), // Clean EOF only if the producer never signalled a failure — else `Ok(0)` // lets fill_extents zero-fill a dead source's rest as "complete". Err(_) if self.producer_failed => Err(crate::error::Error::SourceTerminated), @@ -408,6 +413,7 @@ mod tests { use super::*; use crate::disc::Extent; use crate::error::Result; + use crate::event::Event; use std::sync::mpsc; use std::time::Duration; @@ -556,7 +562,8 @@ mod tests { start_lba: 0, sector_count: 3, }]; - let s = PrefetchedSectorSource::new(Reports(vec![m2ts1()]), ext, 3, None).unwrap(); + let s = + PrefetchedSectorSource::new(Reports(vec![m2ts1()]), ext, 3, &Ctx::default()).unwrap(); assert_eq!(unmapped_paths(&s), ["/BDMV/STREAM/00001.m2ts"]); } @@ -575,7 +582,7 @@ mod tests { PatternSource { capacity: 9999 }, extents, 3, - Some(halt.clone()), + &Ctx::new(halt.clone()), ) .expect("spawn"); // Drop without draining a single batch — the old Drop deadlocked here. @@ -590,8 +597,9 @@ mod tests { fn into_channels_drop_releases_producer() { let _serial = serial(); within(10, || { - let src = PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, None) - .expect("spawn"); + let src = + PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, &Ctx::default()) + .expect("spawn"); let (rx, recycle_tx, shell) = src.into_channels(); // Consumer goes away early (halt / abort analogue): drop both // channel endpoints without draining to EOF. @@ -610,8 +618,9 @@ mod tests { let _serial = serial(); let before = crate::halt::live_drive_holders(); within(10, || { - let src = PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, None) - .expect("spawn"); + let src = + PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, &Ctx::default()) + .expect("spawn"); let (rx, recycle_tx, shell) = src.into_channels(); drop(rx); drop(recycle_tx); @@ -637,7 +646,7 @@ mod tests { sector_count: count, }]; let src = PatternSource { capacity: 1000 }; - let pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let pf = PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); let (rx, recycle_tx, shell) = pf.into_channels(); let mut got = Vec::new(); while let Ok(batch) = rx.recv() { @@ -668,9 +677,13 @@ mod tests { let _serial = serial(); within(10, || { let halt = Halt::new(); - let src = - PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, Some(halt.clone())) - .expect("spawn"); + let src = PrefetchedSectorSource::new( + EndlessZeroSource, + big_extent(), + 3, + &Ctx::new(halt.clone()), + ) + .expect("spawn"); let (rx, recycle_tx, shell) = src.into_channels(); // Drain the forward channel so the producer's sends always // make progress and it can reach the halt check at the loop @@ -714,9 +727,13 @@ mod tests { let _serial = serial(); let before = crate::halt::live_drive_holders(); let halt = Halt::new(); - let pf = - PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, Some(halt.clone())) - .expect("spawn"); + let pf = PrefetchedSectorSource::new( + EndlessZeroSource, + big_extent(), + 3, + &Ctx::new(halt.clone()), + ) + .expect("spawn"); assert_eq!( crate::halt::live_drive_holders(), before + 1, @@ -744,17 +761,25 @@ mod tests { fn sector_prefetcher_drop_after_cancel_returns() { let _serial = serial(); let halt = Halt::new(); - let pf = - PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, Some(halt.clone())) - .expect("spawn"); + let pf = PrefetchedSectorSource::new( + EndlessZeroSource, + big_extent(), + 3, + &Ctx::new(halt.clone()), + ) + .expect("spawn"); std::thread::sleep(Duration::from_millis(50)); halt.cancel(); within(1, move || drop(pf)); let halt = Halt::new(); - let pf = - PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, Some(halt.clone())) - .expect("spawn"); + let pf = PrefetchedSectorSource::new( + EndlessZeroSource, + big_extent(), + 3, + &Ctx::new(halt.clone()), + ) + .expect("spawn"); let (rx, recycle_tx, shell) = pf.into_channels(); std::thread::sleep(Duration::from_millis(50)); halt.cancel(); @@ -770,9 +795,13 @@ mod tests { let _serial = serial(); with_watchdog(Duration::from_secs(10), || { let halt = Halt::new(); - let mut pf = - PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 3, Some(halt.clone())) - .expect("spawn"); + let mut pf = PrefetchedSectorSource::new( + EndlessZeroSource, + big_extent(), + 3, + &Ctx::new(halt.clone()), + ) + .expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; assert_eq!(pf.read_sectors(0, 3, &mut buf, false).unwrap(), 3 * 2048); halt.cancel(); @@ -797,8 +826,8 @@ mod tests { sector_count: 6, }]; let halt = Halt::new(); - let mut pf = - PrefetchedSectorSource::new(EndlessZeroSource, ext, 3, Some(halt)).expect("spawn"); + let mut pf = PrefetchedSectorSource::new(EndlessZeroSource, ext, 3, &Ctx::new(halt)) + .expect("spawn"); let (_, last) = drain_direct(&mut pf, 3, 8); assert_eq!(last.unwrap(), 0); }); @@ -809,7 +838,7 @@ mod tests { #[test] fn zero_batch_rejected() { let _serial = serial(); - let res = PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 0, None); + let res = PrefetchedSectorSource::new(EndlessZeroSource, big_extent(), 0, &Ctx::default()); assert_eq!(res.err().map(|e| e.code()), Some(9085)); } @@ -823,9 +852,13 @@ mod tests { start_lba: 0, sector_count: 6, }]; - let mut pf = - PrefetchedSectorSource::new(PatternSource { capacity: 6 }, ext, batch, None) - .expect("spawn"); + let mut pf = PrefetchedSectorSource::new( + PatternSource { capacity: 6 }, + ext, + batch, + &Ctx::default(), + ) + .expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; for _ in 0..2 { let n = pf.read_sectors(0, 3, &mut buf, false).unwrap(); @@ -845,7 +878,8 @@ mod tests { start_lba: 0, sector_count: 6, }]; - let mut pf = PrefetchedSectorSource::new(EndlessZeroSource, ext, 3, None).unwrap(); + let mut pf = + PrefetchedSectorSource::new(EndlessZeroSource, ext, 3, &Ctx::default()).unwrap(); assert!(!pf.random_access()); let by_ref: &mut dyn SectorSource = &mut pf; assert!(!SectorSource::random_access(&by_ref)); @@ -856,13 +890,12 @@ mod tests { #[test] fn zero_unit_align_rejected() { let _serial = serial(); - let res = PrefetchedSectorSource::new_with_events( + let res = PrefetchedSectorSource::with_alignment( EndlessZeroSource, big_extent(), 4096, 0, - None, - None, + &Ctx::default(), ); let Err(crate::error::Error::IoError { source }) = res else { panic!("zero unit_align must be rejected with InvalidInput"); @@ -884,7 +917,8 @@ mod tests { sector_count: 24, }]; let src = PatternSource { capacity: 24 }; - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; let mut total = 0usize; @@ -899,11 +933,11 @@ mod tests { }); } - // The producer-thread `event_fn` must fire a `BytesRead` event per batch, cumulative and + // The producer thread must emit a `BytesRead` event per batch, cumulative and // non-decreasing, reaching the full extent size at EOF — the contract autorip's progress // bar + stall watchdog depend on. #[test] - fn event_fn_fires_bytes_read_per_batch() { + fn producer_emits_bytes_read_per_batch() { let _serial = serial(); with_watchdog(Duration::from_secs(10), || { // Two 12-sector extents = 8 aligned units; batch of 3 gives 8 batches. @@ -921,21 +955,13 @@ mod tests { let seen = Arc::new(Mutex::new(Vec::<(u64, u64)>::new())); let seen_cb = seen.clone(); - let event_fn: EventFn = Box::new(move |ev: Event| { - if let EventKind::BytesRead { bytes, total } = ev.kind { + let ctx = Ctx::default().with_events(Arc::new(move |ev: &Event<'_>| { + if let Event::BytesRead { bytes, total } = *ev { seen_cb.lock().unwrap().push((bytes, total)); } - }); + })); - let mut pf = PrefetchedSectorSource::new_with_events( - src, - extents, - 3, - SECTOR_ALIGNMENT, - None, - Some(event_fn), - ) - .expect("spawn"); + let mut pf = PrefetchedSectorSource::new(src, extents, 3, &ctx).expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; let mut total = 0usize; @@ -968,7 +994,8 @@ mod tests { sector_count: 8, }]; let src = PatternSource { capacity: 200 }; - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; // First two reads: the 6 unit-aligned sectors come through @@ -1007,7 +1034,8 @@ mod tests { capacity: 9, first: true, }; - let mut pf = PrefetchedSectorSource::new(src, extents, 9, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 9, &Ctx::default()).expect("spawn"); let mut buf = vec![0u8; 9 * 2048]; let mut total = 0usize; @@ -1147,7 +1175,7 @@ mod tests { }, ]; let src = PatternSource { capacity: 9999 }; - let pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let pf = PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); // 9 + 6 + 3 = 18, independent of inner source capacity. assert_eq!(pf.capacity_sectors(), 18); // Release without draining via the production zero-copy path: peel @@ -1180,8 +1208,8 @@ mod tests { // batch=3 so the producer makes forward progress on the // EndlessZeroSource; we only care about the construction-time // capacity computation here, then we drop to join. - let pf = - PrefetchedSectorSource::new(EndlessZeroSource, extents, 3, None).expect("spawn"); + let pf = PrefetchedSectorSource::new(EndlessZeroSource, extents, 3, &Ctx::default()) + .expect("spawn"); assert_eq!( pf.capacity_sectors(), u32::MAX, @@ -1219,7 +1247,8 @@ mod tests { capacity: 99999, calls: calls.clone(), }; - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); let (got, last) = drain_direct(&mut pf, 3, 16); assert_eq!(last.unwrap(), 0, "should reach EOF"); assert_eq!(got.len(), (6 + 3) * 2048); @@ -1253,7 +1282,8 @@ mod tests { }; // batch=5: each read must be trimmed to 3 (one unit), so // 9 sectors take three reads of 3, never a 5/4-sector read. - let mut pf = PrefetchedSectorSource::new(src, extents, 5, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 5, &Ctx::default()).expect("spawn"); let (got, last) = drain_direct(&mut pf, 5, 16); assert_eq!(last.unwrap(), 0); assert_eq!(got.len(), 9 * 2048); @@ -1283,7 +1313,8 @@ mod tests { sector_count: 12, }]; let src = PatternSource { capacity: 100 }; - let mut pf = PrefetchedSectorSource::new(src, extents, 6, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 6, &Ctx::default()).expect("spawn"); let (got, last) = drain_direct(&mut pf, 6, 16); assert_eq!( last.unwrap(), @@ -1304,7 +1335,8 @@ mod tests { start_lba: 0, sector_count: 3, }]; - let mut pf = PrefetchedSectorSource::new(ErrorSource, extents, 3, None).expect("spawn"); + let mut pf = PrefetchedSectorSource::new(ErrorSource, extents, 3, &Ctx::default()) + .expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; let r = pf.read_sectors(0, 3, &mut buf, false); let err = r.expect_err("reader error must surface as Err, not EOF"); @@ -1344,8 +1376,8 @@ mod tests { start_lba: 0, sector_count: 9, }]; - let mut pf = - PrefetchedSectorSource::new(QuitsEarlySource, extents, 3, None).expect("spawn"); + let mut pf = PrefetchedSectorSource::new(QuitsEarlySource, extents, 3, &Ctx::default()) + .expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; let mut last = pf.read_sectors(0, 3, &mut buf, false); // Whatever the first answer, no call may ever settle on a clean @@ -1383,7 +1415,8 @@ mod tests { sector_count: 9, }]; let mut pf = - PrefetchedSectorSource::new(PartialSectorSource, extents, 3, None).expect("spawn"); + PrefetchedSectorSource::new(PartialSectorSource, extents, 3, &Ctx::default()) + .expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; let r = pf.read_sectors(0, 3, &mut buf, false); let err = r.expect_err("split-sector read must be rejected"); @@ -1409,7 +1442,8 @@ mod tests { }]; let src = PatternSource { capacity: 6 }; // batch=3 → producer fills 3 sectors (6144 bytes) per batch. - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); // Caller buffer holds only 1 sector — far too small. let mut tiny = vec![0u8; 2048]; let r = pf.read_sectors(0, 1, &mut tiny, false); @@ -1436,7 +1470,8 @@ mod tests { let src = PatternSource { capacity: 30 }; // batch=3 → producer fills 3 sectors (6144 bytes) per batch. // Pool depth is PREFETCH_CHANNEL_DEPTH+1 = 3. - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); // Caller buffer holds only 1 sector — far too small for a 3-sector batch. let mut tiny = vec![0u8; 2048]; @@ -1467,7 +1502,8 @@ mod tests { sector_count: count, }]; let src = PatternSource { capacity: 1000 }; - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); let (got, last) = drain_direct(&mut pf, 3, 16); assert_eq!(last.unwrap(), 0); assert_eq!(got.len(), (count as usize) * 2048); @@ -1491,8 +1527,8 @@ mod tests { fn empty_extents_eof_immediately() { let _serial = serial(); with_watchdog(Duration::from_secs(10), || { - let pf = - PrefetchedSectorSource::new(EndlessZeroSource, Vec::new(), 3, None).expect("spawn"); + let pf = PrefetchedSectorSource::new(EndlessZeroSource, Vec::new(), 3, &Ctx::default()) + .expect("spawn"); assert_eq!(pf.capacity_sectors(), 0); let mut pf = pf; let mut buf = vec![0u8; 3 * 2048]; @@ -1527,7 +1563,8 @@ mod tests { capacity: 9999, calls: calls.clone(), }; - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); let (got, last) = drain_direct(&mut pf, 3, 16); assert_eq!(last.unwrap(), 0); assert_eq!(got.len(), 6 * 2048, "two non-empty extents = 6 sectors"); @@ -1555,7 +1592,8 @@ mod tests { }]; let src = PatternSource { capacity: 100 }; // batch=9 (>4) so the first iter requests 4, trims to 3. - let mut pf = PrefetchedSectorSource::new(src, extents, 9, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 9, &Ctx::default()).expect("spawn"); let mut buf = vec![0u8; 9 * 2048]; let n0 = pf.read_sectors(0, 9, &mut buf, false).unwrap(); assert_eq!(n0, 3 * 2048, "first batch must be exactly one unit"); @@ -1582,7 +1620,8 @@ mod tests { }) .collect(); let src = PatternSource { capacity: 999999 }; - let mut pf = PrefetchedSectorSource::new(src, extents, 3, None).expect("spawn"); + let mut pf = + PrefetchedSectorSource::new(src, extents, 3, &Ctx::default()).expect("spawn"); let (got, last) = drain_direct(&mut pf, 3, 64); assert_eq!(last.unwrap(), 0); assert_eq!(got.len(), 30 * 2048, "all 10 extents must be drained"); @@ -1622,8 +1661,13 @@ mod tests { start_lba: 100, sector_count: 9, }]; - let mut pf = PrefetchedSectorSource::new(FailingSource { status, sense }, extents, 3, None) - .expect("spawn"); + let mut pf = PrefetchedSectorSource::new( + FailingSource { status, sense }, + extents, + 3, + &Ctx::default(), + ) + .expect("spawn"); let mut buf = vec![0u8; 3 * 2048]; pf.read_sectors(100, 3, &mut buf, false) .expect_err("the producer's read failure must surface") diff --git a/src/sector/read_stage.rs b/src/sector/read_stage.rs new file mode 100644 index 00000000..197797c3 --- /dev/null +++ b/src/sector/read_stage.rs @@ -0,0 +1,466 @@ +//! The Read stage (pipeline design §2.2, slice 6): walks a title's extents over a +//! (decrypting) sector source and yields chunks, under one [`ReadPolicy`]. The prefetch +//! producer runs it for every sector input — a live drive and an image alike — so there is +//! one read loop and one demux behind it. +//! +//! The policy is the transport's (X-1): an image reads fixed batches and stops on the first +//! error; a live drive shrinks its batch on a failing zone, regrows it after a clean streak, +//! gives a unit that still fails one full ECC recovery read, then zero-fills and counts it +//! (`skip_errors`) or fails the read. + +use crate::ctx::Ctx; +use crate::disc::Extent; +use crate::drive::extract_scsi_context; +use crate::error::{Error, Result}; +use crate::event::{BatchSizeReason, Event}; +use crate::sector::SectorSource; +use std::sync::Arc; +use std::sync::atomic::{AtomicU64, Ordering}; + +// Ramp back to the preferred batch size after this many clean sectors (100 +// MiB = 51,200 sectors) — long enough that noisy zones can't trigger a +// premature probe, short enough an isolated failure doesn't lock size 1. +const PROBE_THRESHOLD_SECTORS: u32 = 100 * 1024 * 1024 / 2048; + +const SECTOR: usize = crate::consts::SECTOR_BYTES; + +/// How the Read stage reads a source (the transport's policy, X-1). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ReadPolicy { + /// An image or file: `batch`-sector reads; any read error ends the run, and an extent + /// tail shorter than one decrypt unit is refused. + Image { batch: u16 }, + /// A live drive: adaptive batches from `batch` down, one ECC recovery read for a unit + /// that still fails, then a zero-filled, counted unit (`skip_errors`) or a `DiscRead`. + Live { batch: u16, skip_errors: bool }, +} + +impl ReadPolicy { + /// The preferred read batch, in sectors. + pub fn batch(&self) -> u16 { + match *self { + ReadPolicy::Image { batch } | ReadPolicy::Live { batch, .. } => batch, + } + } +} + +/// Read loss the stage counted (skip events and zero-filled bytes), shared with the PES +/// stream that reports it as `errors()` / `lost_bytes()`. +#[derive(Debug, Default)] +pub struct ReadLoss { + skips: AtomicU64, + bytes: AtomicU64, +} + +impl ReadLoss { + /// Read errors skipped past. + pub fn skips(&self) -> u64 { + self.skips.load(Ordering::Relaxed) + } + + /// Bytes zero-filled in their place. + pub fn bytes(&self) -> u64 { + self.bytes.load(Ordering::Relaxed) + } + + fn add(&self, bytes: u64) { + self.skips.fetch_add(1, Ordering::Relaxed); + self.bytes.fetch_add(bytes, Ordering::Relaxed); + } +} + +/// Halve a batch size, keeping 3-sector alignment when >= 6 +/// (3-sector alignment = one AACS unit). At sizes < 6 we descend +/// through 3 → 1 without intermediate unaligned sizes. +fn halve_batch_size(size: u16) -> u16 { + let h = (size / 2).max(1); + if h >= 6 { h - (h % 3) } else { h } +} + +/// Double a batch size toward a preferred max, keeping 3-sector alignment +/// when the result is >= 6. +fn double_batch_size(size: u16, preferred: u16) -> u16 { + let d = size.saturating_mul(2).min(preferred); + if d >= 6 { d - (d % 3) } else { d } +} + +/// Adaptive batch sizer. Shrinks on read failure, grows after a sustained +/// clean streak. Amortizes the cost of entering a bad zone — descent happens +/// once, not once per bad sector. +#[derive(Debug)] +struct AdaptiveBatch { + preferred: u16, + current: u16, + streak_sectors: u32, +} + +impl AdaptiveBatch { + fn new(preferred: u16) -> Self { + Self { + preferred, + current: preferred, + streak_sectors: 0, + } + } + + fn current(&self) -> u16 { + self.current + } + + /// Record a successful read of `sectors`. Returns an event if the + /// sizer probed up to a larger batch size. + fn on_success(&mut self, sectors: u16) -> Option> { + self.streak_sectors = self.streak_sectors.saturating_add(sectors as u32); + if self.current < self.preferred && self.streak_sectors >= PROBE_THRESHOLD_SECTORS { + let new_size = double_batch_size(self.current, self.preferred); + if new_size != self.current { + self.current = new_size; + self.streak_sectors = 0; + return Some(Event::BatchSizeChanged { + new_size, + reason: BatchSizeReason::Probed, + }); + } + } + None + } + + /// Record a read failure. Returns an event if the sizer shrank. + /// Does nothing at size 1 (caller handles skip/error). + fn on_failure(&mut self) -> Option> { + self.streak_sectors = 0; + if self.current <= 1 { + return None; + } + let new_size = halve_batch_size(self.current); + self.current = new_size; + Some(Event::BatchSizeChanged { + new_size, + reason: BatchSizeReason::Shrunk, + }) + } +} + +// The key set's on-arrival loud stop: E7022 (a title) or E7032 (an image or folder). +fn is_key_stop(e: &Error) -> bool { + matches!(e, Error::NoDiscKey { .. } | Error::WholeDiscKeyMissing) +} + +/// One title's extent walk under a [`ReadPolicy`]: each [`next`](Self::next) fills a buffer +/// with the next chunk (today's read batch for the source kind). +pub struct ExtentWalk { + extents: Vec, + idx: usize, + offset: u32, + unit_align: u32, + policy: ReadPolicy, + adaptive: AdaptiveBatch, + bytes_read: u64, + bytes_total: u64, + loss: Arc, + ctx: Ctx, +} + +impl ExtentWalk { + /// Walk `extents` in `unit_align`-sector decrypt units (3 for AACS, 1 for CSS/clear). + /// A zero batch or alignment is refused (it would never advance). + pub fn new( + extents: Vec, + policy: ReadPolicy, + unit_align: u16, + ctx: &Ctx, + ) -> Result { + if policy.batch() == 0 { + return Err(Error::MuxBatchSectorsZero); + } + if unit_align == 0 { + return Err(Error::IoError { + source: std::io::Error::from(std::io::ErrorKind::InvalidInput), + }); + } + // Extents come from untrusted nav/MPLS/UDF data: sum in u64. + let bytes_total = extents + .iter() + .map(|e| e.sector_count as u64 * crate::consts::SECTOR_BYTES_U64) + .sum(); + Ok(ExtentWalk { + extents, + idx: 0, + offset: 0, + unit_align: unit_align as u32, + policy, + adaptive: AdaptiveBatch::new(policy.batch()), + bytes_read: 0, + bytes_total, + loss: Arc::default(), + ctx: ctx.clone(), + }) + } + + /// The loss this walk counts, for the stream that reports it. + pub fn loss(&self) -> Arc { + self.loss.clone() + } + + /// Total sectors across the extents (advisory; clamped to `u32`). + pub fn total_sectors(&self) -> u32 { + self.extents + .iter() + .map(|e| e.sector_count as u64) + .sum::() + .min(u32::MAX as u64) as u32 + } + + // The next extent with sectors left, as `(start, sectors, remaining)`; `None` at the end. + // Iterative, not recursive: a malformed disc can declare thousands of empty extents. + fn current(&mut self) -> Option<(u32, u32, u32)> { + loop { + let ext = self.extents.get(self.idx)?; + let remaining = ext.sector_count.saturating_sub(self.offset); + if remaining > 0 { + return Some((ext.start_lba, ext.sector_count, remaining)); + } + self.idx += 1; + self.offset = 0; + } + } + + fn advance(&mut self, sectors: u32, ext_sectors: u32) { + self.offset = self.offset.saturating_add(sectors); + if self.offset >= ext_sectors { + self.idx += 1; + self.offset = 0; + } + } + + /// Fill `buf` with the next chunk. `Ok(false)` once every extent is read. A Stop is + /// `Err(Halted)`, never a skipped sector. + pub fn next(&mut self, reader: &mut dyn SectorSource, buf: &mut Vec) -> Result { + let Some((start, ext_sectors, remaining)) = self.current() else { + return Ok(false); + }; + // AACS aligned units anchor at this extent's start LBA, not absolute disc LBA 0. + // No-op for CSS / clear sources. + reader.set_unit_base(start); + // `start + offset` derives from untrusted extent data: saturate. + let lba = start.saturating_add(self.offset); + match self.policy { + ReadPolicy::Image { batch } => { + self.next_image(reader, buf, lba, remaining, ext_sectors, batch) + } + ReadPolicy::Live { skip_errors, .. } => { + self.next_live(reader, buf, lba, remaining, ext_sectors, skip_errors) + } + } + } + + fn read_progress(&mut self, got: usize) { + self.bytes_read = self.bytes_read.saturating_add(got as u64); + self.ctx.emit(Event::BytesRead { + bytes: self.bytes_read, + total: self.bytes_total, + }); + } + + fn next_image( + &mut self, + reader: &mut dyn SectorSource, + buf: &mut Vec, + lba: u32, + remaining: u32, + ext_sectors: u32, + batch: u16, + ) -> Result { + self.ctx.halt.check()?; + let align = self.unit_align; + // Decrypt processes only full units, so a tail below one unit can't decrypt: + // refuse it rather than emit encrypted bytes. + if remaining < align { + return Err(Error::ExtentNotUnitAligned); + } + let mut sectors = remaining.min(batch as u32); + // Trim to whole units; a window landing on a sub-unit boundary reads one unit. + sectors = if sectors >= align { + sectors - sectors % align + } else { + align + }; + let bytes = sectors as usize * SECTOR; + buf.resize(bytes, 0); + let n = reader.read_sectors(lba, sectors as u16, &mut buf[..bytes], false)?; + // A short read must not desync the stream: advance by sectors actually read, and + // reject a non-whole-sector count. + if n % SECTOR != 0 { + return Err(Error::ExtentNotUnitAligned); + } + let read = (n / SECTOR) as u32; + // A zero-byte read with extents left is not EOF and would spin forever. + if read == 0 { + return Err(Error::SourceTerminated); + } + buf.truncate(n); + self.read_progress(n); + self.advance(read, ext_sectors); + Ok(true) + } + + // A failed read that Stop caused: the reader reports Halted, or the token fired. + fn stopped(&self, e: &Error) -> bool { + matches!(e, Error::Halted) || self.ctx.halt.is_cancelled() + } + + // The read error to report for a unit that failed at `lba`. + fn disc_read(lba: u32, e: Option<&Error>) -> Error { + let (status, sense) = e.map(extract_scsi_context).unwrap_or((0, None)); + Error::DiscRead { + sector: lba as u64, + status: Some(status), + sense, + } + } + + // A read failure no policy may skip or retry past (`Err`): Stop, the key set's loud + // stop, a wedged transport, a dead source, or an image that ends before the read. + // `Ok` hands a soft failure back. + fn soft(&self, lba: u32, e: Error) -> std::result::Result { + if is_key_stop(&e) || matches!(e, Error::ImageEndsBeforeRead { .. }) { + return Err(e); + } + if self.stopped(&e) { + return Err(Error::Halted); + } + // A transport failure (status 0xFF: bridge crash/disconnect) is NOT a skippable bad + // sector: every later read fails too, so skipping marches the disc producing nothing. + if e.is_scsi_transport_failure() { + return Err(Self::disc_read(lba, Some(&e))); + } + // The read SOURCE is gone, not bad media: skipping would zero-fill the rest. + if e.is_source_terminated() { + return Err(Error::SourceTerminated); + } + Ok(e) + } + + // Commit a read of `got` of `bytes`: a short read is a `DiscRead`, or zero-filled and + // counted under `skip_errors` (never a silent hole). + fn commit( + &mut self, + buf: &mut [u8], + lba: u32, + got: usize, + bytes: usize, + skip_errors: bool, + ) -> Result<()> { + if got < bytes { + if !skip_errors { + return Err(Error::DiscRead { + sector: lba as u64, + status: None, + sense: None, + }); + } + // Clear the tail the source did not write: it holds stale bytes from a + // previous batch, which would mux as plausible garbage. + buf[got..bytes].fill(0); + self.skip(lba, (bytes - got) as u64); + } + // Only the bytes the source delivered count as read; the zero-filled tail is loss. + self.read_progress(got); + Ok(()) + } + + fn skip(&mut self, lba: u32, bytes: u64) { + self.loss.add(bytes); + self.ctx.stats.add_skip(bytes); + self.ctx.emit(Event::SectorSkipped { lba: lba as u64 }); + } + + fn next_live( + &mut self, + reader: &mut dyn SectorSource, + buf: &mut Vec, + lba: u32, + remaining: u32, + ext_sectors: u32, + skip_errors: bool, + ) -> Result { + let align = self.unit_align; + // Shrink on failure, grow on success; one attempt per size, no retry loops. Stop + // is checked every iteration: a bad zone can spend minutes here. + loop { + if self.ctx.halt.is_cancelled() { + return Err(Error::Halted); + } + // Every read buffer starts on a real on-disc unit boundary: AACS decrypts whole + // units keyed off the buffer's first bytes. + let want = remaining.min(self.adaptive.current() as u32); + let sectors: u16 = if align <= 1 { + want as u16 + } else if remaining < align { + remaining as u16 + } else if want < align { + align as u16 + } else { + (want - want % align) as u16 + }; + let bytes = sectors as usize * SECTOR; + buf.resize(bytes, 0); + let res = reader.read_sectors(lba, sectors, &mut buf[..bytes], false); + match res { + Ok(got) => { + debug_assert!(got <= bytes, "read_sectors over-reported byte count"); + if let Some(ev) = self.adaptive.on_success(sectors) { + self.ctx.emit(ev); + } + self.commit(buf, lba, got.min(bytes), bytes, skip_errors)?; + self.advance(sectors as u32, ext_sectors); + return Ok(true); + } + Err(e) => { + self.soft(lba, e)?; + if (sectors as u32) > align { + // Shrink and retry at the same LBA with a smaller batch. + if let Some(ev) = self.adaptive.on_failure() { + self.ctx.emit(ev); + } + continue; + } + // Bottomed out with no later pass to recover: one full ECC recovery + // read (~60 s) before skip/fail — a single bounded read, never a loop. + tracing::debug!( + target: "mux", + "last-chance recovery read at LBA {lba} ({sectors} sectors, 60s ECC)" + ); + let rec = reader.read_sectors(lba, sectors, &mut buf[..bytes], true); + match rec { + Ok(got) => { + debug_assert!(got <= bytes, "recovery read over-reported byte count"); + if let Some(ev) = self.adaptive.on_success(sectors) { + self.ctx.emit(ev); + } + self.commit(buf, lba, got.min(bytes), bytes, skip_errors)?; + self.advance(sectors as u32, ext_sectors); + return Ok(true); + } + Err(rec) => { + let rec = self.soft(lba, rec)?; + if !skip_errors { + return Err(Self::disc_read(lba, Some(&rec))); + } + // Skip the WHOLE failed unit: advancing by the full unit keeps + // the cursor AACS-aligned. + buf[..bytes].fill(0); + self.skip(lba, bytes as u64); + self.advance(sectors as u32, ext_sectors); + return Ok(true); + } + } + } + } + } + } +} + +#[cfg(test)] +#[path = "read_stage_tests.rs"] +mod live_tests; diff --git a/src/sector/read_stage_tests.rs b/src/sector/read_stage_tests.rs new file mode 100644 index 00000000..47e7f5f7 --- /dev/null +++ b/src/sector/read_stage_tests.rs @@ -0,0 +1,1107 @@ +//! The live Read policy, driven one chunk at a time through `LiveRead` (the read loop +//! the live-drive mux runs in its prefetch producer). +use super::*; +use crate::disc::{ContentFormat, DiscTitle}; +use crate::halt::Halt; +use std::io; + +// The live Read stage over a decrypting reader, one chunk per `fill_extents`, with the +// counters the tests read. `skip_errors` applies from the first fill. +struct LiveRead { + walk: Option, + extents: Vec, + batch: u16, + unit_align: u16, + ctx: Ctx, + reader: crate::sector::DecryptingSectorSource>, + skip_errors: bool, + read_buf: Vec, + buf_valid: usize, + current_offset: u32, + errors: u64, + lost_bytes: u64, + bytes_read_total: u64, +} + +impl LiveRead { + fn new( + reader: Box, + title: DiscTitle, + keys: crate::decrypt::DecryptKeys, + batch: u16, + _format: ContentFormat, + _raw: bool, + ctx: &Ctx, + ) -> io::Result { + if batch == 0 { + return Err(Error::MuxBatchSectorsZero.into()); + } + let unit_align = match keys { + crate::decrypt::DecryptKeys::Aacs { .. } => 3, + _ => 1, + }; + Ok(LiveRead { + walk: None, + extents: title.extents.clone(), + batch, + unit_align, + ctx: ctx.clone(), + reader: crate::sector::DecryptingSectorSource::new(reader, keys), + skip_errors: false, + read_buf: Vec::new(), + buf_valid: 0, + current_offset: 0, + errors: 0, + lost_bytes: 0, + bytes_read_total: 0, + }) + } + + fn fill_extents(&mut self) -> io::Result { + if self.walk.is_none() { + let policy = ReadPolicy::Live { + batch: self.batch, + skip_errors: self.skip_errors, + }; + self.walk = Some(ExtentWalk::new( + self.extents.clone(), + policy, + self.unit_align, + &self.ctx, + )?); + } + let w = self.walk.as_mut().expect("built above"); + let r = w.next(&mut self.reader, &mut self.read_buf); + let loss = w.loss(); + self.errors = loss.skips(); + self.lost_bytes = loss.bytes(); + self.bytes_read_total = w.bytes_read; + self.buf_valid = self.read_buf.len(); + self.current_offset = w.offset; + r.map_err(io::Error::from) + } +} + +// Trivial SectorSource yielding zeroed sectors. Empty title -> no PES +// frames -> read() walks to EOF -> Ok(None); enough to exercise the +// trait-object dispatch (the goal is the bridge, not the demuxer). +struct ZeroReader { + capacity: u32, +} + +impl crate::sector::SectorSource for ZeroReader { + fn read_sectors( + &mut self, + _lba: u32, + count: u16, + buf: &mut [u8], + _recovery: bool, + ) -> crate::error::Result { + let bytes = count as usize * 2048; + buf[..bytes].fill(0); + Ok(bytes) + } + + fn capacity_sectors(&self) -> u32 { + self.capacity + } +} + +// A SectorSource that UNDER-reports: writes the whole requested span but +// reports only the first sector valid, modeling a non-full-or-error +// source — the case fill_extents handled inconsistently pre-fix. +struct ShortReader { + capacity: u32, +} + +impl crate::sector::SectorSource for ShortReader { + fn read_sectors( + &mut self, + _lba: u32, + count: u16, + buf: &mut [u8], + _recovery: bool, + ) -> crate::error::Result { + let bytes = count as usize * 2048; + // Stale marker across the whole span; only the first sector is + // reported as actually delivered. + buf[..bytes].fill(0xAB); + Ok(2048usize.min(bytes)) + } + + fn capacity_sectors(&self) -> u32 { + self.capacity + } +} + +fn short_read_stream(skip_errors: bool) -> LiveRead { + short_read_stream_in(skip_errors, &crate::ctx::Ctx::default()) +} + +fn short_read_stream_in(skip_errors: bool, ctx: &crate::ctx::Ctx) -> LiveRead { + let mut s = LiveRead::new( + Box::new(ShortReader { capacity: 64 }), + synthetic_title(64), + crate::decrypt::DecryptKeys::None, + 8, // request 8 sectors (16384 B); the source delivers 1 (2048 B) + ContentFormat::BdTs, + false, + ctx, + ) + .unwrap(); + s.skip_errors = skip_errors; + s +} + +// A short read must never become a silent gap: advancing current_offset +// by the full requested count would drop the undelivered tail with no +// error. Without skip_errors it's a hard read error: E6000. +#[test] +fn short_read_without_skip_errors_is_reported_not_silently_skipped() { + let mut s = short_read_stream(false); + let err = s + .fill_extents() + .expect_err("a short read must not be reported as a clean fill"); + assert_eq!(err.kind(), std::io::ErrorKind::InvalidData); + assert!( + err.to_string().contains("E6000"), + "expected the disc-read code E6000, got {err}" + ); +} + +// With skip_errors the undelivered tail is zero-filled and accounted, not +// left stale or dropped silently. current_offset still advances by the +// full request to stay on an AACS unit boundary. +#[test] +fn short_read_with_skip_errors_is_zero_filled_and_accounted() { + let mut s = short_read_stream(true); + assert!( + s.fill_extents() + .expect("skip_errors absorbs the short read") + ); + + // 8 sectors requested at 2048 B/sector (the ECMA-167 / UDF logical + // sector size) = 16384 B of buffer; the source delivered 2048 B. + assert_eq!(s.buf_valid, 16_384, "the whole requested span stays valid"); + assert_eq!(s.current_offset, 8, "the cursor advances a whole unit span"); + assert_eq!( + s.lost_bytes, 14_336, + "16384 requested - 2048 delivered = 14336 lost bytes must be counted" + ); + assert_eq!(s.errors, 1, "the gap is counted as one skip event"); + // The delivered sector survives; the undelivered tail is zeroed, not + // the stale 0xAB the source left behind. + assert!(s.read_buf[..2048].iter().all(|&b| b == 0xAB)); + assert!( + s.read_buf[2048..16_384].iter().all(|&b| b == 0), + "the undelivered tail must be zero-filled, not stale bytes" + ); +} + +// A malformed disc can declare thousands of zero-sector extents; the old self-recursive +// skip overflowed the stack (fatal, unlike iterative EOF). +#[test] +fn a_long_run_of_empty_extents_does_not_recurse_per_extent() { + let title = DiscTitle { + extents: (0..50_000) + .map(|_| crate::disc::Extent { + start_lba: 0, + sector_count: 0, + }) + .collect(), + ..DiscTitle::empty() + }; + let handle = std::thread::Builder::new() + .stack_size(256 * 1024) + .spawn(move || { + let mut s = LiveRead::new( + Box::new(ZeroReader { capacity: 0 }), + title, + crate::decrypt::DecryptKeys::None, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + s.fill_extents() + }) + .unwrap(); + assert!( + !handle.join().unwrap().unwrap(), + "an all-empty extent list is EOF, not an error" + ); +} + +fn synthetic_title(sector_count: u32) -> DiscTitle { + DiscTitle { + extents: vec![crate::disc::Extent { + start_lba: 0, + sector_count, + }], + ..DiscTitle::empty() + } +} + +// MuxOptions::default() carries batch_sectors 0: a zero batch reads 0 sectors +// and never advances, so it must be refused up front, not spin until Stop. +#[test] +fn zero_batch_is_rejected_not_an_endless_zero_read() { + let dir = tempfile::tempdir().unwrap(); + let iso = dir.path().join("t.iso"); + std::fs::write(&iso, vec![0u8; 4 * 2048]).unwrap(); + let src = crate::io::file_sector_source::FileSectorSource::open(&iso).unwrap(); + let res = LiveRead::new( + Box::new(src), + synthetic_title(4), + crate::decrypt::DecryptKeys::None, + 0, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ); + let err = res.err().expect("batch_sectors 0 must be rejected"); + assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput); + assert_eq!(crate::error::error_code(&err), Some(9085)); +} + +// A truncated ISO is not a bad sector: even with skip_errors the missing tail +// must abort with the image error, not be zero-filled and reported as success. +#[test] +fn truncated_image_aborts_even_with_skip_errors() { + let dir = tempfile::tempdir().unwrap(); + let iso = dir.path().join("short.iso"); + std::fs::write(&iso, vec![0u8; 4 * 2048]).unwrap(); + let src = crate::io::file_sector_source::FileSectorSource::open(&iso).unwrap(); + let mut stream = LiveRead::new( + Box::new(src), + synthetic_title(16), + crate::decrypt::DecryptKeys::None, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + stream.skip_errors = true; + let mut outcome = Ok(true); + for _ in 0..64 { + outcome = stream.fill_extents(); + if !matches!(outcome, Ok(true)) { + break; + } + } + let err = outcome.expect_err("a truncated image must abort the read"); + assert!( + err.to_string() + .starts_with(&format!("E{}", crate::error::E_IMAGE_ENDS_BEFORE_READ)), + "got {err}" + ); + assert_eq!(stream.errors, 0, "must not count as a skipped sector"); + assert_eq!(stream.lost_bytes, 0, "must not zero-fill the missing tail"); +} + +// Every read fails; the `stop_at`-th read cancels `halt` (Stop pressed inside a +// bad zone) and fails with `Halted` (a cancelled drive) or a media error. +struct StopInBadZoneReader { + halt: Halt, + stop_at: usize, + reader_halted: bool, + log: std::sync::Arc>>, +} + +impl crate::sector::SectorSource for StopInBadZoneReader { + fn read_sectors( + &mut self, + lba: u32, + _count: u16, + _buf: &mut [u8], + recovery: bool, + ) -> crate::error::Result { + let mut log = self.log.lock().unwrap(); + log.push(recovery); + if log.len() >= self.stop_at { + self.halt.cancel(); + if self.reader_halted { + return Err(crate::error::Error::Halted); + } + } + Err(crate::error::Error::DiscRead { + sector: lba as u64, + status: Some(0x02), + sense: None, + }) + } + + fn capacity_sectors(&self) -> u32 { + 64 + } +} + +fn stop_in_bad_zone( + batch: u16, + stop_at: usize, + reader_halted: bool, + share_halt: bool, + skip_errors: bool, +) -> (io::Result, LiveRead, Vec) { + let halt = Halt::new(); + let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let reader = StopInBadZoneReader { + halt: halt.clone(), + stop_at, + reader_halted, + log: log.clone(), + }; + let ctx = if share_halt { + crate::ctx::Ctx::new(halt.clone()) + } else { + crate::ctx::Ctx::default() + }; + let mut s = LiveRead::new( + Box::new(reader), + synthetic_title(64), + crate::decrypt::DecryptKeys::None, + batch, + ContentFormat::BdTs, + false, + &ctx, + ) + .unwrap(); + s.skip_errors = skip_errors; + if stop_at == 0 { + halt.cancel(); + } + let res = s.fill_extents(); + let log = log.lock().unwrap().clone(); + (res, s, log) +} + +// A drive cancelled by Stop fails its read with Halted: that is a stop, not a +// bad sector — no 60s recovery read, no skip accounting, no DiscRead. +#[test] +fn a_halted_read_at_the_bottomed_out_unit_is_a_stop_not_a_bad_sector() { + for share_halt in [true, false] { + for skip_errors in [true, false] { + let (res, s, log) = stop_in_bad_zone(1, 1, true, share_halt, skip_errors); + let err = res.expect_err("a halted read must not fill"); + assert!( + crate::error::is_halt(&err), + "share={share_halt} skip={skip_errors}: got {err}" + ); + assert_eq!(log, vec![false], "no recovery read after a halted read"); + assert_eq!((s.errors, s.lost_bytes), (0, 0), "no bogus skip"); + } + } +} + +// Stop pressed while the read stalls in a failing region returns Halted at once, +// never walking the shrink/recovery ladder first. +#[test] +fn stop_in_a_failing_region_returns_halted_without_more_reads() { + let (res, s, log) = stop_in_bad_zone(32, 0, false, true, true); + assert!(crate::error::is_halt(&res.expect_err("halted before read"))); + assert!(log.is_empty(), "no read once already halted: {log:?}"); + assert_eq!(s.errors, 0); + + let (res, s, log) = stop_in_bad_zone(32, 2, false, true, true); + let err = res.expect_err("a stop mid bad zone must not fill"); + assert!(crate::error::is_halt(&err), "got {err}"); + assert_eq!(log, vec![false, false], "no read after the stop"); + assert_eq!((s.errors, s.lost_bytes), (0, 0), "no bogus skip"); +} + +// Recording SectorSource: logs every (lba, count) request, errors when the +// range covers bad_sector. Successful reads return zeroed sectors (not +// flagged scrambled, so DecryptingSectorSource passes them through as-is). +struct RecordingReader { + capacity: u32, + bad_sector: u32, + log: std::sync::Arc>>, +} + +impl crate::sector::SectorSource for RecordingReader { + fn read_sectors( + &mut self, + lba: u32, + count: u16, + buf: &mut [u8], + _recovery: bool, + ) -> crate::error::Result { + self.log.lock().unwrap().push((lba, count)); + let end = lba + count as u32; + if self.bad_sector >= lba && self.bad_sector < end { + return Err(crate::error::Error::DiscRead { + sector: self.bad_sector as u64, + status: Some(0x02), + sense: None, + }); + } + let bytes = count as usize * 2048; + buf[..bytes].fill(0); + Ok(bytes) + } + + fn capacity_sectors(&self) -> u32 { + self.capacity + } +} + +// SectorSource that fails every read covering bad_sector with a SCSI +// transport failure (status=0xFF, the USB-bridge-crash sentinel Drive::read +// surfaces). Logs each (lba, count) so a test can prove it wasn't retried. +struct TransportFailReader { + capacity: u32, + bad_sector: u32, + log: std::sync::Arc>>, +} + +impl crate::sector::SectorSource for TransportFailReader { + fn read_sectors( + &mut self, + lba: u32, + count: u16, + buf: &mut [u8], + _recovery: bool, + ) -> crate::error::Result { + self.log.lock().unwrap().push((lba, count)); + let end = lba + count as u32; + if self.bad_sector >= lba && self.bad_sector < end { + return Err(crate::error::Error::DiscRead { + sector: self.bad_sector as u64, + status: Some(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE), + sense: None, + }); + } + let bytes = count as usize * 2048; + buf[..bytes].fill(0); + Ok(bytes) + } + + fn capacity_sectors(&self) -> u32 { + self.capacity + } +} + +// SectorSource mirroring a marginal sector recoverable only with the +// drive's full ECC budget: fails while recovery=false, succeeds once +// recovery=true. Exercises fill_extents' last-chance recovery-read branch. +struct RecoverableReader { + capacity: u32, + bad_sector: u32, + /// `(lba, count, recovery)` for every issued read. + log: std::sync::Arc>>, +} + +impl crate::sector::SectorSource for RecoverableReader { + fn read_sectors( + &mut self, + lba: u32, + count: u16, + buf: &mut [u8], + recovery: bool, + ) -> crate::error::Result { + self.log.lock().unwrap().push((lba, count, recovery)); + let end = lba + count as u32; + let covers_bad = self.bad_sector >= lba && self.bad_sector < end; + // Fail on the fast (non-recovery) pass; the 60s ECC recovery read + // succeeds. Distinct non-0x02 sense byte so a transport-failure + // re-check (status 0xFF) is provably NOT triggered here. + if covers_bad && !recovery { + return Err(crate::error::Error::DiscRead { + sector: self.bad_sector as u64, + status: Some(0x02), + sense: None, + }); + } + let bytes = count as usize * 2048; + buf[..bytes].fill(0); + Ok(bytes) + } + + fn capacity_sectors(&self) -> u32 { + self.capacity + } +} + +// Coverage for the recovery-read SUCCESS branch (rc.5.2 audit #3): a +// sector failing the fast read but clean on the 60s ECC pass must mux the +// RECOVERED data — cursor advances, no skip counted. unit_align=1 (None). +#[test] +fn recovery_read_success_muxes_recovered_data_no_skip() { + const COUNT: u32 = 10; + let bad = 4u32; + let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let reader = RecoverableReader { + capacity: COUNT, + bad_sector: bad, + log: log.clone(), + }; + let mut stream = LiveRead::new( + Box::new(reader), + synthetic_title(COUNT), + crate::decrypt::DecryptKeys::None, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + // skip_errors=false: if the recovery read did NOT succeed, fill_extents + // would return Err — so reaching EOF cleanly proves recovery worked. + stream.skip_errors = false; + + let mut guard = 0; + loop { + match stream.fill_extents() { + Ok(true) => {} + Ok(false) => break, + Err(e) => panic!("recovery read should have succeeded, got: {e}"), + } + guard += 1; + assert!(guard < 1000, "fill_extents did not reach EOF"); + } + + // No skip counted: the recovered unit was muxed, not zero-filled. + assert_eq!( + stream.errors, 0, + "a successful recovery read must not count as a skipped sector" + ); + assert_eq!( + stream.lost_bytes, 0, + "a successful recovery read loses no bytes" + ); + // All COUNT sectors' worth of bytes were read through to the cursor end. + assert_eq!( + stream.bytes_read_total, + COUNT as u64 * 2048, + "every sector (including the recovered one) must be counted as read" + ); + + // The bad sector was retried with recovery=true and that read SUCCEEDED. + let reads = log.lock().unwrap(); + assert!( + reads + .iter() + .any(|&(lba, count, rec)| rec && lba == bad && count == 1), + "expected a recovery=true single-sector read at the bad sector; got {reads:?}" + ); + // And the fast pass at the bad sector did happen with recovery=false. + assert!( + reads.iter().any(|&(lba, _c, rec)| !rec && lba == bad), + "expected a non-recovery read to have first failed at the bad sector" + ); +} + +// SectorSource that fails the fast read at bad_sector with status 0x02, +// then fails the 60s ECC recovery read with transport failure (0xFF) — +// models a bridge wedging during the last-chance recovery read. +struct RecoveryTransportFailReader { + capacity: u32, + bad_sector: u32, + log: std::sync::Arc>>, +} + +impl crate::sector::SectorSource for RecoveryTransportFailReader { + fn read_sectors( + &mut self, + lba: u32, + count: u16, + buf: &mut [u8], + recovery: bool, + ) -> crate::error::Result { + self.log.lock().unwrap().push((lba, count, recovery)); + let end = lba + count as u32; + if self.bad_sector >= lba && self.bad_sector < end { + let status = if recovery { + crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE + } else { + 0x02 + }; + return Err(crate::error::Error::DiscRead { + sector: self.bad_sector as u64, + status: Some(status), + sense: None, + }); + } + let bytes = count as usize * 2048; + buf[..bytes].fill(0); + Ok(bytes) + } + + fn capacity_sectors(&self) -> u32 { + self.capacity + } +} + +// Regression (rc.5.2 audit #2): a transport failure on the 60s ECC +// RECOVERY read (not just the initial read) must ABORT even under +// skip_errors=true, not fall into skip/advance and march forever. +#[test] +fn transport_failure_on_recovery_read_aborts_even_with_skip_errors() { + const COUNT: u32 = 10; + let bad = 4u32; + let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let reader = RecoveryTransportFailReader { + capacity: COUNT, + bad_sector: bad, + log: log.clone(), + }; + let mut stream = LiveRead::new( + Box::new(reader), + synthetic_title(COUNT), + crate::decrypt::DecryptKeys::None, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + stream.skip_errors = true; + + // Drive fill_extents across batches: the batch covering the bad sector + // shrinks to size 1, fails the fast read (0x02), then the bottom-out + // recovery read returns the transport failure (0xFF), which must abort. + let mut res = Ok(true); + for _ in 0..1000 { + res = stream.fill_extents(); + if !matches!(res, Ok(true)) { + break; + } + } + assert!( + res.is_err(), + "a transport failure on the recovery read must abort fill_extents, got {res:?}" + ); + assert_eq!( + stream.errors, 0, + "a recovery-read transport-failure abort must NOT count as a skip" + ); + assert_eq!( + stream.lost_bytes, 0, + "a transport-failure abort zero-fills nothing" + ); + // Prove the bottom-out recovery read was actually reached and aborted on. + let reads = log.lock().unwrap(); + assert!( + reads + .iter() + .any(|&(lba, count, rec)| rec && lba == bad && count == 1), + "expected a recovery=true read at the bad sector to have been attempted; got {reads:?}" + ); +} + +// Fast read fails at bad_sector (0x02); the recovery read returns `rec_err()`. +struct RecoveryErrReader { + bad_sector: u32, + rec_err: fn() -> crate::error::Error, +} + +impl crate::sector::SectorSource for RecoveryErrReader { + fn read_sectors( + &mut self, + lba: u32, + count: u16, + buf: &mut [u8], + recovery: bool, + ) -> crate::error::Result { + if self.bad_sector >= lba && self.bad_sector < lba + count as u32 { + if recovery { + return Err((self.rec_err)()); + } + return Err(crate::error::Error::DiscRead { + sector: self.bad_sector as u64, + status: Some(0x02), + sense: None, + }); + } + let bytes = count as usize * 2048; + buf[..bytes].fill(0); + Ok(bytes) + } + + fn capacity_sectors(&self) -> u32 { + 10 + } +} + +// A key stop or dead source on the RECOVERY read must abort even under +// skip_errors, never be zero-filled as a skipped sector. +#[test] +fn key_stop_or_dead_source_on_recovery_read_aborts_even_with_skip_errors() { + type MkErr = fn() -> crate::error::Error; + let cases: [(&str, MkErr); 3] = [ + ("NoDiscKey", || crate::error::Error::NoDiscKey { + disc_hash: String::new(), + }), + ("WholeDiscKeyMissing", || { + crate::error::Error::WholeDiscKeyMissing + }), + ("SourceTerminated", || crate::error::Error::SourceTerminated), + ]; + for (name, rec_err) in cases { + let mut stream = LiveRead::new( + Box::new(RecoveryErrReader { + bad_sector: 4, + rec_err, + }), + synthetic_title(10), + crate::decrypt::DecryptKeys::None, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + stream.skip_errors = true; + let mut res = Ok(true); + for _ in 0..1000 { + res = stream.fill_extents(); + if !matches!(res, Ok(true)) { + break; + } + } + assert!( + res.is_err(), + "{name} on the recovery read must abort, got {res:?}" + ); + assert_eq!(stream.errors, 0, "{name}: must not count as a skip"); + assert_eq!(stream.lost_bytes, 0, "{name}: must not zero-fill"); + } +} + +// Regression: a USB-bridge transport crash (0xFF) during a single-pass +// disc://->mkv:// rip must ABORT immediately, even under skip_errors=true +// — mirrors the multipass sweep's short-circuit; exactly one read issued. +#[test] +fn transport_failure_aborts_single_pass_even_with_skip_errors() { + const COUNT: u32 = 10; + let bad = 4u32; + let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let reader = TransportFailReader { + capacity: COUNT, + bad_sector: bad, + log: log.clone(), + }; + let mut stream = LiveRead::new( + Box::new(reader), + synthetic_title(COUNT), + crate::decrypt::DecryptKeys::None, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + stream.skip_errors = true; + + let res = stream.fill_extents(); + assert!( + res.is_err(), + "transport failure must abort fill_extents, not skip past it" + ); + assert_eq!( + stream.errors, 0, + "a transport-failure abort must NOT count as a skipped sector" + ); + let reads = log.lock().unwrap(); + assert_eq!( + reads.len(), + 1, + "transport failure must abort after the first failed read with no \ + shrink/retry/skip-ahead; got reads {reads:?}" + ); +} + +// AACS unit-alignment skip: with unit_align=3 and skip_errors=true, a +// single bad mid-extent sector must not desync the title — every read +// starts on a 3-sector unit boundary and skips advance a whole unit. +#[test] +fn aacs_reads_stay_unit_aligned_and_skip_whole_units() { + const COUNT: u32 = 30; + const ALIGN: u32 = 3; + // Bad sector at offset 13 — inside unit 4 (offsets 12,13,14). The + // whole unit must be skipped, keeping the cursor unit-aligned. + let bad = 13u32; + let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let reader = RecordingReader { + capacity: COUNT, + bad_sector: bad, + log: log.clone(), + }; + let title = synthetic_title(COUNT); + let keys = crate::decrypt::DecryptKeys::Aacs { + unit_keys: vec![(0, [0u8; 16])], + format: crate::disc::ContentFormat::BdTs, + }; + let stream = LiveRead::new( + Box::new(reader), + title, + keys, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + // AACS decrypts only through a key map; without one every read is a decrypt refusal. + let map = crate::keys::test_key_map(vec![(0, COUNT, 0, crate::decrypt::Phase::All)]); + let mut stream = stream; + stream.reader = crate::keys::test_keyed_source(stream.reader, map); + stream.skip_errors = true; + assert_eq!( + stream.unit_align, ALIGN as u16, + "AACS keys must set unit_align=3" + ); + + // Drive fill_extents to EOF (no PES demux needed — we observe the + // raw read pattern directly). + let ext_start = 0u32; + let mut guard = 0; + loop { + match stream.fill_extents() { + Ok(true) => {} + Ok(false) => break, + Err(e) => panic!("fill_extents errored unexpectedly: {e}"), + } + guard += 1; + assert!(guard < 1000, "fill_extents did not reach EOF"); + } + + let reads = log.lock().unwrap(); + assert!(!reads.is_empty(), "expected at least one read"); + for &(lba, count) in reads.iter() { + assert_eq!( + (lba - ext_start) % ALIGN, + 0, + "read at lba {lba} is not unit-aligned (offset {} % {ALIGN} != 0)", + lba - ext_start + ); + // Non-tail reads must be a whole number of units; the only permitted + // short read is a final partial unit. A mid-stream non-unit-multiple + // read would straddle AACS unit boundaries and decrypt wrong. + assert!( + (count as u32).is_multiple_of(ALIGN) || (count as u32) < ALIGN, + "read count {count} is neither a whole number of units nor a sub-unit tail" + ); + } + + // At least one error was skipped (the bad unit) and a SectorSkipped + // event was emitted; errors counter advanced by exactly the bad units. + assert!(stream.errors >= 1, "expected the bad unit to be skipped"); + + // Regression: `lost_bytes` must account for the WHOLE skipped unit + // (3 sectors = 6144 bytes), not one sector — `errors * 2048` would + // undercount AACS loss ~3x, the single-pass abort-gate bug this guards. + assert_eq!( + stream.lost_bytes, + stream.errors * ALIGN as u64 * 2048, + "AACS skip must record a whole unit (6144 B) per skip event, not 2048" + ); + assert!( + stream.lost_bytes > stream.errors * 2048, + "lost_bytes must exceed the errors*2048 undercount for AACS units" + ); + + // Crucial anti-desync assertion: the read that bottomed out and was + // skipped must have been a single 3-sector unit starting at offset 12 + // (the unit boundary at/below bad sector 13), NOT a 1-sector read at 13. + assert!( + reads + .iter() + .any(|&(lba, count)| lba == 12 && count == ALIGN as u16), + "expected a unit-aligned (lba=12,count=3) read over the bad unit; got {reads:?}" + ); + // And NO single-sector read at the bad sector itself (would be a desync). + assert!( + !reads.iter().any(|&(lba, count)| lba == bad && count == 1), + "a 1-sector read at the bad sector {bad} would desync the AACS unit stream" + ); +} + +/// `unit_align == 1` (DecryptKeys::None) variant: single-sector skips +/// still work (CSS/raw is self-synchronizing, so a 1-sector skip is +/// correct there — contrast with the AACS whole-unit skip above). +#[test] +fn unencrypted_single_sector_skip_works() { + const COUNT: u32 = 10; + let bad = 4u32; + let log = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let reader = RecordingReader { + capacity: COUNT, + bad_sector: bad, + log: log.clone(), + }; + let mut stream = LiveRead::new( + Box::new(reader), + synthetic_title(COUNT), + crate::decrypt::DecryptKeys::None, + 8, + ContentFormat::BdTs, + false, + &crate::ctx::Ctx::default(), + ) + .unwrap(); + stream.skip_errors = true; + assert_eq!(stream.unit_align, 1, "None keys must leave unit_align=1"); + + let mut guard = 0; + loop { + match stream.fill_extents() { + Ok(true) => {} + Ok(false) => break, + Err(e) => panic!("fill_extents errored unexpectedly: {e}"), + } + guard += 1; + assert!(guard < 1000, "fill_extents did not reach EOF"); + } + + let reads = log.lock().unwrap(); + // The bad sector must have been retried down to a single sector and + // skipped at count==1 — the self-synchronizing per-sector path. + assert!( + reads.iter().any(|&(lba, count)| lba == bad && count == 1), + "align=1 must bottom out at a 1-sector read over the bad sector; got {reads:?}" + ); + assert!(stream.errors >= 1); + // align=1: a skip event covers exactly one sector, so lost_bytes + // and errors*2048 agree (the AACS undercount does not apply here). + assert_eq!( + stream.lost_bytes, + stream.errors * 2048, + "single-sector (align=1) skip must record exactly 2048 B per event" + ); +} + +// ── AdaptiveBatch ───────────────────────────────────────────────── + +// Batch sizes >= 6 must stay a multiple of 3 (AACS units) or the next +// read straddles a boundary; below 6 the ladder descends 3 -> 1. +#[test] +fn halve_batch_size_keeps_unit_alignment_and_bottoms_out_at_one() { + assert_eq!( + halve_batch_size(64), + 30, + "32 rounded down to a unit multiple" + ); + assert_eq!(halve_batch_size(30), 15); + assert_eq!(halve_batch_size(12), 6, "6 is already unit-aligned"); + assert_eq!(halve_batch_size(11), 5, "below 6: no alignment rounding"); + assert_eq!(halve_batch_size(6), 3); + assert_eq!(halve_batch_size(3), 1); + assert_eq!(halve_batch_size(2), 1); + assert_eq!( + halve_batch_size(1), + 1, + "must never reach 0 — a 0-sector read" + ); + for size in 1u16..=4096 { + let h = halve_batch_size(size); + assert!(h >= 1, "halve({size}) must never be 0"); + assert!(h <= size, "halve({size}) = {h} must not grow"); + assert!( + h < 6 || h.is_multiple_of(3), + "halve({size}) = {h} is unit-unaligned" + ); + } +} + +#[test] +fn double_batch_size_grows_toward_preferred_without_breaking_alignment() { + assert_eq!(double_batch_size(30, 64), 60); + assert_eq!( + double_batch_size(4, 64), + 6, + "8 rounded down to a unit multiple" + ); + assert_eq!( + double_batch_size(1, 64), + 2, + "below 6: no alignment rounding" + ); + assert_eq!( + double_batch_size(60, 64), + 63, + "clamped to preferred, then aligned" + ); + for size in 1u16..=2048 { + let d = double_batch_size(size, 4096); + assert!(d >= size, "double({size}) = {d} must not shrink"); + assert!( + d < 6 || d.is_multiple_of(3), + "double({size}) = {d} is unit-unaligned" + ); + } +} + +// The sizer must probe back up: after a failure drops the batch, a +// sustained clean run must return BatchSizeChanged{Probed} AND raise +// current — never probing locks the rip at reduced size forever. +#[test] +fn on_success_probes_up_after_a_sustained_clean_run_and_resets_the_streak() { + let mut b = AdaptiveBatch::new(64); + assert!( + matches!( + b.on_failure(), + Some(Event::BatchSizeChanged { + new_size: 30, + reason: BatchSizeReason::Shrunk + }) + ), + "a failure must shrink 64 -> 30" + ); + assert_eq!(b.current(), 30); + + // Just under the 51200-sector probe threshold: still silent. + let mut fed = 0u32; + while fed + 30 < PROBE_THRESHOLD_SECTORS { + assert!( + b.on_success(30).is_none(), + "no probe before {PROBE_THRESHOLD_SECTORS} clean sectors (at {fed})" + ); + fed += 30; + } + assert_eq!(b.current(), 30, "still at the reduced size"); + + // The read that crosses the threshold probes up. + let ev = b + .on_success(30) + .expect("a sustained clean run must probe the batch size back up"); + assert!( + matches!( + ev, + Event::BatchSizeChanged { + new_size: 60, + reason: BatchSizeReason::Probed + } + ), + "expected a Probed grow to 60, got {ev:?}" + ); + assert_eq!( + b.current(), + 60, + "the sizer must actually adopt the new size" + ); + assert_eq!( + b.streak_sectors, 0, + "the streak resets so the next probe needs a fresh clean run" + ); + + // A failure mid-streak must reset it: no probe right after a failure. + let mut b = AdaptiveBatch::new(64); + b.on_failure(); + b.on_success(30); + assert!(b.streak_sectors > 0); + b.on_failure(); + assert_eq!(b.streak_sectors, 0, "a failure must reset the clean streak"); + + // At the preferred size a clean run must NOT keep firing events. + let mut b = AdaptiveBatch::new(64); + for _ in 0..(PROBE_THRESHOLD_SECTORS / 64 + 2) { + assert!( + b.on_success(64).is_none(), + "no probe is possible when already at the preferred size" + ); + } + assert_eq!(b.current(), 64); +} diff --git a/src/sector/stage.rs b/src/sector/stage.rs index 4b2c0ce1..73b4a56a 100644 --- a/src/sector/stage.rs +++ b/src/sector/stage.rs @@ -2,7 +2,7 @@ //! transitional: the pipeline's chunk stream replaces the sector/byte split). //! //! Every `input()` source passes the stage. A sector source (`mpg://`, `m2ts://`) is wrapped -//! by [`DecryptingSectorSource::detecting`](super::DecryptingSectorSource::detecting); a byte +//! by the content-detected [`DecryptingSectorSource`](super::DecryptingSectorSource); a byte //! reader (`mkv://`, `mp4://`, `network://`, `stdio://`) by [`Stage`], which hands a clear //! container through untouched and refuses encrypted content it cannot decrypt. The verdict //! comes from the bytes alone ([`classify`]), never from the URL scheme. @@ -287,6 +287,23 @@ impl SectorBytes { } } +impl SectorBytes { + /// The stage back, after a head read: the bytes it holds past the read position (up to + /// the real length) and the sector its next refill would read. + pub(crate) fn into_rest(self) -> (S, Vec, u32) { + let buf_end = self.buf_start + self.buf.len() as u64; + let held = self.pos.clamp(self.buf_start, buf_end)..buf_end.min(self.len); + let rest = match held.start < held.end { + true => self.buf + [(held.start - self.buf_start) as usize..(held.end - self.buf_start) as usize] + .to_vec(), + false => Vec::new(), + }; + let next = (buf_end / SECTOR_BYTES as u64) as u32; + (self.src, rest, next) + } +} + impl Read for SectorBytes { fn read(&mut self, out: &mut [u8]) -> io::Result { if self.pos >= self.len || out.is_empty() { diff --git a/src/session.rs b/src/session.rs index 749289d1..226cffb6 100644 --- a/src/session.rs +++ b/src/session.rs @@ -10,9 +10,9 @@ use crate::disc::{Disc, DiscId, DriveCredentials, ScanOptions}; use crate::drive::{Drive, find_drive}; use crate::error::{Error, Result}; -use crate::halt::{Halt, Progress}; +use crate::halt::{Halt, Liveness}; use crate::keysource::KeySource; -use crate::sector::{FileSectorSource, SectorSource}; +use crate::sector::SectorSource; use std::path::{Path, PathBuf}; use std::sync::Arc; @@ -20,11 +20,11 @@ use std::sync::Arc; /// /// libfreemkv builds no key sources itself (the `freemkv_keysources` crate that /// implements [`KeySource`] depends on libfreemkv, not the other way round), so -/// the consumer hands in a way to build its sources. [`ResolvedKeySet::resolve`] +/// the consumer hands in a way to build its sources. [`KeyRing::resolve`] /// calls it once and keeps nothing; it stays `Send + Sync` without requiring /// `KeySource: Send`. /// -/// [`ResolvedKeySet::resolve`]: crate::keys::ResolvedKeySet::resolve +/// [`KeyRing::resolve`]: crate::keys::KeyRing::acquire pub type KeySourceFactory = Arc Vec> + Send + Sync>; /// Which optical device a [`DiscSession`] should open. @@ -77,7 +77,7 @@ pub struct KeySpec { /// /// Owns the [`Drive`] by value. Consumers that still need the raw drive (e.g. /// to sample ciphertext for key validation, or to move it into a -/// `DiscStream`) reach it via [`Self::into_drive`]; the +/// live mux) reach it via [`Self::into_drive`]; the /// scanned [`Disc`] comes out via [`Self::disc`] / [`Self::take_disc`]. pub struct DiscSession { /// The opened drive. `Some` from [`Self::open`] until @@ -99,7 +99,7 @@ pub struct DiscSession { halt: Option, /// The op's progress from [`Self::attach_progress`] (T29): the drive's CDBs and each /// key-source call report to it. - progress: Option, + progress: Option, } // Overlay the session's key material onto `opts` without clobbering what the @@ -147,7 +147,7 @@ impl DiscSession { /// [`Self::open`] under the caller's op token (stop design §2.2): the drive checks /// `halt` on every CDB, a Stop during the bring-up ends it `Halted` (the handle is /// closed), and the session keeps the token for [`Self::scan_with`] and - /// [`Self::resolve_key_set`]. + /// [`Self::acquire_keys`]. pub fn open_with(target: DeviceTarget, spec: KeySpec, halt: &Halt) -> Result { let drive = match target { DeviceTarget::Path(ref path) => Drive::open_with(path, halt)?, @@ -233,8 +233,8 @@ impl DiscSession { } /// Report the op's forward progress to `p` (T29): every drive CDB (as - /// [`Drive::attach_progress`]) and every key-source call in [`Self::resolve_key_set`]. - pub fn attach_progress(&mut self, p: &Progress) { + /// [`Drive::attach_progress`]) and every key-source call in [`Self::acquire_keys`]. + pub fn attach_progress(&mut self, p: &Liveness) { if let Some(drive) = self.drive.as_mut() { drive.attach_progress(p); } @@ -338,36 +338,37 @@ impl DiscSession { } } - /// Resolve the rip's key set for `scope` up front (KU §3.1): one - /// [`ResolvedKeySet::resolve`](crate::keys::ResolvedKeySet::resolve) through the - /// session's staged reader, else its drive. The session keeps nothing: the set is the + /// Acquire the rip's key ring for `scope` up front (KU §3.1): one + /// [`KeyRing::acquire_for_disc`](crate::keys::KeyRing::acquire_for_disc) through the + /// session's staged reader, else its drive. The session keeps nothing: the ring is the /// caller's, and no source is retained. Requires [`Self::scan`] to have run. - pub fn resolve_key_set( + pub fn acquire_keys( &mut self, scope: crate::keys::KeyScope, sources: &KeySourceFactory, - opts: crate::keys::ResolveKeysOptions, + opts: crate::keys::AcquireOptions, + ctx: &crate::ctx::Ctx, ) -> Result { - // Under `open_with` the session's op token governs the resolve too (§2.12). + // Under `open_with` the session's op token governs the acquire too (§2.12). let (op, progress) = (self.halt.clone(), self.progress.clone()); - if let Some(h) = &op { - h.check()?; - } - let mut opts = opts; - if let Some(h) = &op { - // As `Drive::alias` (§2.2): the session's op token wins over the caller's. - if opts - .halt - .is_some_and(|c| !Arc::ptr_eq(c.as_arc(), h.as_arc())) - { - tracing::warn!( - target: "freemkv::session", - phase = "halt_alias", - "ResolveKeysOptions.halt differs from the session's op token; the session token wins" - ); + let ctx = match op { + Some(h) => { + h.check()?; + if !Arc::ptr_eq(ctx.halt.as_arc(), h.as_arc()) { + // As `Drive::alias` (§2.2): the session's op token wins over the caller's. + tracing::warn!( + target: "freemkv::session", + phase = "halt_alias", + "the run context's halt differs from the session's op token; the session token wins" + ); + } + crate::ctx::Ctx { + halt: h, + ..ctx.clone() + } } - opts.halt = Some(h); - } + None => ctx.clone(), + }; let Some(disc) = self.disc.as_ref() else { return Err(Error::DeviceNotReady { path: self.device.clone(), @@ -382,12 +383,9 @@ impl DiscSession { }); } }; - match &progress { - Some(p) => crate::keys::ResolvedKeySet::resolve_with_progress( - disc, reader, scope, sources, opts, p, - ), - None => crate::keys::ResolvedKeySet::resolve(disc, reader, scope, sources, opts), - } + let mut opts = opts; + opts.liveness = opts.liveness.or(progress.as_ref()); + crate::keys::KeyRing::acquire_for_disc(disc, reader, scope, sources, opts, &ctx) } /// The scanned disc, if [`Self::scan`] has run. @@ -422,8 +420,8 @@ impl DiscSession { } } - /// Consume the session, returning the owned drive (e.g. to move into a - /// `DiscStream` for a live-drive mux). + /// Consume the session, returning the owned drive (e.g. to move into + /// the Read stage for a live-drive mux). /// /// # Errors /// @@ -437,7 +435,7 @@ impl DiscSession { /// Stage the owned drive as the session's boxed sector source so a live /// single-pass mux can drive it through - /// [`MuxSource::Session`](crate::mux::MuxSource::Session). Moves the `Drive` + /// [`Source::from_session`](crate::mux::Source::from_session). Moves the `Drive` /// (itself a [`SectorSource`]) into the `reader` slot; the cached /// [`Self::device_path`] keeps the device name available afterward. A no-op /// if the drive was already staged or moved out. @@ -465,7 +463,7 @@ impl DiscSession { /// Take the staged sector source out of the session by mutable borrow, /// leaving `None` behind. Used by [`crate::mux::mux_with_keys`]'s - /// [`MuxSource::Session`](crate::mux::MuxSource::Session) arm, which drives + /// [`Source::from_session`](crate::mux::Source::from_session) arm, which drives /// the mux from `&mut DiscSession` and so cannot consume the whole session. /// A second call (or a call before the reader is staged) returns `None`, and /// the driver maps that to a clean error rather than a panic (see Q2 of the @@ -519,15 +517,12 @@ impl DiscSession { /// [`Disc`] together with a reusable [`SectorSource`] over the same file. /// /// This is the file-backed counterpart to [`DiscSession::scan`]: it opens a -/// [`FileSectorSource`], reads its capacity, and runs [`Disc::scan_image`]. +/// [`crate::sector::FileSectorSource`], reads its capacity, and runs [`Disc::scan_image`]. /// No SCSI, no handshake, no key resolution beyond what `opts` already /// carries. The returned reader is a fresh handle at the start of the image, /// reusable by callers that need to sample ciphertext or feed a mux. pub fn scan_iso(path: &Path, opts: ScanOptions) -> Result<(Disc, Box)> { - let mut reader = FileSectorSource::open(path)?; - let capacity = reader.capacity_sectors(); - let disc = Disc::scan_image(&mut reader, capacity, &opts)?; - Ok((disc, Box::new(reader))) + crate::mux::source::probe_image(path, false, &opts) } // Sampled 6144-byte aligned units when judging whether a folder with `AACS/` @@ -543,14 +538,9 @@ const AACS_PROBE_UNITS: usize = 8; /// content is sampled and judged by `aacs_unit_needs_decrypt`: /// /// * none need decryption → `encrypted` is forced false, reason logged. -/// * any unit does → [`Error::DirImageEncrypted`] (`dir://` doesn't support it). +/// * any unit does → the folder keeps its AACS verdict and is keyed like an image. pub fn scan_dir(path: &Path, opts: ScanOptions) -> Result<(Disc, Box)> { - let mut reader = crate::dirimage::DirImage::open(path)?; - let capacity = reader.capacity_sectors(); - let mut disc = Disc::scan_image(&mut reader, capacity, &opts)?; - - apply_folder_encryption_verdict(&mut reader, &mut disc)?; - Ok((disc, Box::new(reader))) + crate::mux::source::probe_image(path, true, &opts) } // Re-judge a FOLDER's encryption verdict from its CONTENT (tree shape alone can be wrong for an @@ -565,7 +555,10 @@ pub(crate) fn apply_folder_encryption_verdict( // Only the AACS-by-tree-shape verdict is re-judged here. if disc.encrypted && disc.css.is_none() && disc.css_error.is_none() { match probe_folder_encryption(reader, disc)? { - true => return Err(Error::DirImageEncrypted), + // Encrypted content: the folder keeps its AACS verdict and is keyed and + // decrypted like an image of the same disc (every input passes the one + // decryption stage). + true => {} false => { tracing::warn!( target: "freemkv::scan", diff --git a/src/session/stop_tests.rs b/src/session/stop_tests.rs index 0773b6b0..c135be99 100644 --- a/src/session/stop_tests.rs +++ b/src/session/stop_tests.rs @@ -129,8 +129,13 @@ fn open_with_succeeds_after_stopped_op_drops() { start_lba: 0, sector_count: 300, }]; - let pf = crate::sector::PrefetchedSectorSource::new(drive, extents, 3, Some(h1.clone())) - .expect("spawn"); + let pf = crate::sector::PrefetchedSectorSource::new( + drive, + extents, + 3, + &crate::ctx::Ctx::new(h1.clone()), + ) + .expect("spawn"); std::thread::sleep(Duration::from_millis(20)); h1.cancel(); drop(pf); diff --git a/src/whole_disc.rs b/src/whole_disc.rs index 7b5a3e8f..a0f76b3b 100644 --- a/src/whole_disc.rs +++ b/src/whole_disc.rs @@ -1,5 +1,5 @@ //! The pieces of the whole-disc decrypting reader -//! ([`ResolvedKeySet::whole_disc_reader`](crate::keys::ResolvedKeySet::whole_disc_reader)) +//! ([`KeyRing::whole_disc_reader`](crate::keys::KeyRing::whole_disc_reader)) //! behind every decrypted disc/image → ISO copy, so the GUI, the CLI and the image path //! share one set of rules. AACS content is every content file (`/BDMV/STREAM`, HD DVD //! `/HVDVD_TS/*.EVO`), not just the kept titles. Reads follow each file's own 3-sector @@ -20,7 +20,7 @@ pub type WholeDiscReader = UnitAligned>; /// The whole-disc reader for a raw (`--raw`) copy. It never decrypts: AACS ciphertext and /// CSS-scrambled sectors pass through byte for byte. A decrypting copy reads through -/// [`ResolvedKeySet::whole_disc_reader`](crate::keys::ResolvedKeySet::whole_disc_reader). +/// [`KeyRing::whole_disc_reader`](crate::keys::KeyRing::whole_disc_reader). pub fn raw_whole_disc_reader(reader: S) -> WholeDiscReader { UnitAligned::new( DecryptingSectorSource::new(reader, crate::decrypt::DecryptKeys::None), diff --git a/tests/disc_tests.rs b/tests/disc_tests.rs index c695051d..0919cf42 100644 --- a/tests/disc_tests.rs +++ b/tests/disc_tests.rs @@ -412,7 +412,7 @@ fn first_batch_sectors(sector_count: u32, batch_sectors: u16) -> usize { sector_count, }], batch_sectors, - None, + &libfreemkv::Ctx::default(), ) .expect("prefetch producer spawns"); let mut buf = vec![0u8; batch_sectors as usize * SECTOR_SIZE]; @@ -519,7 +519,7 @@ fn batch_count_zero_remaining() { sector_count: 0, }], 60, - None, + &libfreemkv::Ctx::default(), ) .expect("prefetch producer spawns"); let mut buf = vec![0u8; 60 * SECTOR_SIZE]; diff --git a/tests/goldens/parity_sinks_clear_m2ts_selection.golden b/tests/goldens/parity_sinks_clear_m2ts_selection.golden index ffc6d852..94f76076 100644 --- a/tests/goldens/parity_sinks_clear_m2ts_selection.golden +++ b/tests/goldens/parity_sinks_clear_m2ts_selection.golden @@ -1,10 +1,9 @@ -mkv outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] -mkv out o.mkv = 58232 bytes sha256:b25ee88e8f6935e99acf42fd476cc71d -m2ts outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] -m2ts out o.m2ts = 80256 bytes sha256:ab5d202fdf7ea5d2e5be98f01840be21 -demux outcome = completed=true opened=true bytes=54504 errors=0 lost=0 streams=3 undelivered=[] +mkv outcome = completed=true opened=true bytes=41192 errors=0 lost=0 streams=2 undelivered=[] +mkv out o.mkv = 43765 bytes sha256:fcc0509e04bf54381592dbcda6e26a28 +m2ts outcome = completed=true opened=true bytes=41192 errors=0 lost=0 streams=2 undelivered=[] +m2ts out o.m2ts = 60288 bytes sha256:93cf375a4eecf1bff5feaa2fe9b3a74e +demux outcome = completed=true opened=true bytes=41192 errors=0 lost=0 streams=2 undelivered=[] demux out o/title t00 MPEG2.m2v = 27880 bytes sha256:be8990c82ea9024deb38cdfb174bea80 -demux out o/title t01 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:904e9e35a2990b1001d2d99baeb1a633 -demux out o/title t02 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:48694b3013866911b7e0f20507ebb311 -json outcome = completed=true opened=true bytes=0 errors=0 lost=0 streams=3 undelivered=[] -json out o.json = 1263 bytes sha256:040a1c5961ab0286333076150fbab37e +demux out o/title t01 und AC3 DELAY 0ms.ac3 = 13312 bytes sha256:48694b3013866911b7e0f20507ebb311 +json outcome = completed=true opened=true bytes=0 errors=0 lost=0 streams=2 undelivered=[] +json out o.json = 935 bytes sha256:07827a0ee32bcb431c16fd96d66362c2 diff --git a/tests/lint_stop.rs b/tests/lint_stop.rs index b5161a27..a6bae9a3 100644 --- a/tests/lint_stop.rs +++ b/tests/lint_stop.rs @@ -384,7 +384,7 @@ fn no_thread_sleep_on_op_paths() { ]; const ALLOW: &[Allow] = &[ ( - "src/disc/extract.rs", + "src/io/tree_sink.rs", "probe_case_insensitive", "20 ms filesystem remove retry; no drive wait, no token in reach", ), @@ -536,17 +536,11 @@ fn no_is_finished_spins_outside_halt() { // The `#[allow]` / `#[expect]` sites present when ST-X2 landed, per (file, attribute): // each is a §5.8 violation awaiting a per-site fix. The list only shrinks. const ALLOW_BASELINE: &[(&str, &str, usize)] = &[ - ("src/aacs/provider.rs", r#"allow(dead_code)"#, 1), ( "src/disc/extract.rs", r#"allow(clippy::too_many_arguments)"#, 1, ), - ( - "src/disc/extract.rs", - r#"allow(clippy::unnecessary_cast,clippy::useless_conversion)"#, - 1, - ), ("src/drive/linux.rs", r#"allow(dead_code)"#, 1), ("src/drive/mod.rs", r#"allow(dead_code)"#, 1), ( @@ -554,6 +548,11 @@ const ALLOW_BASELINE: &[(&str, &str, usize)] = &[ r#"cfg_attr(not(target_os="linux"),allow(unused_variables))"#, 1, ), + ( + "src/io/tree_sink.rs", + r#"allow(clippy::unnecessary_cast,clippy::useless_conversion)"#, + 1, + ), ( "src/keys/arrival.rs", r#"allow(clippy::too_many_arguments)"#, @@ -592,11 +591,8 @@ const ALLOW_BASELINE: &[(&str, &str, usize)] = &[ ("src/mpls.rs", r#"allow(dead_code)"#, 2), ("src/mux/demux_sink.rs", r#"allow(dead_code)"#, 2), ("src/mux/demux_thread.rs", r#"allow(dead_code)"#, 1), - ("src/mux/fmp4/mod.rs", r#"allow(dead_code)"#, 1), - ("src/mux/hevc/mod.rs", r#"allow(dead_code)"#, 1), - ("src/mux/m2ts_mux/mod.rs", r#"allow(dead_code)"#, 1), ("src/mux/mkv.rs", r#"allow(clippy::too_many_arguments)"#, 3), - ("src/mux/mod.rs", r#"allow(dead_code)"#, 6), + ("src/mux/mod.rs", r#"allow(dead_code)"#, 3), ("src/mux/pipelined_stream.rs", r#"allow(dead_code)"#, 1), ( "src/mux/resolve.rs", diff --git a/tests/spec_sites.txt b/tests/spec_sites.txt index 573e7fcc..850319a5 100644 --- a/tests/spec_sites.txt +++ b/tests/spec_sites.txt @@ -31,7 +31,7 @@ src/drive/mod.rs note_ledger SS-7 src/io/artifact_lock.rs acquire_ids SS-8 src/io/artifact_lock.rs acquire_ids SS-9 src/io/artifact_lock.rs acquire_ids SS-10 -src/keys/resolve.rs resolve_observed KS-14 +src/keys/resolve.rs acquire KS-14 src/keys/resolve.rs borrow_samples KS-10 src/keys/resolve.rs apply_single_unit_rule KS-14 src/keys/arrival.rs decrypt_proven KS-5 diff --git a/tests/streams.rs b/tests/streams.rs index a6a51181..48b4675a 100644 --- a/tests/streams.rs +++ b/tests/streams.rs @@ -1,7 +1,7 @@ //! Integration tests for the PES stream pipeline. use libfreemkv::mux::meta::M2tsMeta; -use libfreemkv::pes::Stream as PesStream; +use libfreemkv::pes::PesSink; use libfreemkv::*; use std::io::{Cursor, Write}; @@ -120,7 +120,11 @@ fn parse_url_m2ts_relative() { #[test] fn open_input_bare_path_errors() { - let result = libfreemkv::input("Movie.mkv", &libfreemkv::InputOptions::default()); + let result = libfreemkv::input( + "Movie.mkv", + &libfreemkv::InputOptions::default(), + &libfreemkv::Ctx::default(), + ); assert!(result.is_err()); let msg = match result { Err(e) => e.to_string(), @@ -143,7 +147,11 @@ fn open_output_bare_path_errors() { #[test] fn open_input_m2ts_empty_path_errors() { - let result = libfreemkv::input("m2ts://", &libfreemkv::InputOptions::default()); + let result = libfreemkv::input( + "m2ts://", + &libfreemkv::InputOptions::default(), + &libfreemkv::Ctx::default(), + ); assert!(result.is_err()); let msg = match result { Err(e) => e.to_string(), @@ -154,7 +162,11 @@ fn open_input_m2ts_empty_path_errors() { #[test] fn open_output_null_input_errors() { - let result = libfreemkv::input("null://", &libfreemkv::InputOptions::default()); + let result = libfreemkv::input( + "null://", + &libfreemkv::InputOptions::default(), + &libfreemkv::Ctx::default(), + ); assert!(result.is_err()); let msg = match result { Err(e) => e.to_string(), @@ -177,7 +189,11 @@ fn open_output_disc_errors() { #[test] fn open_input_network_no_port_errors() { - let result = libfreemkv::input("network://192.0.2.1", &libfreemkv::InputOptions::default()); + let result = libfreemkv::input( + "network://192.0.2.1", + &libfreemkv::InputOptions::default(), + &libfreemkv::Ctx::default(), + ); assert!(result.is_err()); let msg = match result { Err(e) => e.to_string(), @@ -284,12 +300,12 @@ fn m2ts_stream_write_read() { data: vec![i; 100], duration_ns: None, }; - PesStream::write(&mut stream, &frame).unwrap(); + PesSink::write(&mut stream, &frame).unwrap(); } - PesStream::finish(&mut stream).unwrap(); + PesSink::finish(&mut stream).unwrap(); // Verify the info is correct - let info = PesStream::info(&stream); + let info = PesSink::info(&stream); assert_eq!(info.streams.len(), 4); assert_eq!(info.duration_secs, 7200.0); } @@ -332,19 +348,11 @@ fn m2ts_implements_pes_stream() { let output = Cursor::new(Vec::new()); let stream = M2tsStream::create(output, &dt).unwrap(); - let boxed: Box = Box::new(stream); + let boxed: Box = Box::new(stream); let meta = boxed.info(); assert_eq!(meta.streams.len(), 4); } -#[test] -fn m2ts_read_returns_error_on_write_stream() { - let dt = sample_disc_title(); - let output = Cursor::new(Vec::new()); - let mut stream = M2tsStream::create(output, &dt).unwrap(); - assert!(PesStream::read(&mut stream).is_err()); -} - // ── DiscTitle::empty ────────────────────────────────────────── #[test] @@ -598,11 +606,11 @@ fn mkvstream_write_finish() { data: vec![i; 100], duration_ns: None, }; - PesStream::write(&mut stream, &frame).unwrap(); + PesSink::write(&mut stream, &frame).unwrap(); } // finish should not panic even without valid codec data - PesStream::finish(&mut stream).unwrap(); + PesSink::finish(&mut stream).unwrap(); } #[test] @@ -611,7 +619,7 @@ fn mkvstream_meta_sets_title() { let writer: Box = Box::new(Cursor::new(Vec::new())); let stream = MkvStream::create(writer, &dt, None).unwrap(); - let info = PesStream::info(&stream); + let info = PesSink::info(&stream); assert_eq!(info.playlist, "Test Movie"); assert_eq!(info.duration_secs, 7200.0); assert_eq!(info.streams.len(), 4); @@ -662,13 +670,13 @@ fn mkvstream_roundtrip_bdts() { data: vec![i; 100], duration_ns: None, }; - PesStream::write(&mut stream, &frame).unwrap(); + PesSink::write(&mut stream, &frame).unwrap(); } - PesStream::finish(&mut stream).unwrap(); + PesSink::finish(&mut stream).unwrap(); // Verify the info is correct - let info = PesStream::info(&stream); + let info = PesSink::info(&stream); assert_eq!(info.streams.len(), 1); assert_eq!(info.playlist, "Audio Only"); } @@ -740,7 +748,7 @@ fn mkvstream_meta_preserves_all_streams() { let writer: Box = Box::new(Cursor::new(Vec::new())); let stream = MkvStream::create(writer, &dt, None).unwrap(); - let info = PesStream::info(&stream); + let info = PesSink::info(&stream); assert_eq!(info.streams.len(), 5, "all 5 streams should be preserved"); assert_eq!(info.playlist, "Stream Test"); assert_eq!(info.duration_secs, 3600.0); @@ -862,7 +870,7 @@ fn mkvstream_e2e_h264_produces_valid_mkv() { data: es_data, duration_ns: None, }; - PesStream::write(&mut stream2, &frame1).unwrap(); + PesSink::write(&mut stream2, &frame1).unwrap(); // Write a second non-IDR frame let frame2 = libfreemkv::pes::PesFrame { @@ -875,8 +883,8 @@ fn mkvstream_e2e_h264_produces_valid_mkv() { data: es_data2, duration_ns: None, }; - PesStream::write(&mut stream2, &frame2).unwrap(); - PesStream::finish(&mut stream2).unwrap(); + PesSink::write(&mut stream2, &frame2).unwrap(); + PesSink::finish(&mut stream2).unwrap(); let data = output2.lock().unwrap().clone().into_inner();