Compare commits

...
50 Commits
Author SHA1 Message Date
Matthew Jackson 6980b04e63 v1.8.1: bump version 2026-10-07 03:56:38 +00:00
Matthew Jackson 777cd6275c docs: 1.8.1 release notes 2026-10-06 20:19:16 -07:00
Matthew Jackson 4d8cba6ff8 feat: report each drive's OS display name
DriveInfo gains display_name: the drive letter on Windows, /dev/srN on
Linux and diskN on macOS while a disc is mounted, or the path when the OS
has no name. path stays the selector to open. Windows lists lettered drives
in letter order, then unlettered drives by CdRomN.
2026-10-06 18:21:41 -07:00
Matthew Jackson 76ddaf5538 style: no new #[allow] for the Windows enumeration helpers
Name STORAGE_DEVICE_NUMBER's fields in snake_case (repr(C) layout is
unchanged) and compile the 36-byte INQUIRY retry only on Windows, which keeps
lint_stop's #[allow] baseline.
2026-10-06 16:53:57 -07:00
Matthew Jackson 0672db23e7 test: allow the allow-path modules' #[path] test side files in the key guard
0b146ef7 moved decrypt.rs and sector/decrypting.rs tests into
decrypt_tests.rs, decrypt_spec_guards_tests.rs and sector/decrypting_tests.rs;
they are the same modules' tests, so they share the allow-path.
2026-10-06 16:28:12 -07:00
Matthew Jackson 6621987c8a style: keep the Linux enumeration comment within the 3-line limit 2026-10-06 16:26:21 -07:00
Matthew Jackson 76507e677d fix: enumerate optical drives on Linux and macOS without limits
Linux also lists optical sr block devices that have no sg node (sg module
not loaded), and without sysfs reads /dev for sg nodes instead of probing
sg0..15. macOS grows its listing buffer instead of stopping at 8 drives.
Changelog covers the Windows enumeration fix as well.
2026-10-06 15:49:35 -07:00
Matthew Jackson c3ceb62f5f feat(macos): log native transport open stages for diagnostics
Report the selected IOKit service, plug-in metadata, each interface
initialization stage, exclusive-access attempts and cleanup results through
tracing, so a failed drive open shows which step failed.
2026-10-06 15:21:02 -07:00
Matthew Jackson d1e0a8a98c fix(windows): list every optical drive without guessing device numbers
Enumeration probed only \\.\CdRom0..15 and fell back to drive letters only
when that found nothing, so drives numbered above 15 (common after USB
replugging) were silently missing. List every CdRomN from QueryDosDeviceW,
add optical drive letters (GetDriveTypeW) not already covered by their
device number, retry INQUIRY with the standard 36 bytes for strict USB
bridges, and log each skipped device with its reason. Also syncs the stale
Cargo.lock with the 1.8.0 manifest.
2026-10-06 15:15:10 -07:00
Matthew Jackson 0b146ef784 Move tests to side files 2026-10-05 20:22:40 -07:00
Matthew Jackson a69a8d9bbd fix(macos): keep optical drive identity stable during exclusive access 2026-10-05 13:04:40 -07:00
Matthew Jackson 92847607b9 v1.8.0: bump version 2026-10-05 02:17:00 +00:00
Matthew Jackson 0a96b1ca81 changelog: 1.8.0 release notes 2026-10-04 18:28:23 -07:00
Matthew Jackson 561a757b60 dvd, uhd, aacs, hddvd: navigation-pack playback, HDR metadata, per-pack HD DVD AACS
DVD titles play the way their navigation packs say: interleaved cells
as their own units, chapter names from the VMG, region mask, IFO stream
labels, a whole-title forced-subtitle probe, DvdPs-only stages. UHD
keeps mastering metadata and hdr_plus_flag and fixes dvcC. HD DVD
decrypts AACS per pack by each EVOBU's CPI and joins PEVOB features;
Class II MKBs are reported.
2026-10-04 12:25:45 -07:00
Matthew Jackson f745a6f2e6 video: drop pictures a decoder cannot reconstruct at a start, join or resync
MPEG-2, H.264, HEVC and VC-1 drop pictures whose references are missing
at a title start, clip join, gap resync or a trimmed clip mark; the
M2TS sink rides the clip-join timeline. An image extent's sub-unit tail
is judged by the decrypt stage instead of refused (E9030).
Wall-clock-sensitive tests hardened; monthly grouped dependabot.
2026-10-02 20:24:26 -07:00
Matthew Jackson f855d604ae audit: second-round library audit fixes
Fixes and tests from the second audit across bdnav, labels, ifo, codec
parameter sets, mkv/mp4/stdio, css, aacs, keysource, keys, decrypt and
sector code. An HD DVD copy already in the clear needs no key, with one
encryption decision for every AACS format; image and folder titles are
named by the meta title, else the playlist, as in 1.7.7.
2026-10-01 22:46:14 -07:00
Matthew Jackson 46ffac8690 pipeline: one run context, KeyRing, Source, Read stage, block and tree sinks
One run context (halt, events, loss counters) serves every stage; keys
come from KeyEvidence into a KeyRing. open_source opens every input,
disc:// included, replacing MuxSource; PesSource/PesSink replace
pes::Stream; one Read stage replaces DiscStream and the byte
prefetcher. iso://, null:// and dir:// are sinks. Dead error variants
(E9009, E9063, E9082) and scaffolding removed.
2026-10-01 08:34:14 -07:00
Matthew Jackson 81ecc8acce decrypt: one content-detected decryption stage; keep going past damage
Every input decrypts through one stage that detects its content, with
strict stale-CPI handling and partial units only at EOF. Damage no
longer ends a rip where usable content remains (E7031, UDF FID,
MPLS/CLPI truncation, whole-disc E7032). M2TS output declares tracks in
an HDMV PMT and paces PCR; network:// allows LAN and loopback and
is_blocked_ip is public; parity goldens pin today's outputs.
2026-09-30 20:11:59 -07:00
Matthew Jackson cf2171e11a mux: MKV read-back, FMKV header metadata, live-stream stop handling
MKV read-back carries display shape, colour, frame rate, content
encodings and BCP47 language, and drops malformed blocks with a count.
The FMKV header carries display aspect, CICP, labels, chapters and
format. Live reads refuse a zero read batch (E9085) and treat a stopped
read as a stop. freemkv-unlock becomes a version dep with a git-tag
override; dependency bumps (#27, #28); LPCM keeps its source bit depth; 1.8.0
changelog.
2026-09-30 18:45:38 -07:00
Matthew Jackson 81b7e3ab4e ts: robust transport-stream demux
Skips duplicate packets, gathers PES headers that span packets, resyncs
after byte slips and across zero-filled gaps, prefers a current
CRC-valid PAT/PMT, and drops the open PES on transport_error_indicator.
M2TS output remaps PIDs into the BD ranges.
2026-09-30 15:52:06 -07:00
Matthew Jackson 5b9dd0fbec error, mux: coded remux errors; MKV writer hardening; damage rules for key checks
E9077-E9084 for engine remux and preflight refusals; MuxOptions loses
its send deadline. The MKV writer drops unwrap/expect and overflow
paths, tags SDR BT.2020 correctly, leaves out streams with no Matroska
CodecID and derives the Dolby Vision level. On-arrival damage with no
vouching partner is blanked, and a garbled head is damage, not a stop.
2026-09-30 15:50:46 -07:00
Matthew Jackson 4031513574 audit: module-by-module code audit fixes and test coverage
Lands the code-audit branches for aacs, css, keys, labels, udf, ifo and
dvd, bdnav/mpls/clpi, disc/bluray, extract, io, scsi, the mkv, mp4,
network and mpg muxers, the audio and video codecs, and CI. Fixes
include MKB and FMTS validation, UDF bounds, IFO fallback to the BUP,
DTS Express framing, MPEG-2 field pairing and mpg re-basing; the rest
pins existing behaviour with tests.
2026-09-29 20:45:05 -07:00
Matthew Jackson 1d78420366 keys: read damage is never a key failure; legacy key API removed
Damaged AACS units are holed, blanked and counted on every path instead
of stopping with E7013; failed batch partners fall back to side reads
and a wrong FMTS key is judged per key. ResolvedKeySet is the only AACS
decrypt door; a raw whole-disc reader never decrypts and refuses titles
with no stream folder (E6003) or a piece no held key opens (E7032).
2026-09-29 17:20:43 -07:00
Matthew Jackson 25fd393c04 session, io: Stop through held sessions, kept partials, <final>.lock
Key-source contexts carry the op token and progress; DiscSession gains
open_with, scan_with and finish on the held handle. A consumer that
closes after a cancel keeps *.partial, and an artifact lock sidecar
guards each final output. Stop rules are enforced as lint tests.
2026-09-28 09:11:34 -07:00
Matthew Jackson fdab162bc4 mux: mpg:// program-stream output and source
A container-neutral fit plan, then mpg:// as an H.222.0 program-stream
sink on the PES IR and as a source through the DVD sector pipeline. PES
and AU placement, start-up hold caps and 20/24-bit LPCM timing follow
the spec; a clear MPEG-1/2 PS is never read as scrambled and CSS is
judged per pack. E9074 for an output with no video track.
2026-09-28 07:25:53 -07:00
Matthew Jackson 9000c83c1b keys: one resolved key set per rip, proven on first read
ResolvedKeySet resolves keys up front, forensic keys included, and every
read surface decrypts through it. A keyless session never muxes
ciphertext; a single declared key keys only the pieces it opens; held
keys are tried before side reads. Resolve reports whether the disc's
VID would help, returns its trace on refusal, and never codes an
unreachable key service as Missing.
2026-09-28 07:18:11 -07:00
Matthew Jackson 1baf9ba061 halt: Stop-aware waits across channels, drive, io, flush and the macOS shim
Waits are halt-aware on a stall timer; every CDB carries the op token;
prefetchers and pipeline joins honour Stop; flush is durable with
measured progress; the macOS shim open is cancellable. Also: CPI is
cleared on every decrypted unit, FMTS phase verification, the HD DVD
declared CPS count, AES work hoisted out of AACS hot paths, and the
last production unsafe removed.
2026-09-28 03:57:56 -07:00
Matthew Jackson 3617ccf733 disc, audio: record unlocatable bus-encrypted files; resync without losing good frames
A bus-encrypted stream file whose File Entry cannot be located is
re-read, then recorded rather than failing the scan; MKV staging ranges
and E6022 for a partial image. Audio framing chains headers while
resyncing, reports every lost slot, keeps good frames beside corrupt
ones and places locks by PTS, with ADTS and MPEG property fuzzing.
2026-09-28 01:11:12 -07:00
Matthew Jackson cbd3db9d0e spec, test_util: specification quote registry, spec guards, test-util fixtures
libfreemkv::spec holds the verbatim specification quotes the code
cites, with guards tying each governed site to its quote. A test-util
feature exposes shared fixtures to downstream tests. M2TS output
derives DTS for reordered video and rounds to whole 90 kHz ticks.
2026-09-28 00:50:09 -07:00
Matthew Jackson 3207856c17 dvd, mux: route DVD audio and subtitles by the PGC tables; MP2 multichannel
DVD subtitles and audio route by PGC SPST_CTL and AST_CTL instead of
stream ordinal, and MPEG audio is fed from its PES id. The DVD MPEG-2
extension stream rides the IR with its channel count verified by CRC.
Audio framing stops false syncs poisoning a track; PGS and DVD subtitle
parser fixes; E9073 and E7034 declared.
2026-09-28 00:31:27 -07:00
Matthew Jackson 18982a7c0e disc, aacs: key files before the handshake; one whole-disc reader
Scan reads UDF and the AACS files before the drive handshake and records
a missing DVD /AACS key file instead of failing. Decryption needs an
explicit AACS unit base and refuses off-grid units; an all-broken
host-cert list is its own error (E7033). Every whole-disc and image to
ISO copy goes through one shared reader that refuses an unkeyable file
up front (E7032). HD DVD XPL tick and timecode fixes; CI action bumps.
2026-09-27 20:17:01 -07:00
Matthew Jackson 503a80884e mux: sink and reader audit fixes; DTS in MP4; halt-aware network receive
Fixes from the mkv, mp4, network and ts sink audit; track timing and
DiscardPadding ride the FMKV wire. MP4 writes DTS per ETSI TS 102 114
(ddts rates and timescales). The MKV PCM depth probe is bounded and
evidence-based. Network receives use keepalive and wait on readiness,
so a Stop or a dead peer ends them.
2026-09-27 12:40:50 -07:00
Matthew Jackson d6396005b8 scsi, lpcm: MMC-6 disc presence and tray handling; LPCM decoded to plain PCM
Disc presence is tri-state per MMC-6, an empty drive fails fast, a
cleaning cartridge reads as no disc, and the tray is unlocked only when
we locked it. BD and DVD LPCM decode to plain PCM, with whole-frame DVD
blocks and the BD layout carried through M2TS.
2026-09-27 11:54:34 -07:00
Matthew Jackson 8c707efbe1 nav, labels: navigation command semantics, BD-J feature detection, CLPI audit
DVD and HDMV navigation commands follow the reference semantics
(SetGPRMMD, set-links, saturating ADD/MUL, unmasked shifts). Label
detection bounds the BD-J menu walk, opens large jars in place and
tightens Paramount, Deluxe and BD-J feature evidence. The CLPI audit
keys streams by clip and PID; channel layouts keep the LFE split and
TrueHD counts for unnamed layouts.
2026-09-27 11:45:48 -07:00
Matthew Jackson 4c5098ce75 keys, disc, io: bus-encryption gates, halt-aware key paths, writeback error latching
Read-time key fetches are validated and source, halt and drive faults
surface. Every BDMV/STREAM file is bus-decrypted per the BEE flag and
unit copy-permission bits; a missing or unparseable cert fails safe.
Extract treats a drive Stop as a halt, not a hole. io latches writeback
errors, fixes the read eviction window and never waits on writeback
during seeks.
2026-09-26 19:53:08 -07:00
Matthew Jackson aa85906a87 udf, mux: post-1.7.7 fixes for UDF chains, AAC headers and codec framing
UDF validates allocation-descriptor chains, maps fragmented metadata
files and tolerates broken subdirectory chains. The muxer holds headers
until every AAC track has its AudioSpecificConfig and backpatches a late
CodecPrivate. HEVC PPS tables are bounded, DTS EXSS aligned and MPEG
free-format frames capped. A no-media drive is released before the next
candidate opens; CI builds freemkv's app as a consumer.
2026-09-26 19:29:26 -07:00
Matthew Jackson ee25d50c11 restore freemkv-unlock path dep for local dev (post-v1.7.7) 2026-09-26 21:16:58 +00:00
Matthew Jackson c8e67f16b4 v1.7.7: bump version (freemkv-unlock git-pinned for the tag) 2026-09-26 21:16:54 +00:00
Matthew Jackson efc9bd705d Link the website FVI specification and simplify PGS comments 2026-09-26 11:58:27 -07:00
Matthew Jackson 68d27be8f8 Clarify send deadline Rustdoc contract 2026-09-26 11:50:01 -07:00
Matthew Jackson a42241eb15 Prepare 1.7.7 with concise comments and shared CI policy 2026-09-26 11:46:32 -07:00
Matthew Jackson 5520890550 Update Scorecard action to use supported GHCR image 2026-09-26 11:23:38 -07:00
Matthew Jackson d0c9df91ef Fix PGS clear events and add FFmpeg QA interoperability gate 2026-09-26 10:47:57 -07:00
Matthew Jackson 1363ba8510 restore freemkv-unlock path dep for local dev (post-v1.7.6) 2026-09-26 04:05:41 +00:00
Matthew Jackson 3f156f1f03 v1.7.6: bump version (freemkv-unlock git-pinned for the tag) 2026-09-26 04:05:38 +00:00
Matthew Jackson df43373f41 docs: neutral wording in changelog [skip-media] 2026-09-25 17:10:05 -07:00
Matthew Jackson a797c544f1 docs: say test corpus, not hoard [skip-media] 2026-09-25 16:48:38 -07:00
Matthew Jackson 584cc6030b tests(macos): hardware drive checks (INQUIRY, reopen, UHD AACS2 state, rescan stability, probe/scan agreement) + non-hardware WritebackFile/preallocate tests 2026-09-25 13:44:22 -07:00
Matthew Jackson 7257559153 chore: absorb qa into dev for lockstep promote [skip-media] 2026-09-25 12:25:45 -07:00
Matthew Jackson 8949b81dfc macos(drive): discover empty-tray drives via IOKit registry-id selectors
Empty-tray optical drives couldn't be opened because the macOS backend
resolved drives by media-child `BSD Name`, which doesn't exist without a
disc loaded. Result: `list_drives()` returned nothing for a drive with no
media, and any attempt to `open("disc://")` on that state failed.

Fix: enumerate `IOBDServices` directly. With media present, the returned
selector is the familiar `/dev/diskN` path (unchanged). With no media,
the selector is an opaque `ioreg:<registry-id>` string that resolves to
the same IOBDServices entry — Disk Arbitration and unmount are skipped
because there's no media node.

- `src/drive/macos.rs`, `src/drive/mod.rs`, `src/scsi/mod.rs`: threads
  the selector through the transport so empty-tray drives are openable
  without punning it as a filesystem path.
- `src/scsi/macos.rs`, `src/scsi/macos_shim.c`: the C shim now accepts
  both `/dev/diskN` and `ioreg:<id>` in `shim_open_exclusive`, and
  `shim_list_drives` reports the correct selector for each drive.
- `tests/macos_drive_tests.rs` (new): covers empty-tray discovery.
- `docs/drive-access.md`, `docs/scsi-macos.md`: document the two-path
  selector model.

[skip-media]
2026-09-25 12:03:02 -07:00
662 changed files with 192797 additions and 139408 deletions
+31 -61
View File
@@ -1,63 +1,33 @@
# Version updates are monthly and grouped into one PR per ecosystem.
# Security updates are separate and run whenever an advisory is published.
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: monthly
open-pull-requests-limit: 5
groups:
all:
patterns:
- '*'
ignore:
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: monthly
open-pull-requests-limit: 5
groups:
all:
patterns:
- '*'
ignore:
- dependency-name: dtolnay/rust-toolchain
- dependency-name: freemkv/.github*
+66 -35
View File
@@ -19,10 +19,11 @@ jobs:
# cannot drift. clippy and `consumers` deliberately run on `stable`.
resolve-msrv:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
@@ -32,8 +33,9 @@ jobs:
# whose entry isn't written yet is not blocked.
changelog:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
@@ -55,21 +57,25 @@ jobs:
lint:
needs: resolve-msrv
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@stable
with:
toolchain: stable
components: clippy
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: rustfmt
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
@@ -80,23 +86,29 @@ jobs:
test:
needs: resolve-msrv
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- run: cargo test --tests
working-directory: libfreemkv
# `--tests` skips doctests; the keys compile_fail doctests enforce spec LK19.
- run: cargo test --doc
working-directory: libfreemkv
# The memory orderings in `scsi::fd_handoff` compile to the same instructions
# The memory orderings in `scsi::fd_handoff` (and `halt`) compile to the same instructions
# on x86_64, so the WRONG ones pass every run; loom enumerates the executions
# the C++20 model permits and is the only regression coverage that code has.
# Its own job because RUSTFLAGS is part of the rust-cache key. Takes the sibling
@@ -105,24 +117,30 @@ jobs:
loom:
needs: resolve-msrv
runs-on: ubuntu-latest
timeout-minutes: 30
env:
RUSTFLAGS: --cfg loom
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: clippy
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- run: cargo test --lib --no-default-features --features scsi fd_handoff
working-directory: libfreemkv
# LT3: `Halt`'s cancel Release / observe Acquire pair (stop design §2.1, SS-23).
- run: cargo test --lib --no-default-features --features scsi loom_halt
working-directory: libfreemkv
# `lint` runs clippy without --cfg loom, so this is the only place the
# model code is linted at all.
- run: cargo clippy --lib --no-default-features --features scsi --profile test -- -D warnings
@@ -130,18 +148,23 @@ jobs:
deny:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@stable
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check advisories bans licenses sources
manifest-path: libfreemkv/Cargo.toml
toolchain: stable
- uses: taiki-e/install-action@cargo-deny
# Run from the repo dir so the generated .cargo/config.toml path patches apply.
- run: cargo deny --all-features check advisories bans licenses sources
working-directory: libfreemkv
# dev is the FAST lane: the platform jobs run on qa/main only (qa.yml also
# covers macOS/Windows). `if` SKIPS (does not queue) so a never-scheduled job
@@ -150,17 +173,20 @@ jobs:
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- run: cargo check
@@ -170,17 +196,20 @@ jobs:
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
# Build the tests: forces full codegen of the Windows-only SPTI transport
@@ -188,23 +217,24 @@ jobs:
- run: cargo build --tests
working-directory: libfreemkv
# Did this change break anything downstream? libfreemkv sits below all five
# Did this change break anything downstream? libfreemkv sits below all four
# dependents; a break here is worth strictly more than anywhere else. Runs
# their suites (not just check) — a behavioural change keeps signatures but
# fails a dependent's tests, which is the shape worth catching. This caught an
# engine fixture break the moment it landed, so it stays on the dev fast lane.
consumers:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with: { path: libfreemkv }
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock freemkv-keysources freemkv-engine freemkv-i18n freemkv autorip bdemu
checkout: freemkv-unlock freemkv-keysources freemkv-engine freemkv-i18n freemkv bdemu
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
for c in freemkv-keysources freemkv-engine freemkv autorip bdemu; do
for c in freemkv-keysources freemkv-engine freemkv bdemu; do
mkdir -p "$c/.cargo"
cat > "$c/.cargo/config.toml" <<'EOF'
[patch.crates-io]
@@ -212,14 +242,11 @@ jobs:
freemkv-keysources = { path = "../freemkv-keysources" }
freemkv-engine = { path = "../freemkv-engine" }
freemkv-i18n = { path = "../freemkv-i18n" }
freemkv-unlock = { path = "../freemkv-unlock" }
EOF
done
# Since freemkv 1.7.6 the `freemkv` consumer's Linux target compiles a
# GTK4 + libadwaita desktop shell, so `cargo test` on it needs the
# pkg-config dev headers the `gtk4-sys`/`glib-sys` build.rs scripts
# probe. Kept local (not in the shared reusable) — only freemkv links
# against GTK; adding the deps to every consumer would tax the six
# that don't.
# freemkv's app build (`--features gui`) compiles a GTK4 + libadwaita
# shell, so testing it needs these pkg-config dev headers.
- name: Install GTK4 + libadwaita development headers (for freemkv)
run: |
sudo apt-get update
@@ -228,13 +255,14 @@ jobs:
# satisfies the highest dependent floor (the library's own lower MSRV, which
# the `test` job verifies, cannot compile them).
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
toolchain: stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: |
freemkv-keysources
freemkv-engine
freemkv
autorip
bdemu
- run: cargo test --tests
working-directory: freemkv-keysources
@@ -242,8 +270,8 @@ jobs:
working-directory: freemkv-engine
- run: cargo test --tests
working-directory: freemkv
- run: cargo test --tests
working-directory: autorip
- run: cargo test --tests --features gui
working-directory: freemkv
- run: cargo test --tests
working-directory: bdemu
@@ -251,24 +279,27 @@ jobs:
coverage:
needs: resolve-msrv
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- uses: taiki-e/install-action@cargo-llvm-cov
- uses: taiki-e/install-action@72f835688cb6c15903b796f704b01e652a2b8835 # cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: libfreemkv
- uses: codecov/codecov-action@v7
- uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: libfreemkv/lcov.info
+1 -1
View File
@@ -9,4 +9,4 @@ concurrency:
jobs:
comment-guard:
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: freemkv/.github/.github/workflows/guard-comment.yml@v1
uses: freemkv/.github/.github/workflows/guard-comment.yml@v2
+4 -3
View File
@@ -1,9 +1,9 @@
name: doc-links
# Thin caller → the canonical reusable doc-links gate in freemkv/.github.
# libfreemkv path-deps freemkv-unlock, so the reusable checks it out as a sibling
# (no [patch] needed — the layout resolves the path dep) and resolves the
# toolchain from rust-version.
# libfreemkv deps freemkv-unlock by version (git-tag override), so the reusable
# checks it out as a sibling and path-patches it, and resolves the toolchain
# from rust-version.
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -15,3 +15,4 @@ jobs:
with:
self: libfreemkv
checkout: freemkv-unlock
patch: freemkv-unlock
+58
View File
@@ -0,0 +1,58 @@
name: FFmpeg validation
# Called by qa.yml on pushes to qa, or manually for a development branch.
# Pure Rust tests remain in dev CI; these tests require external codecs.
on:
workflow_call:
workflow_dispatch:
permissions:
contents: read
jobs:
validate:
runs-on: ubuntu-24.04
timeout-minutes: 20
env:
FREEMKV_FFMPEG_ARTIFACT_DIR: ${{ github.workspace }}/ffmpeg-artifacts
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- name: Read the crate's MSRV
id: msrv
working-directory: libfreemkv
run: echo "v=$(sed -n 's/^rust-version = "\(.*\)"/\1/p' Cargo.toml)" >> "$GITHUB_OUTPUT"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.msrv.outputs.v }}
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- name: Install software encoders and decoders
run: sudo apt-get update && sudo apt-get install -y ffmpeg
- name: Record tool versions
run: |
mkdir -p "$FREEMKV_FFMPEG_ARTIFACT_DIR"
ffmpeg -version > "$FREEMKV_FFMPEG_ARTIFACT_DIR/ffmpeg-version.txt"
ffprobe -version > "$FREEMKV_FFMPEG_ARTIFACT_DIR/ffprobe-version.txt"
rustc --version > "$FREEMKV_FFMPEG_ARTIFACT_DIR/rustc-version.txt"
- name: Generate fixtures and validate libfreemkv output
working-directory: libfreemkv
shell: bash
run: |
set -o pipefail
cargo test --release --lib ffmpeg_ -- --ignored --nocapture 2>&1 | tee "$FREEMKV_FFMPEG_ARTIFACT_DIR/tests.log"
- name: Upload fixtures, probes, hashes and logs
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: ffmpeg-validation-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ env.FREEMKV_FFMPEG_ARTIFACT_DIR }}
if-no-files-found: warn
retention-days: 14
+42 -14
View File
@@ -33,10 +33,11 @@ jobs:
# clippy (cross-lint) runs on stable instead.
resolve-msrv:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
@@ -56,11 +57,14 @@ jobs:
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
# A manual dispatch from another branch must not stamp a candidate.
if: github.ref_name == 'qa'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
fetch-depth: 0
- name: Stamp the next rc
@@ -71,10 +75,16 @@ jobs:
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
# Retry on a lost race for the same ordinal: refetch tags, recompute.
for attempt in 1 2 3 4 5; do
git fetch --tags --force origin
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag" && break
git tag -d "$tag"
[ "$attempt" -lt 5 ] || { echo "could not push an rc tag" >&2; exit 1; }
done
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
@@ -88,44 +98,59 @@ jobs:
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- run: cargo test --release --tests
working-directory: libfreemkv
- run: cargo test --release --doc
working-directory: libfreemkv
# Independent decoder/remux verification uses generated media and belongs
# on qa, alongside the release suite. Also runnable manually on a branch.
ffmpeg-validation:
uses: ./.github/workflows/ffmpeg-validation.yml
permissions:
contents: read
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
# clippy runs on stable (matches ci.yml's lint job); build/test stay on
# the MSRV pin. See ci.yml for why the lint toolchain need not be the MSRV.
- uses: dtolnay/rust-toolchain@stable
with:
toolchain: stable
components: clippy
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
- run: cargo clippy --all-targets -- -D warnings
working-directory: libfreemkv
# Windows compiles the tests but does not run them, matching the policy
@@ -135,17 +160,20 @@ jobs:
windows-build:
needs: resolve-msrv
runs-on: windows-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: libfreemkv
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: freemkv-unlock
patch: freemkv-unlock
self: libfreemkv
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
workspaces: libfreemkv
- run: cargo build --release --tests
+15 -10
View File
@@ -11,13 +11,14 @@ on:
- '!v*-rc*'
permissions:
contents: write
contents: read
jobs:
verify:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -28,23 +29,24 @@ jobs:
echo "Version match: $CARGO_VER"
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# release job does NOT `needs:` this job. The tag decision was already
# gated by CI (ci.yml on dev, then a green qa.yml on the same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
# the instant this tag exists — so this test job must NOT sit on their
# critical path.
test:
needs: verify
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
# Cargo.lock is now committed (issue #15), but --locked is still omitted:
# the sibling path-patch layout resolves the freemkv crates as path
# packages, which --locked can't satisfy.
@@ -58,12 +60,15 @@ jobs:
release:
# Only needs `verify`; the GitHub Release can be cut as soon as the version
# check passes, in parallel with test + publish.
# check passes, in parallel with test.
needs: verify
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: Create GitHub Release
uses: softprops/action-gh-release@v3
uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3
with:
generate_release_notes: true
+3 -3
View File
@@ -14,14 +14,14 @@ jobs:
security-events: write
id-token: write
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: ossf/scorecard-action@v2.4.0
- uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4
with:
results_file: results.sarif
results_format: sarif
publish_results: true
- uses: github/codeql-action/upload-sarif@v3
- uses: github/codeql-action/upload-sarif@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4
with:
sarif_file: results.sarif
+149 -2
View File
@@ -1,6 +1,153 @@
# Changelog
## [1.7.6] — 2026-09-25
## [1.8.1] — 2026-10-07
- `DriveInfo::display_name` is the name the OS shows users: the drive letter (`E:`) on Windows, `/dev/srN` on Linux and `diskN` on macOS while a disc is mounted; it equals `path` when the OS has no name. `path` remains the selector to open. Windows lists lettered drives in letter order, then unlettered drives by `CdRomN`.
- Drive listing enumerates instead of guessing device numbers. Windows lists every `CdRomN` from `QueryDosDeviceW` plus optical drive letters (previously only `CdRom0`–`15`), retries INQUIRY with 36 bytes for strict USB bridges, and logs skipped devices. Linux also lists optical `sr` devices that have no `sg` node (the `sg` module is not loaded) and, without sysfs, reads `/dev` instead of probing `sg0`–`15`. macOS no longer stops at 8 drives.
- macOS CLI/GUI/server transport diagnostics include selected IOKit service, plug-in metadata, interface initialization stages, exclusive-access attempts and cleanup results.
- macOS optical-drive enumeration keeps the same IOKit service selector with or without mounted media, so exclusive access during a rip does not turn one drive into two identities.
## [1.8.0] — 2026-10-05
### Breaking
- **AACS keys:** `keys::KeyRing` is the only door to AACS keys. Removed: crate-root `AacsKeyMap`, `DecryptKeys`, `resolve_and_apply`, `KeyFetch`; `resolve_and_apply_traced`, `keysource::{fetch_unit_keys, fetch_fmts_indexes, key_fetch}`, `sector::{KeyFetch, KeyFetchFn}`, `session::{resolve_keys_for, ResolvedKeys}`, `DiscSession::{resolve_keys, key_fetch}`, `Disc::{decrypt_keys, decrypt_keys_for_title, decrypt_with, ensure_decryptable, ensure_decryptable_keys, ensure_title_decryptable}`, `InputOptions::{unit_keys, key_fetch}`, `DiscStream::with_key_map`. `DecryptKeys` is `#[non_exhaustive]`; `AacsKeyMap::from_ranges*`, `DecryptingSectorSource::{with_key_map, set_key_map}` and `aacs::content::{decrypt_unit, decrypt_bus}` are no longer public. Acquire once with `DiscSession::resolve_key_set` / `KeyRing::acquire`.
- **Mux entry point:** `mux_stream` and `MuxInput` are replaced by `mux_with_keys(Source, Option<&KeyRing>, dest_url, opts, ctx)`; a `Source` carries no key material. `InputOptions::keys: Option<KeyRing>` replaces `unit_keys` / `key_fetch`; an AACS image opened without a set is refused before any output. Crate-root `build_iso_pipeline` and `resolve_mux_key_map` are removed.
- **One run context for every stage:** `Ctx { halt, events, stats, diag }` is passed to `input(url, opts, ctx)`, `mux_with_keys(.., ctx)`, `Disc::extract_tree(reader, dest, opts, ctx)`, `write_image(reader, dest, sectors, ctx)` and `PrefetchedSectorSource::new(.., ctx)`. One `Events` trait over one `Event` enum (`BytesRead`, `SectorSkipped`, `UnitBlanked`, `BatchSizeChanged`, `Pass`, `BytesWritten`, `BytesDurable`, `OutputOpened`) replaces `MuxEvents`, `event::{Event, EventKind}`, `progress::Progress`, `EventFn` / `PrefetchedSectorSource::new_with_events`, `DiscStream::{on_event, with_halt}` and `Drive::on_event`; `ExtractOptions::{halt, progress}` are removed (Stop is `ctx.halt`; extract progress is `PassKind::Extract`). `Stats` / `LossReport` count read skips, lost bytes, blanked units and resync drops across the run. `halt::Progress` is renamed `halt::Liveness`. `FREEMKV_SKIP_PARSE` / `FREEMKV_PROFILE` take effect only through `Diag::from_env()` on the caller's `Ctx`.
- **Key acquisition is not bound to a scanned disc:** `ResolvedKeySet` is renamed `KeyRing` and `ResolveKeysOptions` `AcquireOptions` (its `halt` is the run's `Ctx`; `resolve_with_progress` is `AcquireOptions::liveness`). `KeyRing::acquire(&KeyEvidence, &mut Sampler, sources, opts, ctx)` takes what a source can tell (`KeyEvidence`: `MediaId`, AACS structures, `Piece`s in scope, main title, `Detector`), and `KeyRing::acquire_for_disc(disc, reader, scope, sources, opts, ctx)` builds that evidence from a scanned disc (`KeyEvidence::from_disc`). `KeyRing::is_for` takes a `&MediaId` (`Disc::media_id()`), which includes the Volume ID fingerprint. Evidence with no disc hash derives it from the title-key file, so a keydb lookup no longer misses silently. The legacy `aacs::resolve` / `aacs::provider` ladder and `disc::Key` are removed; `KeyOrigin` lives in `aacs::trace` (still re-exported at the crate root).
- **One way to open any input:** `open_source(url, ScanOptions, ctx)` opens every input scheme, `disc://` included (the drive is brought up and scanned; `input("disc://…")` now opens it instead of returning `DiscUrlNotDirect`), and `Source::layout()` is the probe's scanned `Disc` for `disc://`, `iso://` and `dir://`. `MuxSource` is removed: `Source::from_session(&mut DiscSession)`, `Source::from_image(path, ScannedTitle)` (a title scanned elsewhere, never rescanned) and `Source::from_reader(reader, ScannedTitle)` replace `Session`, `Iso` and `Live`, and `mux_url(url, keys, dest, opts, ctx)` replaces `Url`. `MuxOptions` gains `title_index` (it replaces `MuxSource::Session::title_index` and `InputOptions::title_index` for a mux), and `MuxOptions::selection` applies to every source. `scan_iso` / `scan_dir` share the one image probe.
- **Sources and sinks are separate traits:** `pes::Stream` (and the `PesStream` re-export) is replaced by `PesSource` (`read`, `info`, `track_timing`, `codec_private`, `headers_ready`, `config_changes`, `errors`, `lost_bytes`) and `PesSink` (`write`, `finish`, `finish_incomplete`, `info`, `set_track_timing`, `set_codec_private`, `undelivered_streams`). `input()` returns `Box<dyn PesSource>` and `output()` `Box<dyn PesSink>`; a wrong-direction call no longer compiles instead of returning `StreamReadOnly` / `StreamWriteOnly`. `MkvStream`, `StdioStream` and `NetworkStream` implement both (and an inherent `info()`); `CountingStream` wraps a `PesSink`. `SinkCaps::of(&StreamUrl)` (`needs_frames`, `carries_mp2_extensions`) is the one table the mux consults for metadata sinks and MPEG-2 extension tracks.
- **One read loop and one demux for every sector source:** `DiscStream` is removed. A live drive is read by the same prefetch producer and demux as an image, under the drive's Read policy (`sector::read_stage::ReadPolicy::Live`: adaptive batches that shrink on a failing zone and regrow after a clean streak, one ECC recovery read for a unit that still fails, then a zero-filled, counted unit under `skip_errors` or a `DiscRead`); an image keeps fixed batches that stop on the first error (`ReadPolicy::Image`). Live reads now overlap demux and parse on their own thread. Read skips are reported in `MuxOutcome::errors` / `lost_bytes` as before.
- **Stop is the only send bound:** `MuxOptions::send_deadline`, `NO_SEND_DEADLINE` and `halt::POLL_INTERVAL` are removed (use `halt::WAIT_SLICE`); a slow sink blocks the mux until it drains or Stop.
- `Drive::clear_halt`, `Drive::scsi_mut` and `Drive::is_unlocked` and the path-based `platform::fs_type::detect` are removed; use `Drive::attach` / `detach` / `halt` / `token` and `DiscSession::source_mut`.
- **Block sinks:** `io::BlockSink` (`write_at(lba, bytes)`, `finish(Finish::{Complete, Incomplete})`) with `IsoSink` (a sector-exact image) and `NullBlockSink`, opened from a URL by `io::open_block_sink("iso://…" | "null://")`; `write_image` writes through `IsoSink`. `io::null_device()` / `is_null_device()` name the platform's null device (`NUL` on Windows), and `Disc::mapfile_for` keys its private mapfile on it, so a whole-disc copy to `null://` works on Windows.
- **Tree sink:** `io::TreeSink` is the `dir://` output, opened by `io::open_tree_sink("dir://…", force)`: host-safe names, collision refusal, `.partial` then rename, and the AACS/CERTIFICATE directories left out. `Disc::extract_into(reader, &mut TreeSink, keys, ctx)` runs the extraction into it; `Disc::extract_tree` is that chain over a sink it creates.
- **One decryption-stage constructor:** `DecryptingSectorSource::new(inner, keying)` takes `impl Into<sector::Keying>` — `DecryptKeys` (as before), a key ring's view, or the content-detected stage — replacing the crate-internal `detecting` constructor and `with_key_map` / `with_arrival` builders.
- **Removed `io::byte_prefetcher`:** `m2ts://` reads through the one Read stage and `PrefetchedSectorSource` (the same reads, in the same ~1 MiB batches, as before); `BytePrefetcher` and its `PrefetchShell` are gone.
- Error variants no path returns any more are removed with their codes: `DiscUrlNotDirect` (E9009; `disc://` opens through `input()`), `DirImageEncrypted` (E9063; an encrypted folder is keyed like its image) and `MultipassRequiresRaw` (E9082; a multipass recovery may decrypt).
- Unused scaffolding is removed: the `io::sink` module (`SequentialSink`, `RandomAccessSink`, `LocalFileSink`, `SocketSink`, `UdpSocketSink`; `WritebackFile` no longer implements them) and the unfinished `fmp4`, `m2ts_mux` and `HevcMux` muxers.
- `DvdAudioAttr::sub_stream_id` (positional id) is removed.
- `M2tsMeta` gains `timings`, `chapters`, `content_format` and `frame_padding` (all `#[serde(default)]`); a struct-literal build of it must name them.
- `css::CrackOutcome` is `#[non_exhaustive]` and gains `Halted` and `Unreadable(Error)`; it no longer derives `Clone`. `css::crack_key` is deprecated for `crack_key_outcome`.
- `AudioChannels` gains 3.0, 3.1, 4.1, 6.0 and 7.0; `SkipReason` is `#[non_exhaustive]`.
- `scsi::linux::SgIoTransport` `fd` / `fd_recovery` are private; `SptiTransport::reset` (Windows) is removed.
- `freemkv-unlock` is a version dependency resolved by a `[patch.crates-io]` git-tag override; consumers must carry the same patch.
### Output-format changes
- **MPEG-2 video starts on a decodable picture:** pictures before the first I-picture, and the leading B-pictures of an open GOP whose forward reference is not in the output (at the start of a title, after a `broken_link` GOP, or where the timeline jumps at a join), are left out instead of showing as a green or corrupt frame. Every kept frame keeps its PTS, so audio sync is unchanged; a title that opens on a closed GOP is unchanged.
- **H.264, HEVC and VC-1 video start on a decodable picture, and every codec resumes cleanly:** pictures before the first random-access picture are left out, and so are leading pictures that predict from a picture the output does not hold: at the start of a title, at a join (the decode clock or PTS moves at the random-access picture), at a VC-1 `BROKEN_LINK`, and at the keyframe the resync gate resumes on after a gap (MPEG-2 too). These are H.264/VC-1 open-GOP leading B-pictures and HEVC RASL pictures (also after a stream-start CRA, which a conforming decoder discards anyway); an MVC dependent-view unit goes with its base picture, and a CRA at any join is written as BLA (before, only after a PTS step back of over 3 s). Where a clip's IN or OUT mark trims a picture another one predicts from, that one goes too, up to the next keyframe (and its leading pictures). Every kept frame keeps its PTS, so audio sync is unchanged; these are not counted as lost data. A closed-GOP or IDR-only title is unchanged.
- **MKV DefaultDuration of MPEG-2 video** follows the kept frames' measured period when that is another standard rate: a soft-telecined NTSC film title (DVD) declares 23.976 fps instead of the sequence's 29.97, so the frame count agrees with the duration. Video at its sequence rate is unchanged.
- **LPCM** (BD and DVD) is decoded to interleaved big-endian PCM in WAVE_FORMAT_EXTENSIBLE channel order (BD pad channel removed, LFE/surround remapped, DVD 20/24-bit groups unpacked) at the source depth: 16-bit stays 16-bit, 20- and 24-bit sources output 24-bit; MKV `BitDepth` matches, and M2TS/MPG output keeps the depth. BD 20-bit packets are kept (24-bit container); reserved header codes are dropped. On M2TS output LPCM is repacked to the BD layout; LPCM BD cannot carry (e.g. 44.1 kHz) is dropped and reported.
- **M2TS:** reordered video (H.264, HEVC, MPEG-2, VC-1) now carries a DTS (H.222.0 2.7.5); PTS round to the nearest 90 kHz tick (a PTS can be 1 tick later than 1.7.7), the first frame lands one second above zero, PCR never decreases with B-frames; output always declares `bd-ts` in its FMKV header. PGS `.sup` ticks use the same rounding.
- **MKV:** DTS, DTS-HD HRA and DTS-HD MA are all `A_DTS`; streams with no Matroska CodecID (text subtitles, unknown codecs) are omitted rather than declared under another codec; SDR BT.2020 video is tagged with the BT.2020 transfer (14), not PQ; the timeline origin is the earliest sample of any selected track (audio and subtitles before the first video keyframe are kept at their true offsets; video starts at its first keyframe), and Blu-ray clip IN applies only to BD-TS sources, whose frames before the IN are dropped and counted; VobSub cues without a stop command end at the next subtitle (10 s cap) instead of 30 s. **MP4:** a track that starts after the earliest sample keeps its offset through an empty edit, and video before its first keyframe is not stored.
- **One decryption stage for every input:** `mpg://`, `m2ts://`, `mkv://`, `mp4://`, `network://` and `stdio://` pass the same content-detected stage as disc/`iso://`/`dir://`. CSS packs are cracked and descrambled, AACS units are decrypted with `InputOptions.keys`, and clear containers pass untouched. An encrypted loose `.m2ts` is decrypted (keys proven on arrival) or refused with E7022 before any output instead of being muxed as ciphertext; `--raw` passes ciphertext. A clear `.m2ts` whose CPI bits were left set still opens, also when a unit has damaged packets (half of them synced suffices). Encrypted content under a clear container's scheme is refused (E7023/E7022). A file whose start is zero-filled is judged from its first written sector; a file judged clear still refuses a scrambled pack (E7023). On `m2ts://` an unflagged unit with no TS structure is blanked and counted in `errors()`.
- **One CSS pack test for disc and file:** `css::is_scrambled_pack` (so the crack scan, `descramble_sector` and `descramble_region`) reads the scramble bits from the pack's first PES header: past pack stuffing and any system header, map, padding or nav packet, and only from MPEG-2 PES flags. Every DVD-Video pack is judged as before; a stuffed, map-first or MPEG-1-PES pack whose byte 0x14 merely looks flagged is no longer descrambled on the disc path, and a stuffed scrambled pack is descrambled by its own flags.
- **A flagged partial AACS unit that ends the source** (a truncated copy) is blanked and counted as damage instead of failing the read.
- **Decrypted BD-TS units have the Copy_permission_indicator cleared** (top 2 bits of each TP_extra_header), so decrypted output differs from 1.7.7 in those bits. HD DVD and units left as ciphertext are untouched.
- **`json://`** audio `sample_rate` is a number in Hz (`null` when unknown), not a string such as `"48kHz"`; a `sample_rates` array lists every rate a stream carries.
- **FMKV (`network://`, `stdio://`) header version 2** is used when a track carries decoder timing (Opus CodecDelay/SeekPreRoll) and adds per-frame DiscardPadding; streams without timing stay version 1. Headers now also carry display aspect, CICP, stream purpose/qualifier, chapters and content format, and PIDs actually written by M2TS. A header declaring more than 256 streams is refused with E9008. An older receiver omits the MKV Channels element for the new channel-layout strings; keep both ends on one version.
- **`dir://` image bytes:** the PVD declares interchange level 3, Unique IDs start at 16 and the volume id round-trips through the d-string encoder; `dir://` sources are stamped as images in the `fvi://` provenance header.
- **Output title names:** a title read from a drive (`disc://`, a server live rip) is named by the disc's meta title (else its volume id) in every container's title field and in `fvi://` provenance; one read from an image or folder (`iso://`, `dir://`, a staged image) by the disc's meta title, else its own playlist name as in 1.7.7, so an image and its extracted folder name a title alike. 1.7.7 named server-live rips by the MPLS playlist file. `fvi://` provenance of an `iso://`/`dir://` source now carries the volume id.
- **TrueHD 7.1 / Atmos labels on every path:** the playlist labels a 7.1 or Atmos TrueHD track 5.1 at 48 kHz; every mux (live disc, staged image, `iso://`/`dir://`, containers) now completes the track's channels, sample rate and label from its first major sync before the output opens, and `json://` / `chapters://` get the completed title too. 1.7.7 corrected only `iso://`/`dir://` URL rips, so live and staged-image rips now label such tracks 7.1 / Atmos.
- **Track selection applies to container inputs:** `selection` keeps a subset of the tracks of an `mkv://`, `mp4://`, `m2ts://`, `network://` or `stdio://` input (renumbered in order, video always kept), as it already did for disc titles; an unknown PID is refused at open.
- **A live DVD extracted to `dir://` is descrambled:** `Disc::extract_tree` finds and cracks each scrambled VTS from its content on every DVD-Video layout. A live drive scan records no disc-wide CSS key, so a live DVD's title VOBs were written still scrambled.
- **An encrypted disc folder is an ordinary `dir://` source:** a folder whose sampled content units are AACS-encrypted keeps its AACS verdict and is keyed and decrypted like an image of the same disc, instead of being refused with E9063; with no key it is refused before any output with E7022.
- **Frames dropped by the resync gate count as errors** in `MuxOutcome::errors` / `lost_bytes`, together with AACS units blanked as damaged.
- **A failed or stopped `network://` / `stdio://` sender resets the connection** (`Stream::finish_incomplete`), so the receiver errors instead of finishing a truncated title.
- **Key-service 404 / 422 answers are E7022** (no key opens the title), and a key-service outage or unreachable source is a source failure, never "no key for this disc".
- Diagnostics log tags: `tag=clip` is now `tag=bd.clip`; `tag=aacs` no longer logs key_source/vuk/unit_keys. `fvi://` and FMKV video `dar` for SD without a declared aspect is `0:1` (unknown), not 720:576. `Error::ImageTruncated` displays as `E<code>: have/want`.
- DVD audio and subtitles are routed by the title's PGC AST_CTL / SPST_CTL; track order, language and set may differ from 1.7.7 on discs whose physical ids are not ordinal. The mux-time AC-3 channel remap is gone.
### Added
- DVD: the region mask from the VMG (`DvdInfo.region_mask`, `DiscRegion::Unknown`), stream labels from the audio and subpicture code-extension byte, chapter names from the TXTDT text data, and a probe of each title's subpictures for forced-display (`FSTA_DSP`) commands.
- HD DVD: AACS decryption per pack, keyed by each EVOBU's CPI; AACS-wrapped playlists and features split across `PEVOB_n` files.
- AACS: a Class II MKB is reported as an unsupported class (E7109) instead of as a missing key.
- UHD: HDR10 mastering metadata is kept when content light level is absent; `hdr_plus_flag` is read from the HEVC stream attributes; the Dolby Vision enhancement-layer track's `dvcC` carries `bl_present=0`.
- `mpg://` as a source and a destination: a DVD-Video / MPEG-2 program stream file reads through the decryption stage and the DVD sector pipeline (a scrambled `.vob` is cracked once, `--raw` keeps the ciphertext; a clear MPEG-1 or stuffed MPEG-2 stream is no longer read as scrambled, and a scrambled pack with no key fails E7023). `MpgSink` writes an H.222.0 program stream with P-STD pack timing, system header and PSM; `Error::MpgNoVideoTrack` (E9074), `Error::MpgUnpacketized` (E9075). `FileSectorSource::open_padded` zero-pads a partial tail sector.
- `keys` module: `KeyRing::acquire(evidence, sampler, sources, opts, ctx)` (`KeyScope::{None, Titles, WholeDisc}`, `AcquireOptions`, `KeyResolution`) asks the key sources once before any output and proves each held key against the disc's ciphertext per stream file; keys and the Volume ID stay in memory (`Debug` redacts). Also `title_reader`, `whole_disc_reader`, `decrypt_status`, `check_decryptable`, `KeySetStatus`, `ForensicState`, `DecryptStatus`, `ProofCache`. A piece that cannot be proven up front is proven on first read; a unit no held key opens stops a title rip with E7022; in an image or folder it is blanked, warned and counted. `ResolveCtx::halt()` / `progress()` give sources the Stop token; key-service retries back off 1 s to 8 s. `MediaKeyVariantError::code()` returns E7100-E7108.
- `disc_root_of(path)`: the AACS disc folder a loose Blu-ray clip sits in (`<root>/BDMV/STREAM/x.m2ts` with `<root>/AACS`), so a caller can resolve the clip's keys on `dir://<root>`.
- `mux::fit_report` / `FitReport` / `SkipReason` (`Mp2Extension`, `NoStreamId`, `UnmappableSubtitle`): a container-neutral pre-mux plan of which streams a destination carries. `mp4_fit_report` is unchanged.
- `DiscSession::{open_with, scan_with, finish, attach_progress, token, from_drive, source_mut}` and `session::Finish::{Release, Unlock, Eject}`: a session runs under the caller's Stop token and ends on the handle it holds. `Drive::{attach, detach, attach_progress, halt, token}`: every CDB is checked against the op token.
- `halt`: waits re-check every `WAIT_SLICE` (20 ms); `Liveness`, `StallTimer`, `join_within`, `spawn_drive_holder`. `halt` is available with only the `scsi` feature.
- `NetworkStream::listen_with_halt` / `accept_from_with_halt`; `Event::BytesDurable { bytes, total }` while the output flushes; `io::{ArtifactLock, durable_sync_file, FlushProgress}` (`<final>.lock` sidecar; E9073 `TimedOut` after 30 s with no change); `Pipeline::{set_op_token, progress}`, `Sink::close_stopped` (a consumer closing after Stop keeps `*.partial`); `ISO_MUX_BATCH_SECTORS` is public.
- MPEG-2 multichannel extension audio (DVD `0xD0|n`) is a dependent track (`AudioStream::is_mp2_extension`) carried in the IR and over FMKV; sinks with no mapping list it in `MuxOutcome::undelivered_streams`, `mp4://` skips it (`Mp4SkipReason::Mp2Extension`). MPEG-1/2 audio (PES `0xC0|n`) is demuxed on DVD.
- Errors: E9085 `MuxBatchSectorsZero`, E6020 `ImageEndsBeforeRead`, E6021 `BusStreamUnmapped`, E6022 `ImageScoped`, E6023 XPL too large, E7031 `AacsKeyFileUnreadable`, E7032 `WholeDiscKeyMissing`, E7033 `AacsNoUsableHostCert`, E7034 `AacsVidNeedsDisc`, E7100-E7108 MKB variant faults, E9071 `StreamClosed`, E9072 `StreamHeaderWritten`, E9073 `TimedOut { op }`, E9074, E9075, and the remux/preflight set E9077 `RemuxVerifyFailed { kind, path }` (`E9077: <kind> <path>`, or `runtime-mismatch <have>/<want>`), E9078 `MuxIncomplete`, E9079 `RemuxStagingInvalid`, E9080 `StagedCopySizeMismatch`, E9081 `WorkerLost { op }`, E9083 `StreamLanguageUnknown`, E9084 `RemuxTargetExists`.
- `whole_disc::{whole_disc_reader, WholeDiscReader}`: the decrypting reader for whole-disc / image to ISO copies; it keys every stream file no title plays (probed at its first unit and up to 32 across it) and blanks, warns and counts the units of a stream file no held key opens (E7032 only when no file opens at all, a source faults, or no key is held). `ScanOptions::raw_copy` is kept but has no effect: an unreadable `Unit_Key_RO.inf` never fails the scan.
- `sector::bus_removal::{UnmappedStreamFile, ensure_image_debussable}`, `BusMap::unmapped`, `SectorSource::unmapped_stream_files`, `Disc::{mkv_staging_ranges, stream_content_ranges, inputs_with_samples}`, `DecryptingSectorSource::{blanked_units, clear_unit_base}`, `DiscPresence` / `disc_presence(path)`, `Playlist::duration_ticks`.
- `probe_mkv` / `probe_mkv_with_cues` read a Matroska file's Info and Tracks (and last Cues time) without reading a cluster; `parse_freemkv_version`.
- `mkv://` read-back carries display shape (aspect, PixelCrop), colour, frame rate, `LanguageBCP47` and chapters and undoes content encodings (header stripping, zlib); `mkv://` writes `MaxBlockAdditionID` for MVC 3D. `Mp4Reader` reports colour (CICP, HDR10, HLG, Dolby Vision), frame rate, duration, QuickTime v1/v2 and ISO v1 audio entries, `dac3`/`dec3` layouts and DTS `ddts` rate.
- M2TS output accepts DVD (MPEG-PS) sources: PIDs BD-TS cannot carry are remapped into BD PID ranges (a track with no free PID is omitted with a warning).
- `mux::codec::ns_to_ticks`, `CodecParser::config_changes()` / `Stream::config_changes`, `Stream::set_codec_private`. DTS Express (EXSS-only) streams are framed; MPEG-1 video is framed per picture.
- `libfreemkv::spec` (quoted spec text with registry check); `test-util` feature: `test_util` disc/image fixtures and `FakeTransport` for consumers' tests. Labels: large BD-J jars read in place, DRA CodingTypes mapped.
### Changed
- **DVD titles are `ContentFormat::DvdPs`**, set only by the DVD scan; `MpegPs` now means HD DVD or a plain program-stream file. DVD-only steps (cell filter and clock joins, CSS title-key crack, forced-subtitle probe) no longer run on HD DVD or `mpg://`. The `json://` `format` field reads `"DvdPs"` for DVD titles.
- **AACS damage policy (E7013):** a flagged AACS unit no key opens (no TS sync, zero-filled or garbled head, lone FMTS verify failure, off-grid) is zero-filled, warned and counted on every path (ISO mux, live mux, extract, whole-disc reader) instead of failing with E7013, with an end-of-rip "N damaged AACS units blanked" log. A wrong key still fails: E7013 at resolve (or when two or more FMTS units fail under one key with none verifying), E7022 on arrival in a title rip (an image or folder blanks the unit instead).
- A BD-TS mux with no AACS key set fails E7022 on the first AACS-flagged unit instead of writing ciphertext; an AACS HD DVD session with no set fails `NoDiscKey`. A key-set stop (E7022) is never shrunk, recovered or skipped.
- A Stop during a mux is always `completed = false, halted = true` (`MuxOutcome::halted`), whether it lands during the open or the pump: `mux_with_keys` no longer returns `Err(Halted)` from a stopped open. A Stop reaches `iso://` and `dir://` sources inside a blocked read, the image scan and the CSS crack (it was seen only between frames), and `iso://`, `dir://` and `mpg://` URL muxes report read progress like the disc arms. A stopped read is a stop, not a skipped sector. A read error repeats on every later `read()` of the highway stream. A halted `PrefetchedSectorSource` close is `Err(Halted)`, not EOF.
- `Disc::scan` follows the standard AACS order (UDF, `Unit_Key_RO.inf`, certificate, MKB by plain reads, then the handshake, only on a disc with an AACS directory), no longer reads the MKB from the drive, and passes `None` to `KeySource::host_certs`. A live AACS disc whose `Unit_Key_RO.inf` is missing or unreadable is warned and scanned on, with E7031 recorded in `Disc::aacs_error` and every key refused (titles that need keys then fail E7022 on the normal key path); image and folder scans record it the same way. A handshake transport fault fails with `Drive::init`'s error and a Stop returns `Halted`. `DiscSession::scan` returns the key sources on failure so a retry keeps them.
- Host-key (cert-route) bus-encrypted discs: every `/BDMV/STREAM` file is de-bussed (BEE flag and per-unit CPI honoured; CPI 0 passes through). A stream file whose extents cannot be read is re-read with FUA (bounded, 5 s pause a Stop ends), then recorded; `ensure_image_debussable` refuses a whole-disc image with E6021 naming each file and cause, `extract_tree` counts such files lost whole, and `stream_content_ranges` fails E6021.
- `DecryptingSectorSource` has no default AACS unit base: a content read before `set_unit_base` fails `DecryptFailed`, as does a flagged BD-TS unit lacking TS sync at byte 4.
- `read_mkb_from_drive` returns `AacsKeyRead` for a short, over-long or header-only pack; record 0x07 is no longer read as a cvalue table; a content certificate type other than 0x00 / 0x10 no longer parses. AACS 2.1 tries every device key before declaring the MKB underivable; an HD DVD VTKF is parsed by its magic. A CSS crack that read no sector is `Unreadable`, never `Unencrypted`.
- Key-source failures return `Err` (never an empty key set); a read-time answer that opens no sample is skipped, and the key pool and per-answer keys are capped.
- `drive_has_disc` (Linux, Windows) classifies TEST UNIT READY sense: `Ok(false)` only for NOT READY 3Ah and 30/03, 30/07 (cleaning cartridge); 04/01 and other NOT READY states are `Ok(true)`; UNIT ATTENTION is re-polled up to 4 times. `Drive::wait_ready` rides out transport failures (5 s), sends one START UNIT on 04/02, fails at once on 30h, and returns 3Ah after 10 consecutive MEDIUM NOT PRESENT answers.
- `list_drives` on Linux without sysfs keeps an sg node unless INQUIRY says not optical, or open fails with ENOENT/ENXIO/ENODEV/EACCES/EPERM. Dropping a `Drive` or transport unlocks the tray only if it locked it. `DriveId::from_drive` reads firmware date and serial from GET CONFIGURATION features and returns `Halted` on Stop.
- macOS: opening a drive is cancellable, each CDB uses the caller's timeout (`timeout_ms` 0 is 60 s), and `find_drive` releases an empty drive before the next candidate. The shim is built against `MACOSX_DEPLOYMENT_TARGET` (11.0 aarch64, 10.12 x86_64).
- `Pipeline::finish_with_halt` waits on consumer progress (the 600 s join is a stall window). `sync_all` on a `WritebackFile` fails E9056 only after 60 s without flush progress and returns `Halted` on cancel; `WritebackFile` flushes in chunks (64 MiB, down to 4 MiB). Linux `sync_file_range` errors are latched; `write_image` fsyncs the parent directory; reads past EOF of an image return `ImageTruncated`.
- `network://` send and receive accept LAN targets: loopback, private (RFC 1918), link-local, ULA and CGNAT addresses connect. Only unspecified, multicast, broadcast and reserved addresses are refused (E9022); a caller that takes targets from untrusted input must vet them itself.
- `network://`: send connects to each vetted address in turn (10 s timeout); both ends enable TCP keepalive (a rejected option only warns); receive no longer blocks on a vanished sender or a Stop. `stdio://` input fails E9008 on a non-FMKV or damaged header once bytes were consumed.
- Container headers (MKV, fragmented MP4, M2TS, network) wait (5 s of source PTS, 2048 frames or half the header buffer) for the AAC AudioSpecificConfig and BD LPCM layout byte; a later config is back-patched into a seekable MKV. `Event::OutputOpened` lists only the streams the sink writes. A mux with `batch_sectors == 0` fails E9085.
- DVD: titles map through `VTS_PTT_SRPT` (TTN 0 rejected), one angle counted, an unusable IFO falls back to its BUP; MP2 `Channels` follows CRC-verified Layer II frames; the live path shares the file-backed path's access-unit assembly and H.264 second-field merge; unmapped PS streams warn once.
- Codec framing: ADTS and MPEG audio resync only on a header confirmed by the next, so a few KB of garbage no longer poisons a track; MPEG-2 field order follows picture_structure; H.264 1080i I/P field pairs are keyframes; HEVC arms CRA-to-BLA on gaps. TS demux skips duplicate packets, resyncs after a byte slip, honours transport_error_indicator.
- `mp4://` DTS follows ETSI TS 102 114 Annex E (`ddts` carries the stream's maximum rate; 96 kHz DTS-HD MA reads back as 96 kHz); video CTS is rounded to the nearest tick; `Mp4Reader` labels MPEG-audio `mp4a` as Mp3/Mp2.
- UDF: a corrupt directory FID is warned and stops that directory listing (the root keeps the entries before it, a subdirectory lists empty); AD chains are validated (`UdfAdChainTooLong`); fragmented Metadata Files are mapped. Directory images report `DirImagePlacement` / `DirImageFanout`. MPLS: a truncated play-item list keeps the items before the damage (warned; `MplsParse` only when none remain); a CLPI cut short of its 60-byte head still parses with an unknown (0) packet count, sized from the clip's extents. bdnav / dvdnav VM follows libbluray / libdvdnav.
- Labels: BD-J detection reads each jar once under a shared inflate budget and abstains on unreadable evidence; `clpi_audit` keys streams by (clip, PID).
- HD DVD: a title naming an absent or truncated clip is dropped; the `X!` AACS directory gates identify and scan and is stripped from extractions; oversized XPL is E6023.
- `Error::DirNameCollision` and `StreamLanguageUnknown` display with control characters escaped.
### Fixed
- A program-stream PES that opens inside one picture times the next picture commencing in it (ISO/IEC 13818-1 §2.4.3.7), not the one it opens in. A program stream packing pictures back to back gave each picture its successor's PTS, so MPEG-2 GOPs re-locked a frame off and an open GOP's leading B-pictures were dropped as after a join. DVD and HD DVD titles, which open a PES at each timed picture, are unchanged.
- `m2ts://` output of a multi-clip title rides the clip-join timeline like the MKV, MP4 and MPG sinks: the PTS no longer restarts at each join, and reordered video keeps its DTS after the first join instead of falling back to PTS only.
- An image title whose extent ends off the 3-sector AACS unit grid (an HD DVD `.EVO`) no longer fails E9030 at that extent's end: the sub-unit tail is read, passed when clear and refused (E7013) when it is part of an encrypted unit.
- Every AACS format takes one encryption decision in `KeyRing::acquire`: content its own detector reads in the clear needs no key and asks no source, whatever key files the disc declares (an HD DVD copy in the clear declaring several keys, and an FMTS disc whose forensic segments are also clear, no longer ask for or refuse on keys); encrypted content is keyed by the format's rules and refused (E7022 / E7032) when no source has the key.
- A zero read batch (`MuxOptions::default()` carries `batch_sectors` 0) is refused with `Error::MuxBatchSectorsZero` (E9085) on every path (live, session, ISO, URL) instead of spinning until Stop.
- A stopped or failed read can no longer finalise a truncated MKV / M2TS as a complete title; `extract` after a drive Stop leaves the file `.partial` instead of finalising zeros.
- DVDs with MPEG audio (common on PAL) ripped an empty track; anamorphic DVDs got wrong or missing subtitle languages; a partial angle-cell filter dropped cells.
- Blu-ray picture-in-picture PG entries no longer appear in the subtitle list; HD DVD triple-layer layer count and combo audio channel counts are correct; HD DVD XPL timecodes at 60fps timeBase run at 60000/1001.
- TrueHD channel labels keep the LFE split (3.0 was "2.1", 7.0 "6.1").
- A cleaning cartridge reads as no disc, so autorip no longer rips it; a drive becoming ready (04/01) is no longer "no disc".
- `mkv://` writer fails every later write and `finish()` after a frame write error and no longer panics; read-back counts malformed blocks and undecodable frames as lost data, rejects duplicate TrackNumbers and over 512 TrackEntries (`MkvSourceInvalid`), and treats chapters, tags and attachments as best-effort. PCM tracks without BitDepth use bounded read-ahead and a multi-block estimate.
- `Mp4Sink::write` after `finish()` fails E9071; `Mp4Reader` truncated 64-bit box size is E9049 and QuickTime v1 `esds` is found; `mp4://` no longer invents 48 kHz for reserved AC-3 / E-AC-3 / DTS rate codes.
- PGS clear events, durations over 30 s and gap handling; DVD-sub orphan continuations; chapter names escaped in WebVTT; TrueHD, DTS EXSS, MPEG audio free-format and `read_ue` edge cases; a first PES starting mid-frame drops the leading fragment.
- UDF stale cache window after a failed read; Stop is honoured in image scan, the FE re-read pause and READ CAPACITY retries; a `/dev/null` mapfile uses a private per-process temp dir; `extract` sanitises reserved file names and checks extent LBA arithmetic.
- `GET CONFIGURATION` honours Data/Additional Length and feature code; MODE SELECT list length is MDL+2; Windows SPTI timeout/alignment/bounce-buffer fixes; macOS DiskArbitration claim leak.
- Labels: Paramount "feature" matching, Criterion duplicate mappings, B/T language match, bdmt setNumber. Diagnostic opening capture no longer aliases track index 256 and above.
- `m2ts://` output now carries a PAT and an HDMV-registered PMT (PID 0x0100, repeated every 100 ms) declaring each track's stream_type, so FFmpeg and players identify every track: BD LPCM (0x80) decodes as `pcm_bluray` at the source depth instead of "unknown"/`mp3`. AAC is re-framed as ADTS from its AudioSpecificConfig (stream_type 0x0F; a config ADTS cannot signal is dropped and reported), MP2/MP3 are 0x03, and both use the MPEG-audio PES stream_id 0xC0.
- `m2ts://` output now carries a program clock: PCR packets on PID 0x1001 (the PMT's PCR_PID, BD-ROM convention) at most 100 ms apart, and every packet's TP_extra_header arrival time stamp on the same 27 MHz clock, at most 1 s ahead of its PES's DTS (it was 0). A timeline jump of more than 10 s restarts the clock with the discontinuity_indicator; PID 0x1001 is never given to a stream.
- `m2ts://` output keeps every raw_data_block of an AAC access unit taken from a multi-block ADTS frame: the ADTS header declares the block count (from the frame's duration) instead of one, which made decoders play only the first 1024 samples.
- Loose `m2ts://` input keeps AAC, MP2 and MP3 that other m2ts muxers declare as PES private data (stream_type 0x06): an MPEG-audio stream_id plus the ES sync names the codec. `mp4://` carries AAC, MP2 and MP3 (`mp4a` + `esds`) instead of skipping them.
## [1.7.7] — 2026-09-26
### Maintenance
- Replace comment-overflow documentation with concise source contracts and README instructions; enforce the shared comment policy in CI.
### Fixed
- Preserve PGS clear events so forced subtitles disappear and FFmpeg remuxing retains valid timestamps (#52).
- Reassemble AAC/MPEG audio frames, recognize standard transport audio types, retain opening audio and preserve MKV decoder delay/padding.
### Added
- FFmpeg interoperability validation on QA, using generated fixtures and decoded-content/timing comparisons.
## [1.7.6] — 2026-09-26
### Changed
@@ -16,7 +163,7 @@
### Fixed
- BD-J / Java-menu main-title selection reworked (#45): a chapter-completeness failsafe, a BD-J menu-walk (parses the BDJO objects and jar manifests), and a stream-richness tiebreak now cooperate so discs driven by a Java menu pick the real main feature — keeping the Dolby Vision enhancement layer and all subtitle tracks — instead of a lower-id sibling playlist.
- PGS subtitles now get a synthesized `BlockDuration` on every block, fixing ffmpeg's "Timestamps are unset" mux error, plus a defensive HEVC NAL length-prefix guard (#52).
- PGS subtitles now get a synthesized `BlockDuration` on every block, fixing the "Timestamps are unset" mux error, plus a defensive HEVC NAL length-prefix guard (#52).
- keydb "no entry" is de-conflated into a true miss versus matched-but-no-usable-VID, and the disc hash is surfaced; adds `KeyStep.matched_entry` and `KeyStep.store_entries` (#46).
### Maintenance
+11
View File
@@ -16,6 +16,17 @@ cargo build
cargo test
```
### Parity goldens
`parity_*` tests pin what every input-to-output path produces today (output hashes, loss
counters, refusal codes) in `tests/goldens/*.golden`. A change that moves one must say why.
Re-bless on purpose, then review the diff:
```bash
FREEMKV_BLESS_GOLDENS=1 cargo test --lib parity_
git diff tests/goldens
```
## Code style
- Follow Rust conventions (`cargo fmt`, `cargo clippy`)
Generated
+33 -20
View File
@@ -124,9 +124,9 @@ dependencies = [
[[package]]
name = "crossbeam-channel"
version = "0.5.16"
version = "0.5.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e"
checksum = "98b0cc327b5bc766e7fda9c9260cc0fa81b43a8e240440422dff70788e3f9ef1"
dependencies = [
"crossbeam-utils",
]
@@ -221,12 +221,13 @@ dependencies = [
[[package]]
name = "freemkv-unlock"
version = "1.7.6"
version = "1.8.0"
dependencies = [
"aes",
"base64",
"num-bigint",
"num-traits",
"pioneer-optical",
"rand",
"serde",
"serde_json",
@@ -317,20 +318,25 @@ checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libfreemkv"
version = "1.7.6"
version = "1.8.0"
dependencies = [
"aes",
"base64",
"crossbeam-channel",
"flate2",
"freemkv-unlock",
"libc",
"libfreemkv",
"loom",
"memchr",
"rayon",
"roxmltree",
"rustix",
"serde",
"serde_json",
"sha1",
"sha2",
"socket2",
"tempfile",
"tracing",
"zip",
@@ -425,6 +431,12 @@ version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "pioneer-optical"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf5074a27cc39fb664d0b994814ffe8530e3e8590aff0d104723332dbcb1abc4"
[[package]]
name = "proc-macro2"
version = "1.0.107"
@@ -505,15 +517,18 @@ checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "roxmltree"
version = "0.20.0"
version = "0.21.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c20b6793b5c2fa6553b250154b78d6d0db37e72700ae35fad9387a46f487c97"
checksum = "f1964b10c76125c36f8afe190065a4bf9a87bf324842c05701330bba9f1cacbb"
dependencies = [
"memchr",
]
[[package]]
name = "rustix"
version = "1.1.4"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
dependencies = [
"bitflags",
"errno",
@@ -626,6 +641,16 @@ version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9be42f50aa861c555654aa3a37f52f4b1074bacf4e48fe0ef7fa584e80f1f0f"
[[package]]
name = "socket2"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "syn"
version = "2.0.119"
@@ -836,15 +861,3 @@ dependencies = [
"log",
"simd-adler32",
]
[[patch.unused]]
name = "freemkv-engine"
version = "1.7.6"
[[patch.unused]]
name = "freemkv-i18n"
version = "1.7.6"
[[patch.unused]]
name = "freemkv-keysources"
version = "1.7.6"
+34 -12
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "1.7.6"
version = "1.8.1"
edition = "2024"
rust-version = "1.88"
license = "MIT"
@@ -27,8 +27,8 @@ codegen-units = 1
# none of the ripping dependency tree.
# `rip` — the full ripping tree (drive lifecycle, disc/UDF/MPLS/CLPI parsing,
# AACS/CSS decrypt, PES/mux pipeline, keydb, freemkv-unlock seam). All
# heavy deps (serde, zip, rayon, roxmltree, rand, crossbeam, memchr,
# sha1, aes, base64, freemkv-unlock, serde_json) live here.
# heavy deps (serde, zip, rayon, roxmltree, crossbeam, memchr,
# sha1, sha2, aes, base64, socket2, rustix, freemkv-unlock, serde_json) live here.
# `default = ["rip", "scsi"]` preserves today's full build byte-for-byte;
# consumers wanting transport only use `default-features = false, features =
# ["scsi"]`. `rip` implies `scsi` because the ripping tree reads through it.
@@ -40,39 +40,54 @@ rip = [
"dep:serde",
"dep:serde_json",
"dep:sha1",
"dep:sha2",
"dep:aes",
"dep:freemkv-unlock",
"dep:zip",
"dep:flate2",
"dep:base64",
"dep:roxmltree",
"dep:crossbeam-channel",
"dep:rayon",
"dep:memchr",
"dep:socket2",
"dep:rustix",
]
# `test-util`: `libfreemkv::test_util` fixtures for consumers' tests (dev-dependency
# only; never in a release build). Implies the rip tree the fixtures build on.
test-util = ["rip"]
[dependencies]
serde = { version = "1", features = ["derive"], optional = true }
serde_json = { version = "1", optional = true }
sha1 = { version = "0.11", optional = true }
# SHA-256 for the key-set fingerprints (VID and legacy key identity, keys-upfront §4.1, J18).
sha2 = { version = "0.11", optional = true }
aes = { version = "0.9", optional = true }
# Interim path dep for local cross-repo dev; `release-orchestrate.yml` re-pins
# this to `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` in the tagged commit
# (so the released tag resolves freemkv-unlock from git, not a sibling path) and
# restores the path dep on the branch tip.
freemkv-unlock = { path = "../freemkv-unlock", optional = true }
# Resolved by the git-tag [patch.crates-io] below (re-pinned by
# release-orchestrate.yml); local dev patches it to ../freemkv-unlock.
freemkv-unlock = { version = "1.8", optional = true }
zip = { version = "8", default-features = false, features = ["deflate"], optional = true }
# zlib for Matroska ContentCompression (mkvmerge compresses S_VOBSUB by default); the
# same backend `zip` already pulls in.
flate2 = { version = "1.1", default-features = false, features = ["zlib-rs"], optional = true }
base64 = { version = "0.23", optional = true }
# Safe TCP keepalive setup for `network://` receive (dead-peer detection);
# `all` provides TcpKeepalive::with_retries.
socket2 = { version = "0.6", features = ["all"], optional = true }
# Safe readiness poll (poll/WSAPoll) so a halt-aware `network://` receive never
# arms SO_RCVTIMEO, which on Windows can drop data when it cancels a recv.
rustix = { version = "1", default-features = false, features = ["std", "event"], optional = true }
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = { version = "0.20", optional = true }
roxmltree = { version = "0.21", optional = true }
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Part of the `scsi` feature — the transport layer traces here.
# Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
# Debug/trace logging is the permitted string output. Consumers (autorip) wire a
# tracing subscriber and pipe events into the JSONL debug log.
tracing = { version = "0.1", optional = true }
# Bounded MPSC channel with kernel-wakeup send_timeout. Used by `io::pipeline`
# so the halt-aware send/finish loops can BLOCK on consumer drain instead of
@@ -99,6 +114,9 @@ libc = "0.2"
[dev-dependencies]
tempfile = "3"
# The crate's own `test_util` for its integration tests (`tests/crypto_tests.rs` decrypts
# through `test_util::decrypt_unit`; the unit decrypt is crate-private, KU §2.2).
libfreemkv = { path = ".", features = ["test-util"] }
# Permutation model checker for the `scsi::fd_handoff` atomics — the fd
# hand-off between a Linux recovery thread and `SgIoTransport::drop`. A normal
@@ -125,6 +143,10 @@ loom = "0.7"
name = "sgio_read"
harness = false
# freemkv-unlock is off crates.io: resolve it from its GIT TAG. Consumers carry
# the same patch; a gitignored .cargo/config.toml overrides it to a local path.
[patch.crates-io]
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.8.1" }
[lints.rust]
# `loom` is set by RUSTFLAGS for the model checker above, never by a feature.
+11
View File
@@ -0,0 +1,11 @@
ISO/IEC 11172-5 reference software (dist10) -- table data
src/mux/codec/testdata/dist10/alloc_0..3 are the Table 3-B.2a-d bit-allocation
tables of ISO/IEC 11172-3 Annex B, as shipped in the ISO/MPEG Audio Subgroup
Software Simulation Group's dist10 reference software (commit ea3c5ee of
joncampbell123/iso-dist10). They are used here only as test data to check the
Layer II parser.
Copyright remains with ISO/IEC and the original authors of that software. No
separate licence text accompanies these files; they are not covered by this
project's MIT licence, and their terms are those of the dist10 distribution.
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Matthew Jackson & Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+41 -14
View File
@@ -18,7 +18,7 @@ DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require disc-specific
Multi-lingual by design — the library outputs structured data and numeric error codes, never English text. Build any UI or localization on top.
**[Source & API](https://github.com/freemkv/libfreemkv)** · **[Technical Docs](docs/)** · **[Changelog](CHANGELOG.md)**
**[Source & API](https://github.com/freemkv/libfreemkv)** · **[Changelog](CHANGELOG.md)**
Part of the [freemkv](https://github.com/freemkv) project.
@@ -52,7 +52,8 @@ for title in &disc.titles {
// Stream pipeline — read PES frames from any source, write to any output
let opts = libfreemkv::InputOptions::default();
let mut input = libfreemkv::input("iso://Disc.iso", &opts)?;
let ctx = libfreemkv::Ctx::default(); // halt, events, loss counters for the run
let mut input = libfreemkv::input("iso://Disc.iso", &opts, &ctx)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
@@ -63,15 +64,14 @@ output.finish()?;
### Multi-pass recovery rip
Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the
ddrescue mapfile, damage classification and the multipass loop now live in the
The sweep/patch strategy, ddrescue mapfile, damage classification and
multipass loop live in the
`freemkv-engine` crate as `freemkv_engine::recovery::{copy, sweep, patch}`.
libfreemkv keeps the layers underneath: the raw single-shot read
(`Drive::read`) and the SCSI-fact translation (`SenseFamily`) that the engine's
strategy is built on. The dependency runs engine → libfreemkv, so this crate
cannot call into it; front-ends get recovery from the engine directly. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) for what stayed here.
cannot call into it; front-ends get recovery from the engine directly.
## What It Does
@@ -87,15 +87,15 @@ cannot call into it; front-ends get recovery from the engine directly. See
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) |
| `disc://` | Yes | -- | Optical drive via SCSI (live Read policy: adaptive batches, recovery read, skip or fail) |
| `iso://` / `dir://` | Yes | -- | Disc image file or folder (image Read policy; written by `freemkv_engine::recovery`) |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
Streams implement a single unified `pes::Stream` trait (re-exported as `PesStream`) exposing `read()` and `write()` on one type. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
Inputs implement `PesSource` (`read()`) and outputs `PesSink` (`write()`); a format that is both implements both. `open_source()` / `input()` open every input URL (`disc://` included) and `output()` every output URL. All URLs use the `scheme://path` format — bare paths are rejected.
### Keys
@@ -120,10 +120,12 @@ Disc — scan titles, streams, AACS/CSS state
├── AACS — key resolution + content decryption
└── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
Streams — unified PES pipeline
├── PesStream — pes::Stream: one trait, read()/write() PES frames
├── DiscStream — sectors → decrypt → TS demux → PES
├── IsoStream — ISO file → decrypt → TS demux → PES
Streams — one PES pipeline
├── Source — open_source(url): drive, image, folder, file, socket, pipe
├── Read stage — extents → chunks under the source's ReadPolicy (live | image)
├── Decrypt stage — AACS / CSS, content-detected, damage blanked and counted
├── Demux — TS / PS → PES (one demux for every sector source)
├── PesSource/PesSink — read()/write() PES frames
├── MkvStream — MKV mux/demux
├── M2tsStream — BD transport stream
├── NetworkStream — TCP with FMKV metadata header
@@ -131,7 +133,8 @@ Streams — unified PES pipeline
└── NullStream — discard sink
```
See [docs/](docs/) for detailed technical documentation on each module.
Build API documentation with `cargo doc --no-deps --open`. The public
[FVI format specification](https://freemkv.org/docs/fvi-format/) describes exported video indexes.
## Error Codes
@@ -187,3 +190,27 @@ The minimum supported Rust version (MSRV) is **1.88**, declared as
## License
MIT
## Validation
Run `cargo test --tests`, `cargo fmt --check`, and
`cargo clippy --all-targets -- -D warnings` before pushing.
The FFmpeg interoperability workflow runs on `qa` pushes or manual dispatch.
With FFmpeg/ffprobe installed, run:
```sh
cargo test --release --lib ffmpeg_ -- --ignored --nocapture
```
It generates synthetic PGS/video/audio fixtures and checks decoded content,
timing and decoder warnings. Set `FREEMKV_FFMPEG_ARTIFACT_DIR` to retain local
fixtures; CI retains fixtures and logs for 14 days. Real-disc QA is separate.
FFmpeg runs as an installed executable; we do not link or distribute its
binaries. This use is permitted by its software licenses; codec patent rights
are separate. See [FFmpeg legal guidance](https://ffmpeg.org/legal.html).
MKV remuxing preserves decoder delay and frame padding locally. Network/stdio
PES serialization does not carry that metadata. Direct `PesFrame` constructors
must set `discard_padding_ns` to zero unless preserving trimming.
+17 -2
View File
@@ -9,8 +9,9 @@ version = 1
# Rust sources, documentation, configuration, workflows, scripts, binary assets,
# and the byte-exact `.golden` test fixtures — is covered here instead.
#
# The whole repository is the project's own work under the MIT licence; there is
# no third-party vendored source here.
# The repository is the project's own work under the MIT licence, except the
# MPEG audio ISO reference tables under src/mux/codec/testdata/dist10, which are
# third-party test data and carry their own annotation at the end of this file.
[[annotations]]
path = [
@@ -35,7 +36,21 @@ path = [
"Makefile",
"LICENSE",
"build.rs",
"**/*.c",
"**/*.mp2",
"**/*.class",
"**/PROVENANCE",
"tests/*.txt",
]
precedence = "aggregate"
SPDX-FileCopyrightText = "2026 Matthew Jackson & Contributors"
SPDX-License-Identifier = "MIT"
# Table 3-B.2a-d allocation data copied from the ISO/IEC 11172-5 reference
# software (dist10, ISO/MPEG Audio Subgroup Software Simulation Group); this is
# not the project's work and is not MIT-licensed.
[[annotations]]
path = "src/mux/codec/testdata/dist10/alloc_*"
precedence = "aggregate"
SPDX-FileCopyrightText = "ISO/IEC JTC1/SC29/WG11 (MPEG) and the ISO/MPEG Audio Subgroup Software Simulation Group"
SPDX-License-Identifier = "LicenseRef-ISO-dist10"
+2 -2
View File
@@ -139,7 +139,7 @@ If your machine has a free SATA port, use it.
### How freemkv Handles This
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
freemkv uses a three-layer recovery model.
- **Pass 1 (`freemkv_engine::recovery::sweep`):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (`freemkv_engine::recovery::patch`):** Targeted re-reads of bad ranges with a long 60-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
@@ -164,7 +164,7 @@ Workarounds:
- **Accept the partial ISO.** freemkv's skip-forward recovery will zero-fill the unreadable blocks and continue. The resulting ISO may be playable with minor glitches in the affected areas.
- **Physical replug between retry passes.** If running multi-pass patch, replug the drive between passes to clear the taint state.
freemkv deliberately does not attempt inline SCSI resets or eject cycles to recover from this state, because those operations were found to make the problem worse on affected hardware (see the design rationale in [`docs/rip-recovery.md`](docs/rip-recovery.md)).
freemkv deliberately does not attempt inline SCSI resets or eject cycles to recover from this state, because those operations were found to make the problem worse on affected hardware.
---
+1 -1
View File
@@ -55,6 +55,7 @@ fn main() {
Ok(_) => {
writer.write_all(&buf).unwrap();
ok += 1;
bytes += buf.len() as u64;
}
Err(e) => {
fail += 1;
@@ -65,7 +66,6 @@ fn main() {
writer.write_all(&buf).unwrap();
}
}
bytes += buf.len() as u64;
if i % 50 == 0 && i > 0 {
let elapsed = start.elapsed().as_secs_f64();
+15 -1
View File
@@ -23,10 +23,24 @@ fn main() {
&target_arch // x86_64 → x86_64
};
// Match rustc's deployment target so the shim does not strongly link
// symbols newer than the app promises to run on (IOMainPort is 12+).
println!("cargo:rerun-if-env-changed=MACOSX_DEPLOYMENT_TARGET");
let min_ver = std::env::var("MACOSX_DEPLOYMENT_TARGET").unwrap_or_else(|_| {
if target_arch == "aarch64" {
"11.0"
} else {
"10.12"
}
.to_string()
});
let min_flag = format!("-mmacosx-version-min={min_ver}");
let cc_status = std::process::Command::new("cc")
.args([
"-arch",
clang_arch,
&min_flag,
"-c",
"src/scsi/macos_shim.c",
"-o",
@@ -61,7 +75,7 @@ fn main() {
/// Bake the git short hash into the build as `GIT_SUFFIX` so any muxed MKV or
/// FVI index is traceable to the exact source revision (e.g. ` (g835cc99)`).
/// Empty when git or the repo is unavailable (e.g. a crates.io tarball build),
/// Empty when git or the repo is unavailable (e.g. a source-archive build),
/// leaving just the package version. Always emitted so `env!("GIT_SUFFIX")`
/// resolves on every target.
fn emit_git_suffix() {
-302
View File
@@ -1,302 +0,0 @@
# FVI — Freemkv Video Index Format
**Specification version:** 1.0 (DRAFT)\
**File extension:** `.fvi`\
**Media type:** `application/vnd.freemkv.fvi+jsonl`\
**Status:** Draft for review. This document is the normative reference for the FVI
format; implementations and downstream tools cite it by section.
---
## 1. Scope and purpose
FVI is an open, codec-agnostic, byte-exact **index of the coded pictures** in a
video bitstream, together with **provenance** back to the source medium.
An FVI document answers, for every picture in a stream, three questions:
1. **Where is it?** — the byte-exact offset of its first byte in the *source*
(the disc/ISO/file), so a reader can extract or seek to any picture without
re-parsing the whole bitstream.
2. **What is it?** — coding type, random-access capability, GOP boundary, and
(where the codec defines them) field/pulldown attributes.
3. **When is it?** — decode and presentation timestamps on a declared timescale.
FVI is **not** a container, a codec, or a copy of the bitstream. It indexes; it
never stores coded samples. It is the serialized form of an indexer's per-picture
truth — carried from the demuxer, **never reconstructed** (§9).
## 2. Conformance
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**,
**SHOULD**, **SHOULD NOT**, **MAY**, and **OPTIONAL** are to be interpreted as
described in BCP 14 (RFC 2119, RFC 8174) when, and only when, they appear in all
capitals.
A **conformant writer** MUST emit a document that satisfies §4–§10. A
**conformant reader** MUST accept any such document and MUST ignore unknown
object members (§11) so that forward-compatible extensions do not break it.
## 3. Terminology
- **Picture** — one coded video frame (or pair of fields coded as a frame). The
unit FVI indexes.
- **Access unit (AU)** — the set of bitstream bytes that decode to exactly one
picture (ISO/IEC 14496-10 §3; ISO/IEC 23008-2 §3).
- **Coded order** — the order pictures appear in the bitstream. FVI records are
emitted in coded order.
- **GOP / coded video sequence** — a self-contained run beginning at a
random-access point.
- **Provenance** — the mapping from an AU back to the exact bytes of the physical
source it was read from (§9).
- **Source position (`src`)** — `{ file, sector, byte }`, the provenance anchor of
an AU.
## 4. Encoding
An FVI document is a sequence of **UTF-8** text lines separated by a single LF
(`U+000A`). Each non-empty line is exactly one JSON value (RFC 8259), forming a
**JSON Lines / NDJSON** stream. A writer MUST NOT emit a UTF-8 BOM. A writer MUST
NOT pretty-print: each JSON value occupies exactly one line.
The first line MUST be the **Header** object (§6). Each subsequent line is one
**Picture record** (§7), in coded order.
Rationale: line-delimited JSON is streamable (a writer appends as it indexes; a
reader processes without loading the whole file), line-addressable (picture *n*
is near line *n+1*), append-safe, and parseable by every language without a
custom grammar — while remaining a precisely specified format, not an ad-hoc dump.
A document MAY be concatenated for multiple elementary streams: each stream is its
own header line followed by its records. Readers MUST treat a Header line as the
start of a new stream section.
## 5. Document structure
```
<header> line 1 (exactly one Header object)
<record> line 2 .. N (one Picture record per picture, coded order)
[<header> <record>…] (OPTIONAL further stream sections)
```
## 6. Header object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `format` | string | MUST | Constant `"freemkv/video-index"`. Signature: a document begins with these bytes. |
| `fvi_version` | integer | MUST | Document format version. This spec defines `1`. |
| `generator` | string | SHOULD | Producing tool + version, e.g. `"freemkv/1.0.0-rc.6"`. |
| `stream` | object | MUST | The indexed elementary stream (§6.1). |
| `source` | object | MUST | Provenance root (§6.2). |
| `timescale` | integer | MUST | Ticks per second for all `pts`/`dts` (§10). E.g. `90000`. |
| `picture_count` | integer | MAY | Total pictures, if known at header time; OMITTED when streaming. |
### 6.1 `stream` object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `codec` | string | MUST | Registered codec id (Appendix B), e.g. `"mpeg2video"`, `"hevc"`. |
| `width`,`height` | integer | MUST | Coded luma dimensions in pixels. |
| `dar` | `[int,int]` | SHOULD | Display aspect ratio as `[num,den]`. |
| `frame_rate` | `[int,int]` | SHOULD | Nominal rate as exact rational `[num,den]` (e.g. `[24000,1001]`). |
| `scan` | string | MUST | `"progressive"`<br>`"interlaced"`<br>`"mbaff"` |
| `colour` | object | SHOULD | CICP per ITU-T H.273: `primaries`, `transfer`, `matrix` (integer CICP codes or registered names)<br>`range`: `"limited"` \| `"full"`<br>HDR: `mastering_display`, `max_cll`, `max_fall` per ITU-T H.273 / SMPTE ST 2086. |
| `language` | string | MAY | BCP 47 tag, if known. |
### 6.2 `source` object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `medium` | string | MUST | `"disc"`<br>`"iso"`<br>`"file"`<br>`"stream"` |
| `path` | string | MAY | Source path/label. |
| `title` | integer | MAY | Title/program number. |
| `playlist` | string | MAY | Playlist/PGC identifier. |
| `volume_id` | string | MAY | Disc volume identifier, if read. |
| `sector_size` | integer | SHOULD | Bytes per `src.sector` unit (e.g. `2048`). Lets readers convert `src` to an absolute byte offset. |
## 7. Picture record
One JSON object per coded picture, in coded order.
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `n` | integer | MUST | Coded-order index, 0-based, contiguous. |
| `src` | object | MUST | Provenance: `{ "file": int?, "sector": uint, "byte": uint }` — the offset of this AU's **first byte** in the source (§9). MUST be carried from demux, never reconstructed. |
| `type` | string | MUST | Coding type:<br>`"I"`<br>`"P"`<br>`"B"`<br>_ISO/IEC 13818-2 §6.3.9; H.264/H.265 slice types collapsed to frame type._ |
| `key` | boolean | MUST | `true` iff this picture is an intra (I) picture / parser-flagged decode-restart point (IDR / IRAP / I-picture).<br>_MPEG-2 open-GOP clean-RAP precision (`closed_gop`) is not currently distinguished — see note below._ |
| `gop` | boolean | SHOULD | `true` iff this picture begins a GOP / coded video sequence.<br>_Omitted when the implementation does not carry a distinct GOP-boundary signal._ |
| `pts` | integer\|null | SHOULD | Presentation timestamp in `timescale` ticks; `null` if unknown. |
| `dts` | integer\|null | MAY | Decode timestamp in `timescale` ticks. |
| `size` | integer | MAY | AU length in bytes; enables byte-range extraction with `src`. |
| `recovered` | boolean | MAY | `true` iff any byte of this AU came from a retried/marginal read (§9.1).<br>_Default `false`._ |
| codec ext | object | MAY | Codec-specific members under the codec's namespace (§8). |
The `type` and `key` members are **codec-agnostic** and MUST be populated for
every codec. `type` is the I/P/B coding type the parser decoded (collapsing
H.264/H.265 slice types to a frame type); where no per-picture coding is carried
(audio / synthetic frames), `type` is `"I"` for a key picture else `"P"`. `key`
is the picture's random-access flag as the codec parser sets it (IDR / IRAP /
I-picture). A writer MUST NOT emit a degraded record (`type:"?"` or `src:null`)
merely because a codec lacks per-picture coding info — those fallbacks are
reserved for a field that is genuinely unavailable (e.g. provenance absent on a
synthetic source).
> **Limitation (honest random-access).** `key` is set from the picture's
> intra / decode-restart flag. The per-picture coding model this index carries
> does **not** distinguish MPEG-2 open-GOP clean random-access points
> (`closed_gop`) from any other I-picture, so `key` is the parser-flagged
> decode-restart point, not a verified clean-RAP claim. A future revision MAY
> tighten `key` for codecs/profiles that carry that signal; readers MUST NOT
> assume present `key` precision beyond "intra / decode-restart point".
### 7.1 Interlace / pulldown fields
Codec-agnostic interlace/pulldown attributes, derived through the indexer's
per-picture coding accessors (MPEG-2: ISO/IEC 13818-2 §6.3.10). Emitted as
top-level members of the record, and ONLY when the codec actually measured the
signal — an OPTIONAL member that is omitted (not defaulted) when unknown:
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `field_order` | string | MAY | Display field order:<br>`"tff"` — top field first<br>`"bff"` — bottom field first<br>`"progressive"` — no field order applies<br>_Omitted when the codec did not signal it._ |
| `progressive` | boolean | MAY | `true` iff the picture is progressive.<br>_Omitted when the codec did not signal it._ |
| `nb_fields` | integer | MAY | Number of displayed field periods this picture occupies (the soft-telecine / 2:3 pulldown basis):<br>`1` for a single field picture<br>`2` for a normal frame<br>`3`/`4`/`6` for `repeat_first_field` pulldown per §6.3.10 |
Codecs that carry only a coding type (e.g. H.264 / HEVC / VC-1 through this
pipeline) omit `field_order` and `progressive` rather than guessing a default.
## 8. Codec model and extensibility
Core record members (§7) are codec-agnostic and present for every codec.
Codec-specific data is either (a) promoted to top-level members for a small,
registered set per codec profile (e.g. MPEG-2 §7.1), or (b) placed under an
`ext` object keyed by codec id for richer/optional data:
```json
{
"n": 42,
"type": "P",
"key": false,
"src": {
"sector": 17,
"byte": 924
},
"ext": {
"hevc": {
"temporal_id": 0,
"nal_type": 1
}
}
}
```
New codecs and members are added through Appendix B (codec registry) without a
breaking version bump, provided readers continue to ignore unknown members (§11).
## 9. Provenance and recovery semantics
`src` is **byte-exact** to the source as read. `src.sector` counts in
`source.sector_size`-byte units; `src.byte` is the offset within that sector of
the AU's first byte. For multi-file sources, `src.file` indexes a writer-declared
file list. Provenance MUST be the value observed at demux time; an implementation
MUST NOT recompute `src` by re-parsing — the point of FVI is to *carry* the truth.
### 9.1 Recovery
Because FVI is provenance-native, it can record reliability. A record with
`"recovered":true` indicates the AU's source bytes required retry/marginal-read
recovery. This lets downstream tools surface or quarantine pictures whose bytes
are not byte-identical to a clean read — a capability legacy index formats lack.
## 10. Time model
All `pts`/`dts` are integers in units of `1/timescale` seconds. `pts` is
presentation (display) time; `dts` is decode time. Records are in **coded**
(decode) order, so `pts` is not necessarily monotonic across records (B-pictures
reorder); `dts` is non-decreasing. Readers needing display order sort by `pts`.
## 11. Versioning and forward compatibility
- `fvi_version` is the document version; this spec defines `1`.
- **Additive** changes (new OPTIONAL members, new registered codecs) do NOT bump
`fvi_version`. Readers MUST ignore members they do not recognize.
- A change that alters the meaning of an existing member or makes a new member
REQUIRED bumps `fvi_version`.
- A reader encountering a higher `fvi_version` than it implements SHOULD process
the members it understands and MUST NOT reject the document solely for the
version being higher, unless a member it relies on is absent.
## 12. Conformance requirements (summary)
A conformant **writer** MUST: emit a Header first; emit records in coded order
with contiguous `n`; populate `src` from demux; use named/registered codec ids;
encode one JSON value per UTF-8 LF-terminated line.
A conformant **reader** MUST: accept any §4–§10 document; ignore unknown members;
not assume `picture_count`, `pts`, or `size` are present unless required above.
---
## Appendix A — JSON Schema (informative)
Header:
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["format", "fvi_version", "stream", "source", "timescale"],
"properties": {
"format": { "const": "freemkv/video-index" },
"fvi_version": { "type": "integer", "minimum": 1 },
"timescale": { "type": "integer", "minimum": 1 },
"stream": { "type": "object", "required": ["codec", "width", "height", "scan"] },
"source": { "type": "object", "required": ["medium"] }
}
}
```
Record:
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["n", "src", "type", "key"],
"properties": {
"n": { "type": "integer", "minimum": 0 },
"type": { "enum": ["I", "P", "B"] },
"key": { "type": "boolean" },
"src": {
"type": "object",
"required": ["sector", "byte"],
"properties": {
"file": { "type": "integer" },
"sector": { "type": "integer", "minimum": 0 },
"byte": { "type": "integer", "minimum": 0 }
}
}
}
}
```
## Appendix B — Registered codec identifiers
| `codec` | Bitstream | Field profile |
|---|---|---|
| `mpeg2video` | ISO/IEC 13818-2 | §7.1 (field_order/progressive/nb_fields) |
| `mpeg1video` | ISO/IEC 11172-2 | §7.1 |
| `h264` | ISO/IEC 14496-10 | core + `ext.h264` |
| `hevc` | ISO/IEC 23008-2 | core + `ext.hevc` |
| `vc1` | SMPTE 421M | core |
## Appendix C — Normative references
- RFC 2119, RFC 8174 — Requirement keywords (BCP 14).
- RFC 8259 — JSON.
- ISO/IEC 13818-2 — MPEG-2 video (picture coding, §6.3.9–6.3.10).
- ISO/IEC 14496-10 — H.264/AVC. ISO/IEC 23008-2 — H.265/HEVC.
- ITU-T H.273 — Coding-independent code points (colour primaries/transfer/matrix).
- SMPTE ST 2086 — Mastering display colour volume (HDR).
- BCP 47 — Language tags.
- RFC 9559 — Matroska (alignment of colour/field-order semantics).
-34
View File
@@ -1,34 +0,0 @@
# libfreemkv Documentation
Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv), the open source optical drive library.
## Start Here
**[Disc to Rip: End-to-End Flow](disc-to-rip.md)** — How the library goes from a disc in the drive to decrypted content. Read this first.
## Reference
| Document | What it covers |
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | What this crate owns of the recovery model: single-shot Drive::read, SenseFamily, DiscStream batch halving (the strategy itself moved to freemkv-engine in 1.6.0) |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
| [CLPI Clip Info](clpi.md) | EP map (coarse + fine entries), timestamp-to-sector mapping, extent calculation |
| [API Design](api-design.md) | Stream API design, PES pipeline, input/output resolution |
## Reading Order
If you want to understand the whole library:
1. **[Disc to Rip](disc-to-rip.md)** — the big picture
2. **[Architecture](architecture.md)** — how modules fit together
3. **[Drive Access](drive-access.md)** — how we talk to hardware
4. **[UDF](udf.md)** → **[MPLS](mpls.md)** → **[CLPI](clpi.md)** — how disc content is structured
5. **[AACS](aacs.md)** — how encryption works and how we break it
## API Documentation
Generated API docs are on [docs.rs/libfreemkv](https://docs.rs/libfreemkv).
-190
View File
@@ -1,190 +0,0 @@
# AACS Content Decryption — Implementation Notes
Rationale and background trimmed out of `src/aacs/content.rs` doc comments to
satisfy `ci/comment-guard.py`'s per-audience prose caps. See
[AACS Encryption](aacs.md) for the public-facing key resolution story and
[AACS Crypto Primitives](aacs-crypto.md) for the low-level AES primitives.
## `is_unit_aligned`
AACS aligned units (6144 B / 3 sectors) are anchored at the start of each
clip's encrypted region, so a read must begin a whole number of units past
that base for `decrypt_sectors` (which anchors units at buffer offset 0) to
align the CBC correctly. This is the single source of truth for the
decrypt-on-read gate, the inline and highway mux read paths, and the
key-validation sample reader — all key off this, never absolute `lba % 3`. A
disc whose clip `start_lba` is not itself 3-aligned would otherwise mis-gate
(reject readable units, then report "Decryption failed") on exactly the
titles whose clips land off a 3-boundary.
`saturating_sub` makes the `lba < unit_base` case well-defined — it clamps
the offset to 0, a unit boundary — rather than the latent `wrapping_sub` trap
where an underflow wraps to ~2^32 and, because `2^32 ≡ 1 (mod 3)`,
mis-reports the alignment (e.g. `lba == unit_base - 1` would falsely read as
aligned).
## `aacs_unit_encrypted`
The mux is the principal consumer of this signal for `BdTs` discs (via
[`is_clean`], see below). It is readable WITHOUT a key. CRITICAL: only
meaningful when `unit` is read at the correct clip-FILE-anchored boundary — a
disc-absolute / mis-aligned read makes the flag byte arbitrary mid-stream
data, which is why per-unit checks run clip-anchored, not in the whole-disc
sweep.
## `aacs_unit_needs_decrypt`
Buffer-iterating sites that may run twice over the same `buf` (the post-fetch
re-decrypt, sample collection, failure diagnosis) need an IDEMPOTENT "does
this still need work?" test, so `aacs_unit_encrypted` (the authoritative flag,
which decryption never rewrites) is composed with a "structure restored?"
check that flips once decrypted: TS syncs come back for `BdTs`; valid
`00 00 01 BA` packs come back for `MpegPs`.
## HD-DVD scramble flag (`PS_SCRAMBLE_OFF` / `PS_SCRAMBLE_MASK`)
BD/UHD/FMTS flag encryption uses the Copy Permission Indicator in the top 2
bits of byte 0 (the M2TS `TP_extra_header`). HD-DVD `.evo` is Program Stream —
byte 0 is a `00 00 01 BA` pack_start_code, NOT a CPI — so AACS reuses the
MPEG-2 `PES_scrambling_control` field instead: pack_header (14 bytes) + PES
start-code/`stream_id` (4) + `PES_packet_length` (2) puts the PES flags byte at
offset 20, with `PES_scrambling_control` in bits 5-4 (`& 0x30`). Non-zero =
encrypted. Derived from BackupHDDVD (`Header[20] & 0x30`) cross-checked
against MPEG-2 systems (ISO/IEC 13818-1).
UNVERIFIED against a real ENCRYPTED HD-DVD disc — none available to confirm
byte-exactly. Two open questions a real disc must settle:
1. A pack carrying an MPEG `system_header` (`00 00 01 BB`) before the PES
packet shifts this offset past 20.
2. Whether offset 20 is even readable pre-decrypt. BD keeps only the first 16
bytes clear (the seed) and AES-CBC-encrypts 16..6144 — under that model
offset 20 is ciphertext. BackupHDDVD reading `Header[20]` pre-decrypt
implies HD-DVD instead encrypts PES *payloads* with *clear* pack/PES
headers (per-PES model). If so, `decrypt_unit`'s 16-byte-seed model also
would not fit HD-DVD and needs its own path. TS is unaffected either way.
## `KEY_PROOF_PACKETS`
Four `0x47` syncs are 32 bits of MPEG-TS structure, but the per-UNIT
false-pass risk is NOT 2^-32: `is_clean_ts` accepts ANY four of the ~31
encrypted packets in a 6144-byte aligned unit, so for a wrong key (uniform
AES noise, `0x47` at 1/256 per packet) it is ≈ C(31,4)·256^-4 ≈ 7e-6, i.e.
~1e-5. 1-in-4-billion is the probability for four SPECIFIC packets and
overstates the margin by ~4000x; at `KEY_PROOF_PACKETS = 3` the per-unit rate
is ≈ C(31,3)·256^-3 ≈ 2.6e-4, so do NOT lower it on the strength of slack
that is not there. It is an ABSOLUTE proof floor, NOT a proportion — a unit
the key opened but whose content is bad-encoded (many non-conforming
packets) is proven by ANY four good packets, not rejected for the bad ones.
## `is_clean_ts`
The mux is its principal consumer for `BdTs` discs, reaching it through
`is_clean`: `mux::resolve`'s multi-CPS `pick` closure selects a unit key by
it, `probe_index_phase` reports each FMTS index's interleave parity by it,
and `decrypt::decrypt_sectors_mapped` uses it as the forensic-range verify
net. (The doc used to say "the mux never calls this", which invited a
maintainer to tighten or loosen the proof rule believing only whole-disc read
verification was affected — while it in fact changes which unit key a
multi-CPS disc muxes with and which phase an FMTS index is muxed at.)
Rule — evidence is ABSOLUTE, scaled to the packets that exist. Over the
ENCRYPTED packets (skip packet 0: its `0x47` sits in the clear 16-byte seed,
so it reads `0x47` for ANY key and is never evidence), let `E` = non-padding
content packets and `synced` = those carrying `0x47`. The key opened the unit
iff `E == 0` (nothing encrypted to prove) OR `synced >= min(E, KEY_PROOF_PACKETS)`.
* WRONG key → ~0 synced → fails (reaching 4 by chance ≈ 1e-5/unit, and every
unit of a clip would have to fluke — astronomically safe).
* RIGHT key, bad-encoded content → any 4 good packets pass; the bad ones are
the muxer's problem. (This is the false-negative the old 75% PROPORTION
caused — a mostly-bad unit the key opened was wrongly rejected.)
* `min(E, 4)` handles the end-of-clip fragment TAIL: a unit with only E=1
real packet (then source-zero padding) needs just that one to sync, so a
sparse-but-valid tail is never false-rejected. Padding (all-zero payload)
is excluded throughout.
## `is_clean_ps`
The Program-Stream arm of `is_clean` (HD-DVD `.evo`): every 2048-byte pack
begins with the pack_start_code `00 00 01 BA`; a 6144-byte AACS unit spans
three packs. Pack 0's start sits in the clear 16-byte seed (present
regardless of the key — mirrors `is_clean_ts` skipping packet 0); packs 1
and 2 (offsets 2048 and 4096) are in the encrypted region, so a wrong key
garbles them and this returns false (64 bits of discrimination). Validated
against real decrypted HD-DVD `.evo` (two real retail titles): pack starts
are exactly 2048-aligned, three per aligned unit, at offsets 0 / 2048 / 4096.
UNVERIFIED-HDDVD-DECRYPT: the pack structure here is confirmed on decrypted
rips, but that a real ENCRYPTED `.evo` decrypts to it via the same
6144-byte aligned unit (16-byte seed + AES-CBC over 16..6144) as BD is the
unverified assumption — we have no encrypted HD DVD. If HD-DVD decryption
yields garbage with a known-good key, the unit granularity is the suspect.
## `decrypt_unit` / `encrypt_unit`
Block Key = AES-128E(Kcu, seed) ⊕ seed (`[BD]` §3.10.1 Fig 3-8: encrypt the
clear 16-byte seed under the CPS Unit Key, XOR the seed back in — the
trailing ⊕ seed is load-bearing); then AES-128-CBC decrypt/encrypt bytes
16..6144 under the AACS IV. `encrypt_unit` is the exact inverse of
`decrypt_unit`, for authoring an encrypted disc image (and for building
genuinely-encrypted read-path fixtures); it does NOT set the encrypted flag,
because where that flag lives is container-specific, and keeping it out
keeps the crypto container-agnostic.
Source-zero padding packets (all 192 bytes zero on disc) are restored to
zero by `decrypt_unit`: their decrypted bytes are AES-noise from decrypting
zeros, but the source WAS zero, so writing the true source back is faithful
and gives the demux a tidy gap. Content packets are left EXACTLY as
decrypted — the decrypt path never rewrites content, so an authored-bad
packet passes through verbatim. `encrypt_unit` deliberately does NOT mirror
this: an all-zero plaintext packet encrypts to ciphertext that is not
all-zero, so it is not mistaken for padding on the way back and the round
trip is still exact. Authoring that wants true source-zero padding leaves
those packets unencrypted instead.
`encrypt_unit` returns `false` — encrypting nothing — when `unit` is shorter
than `ALIGNED_UNIT_LEN`. That case MUST be checked: the caller has already
set the container's encrypted flag by then (the contract requires
flag-before-crypto, since the header is the key seed), so ignoring the
result leaves a unit marked encrypted while still carrying plaintext, which
is the worst possible outcome for an authoring tool.
## `decrypt_bus` key-schedule regression (test: `decrypt_bus_expands_the_read_data_key_once_per_unit`)
MEASURED, not reasoned: `decrypt_bus` used to call `aes_cbc_decrypt` once per
2048-byte sector, and each call built its own AES-128 key schedule, so a
6144-byte aligned unit performed THREE key expansions under the same
loop-invariant `read_data_key`. On a 90 GB UHD read on a stock
(non-firmware-unlocked) drive — ~14.6 million aligned units — that is ~29
million redundant expansions on the per-unit decrypt hot path, for a key
that is constant for the whole disc. The counter is incremented inside
`crypto::new_cipher`, the single construction site.
## `IV0_PUBLISHED` (test fixture)
`[C]` §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation. Both
IV tests used to compute their expected value from `crypto::AACS_IV` itself,
so the constant was asserted against itself and NOTHING in the suite pinned
its bytes: swapping `AACS_IV` for `[0u8; 16]` left both tests passing (one
builds its ciphertext with the same value and the other cancels the change
in a triple XOR) while every real AACS disc decrypted to noise — block 0 of
every 6144-byte aligned unit and of every bus-encrypted sector XORed with the
wrong IV. `IV0_PUBLISHED` is the independent witness the tests assert
against instead.
## Encrypted-flag readers (`aacs_unit_seed_encrypted`, `aacs_unit_encrypted`)
`aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the guard
that stops a truncated encrypted fragment from being emitted as clear
content. It reads ONLY the two Copy Permission Indicator bits (`[BD]`
§3.10.2, byte 0 bits 6-7); the remaining six bits are `TP_extra_header`
arrival-timestamp bits and carry no encryption meaning. Both failure
directions are damaging and silent: a reader that answers "encrypted" for a
clear fragment discards good content, and one that answers "clear" for an
encrypted fragment writes ciphertext into the output as if it were video.
`aacs_unit_encrypted` is the AUTHORITATIVE gate and requires a WHOLE
6144-byte aligned unit: on anything shorter the flag byte is not guaranteed
to be the unit's, so it must answer `false` and leave the partial-unit case
to `aacs_unit_seed_encrypted`. A reversed length guard would both classify
fragments off arbitrary mid-stream bytes and, on an empty slice, index out
of bounds.
-89
View File
@@ -1,89 +0,0 @@
# AACS Crypto Primitives — Implementation Notes
Rationale and background for the internal (non-public) primitives in
`src/aacs/crypto.rs`. See [AACS Encryption](aacs.md) for the public-facing
key resolution and decryption story.
## `aes_cbc_encrypt`
Forward direction of `aes_cbc_decrypt`, and its exact inverse (`[C]` §2.1.2,
`AES-128CBCE`). `data.len()` must be a multiple of 16; the `debug_assert!`
documents/enforces that contract.
Constructs the cipher ONCE for the whole slice. Driving this from the
single-block `aes_ecb_encrypt` instead rebuilds the AES key schedule per
16-byte block, which for a 6144-byte aligned unit is 383 redundant key
expansions.
## `aes_cbc_decrypt`
AES-128-CBC decrypt in-place with the fixed AACS IV (`[C]` §2.1.2,
`AES-128CBCD`). `data.len()` must be a multiple of 16; any trailing partial
block is silently ignored. All callers pass aligned regions (6128 and 2032
bytes), and the `debug_assert!` documents/enforces that contract.
This doc was once orphaned onto `aes_cbc_encrypt` above it, when that
function was inserted directly after it with no separating blank line:
rustdoc rendered the crate's only forward-direction AACS primitive as
"decrypt" and cited the spec's DECRYPT clause for it, while this function
had no doc at all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in
`content.rs` pins the directions behaviourally, so a maintainer "fixing" the
contradiction by swapping the two bodies fails the suite instead of shipping
a second decryptor behind an already-set encrypted flag.
## `cbc_decrypt_blocks`
Split out of `aes_cbc_decrypt` so a caller that decrypts several regions
under one loop-invariant key expands the schedule once. `decrypt_bus`
(`super::content::decrypt_bus`) is that caller: bus encryption (`[C]` §4.2 /
the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY 2048-byte
sector, so a 6144-byte aligned unit is three regions under one
`read_data_key` — three key schedules where one suffices, on the per-unit
decrypt hot path of a whole 90 GB read.
## `aes_g`
AES-G(x1, x2) = AES-128D(x1, x2) XOR x2 (`[C]` §2.1.3, note: uses
AES-128**D**). The Media Key Variant chain uses AES-G to derive both the
variant number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
(`Kvu = AES-G(Km, VID)`). See `super::derive::derive_vuk` for the classical
VUK form — the math is identical, this exposes it as a neutral primitive
for the variant chain.
## `aesg3`
AACS-G3 (`[C]` §3.2.2, Triple AES Generator): `left = D(k,s0)⊕s0` inc 0,
`pk = D(k,s0+1)⊕(s0+1)` inc 1, `right = D(k,s0+2)⊕(s0+2)` inc 2. Shared
with `super::variant` (its variant chain runs the same SD tree); a single
definition keeps the two walks byte-identical.
## Test: `S0`
The AACS-G3 seed `s0`, transcribed independently from `[C]` §3.2.2 rather
than read from `AESG3_SEED` — a test that sourced the seed from the
production constant would assert that constant against itself and would
still pass if it were edited.
## Test: `aesg3_inverts_to_the_spec_seed_under_aes_encrypt`
`aesg3` is the node function of the AACS subset-difference tree: every
Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every descent
step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that returned a
fixed block would make every device key in the crate derive the SAME
Processing Key, and a `^` that became `|` or `&` would derive a
wrong-but-plausible one — in both cases the MKB walk simply stops finding
Media Keys, with no error to say why.
Pinned through the spec relation rather than a re-implementation: `[C]`
§3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for `s = s0 + inc` (added
into the last seed byte), so applying the FORWARD primitive
`aes_ecb_encrypt` — a different function from the one under test — to
`aesg3(k, inc) XOR s` must reproduce `s` exactly.
## Test: `aesg3_yields_three_distinct_subkeys_for_the_three_increments`
The Triple Generator's three outputs (`[C]` §3.2.2: left = inc 0, the
Processing Key = inc 1, right = inc 2) are the two child node keys and the
Processing Key of ONE tree node. They must be three different keys — if
`inc` were ignored, a descent would revisit its own parent and the walk
would derive the same key at every level of the tree.
-68
View File
@@ -1,68 +0,0 @@
# `src/aacs/derive.rs` — key derivation notes
Rationale and background relocated here from doc comments in `derive.rs` to
keep those comments within the repo's comment-length limits.
## `derive_media_key_from_pk` — why the direct PK × cvalue scan
A Processing Key is **terminal**: it is the key at its Subset-Difference
node, one `AES-G` from the Media Key. So the fast path tries each PK
*directly* against the MKB cvalue tables (no tree descent). On a large
AACS 2.x UHD MKB (~181k cvalues) this is ~15x faster than treating a PK as
a device-node label and walking the tree.
MKB record types relevant to this path:
| Type | Meaning |
|------|---------|
| `0x10` | Type and Version Record (has MKB version) |
| `0x81` | Verify Media Key Record, AACS 1.0 (has `mk_dv`) |
| `0x86` | Verify Media Key Record, AACS 2.0/2.1 (has `mk_dv`) |
| `0x04` | Subset-Difference Index (has UVS entries) |
| `0x05` | Media Key Data Record (cvalues, 1:1 with `0x04`) |
| `0x07` | Explicit Subset-Difference Record (NOT cvalues) |
## `recover_dk_position` — search strategy and cost
This finds a device key's tree node empirically: for each MKB
subset-difference record, it tries the device at the record's node AND at
every ancestor v-position (the device may sit one or more levels ABOVE the
record, descending via AES-G3 to reach it), deriving the candidate
Processing Key DIRECTLY (one `calc_pk_from_dk` per candidate, no full
re-walk) and checking it validates against that record's cvalue.
Cost is `O(slots × tree_depth)` — linear in the MKB's subset-difference
index, not the quartic cost of re-deriving per candidate. Slots are
independent so the scan parallelizes (~181k slots, ~26s serial on a UHD
MKB); `find_map_any` returns the first match and cancels the rest.
## `resolve_candidate` — composed primitives and `None` conditions
Composes the raw derivation primitives (`derive_media_key_from_pk`,
`derive_media_key_and_pk_from_dk`, `derive_vuk`, `derive_unit_keys`) and
parses `Unit_Key_RO.inf` at the version the disc's MKB declares, so a
multi-CPS disc yields all its unit keys from the one candidate.
Validate `unit_keys` against a real encrypted unit with `decrypt_unit` +
`is_clean_ts` to prove the candidate actually opens the disc — this function
is pure derivation, no sampling or validation.
Returns `None` only when derivation itself cannot proceed: a PK its MKB
rejects, a `Dk` the MKB can't process, a missing VID on a path that needs
one, or an unparseable/empty `Unit_Key_RO.inf`.
## Test fixtures — inverting the AACS relations
There are no published AACS test vectors, but none are needed: the AACS
relations (`[C]` §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
key can be constructed with `aes_ecb_encrypt` and the same `aesg3` the walk
uses as its node function. `plant_mkb` and its siblings in
`position_recovery_tests` do exactly this — no real key material, and the
assertions check the DERIVED Media Key, not any intermediate the code under
test also produces.
`plant_mkb`'s single-slot fixture picks `uv = 0x0400` (lowest set bit 10)
with `u_mask_shift = 12` so a gating device node exists: `resolve_dk_node`
flips one bit `b < 12`, and the walk's gate (`[C]` §3.2.4) needs that bit
inside `v_mask` (`0xFFFF_FFFF << 11`) and outside `u_mask`
(`0xFFFF_FFFF << 12`) — i.e. `b == 11`.
-78
View File
@@ -1,78 +0,0 @@
# `src/aacs/resolve.rs` — internal rationale
Overflow detail relocated here by the comment-guard pass; each section is
pointed to by a short `//` comment at the named site in `resolve.rs`.
## `resolve_keys_classical`
Resolve all AACS keys for a disc using the classical (single-stage Media Key
derivation) paths. Used by both V10 and V20.
Paths run in root-of-trust -> per-disc-leaf order. A match at any path
returns immediately:
1. MKB + device keys -> processing key -> media key -> VUK
2. MKB + processing keys -> media key -> VUK
3. KEYDB MK + matching VID -> derived VUK
4. KEYDB disc-hash -> VUK
5. KEYDB disc-hash -> pre-decrypted unit keys (no VUK)
## `unique_verifying_mk`
The MK-pool selection rule of path 2.5, split out of `resolve_keys_v1` so the
ambiguity guard has a reachable test.
`verifies` is the MKB check -- in production
`AES-D(mk, mk_dv)[..8] == MK_VERIFY_MAGIC`. Returns a Media Key only when
EXACTLY ONE DISTINCT candidate passes. Duplicates of the same key are one
candidate (a pool aggregated across providers routinely repeats a key), but
two DIFFERENT keys that both verify mean the pool cannot say which is this
disc's Km: picking either derives a wrong VUK, and a wrong VUK decrypts to
plausible-looking garbage rather than failing loudly. Bail and let the later
hash/VID paths answer instead.
The predicate is a parameter rather than the inlined AES check because a
genuine two-key multi-hit cannot be synthesised: it needs one ciphertext
that decrypts under two distinct AES-128 keys to plaintexts sharing a 64-bit
prefix -- a 2^64 search. Injecting the verifier is the only way the
ambiguity branch is reachable from a test at all.
## Test: `mk_pool_ambiguity_bails_rather_than_picking_a_media_key`
Path 2.5's ambiguity guard: when MORE THAN ONE DISTINCT pooled Media Key
verifies against the MKB, the resolver must return no key at all rather than
pick one. A wrong Km derives a wrong VUK, and a wrong VUK does not fail
loudly -- it decrypts the title to garbage that muxes and plays as a corrupt
rip.
The real MKB check cannot be forced into a multi-hit: two distinct AES-128
keys decrypting one `mk_dv` to plaintexts that share the 64-bit verify magic
is a 2^64 search, not a fixture. So the rule is tested through
`unique_verifying_mk`, whose verifier is a parameter -- the same function
`resolve_keys_v1` calls, with the same pool semantics.
## Test: `resolve_keys_v21_treats_the_all_zero_volume_id_as_no_vid`
`resolve_keys_v21` gates paths 1 and 3 on `has_vid`, and an all-zero Volume
ID is the crate's "the VID was never read" sentinel -- the SCSI handshake
leaves the buffer zeroed when it does not run or fails.
Both directions matter and both fail silently:
- treating the zero sentinel as a real VID runs path 3 and derives
`Kvu = AES-G(Km, 0...0)`, a perfectly well-formed but WRONG VUK. It
unwraps the title keys to garbage, and nothing downstream errors -- the
rip just decodes to noise.
- treating a real VID as absent skips paths 1 and 3 entirely, so a disc
that could have been resolved from its Media Key reports no key.
Asserted through the final VUK, not through the flag.
## Test: `classify_vid_present_with_material_is_no_material_not_vid`
A NON-zero VID present, but resolution still fails (the providers carry
material that doesn't resolve this disc). The VID is available, so the
failure is NOT "VID unavailable" -- it must classify NoMaterial regardless
of how much derivation material is present, because re-acquiring the VID is
not the fix. This is the `has_vid == true` short-circuit, which no existing
test covers (the gate test only uses the zero-VID sentinel).
-44
View File
@@ -1,44 +0,0 @@
# AACS resolution trace shape
`src/aacs/trace.rs` defines the structured trace of a resolution attempt (the
unlock phase, then the key-resolution phase).
- No user-facing English: every step's STATE is a typed enum variant.
Applications render these into localized text — the library never does.
This module only defines the shape and is wired through the
resolve/handshake return path far enough to compile.
- The `who` of each step is the source's `label()` / unlocker's `name()` — a
stable identifier string (a NAME, like a codec id, NOT user-facing prose),
carried verbatim so an app renderer never has to match an enum back to a
name it already has. Only the OUTCOME / path enums are structured states
the app maps to i18n English.
## `KeyStep` — a source's contribution to the key phase
Each source that ran contributes one `KeyStep`: `who` (its `label()`), `path`
(the `Vec<KeyNode>` it walked), and `outcome` (`KeyOutcome`). Two fields
de-conflate a keyless result — WHY a matched disc produced no usable key:
- `matched_entry: Option<MatchedEntry>` — the shape of the source's MATCHED
entry when it matched this disc; `None` for a true miss or a source kind
with no such shape. An app MAY log it verbatim and render a
matched-but-no-key verdict.
- `store_entries: Option<usize>` — per-disc entries loaded in the source's
store, when known, so a true-miss verdict can name the store size (`… not in
keydb (N entries loaded)`). `None` for a source that carries no such count.
## `MatchedEntry` — booleans-and-lengths, never material
A shape summary of a matched entry — safe to print at any log level, no key
MATERIAL: `has_vuk`, `has_unit_keys`, `unit_keys_len`, `has_media_key`,
`has_keydb_vid`, `enc_title_keys_len`, `vid_available`. It answers "the disc
WAS found — so why no key?" (e.g. a Media Key present but no VID on this path).
## `KeyNode::NoDerivableKey`
A derivation-path node distinct from `NoEntry`: the source MATCHED this disc
but could not derive any usable key from the matched entry (no derivation
material for the path taken). `NoEntry` = the disc was not found at all;
`NoDerivableKey` = found, but the key was not — the node that turns a flat
`no entry` verdict into `matched disc > … > NO KEY`. See docs/keysource.md
(`resolve_unit_keys`) for how a source populates it.
-216
View File
@@ -1,216 +0,0 @@
# AACS Encryption Support
## Overview
AACS (Advanced Access Content System) is the encryption layer used by Blu-ray
and UHD 4K discs to protect content. libfreemkv implements AACS decryption so
disc access is transparent to the application.
There are two major versions:
- **AACS 1.0** -- Used by standard Blu-ray discs.
- **AACS 2.0 / 2.1** -- Used by UHD 4K Blu-ray discs. Adds a per-sector bus
encryption layer on top of the standard content encryption. UHD drives accept
AACS 1.0 host credentials for backward compatibility.
All versions use AES-128 for content decryption. The library reads the keys it
needs from `keydb.cfg`, walks the disc's Media Key Block (MKB) to resolve the
disc's key, and decrypts the content stream. AACS-encrypted discs therefore
require a `keydb.cfg`; CSS-protected DVDs do not (see the CSS notes in the
library docs).
## How it works (feature level)
When a disc is scanned, the library:
1. Reads the disc's AACS key-input files from the `/AACS/` directory.
2. Resolves the disc's key from `keydb.cfg` — either directly from a per-disc
entry, or by walking the MKB with the keys present in the keydb.
3. Performs the drive-level SCSI authentication handshake needed to obtain the
Volume ID and, for UHD, the bus-decryption key.
4. Decrypts the content stream as titles are read.
A resolved key is verified against actual disc content before it is applied, so
a stale or wrong key fails loudly rather than producing silent garbage. If no
usable key is available for an AACS-encrypted disc, the library surfaces a
specific error (the E70xx family) describing which part of the chain was
missing, and a missing `keydb.cfg` surfaces as `Error::KeydbLoad` with the
sentinel path `<no keydb in search paths>`.
## API Usage
AACS decryption is transparent to the application. `Disc::scan()` handles
everything automatically:
```rust
use libfreemkv::{Drive, Disc};
use libfreemkv::disc::ScanOptions;
use std::path::Path;
let mut drive = Drive::open(Path::new("/dev/sg4")).unwrap();
drive.wait_ready().unwrap();
drive.init().unwrap();
let disc = Disc::scan(&mut drive, &ScanOptions::default()).unwrap();
// Check encryption state
if disc.encrypted {
if let Some(ref aacs) = disc.aacs {
println!("AACS {}.0", aacs.version);
println!("Key source: {}", aacs.key_source.name());
if let Some(mkb_ver) = aacs.mkb_version {
println!("MKB version: {}", mkb_ver);
}
} else {
println!("Encrypted but keys not available");
}
}
// Read content -- decryption is applied on read by the DiscStream decorator.
// Live disc does NOT go through the URL resolver: `input("disc://...")` returns
// Error::DiscUrlNotDirect by design.
let keys = disc.decrypt_keys();
let mut stream = DiscStream::new(
Box::new(drive),
disc.titles[0].clone(),
keys,
batch_sectors,
disc.titles[0].content_format,
false, // raw: false → decrypt on read
None, // halt
)?;
while let Ok(Some(frame)) = stream.read() {
// decrypted PES frames
}
```
The application never calls decryption functions and never manages the
drive-level handshake. It DOES own key resolution — see below.
### Key resolution is the caller's job
`libfreemkv` is **lookup-free: it resolves no keys and reads no keydb.** There is
no `ScanOptions::with_keydb`, and `ScanOptions` has no keydb field — its only
scan input is the optional drive credentials for the live-drive authenticated
handshake.
The caller resolves a key out-of-band through a key source and applies it with
[`Disc::decrypt_with`]. `freemkv-keysources` is the crate that implements the
keydb and key-server sources; `ScanOptions::key_sources` takes them as
`Box<dyn KeySource>`.
### AacsState
After a successful scan, `disc.aacs` contains an `AacsState`:
| Field | Type | Description |
|-------|------|-------------|
| `version` | `u8` | AACS version (1 or 2) |
| `bus_encryption` | `bool` | Whether bus encryption is active |
| `mkb_version` | `Option<u32>` | MKB version from disc |
| `disc_hash` | `String` | Identifier for the disc's key-input files |
| `key_source` | `KeyOrigin` | How the disc's key was resolved |
## keydb.cfg
`keydb.cfg` is the single source of AACS key material. It is a text file (lines
starting with `;` or `#` are comments) holding the host credentials and per-disc
entries the library uses to resolve a disc. autorip can auto-download and
refresh it from a configured URL. The library does not ship any AACS keys
compiled into the binary.
## `UnitKey::is_default_index` test rationale
`is_default_index` is the public predicate that separates ordinary (index-0)
content keys from FMTS forensic index keys (see `UnitKey` docs; AACS 2.1
`IndividualSegment.tbl` tagging). A body answering `true` for everything would
present a forensic index key as an ordinary content key -- the caller would
decrypt the bulk of the title with a key that only opens 1/32nd of it;
answering `false` for everything would hide every ordinary key.
`is_default_index_separates_the_two_constructors` is pinned against the two
named constructors, which are the contract: `UnitKey::new` builds the ordinary
key, `UnitKey::forensic` builds an index key for `1..=32`.
`is_default_index_agrees_with_the_forensic_index_resolver` checks the
predicate agrees with the one consumer of `index_number` in the crate:
`crate::aacs::index_select::resolve_disc_index` resolves the disc's forensic
index from exactly the keys that are NOT default. If the two disagree, a disc
resolves an index whose key the rest of the pipeline treats as ordinary (or
vice versa).
## `src/aacs/mod.rs` module notes
### KEYDB.cfg line format
```
| DK | DEVICE_KEY 0x... | DEVICE_NODE 0x... | KEY_UV 0x... | KEY_U_MASK_SHIFT 0x...
| PK | 0x...
| HC | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
| HC2 | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
0x<disc_hash> = <title> | D | <date> | M | 0x<media_key> | I | 0x<disc_id> | V | 0x<vuk> | U | <unit_keys>
```
### Spec provenance tags
The crypto in `src/aacs/` carries `[TAG] §x.y` citations back to the published
AACS specification (Final Rev 0.953), so each primitive links to the section it
implements:
- `[C]` -- AACS Introduction and Common Cryptographic Elements Book (primitives,
MKB/key-management).
- `[PR]` -- AACS Pre-recorded Video Book (Volume/Title Key layer).
- `[BD]` -- AACS Blu-ray Disc Pre-recorded Book (CPS Unit Key, Aligned Unit,
Block Key).
- `[RE]` -- reverse-engineered from real discs, cited only where the public
spec is silent (the `0x86` verify record and the Category-C MKB type values).
### AACS key-input path discovery (`AacsRole`, `role_paths`, `find_hddvd_aacs_dir`)
BD and UHD keep their key material under a fixed `/AACS/...` tree, so those
paths are constants. HD DVD keeps the equivalents in a reserved root directory
whose NAME is authoring-house-specific -- observed `ANY!` (Dukes of Hazzard)
and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK` mirror -- and
whose title-key file is NOT always `VTKF000.AACS` (Freedom ships
`VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED from the
parsed UDF tree (`find_hddvd_aacs_dir` + `role_paths`), never hardcoded.
Each key role (`AacsRole`) resolves to an ordered candidate list -- the BD/UHD
constants first, then whatever the HD DVD directory actually holds -- which
every reader walks with `read_first`, first-that-reads. No reader ever
branches on disc type: a BD/UHD disc has the `/AACS/` files so those win; an
HD DVD has none of them, so it falls through to the discovered entries.
Centralised so `resolve_vid_only`, `read_aacs_inputs`, `read_mkb_content`, and
`read_aacs_version` can never silently diverge the disc_hash / MKB / VID that
another reader feeds a key service.
`find_hddvd_aacs_dir` identifies the HD DVD AACS directory structurally, NOT by
a hardcoded name: the root child directory whose name ends in `!` (so the
`<name>!_BAK` backup mirror, which also ends in a non-`!` char, is not
mistaken for it) and which contains `MKBROM.AACS`. Observed real names:
`ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD disc has no such
directory -> `None`.
`role_paths` builds the ordered candidate list: fixed BD/UHD paths first, then
the discovered HD DVD files. For `AacsRole::UnitKey` every `VTKF*.AACS` in the
directory is appended in sorted name order -- a disc may carry more than one
variant (Freedom: `VTKF090` + `VTKF100`), not just `VTKF000`.
`read_first` walks an AACS role's candidate paths and returns the first that
reads; `read` performs the actual per-path read (full file or bounded prefix),
so callers share the same first-present walk regardless of read style. Returns
`Error::AacsNoKeys` if no candidate is present. Generic over the path element
(`&str` or owned `String`) so it accepts the `Vec<String>` that `role_paths`
builds from the discovered HD DVD directory.
### Test: `a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory`
The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required -- the
discovery is a conjunction, not a disjunction. The existing fixtures only ever
present a directory that satisfies both (`AAC!` with `MKBROM.AACS`) alongside
one that satisfies neither (`AAC!_BAK` -- which contains `MKBROM.AACS` but is
ALSO reached only after the real dir), so either half of the conjunction could
be dropped and the same directory would still be found. This test's fixture
has a directory that satisfies the name half and NOT the contents half: it
must not be picked. If it were, the HD DVD path would resolve `MKBROM.AACS`,
`CONTENT_CERT.AACS` and the title-key file under a directory that holds none
of them -- the disc reports "no AACS key files" and never rips.
-220
View File
@@ -1,220 +0,0 @@
# AC3 / E-AC3 parser notes (`src/mux/codec/ac3.rs`)
## Why a frame set is one sample, never split into separate tracks
A legacy AC-3 syncframe is a complete access unit on its own, but an E-AC-3
access unit (ETSI TS 102 366 / ATSC A/52 Annex E) is a whole FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream frame that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7) and their own dependents. A decoder needs the whole set
to reconstruct the programme (a 5.1 independent substream plus a dependent
substream carrying the extra channels of a 7.1 programme, and the AC-3-core +
E-AC-3-dependent arrangement used for backwards-compatible Dolby Digital
Plus), and every substream of the frame set covers the SAME time period.
WHY the whole frame set is ONE sample, rather than splitting an additional
independent substream (an associated / commentary service) into a track of
its own: Annex E orders the substreams of a frame set inside a single
elementary stream, all covering one time period, and a decoder handed a frame
set renders the programme it is asked for from it — that is exactly what the
container's E-AC-3 sample and its `dec3`/`num_ind_sub` description denote.
Splitting would mean rewriting the bitstream (re-numbering `substreamid` so
the extracted service becomes substream 0, and rebuilding its frame sets),
because a substream numbered 1..7 with no substream 0 is not a conforming
stream. That is a transcode, not a remux; this parser is lossless, so the
frame set stays whole and player-side programme selection decides what is
heard.
## `SAMPLE_RATES` / fscod2
Sample rates indexed by fscod (0=48kHz, 1=44.1kHz, 2=32kHz). fscod=3 is
reserved in AC-3; in E-AC-3 it signals "fscod2" (reduced rates: 24/22.05/16
kHz, selected by byte-4 bits `[5:4]`). `frame_sample_rate` decodes fscod2 in
the E-AC-3 case; the table's index-3 entry (48 kHz) is only the fallback when
the header is too short to read fscod2.
## `MAX_AC3_BUF` derivation
An AC-3/E-AC-3 syncframe is at most 8192 bytes (the `frame_size > 8192`
reject) and an access unit is a whole frame set: up to 8 independent
substreams, each with up to 8 dependent substreams (ETSI TS 102 366 Annex E)
— 72 syncframes worst case. A cap of one straddling frame set plus slack
therefore has to exceed 72 × 8192 = 576 KiB. If the buffer grows past the cap
without yielding a frame (pathological / never-syncing input) it is dropped
and resync begins, rather than accumulating one PES worth of data per call
for the whole title.
## `scan_access_units`
Per ETSI TS 102 366 (ATSC A/52) Annex E an access unit is one FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7 — associated/commentary services) with their own
dependents. The access unit therefore closes only when the next
`substreamid`-0 independent substream (or, with `at_eos`, the end of the
stream) is seen, and it carries THAT substream's PTS and duration: every
other substream of the frame set describes the SAME time period and adds no
duration of its own.
`base_pts_ns` times the access unit that begins at `data[0]` — i.e. the
running cadence carried over from the previous call. `anchor` re-anchors the
running PTS to this PES's own timestamp at the first access unit that STARTS
in the newly-appended bytes: a PES timestamp applies to the first access unit
beginning in that PES, never to one that began in an earlier PES and is only
being completed (or was held) here.
Returns the emitted access units, the offset in `data` from which bytes must
be carried over to the next call, and the PTS to stamp on the access unit
that begins that carry-over.
## `substream_role`
Byte 2 of an E-AC-3 syncframe is `strmtyp(2) | substreamid(3) | frmsiz[10:8]`
(ETSI TS 102 366 Annex E BSI). `strmtyp` 0 and 2 are independent substreams
(type 2 being an independent substream that is not the first of the bit
stream); `strmtyp` 1 is the dependent substream.
Annex E defines a FRAME SET as independent substream 0 — mandatory, always
first — with its dependent substreams, followed by the OPTIONAL additional
independent substreams 1..7, each with their own dependents; all of them
carry the same time period. So the access-unit boundary is an independent
substream with `substreamid` == 0, and NOT merely "an independent substream":
an additional independent substream (an associated or commentary service)
sits INSIDE the frame set already open. Keying the boundary on `strmtyp`
alone made every such substream close the access unit and advance the
running PTS a second time over the same ~32 ms, doubling the timeline (about
a second of A/V drift per second of audio).
`strmtyp` 3 is reserved: its BSI layout is not defined, so its `substreamid`
bits cannot be trusted and it is treated as starting a fresh access unit — an
unknown frame is never merged into an unrelated programme, and never
discarded as an orphan either.
Legacy AC-3 (`bsid < 11`) has no substream structure — byte 2 there is the
crc1 field, never `strmtyp` — so it always starts an access unit.
A frame set MAY carry a substream as several consecutive syncframes of fewer
than six blocks each (Annex E allows numblkscod < 3). Those extra syncframes
are `substreamid` 0 too, so each starts its own access unit here rather than
merging into one frame set. That is deliberate: each carries its OWN
numblkscod-derived duration (see `frame_duration_ns`), so the timeline total
stays exact, and the substreams are still delivered in bitstream order.
## `frame_crc_ok`
Per ETSI TS 102 366 (ATSC A/52) the frame carries a CRC-16/ANSI (poly 0x8005,
init 0, non-reflected) over the bytes after the 2-byte syncword — i.e.
`crc16_ansi(&buf[2..]) == 0` covers `frame_size - 2` bytes; the trailing crc
word makes a clean frame's residue zero. A nonzero residue is a
~1-in-65536-certain sign of payload corruption, so the frame is dropped
(silence gap) rather than shipped as a glitch.
## `acmod_channels` bit layout
This is the AUTHORITATIVE channel count for the track header: the DVD IFO
`audio_attr_t.channels` nibble is a well-known unreliable/stale field, so the
muxer prefers this over the IFO-claimed count (the bitstream acmod is
authoritative; the IFO audio nibble is not trusted). LFE adds one channel
(e.g. acmod=7 + lfeon → 6 = 5.1).
Bit layout from the syncword (A/52 §5.3.2 BSI):
```text
byte 5: bsid(5) | bsmod(3)
byte 6: acmod(3) | [cmixlev(2) if acmod has a centre and acmod!=1]
| [surmixlev(2) if acmod has surround]
| [dsurmod(2) if acmod==2] | lfeon(1) | ...
```
`acmod` therefore always occupies byte-6 bits 7-5; `lfeon` follows a variable
number of optional 2-bit fields, so the decoder tracks the bit cursor.
## `ACMOD_CHANNELS` table
Base channel count per AC-3 `acmod` (A/52 Table 5.8), BEFORE the LFE. Index is
the 3-bit acmod value; add 1 when `lfeon` is set.
```text
0 = 1+1 (Ch1, Ch2) -> 2 4 = 2/1 (L,R,S) -> 3
1 = 1/0 (C, mono) -> 1 5 = 3/1 (L,C,R,S) -> 4
2 = 2/0 (L, R) -> 2 6 = 2/2 (L,R,SL,SR) -> 4
3 = 3/0 (L,C,R) -> 3 7 = 3/2 (L,C,R,SL,SR) -> 5
```
## Test helpers and coverage notes
### `finalize_ac3_crc`
Relies on the CRC-16/ANSI residue property: appending `crc16([2..n-2])`
(big-endian) zeroes the register over `[2..n]`.
### `the_working_buffer_is_reused_across_packets_not_reallocated`
`parse` runs once per PES packet — of the order of 10^5 times on a feature's
audio track — and used to build a fresh `Vec` every call. The copy itself
cannot go without restructuring the borrow relationship, but the allocation
can: the buffer is taken out of `self`, filled, and put back with its
capacity intact. Asserting on capacity is deliberately white-box, because
that is exactly what regresses if someone reverts to `to_vec()` — the
behaviour would be identical and no other test would notice.
### `make_eac3_frame` byte layout
ETSI TS 102 366 Annex E: byte 2 is strmtyp(2) | substreamid(3) | frmsiz[10:8],
byte 3 is frmsiz[7:0], and the frame is `(frmsiz + 1) * 2` bytes. byte 4 =
fscod 0 (48 kHz), numblkscod 3 (6 blocks → 1536 samples → 32 ms), acmod 7 +
lfeon (5.1); byte 5 = bsid 16 so the E-AC-3 paths are taken.
### `a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet`
An access unit closes only at the next `substreamid`-0 independent substream,
so one that keeps gaining dependent substreams stays OPEN across PES
boundaries and its bytes stay in the carry-over. The carry-over must not be
re-scanned — and re-CRCed — from the access unit's first byte on every
packet: the buffer only stops growing at `MAX_AC3_BUF` (1 MiB), and a PES on
a DVD is about 2 KiB, so re-deriving the held access unit costs work
quadratic in the packets fed.
Measured directly, because a work bound has no frame-level shadow:
`frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
Mutation: pass `None` for `held` in `parse` (or drop the `if let Some(h) =
held` resume) — the count returns to the quadratic figure.
### `a_discontinuity_drops_the_held_access_unit_with_its_bytes`
`parse` clears `self.acc` on a discontinuity because the buffered bytes are a
truncated frame. The held access unit is described by OFFSETS into exactly
those bytes, so it has to go with them. Without `self.held = None`, the next
packet resumes a HeldAu whose `start`/`end` were computed against the
pre-gap buffer but are applied to the unrelated post-gap bytes — splicing
audio across the gap at best, and indexing past the end of the new, shorter
buffer at worst.
The two existing discontinuity tests use plain AC-3 (bsid < 11), which never
holds an access unit open, so neither of them reaches this reset.
### `a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes`
Mutation this catches: deleting (or inverting) the resume-path re-check `if
drop_reason.is_none() && self.tally.is_poisoned()` at the top of
`scan_access_units`. The verdict on a held access unit is frozen at the
moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
previous access unit is closed — so the very drop that crosses the poison
threshold lands after the next unit's verdict was already taken as `None`.
Without the re-check that unit is emitted as an ordinary frame after the
track has been judged too damaged to mux: corrupt audio passed through as
success, and worse, it is the ONE frame that escapes a whole-track fallback
whose entire point is that nothing after the verdict ships.
Neither existing held-AU test reaches this: both keep a pristine tally.
The fixture drives the exact interleaving above. `DropTally` poisons once
`verified_dropped * 2 > kept + dropped` past the 200-AU gate, so one PES
carries 200 CRC-failing `substreamid`-0 syncframes followed by a clean one.
Closing corrupt unit #200 (which happens only when the clean frame is
reached) latches the poison — after that clean frame's own verdict was
computed. Being an E-AC-3 unit that can still gain substreams, it is then
HELD across the boundary with `drop_reason: None`, which is precisely the
state the re-check exists for.
-37
View File
@@ -1,37 +0,0 @@
# ADTS decodability gate
## Module rationale (`src/mux/codec/adts.rs`)
Per the ADTS framing defined in ISO/IEC 13818-7 / ISO/IEC 14496-3, a header
is structurally invalid in exactly three ways this gate treats as hard
rejects: syncword != 0xFFF, a reserved `sampling_frequency_index` (the sample
rate table has 13 valid entries, so index >= 13 is reserved), and
`aac_frame_length < 7` (shorter than the fixed+variable header itself). The
optional 16-bit ADTS CRC is not verified here — it is simply skipped. So the
gate enforces those three rejects: a packet that begins with the ADTS sync
but is otherwise malformed is dropped; a packet with no ADTS sync is raw AAC
(e.g. from an MP4 container, which carries no ADTS header) or a continuation
and passes through unchanged — never false-dropped. Raw AAC has no per-frame
integrity data, so like LPCM it cannot be gated.
## Test: `a_crc_present_header_shorter_than_its_own_crc_is_invalid`
A header that CLAIMS a CRC (protection_absent = 0) but declares a frame
length too short to contain one.
`aac_frame_length` counts the header and the CRC, not just the payload,
so with a CRC present the smallest structurally possible frame is 9
bytes: the 7-byte fixed+variable header plus the 16-bit crc_check.
The gate compared against a flat 7 and never read protection_absent at
all, so a frame whose own header says it is impossible was classified
Valid and forwarded to the muxer.
## Test: `dropped_frames_are_counted_but_their_duration_is_not_invented`
A dropped ADTS frame is dropped BECAUSE its header failed validation, so
the very fields a duration would come from (sampling_frequency_index, and
the 1024-samples-per-AAC-frame constant applied to it) are the ones known
to be untrustworthy. This gate therefore reports the drop's duration as
zero rather than deriving a number from a header it has just rejected —
the honest answer, and the one the count alongside it must be read with.
A nonzero constant here would report silence that was never measured.
-237
View File
@@ -1,237 +0,0 @@
# libfreemkv API Design
## Principles
1. Lib provides building blocks. App composes them.
2. No English text in lib. Error codes only. App handles i18n.
3. No display logic in lib. App decides what to show.
4. Streams are the pipeline. Each stage wraps the next.
5. Lib fires events. App listens.
## Core API
```rust
// Open drive — explicit steps, app prints between them
let mut drive = Drive::open(path)?;
drive.wait_ready()?;
drive.init()?;
drive.probe_disc()?;
// Scan disc
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
// Browse
disc.titles // Vec<DiscTitle>
disc.format // BD / UHD / DVD
disc.capacity_gb()
```
## PES Pipeline (primary API)
The PES pipeline is the main way to move content. All streams produce/consume
PES frames. The pipeline just reads frames and writes frames.
```rust
// URL-based — any source to any destination
let opts = InputOptions::default();
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
output.write(&frame)?;
}
output.finish()?;
```
The `FrameSource` and `FrameSink` traits — direction is type-checked, so
calling `read()` on a write-only sink (or `write()` on a read-only source)
is a compile error rather than a runtime fault:
```rust
pub trait FrameSource: Send {
fn read(&mut self) -> Result<Option<PesFrame>, Error>;
fn info(&self) -> &DiscTitle;
fn codec_private(&self, track: usize) -> Option<Vec<u8>> { None }
fn headers_ready(&self) -> bool { true }
}
pub trait FrameSink: Send {
fn write(&mut self, frame: &PesFrame) -> Result<(), Error>;
fn finish(self: Box<Self>) -> Result<(), Error>;
fn info(&self) -> &DiscTitle;
}
```
## Streams
All streams implement `FrameSource` (read) and/or `FrameSink` (write); the
directional split prevents runtime "wrong-direction" errors. URL-based
resolvers open any stream by string.
| Stream | Input | Output | URL | Transport |
|--------|-------|--------|-----|-----------|
| DiscStream | Yes | -- | `disc://` `disc:///dev/sg4` | Optical drive via SCSI |
| IsoStream | Yes | Yes | `iso://path.iso` | Blu-ray ISO image |
| MkvStream | Yes | Yes | `mkv://path` | Matroska container |
| M2tsStream | Yes | Yes | `m2ts://path` | BD-TS with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | `network://host:port` | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | `stdio://` | Raw byte pipe |
| NullStream | -- | Yes | `null://` | Discard sink (byte counter) |
All URLs require a `scheme://path` format. Bare paths are rejected.
```rust
// PES pipeline (frame-level) — input() returns Box<dyn FrameSource>,
// output() returns Box<dyn FrameSink>.
let input = libfreemkv::input("disc:///dev/sg4", &opts)?; // DiscStream
let input = libfreemkv::input("iso://Movie.iso", &opts)?; // IsoStream
let output = libfreemkv::output("mkv://Movie.mkv", &title)?; // MkvOutputStream
let output = libfreemkv::output("m2ts://Movie.m2ts", &title)?; // M2tsOutputStream
let output = libfreemkv::output("network://192.0.2.10:9000", &title)?; // NetworkOutputStream
let output = libfreemkv::output("null://", &title)?; // NullOutputStream
```
### FMKV Metadata Header
M2tsStream and NetworkStream embed a JSON metadata header before the BD-TS data:
```
[8B magic "FMKV\0\0\0\0"][4B JSON length][JSON metadata][padding to 192B boundary][BD-TS data...]
```
The header carries title name, duration, codec_privates, and full stream layout
(PIDs, codecs, languages, labels). This allows the receiving end to set up
demuxing and track metadata without scanning the TS.
## Events
Lib fires events during operations. App provides a callback. No display, no text.
```rust
pub struct Event {
pub kind: EventKind,
}
pub enum EventKind {
// Init / scan
DriveOpened { device: String },
DriveReady,
InitComplete { success: bool },
ProbeComplete { success: bool },
ScanComplete { titles: usize },
// Read pipeline
BytesRead { bytes: u64, total: u64 },
ReadError { sector: u64, error: Error },
SpeedChange { speed_kbs: u16 },
ExtentStart { index: usize, start_sector: u64, sector_count: u64 },
SectorSkipped { sector: u64 },
BatchSizeChanged { new_size: u16, reason: BatchSizeReason },
Complete { bytes: u64, errors: u32 },
// Kept for forward-compat; not emitted in 0.13.6+
Retry { attempt: u32 },
SectorRecovered { sector: u64 },
}
```
Emission notes:
- `BytesRead { bytes, total }` is emitted from `DiscStream::fill_extents`
after each successful sector read. `bytes` is the cumulative running
total; `total` is the precomputed extent sum (0 if unknown).
- `SpeedChange` is emitted from the public `Drive::set_speed` API path.
It is no longer emitted from a recovery hot loop (recovery loop removed
in 0.13.6).
- `BatchSizeChanged` fires from the `DiscStream` adaptive sizer on shrink
(read failed at a larger size) and on probe-up (clean-read streak hit
the threshold). Consumers use it to display a "recovering" state
distinct from "ripping normally".
- `Retry` and `SectorRecovered` are NOT emitted in 0.13.6+. They were
tied to the inline `Drive::read` recovery phases that were removed; the
variants are kept for forward compatibility so consumers' match arms
don't need conditional compilation.
Events report what happened. App decides what to do. GUI shows a dialog. CLI
prints a line. Server logs to file.
## File Layout
```
libfreemkv/src/
├── lib.rs Public exports
├── error.rs Error codes (no English)
├── event.rs Event types for callbacks
├── halt.rs Halt cancellation token (Arc<AtomicBool> wrapper)
├── io/ Pipeline + WritebackFile primitives
│ ├── mod.rs Re-exports WritebackFile, Pipeline, Sink, Flow
│ ├── pipeline.rs Generic Pipeline<I, R> + Sink trait
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), eject
│ ├── capture.rs Raw drive SCSI capture (INQUIRY/GET_CONFIG) for contribution
│ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup, per-format parsing)
│ ├── mod.rs Disc struct, scan, titles, formats
│ ├── bluray.rs Blu-ray / UHD scanning (MPLS/CLPI-driven)
│ ├── dvd.rs DVD-Video scanning (IFO-driven)
│ ├── hddvd.rs HD-DVD scanning
│ ├── extract.rs Per-extent content extraction
│ ├── encrypt.rs Encrypted-range mapping for content reads
│ ├── dvd_audio_probe.rs DVD audio-stream probing
│ └── pgs_forced_probe.rs PGS forced-subtitle probing
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── unlock.rs Unlocker trait + registry (pluggable unlock seam)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
├── css/ DVD CSS cipher
├── decrypt.rs Unified decrypt dispatcher (AACS/CSS/None)
├── pes.rs PES frame types, FrameSource / FrameSink traits
├── sector/ Sector I/O
│ ├── mod.rs SectorSource, SectorSink traits
│ ├── file.rs FileSectorSource, FileSectorSink (ISO-backed)
│ └── decrypting.rs DecryptingSectorSource decorator
├── udf.rs UDF 2.50 filesystem parser
├── mpls.rs MPLS playlist parser
├── clpi.rs CLPI clip info parser
├── ifo.rs DVD IFO parser
├── labels/ BD-J label extraction (5 format parsers)
├── keydb.rs KEYDB download, parse, save
├── identity.rs DriveId from INQUIRY
├── speed.rs DriveSpeed enum
├── mux/
│ ├── mod.rs Public mux exports
│ ├── resolve.rs URL parser + input/output (Box<dyn FrameSource/Sink>)
│ ├── meta.rs FMKV header format
│ ├── disc.rs DiscStream (optical drive → PES)
│ ├── iso.rs IsoStream (ISO image read)
│ ├── isowriter.rs ISO image writer (UDF, AVDP, multi-extent)
│ ├── mkvstream.rs MkvStream (bidirectional Matroska)
│ ├── mkvout.rs MkvOutputStream (PES → MKV)
│ ├── m2ts.rs M2tsStream (BD-TS)
│ ├── pesout.rs PES output streams (M2ts, Network, Stdio, Null)
│ ├── network.rs NetworkStream (TCP + FMKV header)
│ ├── stdio.rs StdioStream (stdin/stdout pipe)
│ ├── null.rs NullStream (discard + byte counter)
│ ├── lookahead.rs LookaheadBuffer (codec header scanning)
│ ├── ts.rs BD-TS demuxer + PAT/PMT scanner
│ ├── tsreader.rs TS reader utilities
│ ├── tsmux.rs TS muxer (PES → BD-TS packets)
│ ├── ps.rs MPEG-2 PS demuxer (DVD)
│ ├── ebml.rs EBML read/write primitives
│ ├── mkv.rs MKV muxer (tracks, clusters, cues)
│ └── codec/ Frame parsers (H.264, HEVC, MPEG-2, VC-1, AC3, EAC3, DTS, TrueHD, LPCM, PGS)
└── ...
freemkv/src/
├── main.rs CLI dispatcher (URL routing)
├── pipe.rs PES pipeline — source → dest copy
├── disc_info.rs Disc/file info display
├── info.rs Drive info + profile submission
├── strings.rs i18n string table
├── output.rs Verbosity-filtered output
└── build.rs Bundled locale code generation
```
-222
View File
@@ -1,222 +0,0 @@
# libfreemkv Architecture
Open source optical drive access library for 4K UHD Blu-ray, Blu-ray, and DVD.
Rust library with profiles bundled and all SCSI communication handled in-process.
AACS decryption requires an external `keydb.cfg` (default
`~/.config/freemkv/keydb.cfg`) — the derivation math is internal, but no AACS key
material is compiled in; DVD CSS player keys are the only compiled-in keys.
**Repository:** <https://github.com/freemkv/libfreemkv>
**License:** MIT
---
## Design Principles
1. **CLI is dumb.** All drive communication, disc parsing, AACS decryption, and
format handling live in the library. CLI binaries are thin wrappers that call
`Drive::open()` and `Disc::scan()`.
2. **Firmware-clean core.** libfreemkv ships no firmware, no unlock CDBs, and no
drive profiles. Drive-unlock logic is plugged in by an external crate through
the `Unlocker` trait + registry (`register_unlocker`); without one the library
still rips via the host-certificate AACS handshake.
3. **Transparent AACS.** The `ContentReader` decrypts on the fly when keys are
available. Callers read cleartext sectors without knowing whether the disc
was encrypted.
4. **Structured errors, no English.** Every error has a numeric code (E1000-E8000).
The library never formats user-facing messages -- applications do that.
5. **Library-agnostic.** No concept of "supported" vs "unsupported" drives at a
policy level. If a profile exists, the library uses it.
6. **Streams are dumb pipes.** Streams read/write PES frames. They don't know
about encryption, transport format, or source type. Decrypt is a stream-internal
concern; the pipeline just moves frames.
---
## Module Map
```
libfreemkv (lib.rs)
│
├── Drive Access
│ ├── drive Drive — open, identify, init, single-shot read
│ ├── scsi ScsiTransport trait + platform backends (sg async, IOKit, SPTI)
│ ├── unlock Unlocker trait + registry — the pluggable unlock seam
│ ├── identity DriveId from INQUIRY + GET_CONFIG 010C
│ ├── speed DriveSpeed enum, SET CD SPEED CDB builder
│ └── event Event system for drive status callbacks
│
├── Disc Scanning
│ ├── disc Disc::scan() — titles, streams, extents, AACS setup
│ ├── udf UDF 2.50 filesystem reader (metadata partitions)
│ ├── mpls MPLS playlist parser — clips, streams, STN table
│ ├── clpi CLPI clip info parser — EP map, sector extents
│ ├── ifo DVD IFO parser — title sets, PGC chains, cell addresses
│ └── labels/ BD-J label extraction (5 formats: Paramount, Criterion, Pixelogic, CTRM, Deluxe)
│
├── Encryption
│ ├── aacs/ AACS handshake, KEYDB, VUK lookup, MKB, unit decryption
│ ├── css DVD CSS cipher — table-driven, no external keys needed
│ └── decrypt decrypt_sectors() — unified AACS/CSS/None dispatcher
│
├── Streaming
│ ├── mux/ Stream implementations (Disc, ISO, MKV, M2TS, Network, Stdio, Null)
│ ├── pes PES frame types; the unified pes::Stream (PesStream) read/write trait
│ └── sector/ SectorSource / SectorSink traits, FileSector{Source,Sink}, DecryptingSectorSource
│
├── I/O Primitives
│ ├── halt Halt cancellation token (one Arc<AtomicBool>, cloneable)
│ └── io/ Pipeline<I, R> + Sink trait + WritebackFile (bounded-cache writer)
│
├── Support
│ ├── keydb KEYDB.cfg download, parse, verify, save
│ └── error Error enum with numeric codes E1000-E8000
│
└── lib.rs Public API re-exports
```
---
## Drive Access Flow
```
Drive::open(Path::new("/dev/sg4"))
│
├─ scsi::open() Open /dev/sg4 (async write/poll/read)
├─ DriveId::from_drive() INQUIRY + GET_CONFIG 010C
└─ Drive ready for init/read
```
After open:
- `init()` -- routes to the matching registered unlocker (if any); otherwise
a no-op and the cert handshake carries the disc
- `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (`READ_TIMEOUT_MS` 10 s vs. `READ_RECOVERY_TIMEOUT_MS` 60 s)
- `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1
(`freemkv_engine::recovery::patch`, in the engine crate) handles bad-range
retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
the stop-wedge postmortem (2026-04-25).
---
## Disc Scanning Flow
```
Disc::scan(&mut drive, &ScanOptions)
│
├─ READ CAPACITY Get disc size in sectors
├─ udf::read_filesystem() Parse UDF 2.50 (AVDP → VDS → metadata → FSD → root)
├─ For each BDMV/PLAYLIST/*.mpls:
│ ├─ mpls::parse() Extract play items, STN streams
│ └─ For each clip:
│ └─ clpi::parse() EP map → sector extents for the clip's time range
├─ labels::detect() Parse BD-J JARs for stream labels
├─ Detect AACS Check for /AACS directory on disc
└─ Disc::setup_aacs() Handshake + KEYDB → VUK → unit keys (if encrypted)
```
For DVD:
```
Disc::scan_dvd(&mut drive, &ScanOptions)
│
├─ ifo::parse() Parse VIDEO_TS.IFO — title sets, PGC chains
├─ CSS detection Check disc structure flag
└─ CSS key cracking Table-driven, no KEYDB needed
```
The result is a `Disc` with:
- `titles: Vec<DiscTitle>` -- sorted by duration, each with streams, sector extents, codec_privates
- `decrypt_keys()` -- DecryptKeys for content decryption
- `encrypted: bool` -- whether the disc uses AACS/CSS
---
## AACS Decryption
Four key resolution paths, tried in order:
| Path | Method | Speed |
|------|--------|-------|
| 1 | VUK lookup by disc hash in KEYDB.cfg | Instant |
| 2 | Media Key + Volume ID from KEYDB → derive VUK | Fast |
| 3 | Processing Keys + MKB → Media Key → VUK | Medium |
| 4 | Device Keys + MKB subset-difference tree → VUK | Slow |
The AACS handshake (`aacs/handshake`) performs ECDH key agreement over the
AACS 1.0 160-bit elliptic curve to obtain:
- **Volume ID** -- needed for VUK derivation (paths 2-4)
- **Read Data Key** -- needed for AACS 2.0 (UHD) bus decryption
Content decryption uses AES-128-CBC on 6144-byte aligned units. The
`ContentReader` handles this transparently. Streams that read sectors
(DiscStream, IsoStream) decrypt internally — the pipeline sees clean bytes.
---
## Error Codes
All errors carry a numeric code for programmatic handling. No user-facing text
is baked into the library.
| Range | Category | Examples |
|-------|----------|----------|
| E1xxx | Device errors | `DeviceNotFound`, `DevicePermission` |
| E2xxx | Profile errors | `UnsupportedDrive`, `ProfileNotFound`, `ProfileParse` |
| E3xxx | Unlock errors | `UnlockFailed`, `SignatureMismatch`, `NotUnlocked`, `NotCalibrated` |
| E4xxx | SCSI errors | `ScsiError`, `ScsiTimeout` |
| E5xxx | I/O errors | `IoError` (wraps `std::io::Error`) |
| E6xxx | Disc format errors | `DiscError` (UDF, MPLS, CLPI parse failures) |
| E7xxx | AACS errors | `AacsError` (key resolution, handshake, decryption) |
| E8xxx | KEYDB errors | `KeydbError` (download, parse, save) |
---
## Platform Support
| Platform | Transport | Status |
|----------|-----------|--------|
| Linux | async sg write/poll/read on `/dev/sg*` | Supported |
| macOS | IOKit SCSITask | Supported |
| Windows | SPTI (`IOCTL_SCSI_PASS_THROUGH_DIRECT`) | Supported |
The `ScsiTransport` trait abstracts the platform. Adding a new platform requires
implementing `execute()` for that OS and wiring it into `scsi::open()`.
---
## Drive Unlock
libfreemkv carries no drive-unlock mechanism. The `Unlocker` trait + registry
(`src/unlock.rs`) is the seam: an external crate implements `Unlocker` and
registers it once via `register_unlocker(...)`. At drive-prep the registry is
walked in order and the first unlocker whose `matches()` is true is asked to
`unlock_drive()` over the raw `ScsiTransport`. If none match, the drive is left
untouched and the host-certificate AACS handshake carries the disc.
The implementor owns everything firmware-specific — drive profiles, vendor CDBs,
variant logic. Concrete unlockers live in the separate
**[freemkv-unlock](https://github.com/freemkv/freemkv-unlock)** repository, never
in libfreemkv. See [`drive-access.md`](drive-access.md#drive-unlock-seam) for the
trait definition and routing.
---
## Build
```
cargo build --release
```
Produces a Rust library crate. The `libc` dependency is unix-only (gated).
All three platforms build and pass CI.
-217
View File
@@ -1,217 +0,0 @@
# `src/mux/au_assembly.rs` — access-unit assembly
## Why this exists
The contract a codec parser converts is `PES → access units (Frames)`. A
*transport* stream hands the parser one AU per PES for free (BD aligns one
access unit per PES; the TS demuxer reassembles to the
`payload_unit_start_indicator`). A *program* stream does not — the PS muxer
chops the elementary stream into fixed-size PES fragments with no AU
alignment, and only the first fragment of an AU carries a PTS. So a parser
that assumes one-AU-per-PES (h264/hevc/vc1, written against TS) mis-frames a
program stream, while `mpeg2` — the DVD/PS codec — must reassemble across PES.
`AuAssembler` is that reassembly, factored out so EVERY program-stream video
parser shares one implementation instead of hand-rolling the buffer. The
h264/hevc/vc1 parsers (`Mode::StartCode` / `Mode::Vc1`) and the MPEG-2 parser
(`Mode::Mpeg2`, via `AuAssembler::mpeg2`) all drive it. It buffers PES-fragment
bytes and emits one AU per codec AU boundary, carrying the AU-start
timing/source forward. Since the boundary is a codec start code, it lives with
the codec parser (which picks the marker); only the generic buffering +
timing-carry is shared here.
This is *inside* the parser, not a pipeline stage: the pipeline stays
`Demuxer → PES → Parser → Frames`, and the demuxer stays codec-agnostic. Every
stream a parser sees runs through one of these — self-framing codecs (MPEG-2,
audio) use `Mode::Passthrough` so the parser code path is uniform.
## `MAX_AU_BUFFER`
Safety cap on a single in-progress access unit. A real coded picture is far
below this; a stream that never yields a second AU boundary is force-flushed
at the cap rather than buffering without bound on hostile/corrupt input.
## `MAX_MARKS`
Cap on buffered timing/discontinuity marks. A real access unit spans a few
hundred PES fragments at most; this bounds the mark deques so a run of
zero-length (or start-code-free) timed fragments — which grow no buffer bytes
and so never trip the `MAX_AU_BUFFER` mark-prune — cannot accumulate marks
without bound on hostile/corrupt disc input.
## `AuAssembler::for_codec`
Video codecs whose parsers assume AU-complete PES (H.264 / HEVC / VC-1) get a
`Mode::StartCode` assembler; MPEG-2 (self-reassembles) and audio/subtitle
codecs (self-framing) get `Mode::Passthrough` so callers can run every stream
through this uniformly.
## `AuAssembler::mpeg2`
The MPEG-2 parser owns one of these directly (rather than hand-rolling the
buffer): the demux layer runs MPEG-2 through `Mode::Passthrough` and hands
each fragment to the parser, which feeds them here to be reframed on picture
boundaries.
## `AuAssembler::push`/`push_owned`
For a self-framing (`Passthrough`) stream the payload is MOVED straight into
the emitted unit with no copy — the common DVD/HD-DVD case (MPEG-2 video, all
audio). A buffering mode copies into `buf` exactly as `push`.
## `take_front`
Detaches `buf[..end]` as the emitted AU's own `Vec` and leaves `buf` holding
the tail.
The AU's bytes are HANDED OVER — `buf`'s allocation becomes the returned `Vec`
and a fresh buffer (pre-sized to the same capacity, so the next AU accumulates
without re-growing) takes its place holding only the short tail.
`buf[..end].to_vec()` + `drain(..end)` instead copied every AU out in full: on
a UHD HEVC title that is a whole-frame memcpy (hundreds of KB) per coded
picture, ~200k times, for bytes that are about to be discarded from `buf`
anyway.
The allocation COUNT is unchanged (one per AU either way — the frame `Vec`
before, the replacement buffer now), so the only difference is the copy that
no longer happens. Nothing depends on `buf` keeping its identity: the only
state tied to `buf[0]`'s position is `base`/`scan_pos`/`opener_pos`, which the
caller updates immediately after.
Falls back to a copy when the buffer's capacity is far larger than the AU (a
small AU after a multi-MB one): handing over would otherwise attach an
oversized idle allocation to a small frame for as long as the frame queues
downstream, trading a copy for resident memory.
That fallback must not become permanent. `buf`'s capacity used to be a one-way
high-water mark — the replacement buffer was created with `cap.max(tail_len)`,
and the copy path's `drain` also preserves `cap` — so once ONE large AU had
been assembled, every later smaller AU satisfied `cap > 2*end` and took the
copy path forever. On a UHD HEVC title the first IDR grows `buf` to ~4-8 MB,
after which each ~200-400 KB P/B AU paid a whole-AU allocation plus a
whole-AU memcpy plus a tail memmove for ~99% of the ~200,000 coded pictures —
tens of GB of exactly the memcpy this handover exists to remove. So the copy
path now also RELEASES the high-water capacity, which re-arms the handover
for the next AU: one copy after a size step down, not one per frame forever.
## `drop_marks_before`
This is the STREAM-START case: bytes ahead of the first AU boundary are the
tail of an access unit that began before we had sync, and there is no prior
AU for them to be discontinuous *from*. Carrying a mark forward here would arm
the resync gate at the head of every title and drop its first GOP.
## `discard_gap_before`
This is the BACKSTOP case: `MAX_AU_BUFFER` bytes accumulated with no AU start
code in them, so the run is unusable and gets thrown away. Unlike the
stream-start trim above, there IS a prior AU here, and whatever follows
definitively does not continue it — a decoder handed the next picture would
resolve its references against frames separated from it by megabytes of
discarded data.
So the discard is itself a discontinuity, whether or not the source signalled
one. It is recorded as a sticky flag rather than an offset mark because a
mark placed at the new base would be retired moments later by the pre-sync
trim that follows resync — the gap has to outlive the bytes that caused it.
It arms the resync gate, which drops to the next keyframe instead of emitting
a picture with dangling references.
Timing marks before `off` are still retired — they describe bytes that no
longer exist, and the AU that eventually emits takes its PTS from the
fragment that actually opened it.
## Tests
### `a_backstop_discard_marks_the_next_au_discontinuous`
The 8 MiB backstop throws away a start-code-free run as unusable. The AU that
eventually emits after that discard MUST be marked discontinuous, whether or
not the source ever signalled a discontinuity: megabytes of the stream are
simply gone, so the next picture cannot resolve its references against the
last one that was emitted.
`discontinuity` is what arms the resync gate downstream (`resync.rs`, driven
from `mux/disc.rs`), which drops to the next keyframe rather than emitting a
picture with dangling references. If the flag is retired with the discarded
bytes, the gate never arms and the broken picture goes out — a silent
corruption, which is the one class of loss this crate refuses to have.
### `a_stream_start_trim_does_not_mark_the_first_au_discontinuous`
The opposite case, and the reason the two call sites are separate. Bytes
ahead of the FIRST access-unit delimiter are the tail of an AU that began
before we had sync. There is no prior AU for them to be discontinuous from, so
retiring the marks there is right — and necessary: marking the first AU of
every title discontinuous would arm the resync gate at the head of each one
and drop its opening GOP.
### `a_source_signalled_discontinuity_reaches_the_au_it_opens`
A source-signalled discontinuity reaches the AU it opens. This is the
`disc_marks` path — the ORIGINAL mechanism, distinct from the sticky
`pending_gap` the backstop sets. Nothing else pins it: the two tests above
drive `pending_gap`, and a mark placed on a fragment that is later discarded
is retired by design.
Deliberately NOT combined with the backstop. A previous version of this test
signalled the discontinuity on the first over-cap push and asserted the flag
on the AU after the discard — but that first run still has the next AU's
delimiter at `buf[0]`, so it force-flushes as an over-long AU, and THAT AU
consumes the mark. The assertion was then satisfied entirely by
`pending_gap`, making the test a duplicate of the one above it under a name
promising something else. The two mechanisms cannot be isolated in one
fixture, so they get one test each.
### `drained_au_takes_over_the_buffer_allocation_without_copying`
MEASURED: a drained AU must be HANDED the accumulation buffer's allocation,
not copied out of it. The emitted `Vec`'s data pointer is the buffer's own
pointer — which is only true if no full-frame copy happened.
(`buf[..end].to_vec()` allocates fresh, so the pointers differ.) One
whole-AU memcpy per coded picture is ~200k memcpys of a few hundred KB each
on a UHD feature.
### `handover_survives_a_large_au_instead_of_copying_every_later_one`
MEASURED: `take_front`'s copy fallback must not become permanent.
`buf`'s capacity used to be a one-way high-water mark, and the copy path's
`drain` preserves it, so after ONE large AU every later smaller AU satisfied
`cap > 2*end` and copied forever. On a UHD HEVC title the first IDR grows
`buf` to multiple MB, after which ~99% of the ~200,000 coded pictures each
paid a whole-AU allocation + whole-AU memcpy + tail memmove — tens of GB of
exactly the copy the handover exists to remove. Counted at the copy path
itself: one copy is expected right after the size step down; a per-frame
copy is the bug.
### `discarded_pre_sync_marks_do_not_time_the_first_access_unit`
After the pre-sync bytes are discarded, the emitted AU must take the timing
of the fragment that ACTUALLY opened it. `drop_marks_before` is what retires
the discarded fragment's marks; a no-op there stamps the first real access
unit with the PTS and source of bytes that were thrown away — a whole-title
A/V sync offset, since every later frame is timed relative to it.
### AU-opener detection: the per-mode start-code rule
`au_opener_from` duplicates a rule each codec parser also encodes; the tests
below pin it to the normative byte values so a drift between copies shows up.
### `au_opener_from_locates_the_real_start_code_per_codec`
The opener offset must be the position of the real start code, never a fixed
0. A constant `Some(0)` makes every pre-sync run of junk bytes look like the
head of an access unit, so the first AU of every stream that does not begin
exactly on a start code is emitted with junk glued to its front.
### `for_codec_routes_each_video_codec_to_its_reassembly_mode`
`for_codec` is the dispatch that decides whether a stream is REASSEMBLED
across PES fragments or passed straight through. Getting it wrong is silent:
an H.264/HEVC/VC-1 stream routed to `Passthrough` on a program source emits
one "frame" per PES fragment — a few hundred bytes of a coded picture, framed
as a whole access unit — and the output plays as corruption, not as an error.
Each mode is identified BEHAVIOURALLY (feed a two-AU stream in two halves and
see whether it reassembles), so the case cannot pass by matching a constant.
-42
View File
@@ -1,42 +0,0 @@
# BDMV disc-library metadata (`bdmt.rs`)
Every commercial Blu-ray carries a disc-library metadata directory with one
XML file per shipped language. The schema is the Blu-ray "disc library
metadata" namespace (`urn:BDA:bdmv;disclibmeta`), conventionally prefixed
`di:`. Fields commonly present:
- `<di:title>` or `<di:name>` — the title string. Vendor practice varies
(Paramount discs tend to use `<di:name>`).
- `<di:description>` — optional synopsis (often absent on retail discs;
common on box sets and special editions).
- `<di:discNumber>` / `<di:numSets>` (or `<di:numberOfSets>`) — set position
for multi-disc releases.
This module is intentionally separate from the BD-J `StreamLabel` parsers
under `labels/*.rs`. The XML here is disc-level (title, description, set
position), not per-stream.
Real-world XML is irregular: missing description elements, multiple title
elements (first one wins), and occasional malformed content. Extraction is
best-effort — a malformed file is treated as "no metadata" (returns `None`
from the helper), and the caller can still get metadata from
sibling-language XML files.
## `MAX_BDMT_BYTES`
The size comes from attacker-controlled UDF metadata; real files are a few
KB, so 1 MiB is generous while preventing a crafted huge-size entry from
triggering an oversized allocation in `read_file`.
## `parse_bdmt_xml`
Title-element preference: `<di:name>` → `<di:title>` →
`<di:tableOfContents>/<di:titleName>` (first match wins, per the
authoring-tool conventions documented above).
## `looks_like_xml`
Rejects candidate description strings that are themselves XML fragments —
observed on a captured disc, where `<di:description>` contained
`<di:thumbnail href="…"/>` child elements and no actual prose. Surfacing
that raw to the JSON output is worse than dropping the field entirely.
-21
View File
@@ -1,21 +0,0 @@
# bdnav — HDMV navigation resolver
Notes relocated from `src/bdnav/mod.rs` doc comments (kept out of the source
to respect the comment-guard's prose caps).
## No per-disc special-casing
There is no per-disc special-casing: one correctly-booted VM resolves
densely-branched dispatchers as well as simple ones. The honest boundary is
BD-J (the feature is chosen by a Java Xlet, not by the HDMV VM); there — and
on any malformed data or non-convergence — the resolver ABSTAINS (`None`)
and selection falls back to the structural/heuristic order.
## `resolve_feature`
Resolves the playlist id the disc's First-Play navigation plays as the
feature, restricted to ids the caller marks as feature candidates
(`is_feature_candidate` — typically video-bearing, non-trivial titles, so a
short logo/pre-roll `PlayPlayList` is skipped). Returns `None` for BD-J
discs, missing/malformed nav data, or when navigation does not converge on
a candidate.
-239
View File
@@ -1,239 +0,0 @@
# src/disc/bluray.rs — extended notes
Overflow prose for comments that were trimmed to the internal 3-line cap in
`src/disc/bluray.rs`. Each section is pointed to by a `// See docs/bluray.md
— <topic>` comment at the corresponding site.
## CLIP_STREAM_EXTS
Stream-file extensions probed for a BD-family playlist clip, in priority
order. A clip is normally `.m2ts`; AACS 2.1 (FMTS) discs name the main
feature `.fmts` (an M2TS transport stream plus forensic variant segments)
and 3D discs use `.ssif`. `.m2ts` is tried first, so a normal clip is
unaffected — the fallback only runs when `.m2ts` is absent (exactly when
`file_extents` errors).
Scope: these are all variants that live in `BDMV/STREAM/` and are reached
through an MPLS playlist. HD-DVD's `.evo` does NOT belong here — HD-DVD is
a different tree (`HVDVD_TS/`) with `.XPL` playlists and needs its own
enumerator (a peer to `parse_playlist`), not another extension in this
list.
## scan_bluray_titles
Cancellation: `halt` is polled between playlists and once more after the
loop, and a read that fails with `Error::Halted` — how a live drive
reports a Stop, since `Drive::checked_exec` fails every command once its
flag is set and `Drive::read` preserves the variant — is propagated rather
than swallowed. `Halted` is the only error this returns from the
enumeration itself; an unreadable or unparseable playlist keeps its
best-effort skip.
It has to be an error and not a short title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it. Before this, a Stop
pressed mid-scan failed every remaining `.mpls` read in turn, each one
silently skipped, and the scan returned `Ok(truncated)` at rc=0.
## parse_playlist
Sums PlayItem durations; returns `Ok(None)` if the playlist is under 30
seconds (skips menu / clip-info stub playlists), fails to parse, or names a
clip that cannot be resolved. Physical sector extents are pulled from the
UDF allocation descriptors of each referenced `.m2ts` (deduplicated by
clip_id).
`Ok(None)` keeps its two benign meanings (unparseable MPLS, sub-30s
playlist) plus the deliberate "drop this title" outcomes below. `Err` means
the scan is over: today that is only `Error::Halted`, the operator's Stop,
which must not be reported as a disc that simply holds fewer titles.
## Test: parse_playlist_keeps_exactly_30_seconds
At exactly 30s the playlist is kept (`< 30.0` is strict). The fixture is a
fully wired BDMV (`make_bdmv_fs`), not the bare `make_min_fs` this used to
use. `make_min_fs` lays no STREAM/CLIPINF, so the play item's `.clpi` read
now fails and the title drops — which is correct behaviour for an
unresolvable clip but has nothing to do with the 30-second boundary this
test exists to pin. Wiring the clip keeps the test measuring exactly one
thing.
## make_bdmv_fs_ssif
Full BDMV with a real Blu-ray 3D layout: `.ssif` files under
`BDMV/STREAM/SSIF/<clip>.ssif` (note the SSIF subdirectory, unlike
`make_bdmv_fs_ext`) plus a matching `.clpi` in CLIPINF. Resolving the SSIF
is what latches `is_3d = true` in `parse_playlist`.
## Test: parse_playlist_fmts_clip_resolves_extent
AACS 2.1: the feature clip is `00001.fmts`, NOT `.m2ts`. The
`CLIP_STREAM_EXTS` fallback in `parse_playlist` must still resolve the
physical extent — before the fix the hard-coded `.m2ts` path errored,
yielding empty extents (a silent empty rip and 0 encrypted samples for key
resolution). Size still comes from the `.clpi`, which parses regardless.
## Test: parse_playlist_dedups_repeated_clip_extents_and_size
THE 0.31.0 DEDUP PATH. A playlist that references the SAME clip_id from
multiple PlayItems (seamless split / looped segment) must count the
physical extents and packet bytes exactly once — mux reads extents in
order, so a duplicate would mux the A/V twice and inflate `size_bytes`
(bluray.rs: `first_ref = seen_clips.insert(...)` gates both `total_size +=`
and the `extents.push`). Per-PlayItem Clip entries are still recorded for
both.
## Test: parse_playlist_missing_clpi_yields_no_title
A clip whose `.clpi` is missing must yield no title.
This test used to assert the opposite — a title with `size_bytes == 0` and
empty extents — and its docstring quoted the buggy control flow ("bluray.rs
only fetches extents inside the `if let Ok(clpi_data)`") as if it were the
specification. It was blessing the defect: the title's `duration_secs` is
summed from the PlayItems before the clip is resolved, so the returned
title advertised the movie's full runtime while carrying none of its
bytes, and the discarded error meant not one log line said so. That is the
flagship failure class of this crate — a failure that looks like success —
reached through an ordinary missing or scratched CLIPINF file.
The correct behaviour is the same as for a clip whose extents cannot be
resolved (see `parse_playlist_unreadable_clip_icb_yields_no_title`): drop
the title and log the read's own error code.
That last clause is now asserted, not merely asked for. The site's comment
insists on `e.code()` because a missing `.clpi` (E6003), a scratched one
(E6000) and a malformed one (E6002) are different populations, but nothing
checked it: putting a literal back compiled and passed. Mutation: `"E6017"`
(or any fixed code) in place of `"E{}", e.code()` fails here, and so does
dropping the warn entirely — a silent drop is the same invisible title
loss this test was written for, one step later.
## Test: parse_playlist_unrecorded_extent_yields_no_title
A clip stream whose ICB declares an UNRECORDED (ECMA-167 4/14.14.1.1
type-1) extent must not yield a title at all.
The extent is allocated to the file but was never written, so the file's
content there is zeros while the media holds whatever was left at those
sectors. Neither answer a `(lba, sector_count)` read plan can give is true
— reading it splices undefined sectors into the rip as content, dropping
it slides every later extent's byte space — so the title is refused rather
than mis-ripped. This fixture is the shape a crafted disc uses to get such
a range into a title's extent list.
(The `sectors > 0 && lba > 0` filter below the resolver stays as defence in
depth; a zero-length AD is only reachable as an unrecorded descriptor,
since a zero-length TYPE 0 one terminates the AD list.)
## Test: parse_playlist_unreadable_clip_icb_yields_no_title
An unrecorded extent was never the only way a clip fails to resolve.
RED BEFORE GREEN: this fixture gives the .m2ts an ICB whose descriptor tag
is neither 261 nor 266, so `file_extents` returns `DiscRead` — the same
variant a scratched sector under a real clip's ICB produces, which is the
ordinary way this happens on real media. Before the fix only
`UdfUnrecordedExtent` set the drop flag, so this fell through to the "file
absent" path and `parse_playlist` returned a title: full declared duration
from the play item, `total_size` already counted from the .clpi, and zero
extents — a movie advertising its runtime with the content missing, and
not one log line. The clip must drop the title exactly as an unrecorded
extent does.
## Test: parse_playlist_logs_a_non_absence_ssif_failure_the_m2ts_fallback_hid
A non-absence SSIF failure that the `.m2ts` fallback then papers over must
be logged, with its own code.
`unresolved` had exactly one reader — `if let (None, Some(code))` — so a
code recorded for `/BDMV/STREAM/SSIF/<clip>.ssif` was thrown away whenever
the fallback succeeded. The disc here IS a 3D disc: the SSIF is present and
carries both eyes, and only its ICB is unreadable (a scratched sector, the
ordinary way this happens). The title shipped base-view 2D at rc=0 with
`is_3d` false and not one log line, so the operator's rip is silently
missing the dependent view and the journal cannot tell this disc apart
from one that was only ever 2D.
The title is deliberately still returned — the base view resolved, so
refusing would trade a degraded rip for no rip. The defect being fixed is
the silence, not the fallback.
Mutations this catches: deleting the new `(Some(_), Some(code))` arm, or
restoring the `if let (None, Some(code))` shape, leaves no event to find;
logging a fixed code instead of `e.code()` fails the code assertion;
making the arm `return Ok(None)` fails the title assertion.
## Test: parse_playlist_id_falls_back_to_zero_when_suffix_is_not_mpls
A filename that is long enough (>= 5 bytes) but does NOT end in ".mpls"
must NOT have its last 5 bytes stripped — the whole string is handed to
the numeric parse instead, which fails and falls back to playlist_id 0
(bluray.rs `filename.len() >= 5 && filename[len-5..].eq_ignore_ascii_case(".mpls")`).
## Test: scan_bluray_titles_skips_non_mpls_extension_file
A non-directory PLAYLIST entry whose name does NOT end in ".mpls" must be
skipped even though its content parses as a perfectly good (long) MPLS
playlist — extension gating, not content sniffing, decides eligibility
(bluray.rs `!entry.is_dir && entry.name...ends_with(".mpls")`).
## HaltingReader
A `SectorSource` that fails every read in `halt_range` with
`Error::Halted` — exactly how a live drive behaves once the operator
presses Stop: `Drive::checked_exec` fails every SCSI command with `Halted`
from then on, and `Drive::read` deliberately preserves the variant. Reads
outside the range still succeed, so a test can aim the cancel at one
structure and leave the scan far enough along to have something to
truncate.
## Test: halted_playlist_read_is_not_reported_as_a_shorter_disc
A Stop on a live drive never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The Blu-ray enumerator must not swallow that into a
successful scan.
RED BEFORE GREEN: with the propagation reverted this returned
`Ok([00800])` — the `if let Ok(mpls_data)` skipped the cancelled read of
00801.mpls, the loop ended, and a half-enumerated disc came back at
success. One title from a two-title disc is indistinguishable from a disc
that genuinely holds one title, and the caller caches and rips from it.
The halt lands on the second playlist deliberately: it is the last
iteration, so nothing after it would poll a flag — only propagating the
read's own error catches it.
## Test: halted_clpi_read_is_not_accounted_as_an_unresolvable_clip
The same cancel landing on a `.clpi` read must not be classified as an
unresolvable clip either.
RED BEFORE GREEN: with the `Err(Error::Halted)` arm removed from the
CLIPINF match, the cancel fell into the generic "clip could not be
resolved" arm, which logs a disc-defect code and drops the title —
accounting an operator Stop as a scratched disc, and (in the scan loop)
dropping every remaining playlist in turn for a truncated `Ok`. With the
fix the cancel is propagated with its own variant intact.
## Test: halted_extent_resolve_is_not_a_title_missing_its_clip
And the same cancel landing on `file_extents` — the clip's ICB, not its
CLIPINF — must propagate too.
RED BEFORE GREEN: `note` used to exempt `Halted` from the unresolved
classification (correctly — a cancel is not an authoring hole) but had no
way to propagate it, so the resolver simply produced no extents and
`parse_playlist` returned a title with the clip's runtime counted, its
`size_bytes` counted from the .clpi, and zero bytes behind it. Measured
with the propagation reverted: `Ok(Some((768000, [])))` — the flagship
defect shape, wearing a cancel.
## Test: read_meta_title_ignores_non_xml_file_regardless_of_content
A non-.xml file must be ignored even if its content looks like a valid
meta XML (contains a `<di:name>`) — extension gating, not content
sniffing, decides eligibility (bluray.rs `!e.is_dir &&
e.name...ends_with(".xml")`).
-61
View File
@@ -1,61 +0,0 @@
# `src/io/bounded.rs` — bounded-syscall primitive
## Why this exists
[`crate::halt::Halt`] is cooperative: callers poll `is_cancelled()`. It
cannot reach inside a syscall the kernel currently owns the thread for —
`libc::sync_file_range`, `libc::fsync`, `File::write` on NFS, and so on.
`/api/stop` from autorip therefore can't unstick a thread sitting in
such a syscall.
`bounded_syscall` is the escape hatch: it runs `op` on a fresh worker
thread, then `recv_timeout`s on a rendezvous channel for the result.
The wait is broken into ~250 ms slices so the calling thread can poll
the supplied `Halt` in between. If the deadline elapses or the halt
fires, the worker is intentionally leaked — the syscall will unwind
whenever the kernel decides, or at process exit, but the caller is
free to fall back to a degraded code path (skip the sync, log loudly,
etc.).
## Trade-offs
- **Thread per call.** Cheap (`std::thread::spawn` is < 100 µs on
Linux/macOS), but not free. Use on coarse-grained finalisation
syscalls (`sync_all`, `sync_file_range(WAIT_AFTER)`), not on hot
inner-loop writes.
- **Leak on timeout.** A wedged syscall keeps a kernel slot and a
user-space thread around for the rest of the process's life. Bounded
by the number of independent rip/mux sessions, which is one per
disc. The alternative — trapping the caller forever — defeats the
entire purpose of `/api/stop`.
- **Halt granularity ~250 ms.** Halt observation is not instant; it's
the worst-case latency of the `recv_timeout` slice. Good enough for
human-driven stop requests; not suitable for hard real-time
deadlines.
## Single source of truth
Do NOT inline this pattern. Every blocking-syscall wrapper in the rip
+ mux pipeline calls this helper, so changes (e.g. swapping the
channel impl, adjusting the poll slice, adding metrics) land in one
place.
## Platform
Pure `std::thread` + `std::sync::mpsc`. No `cfg(target_os)` needed
here — the helper itself is platform-agnostic. Callers that wrap
Linux-only syscalls (`sync_file_range`) still need their own
`#[cfg(target_os = "linux")]` gates; this helper does not.
## `bounded_syscall` contract details
- `halt` is polled at `POLL_INTERVAL` granularity. Pass `None` for
callers that don't (yet) have a halt token plumbed through —
behaviour degrades to deadline-only, matching the 0.20.5
`wait_after_with_timeout` shape this helper generalises.
- `op` returns `R: Send + 'static`. The closure must own everything it
touches because it may outlive this call (timeout / halt cases).
- On `Halted` / `Timeout` the worker thread is intentionally leaked:
the syscall will unwind whenever the kernel decides, or when the
process exits. The calling thread is never trapped inside a kernel
call.
-111
View File
@@ -1,111 +0,0 @@
# `byte_prefetcher` — design notes and test rationale
`BytePrefetcher` (`src/io/byte_prefetcher.rs`) is the `std::io::Read`
analogue of `crate::sector::PrefetchedSectorSource`. It spawns a
producer thread that fills a bounded pool of `Vec<u8>` chunks from the
underlying reader and ships them through a channel; the consumer pulls
filled chunks, uses them, and sends the empty `Vec<u8>` back through a
recycle channel so the producer can re-fill in place. Result: zero
allocations and zero cross-thread frees in the steady-state hot loop.
This is the byte-stream half of the freemkv mux highway —
`BytePrefetcher` feeds `crate::mux::demux_thread::DemuxThread` for
`m2ts://` (the only in-tree caller today, via `crate::mux::resolve`),
and works for any stream whose source is an `io::Read` rather than a
`SectorSource`.
## `PrefetchShell`
Drop blocks the calling thread until the producer exits. To guarantee
a prompt exit, drop the forward receiver and the recycle sender first
so the producer observes channel disconnection (or cancel the `Halt`
passed to `BytePrefetcher::new`, which the producer polls at
`POLL_INTERVAL` granularity even while parked on a channel op).
## Test coverage rationale
- **`recycle_depth_is_forward_depth_plus_one`** — `RECYCLE_DEPTH` must
be one MORE than `FORWARD_DEPTH` per its own doc comment: the
producer needs at least one buffer to fill while the consumer holds
the other `FORWARD_DEPTH`-worth in flight. A `+` -> `*`/`-` mutation
on `FORWARD_DEPTH + 1` would under-size the recycle channel (e.g.
`FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one short), which starves the
producer of a spare buffer.
- **`into_channels_drop_releases_producer`** — the CRITICAL
regression: after `into_channels`, dropping the returned forward
receiver + recycle sender must let the producer observe
disconnection and exit, so dropping the `PrefetchShell` (which
joins the producer) returns promptly. With the old clone+forget the
leaked endpoints kept the producer blocked and this join hung
forever.
- **`delivers_all_bytes_in_order_across_chunks`** — CORE CONTRACT: the
prefetcher must deliver every source byte, in order, exactly once —
never silently truncate or duplicate. Source is 5000 bytes; chunk
size 1024 forces multiple chunks (4 full + 1 short of 904). The
reassembled stream must equal the source. Mutation: replacing
`buf.truncate(n)` with a no-op would over-report bytes on the final
short read and this fails.
- **`short_read_truncates_to_actual_length`** — a reader that returns
fewer bytes than requested per call must NOT leave stale tail bytes
in the delivered chunk. Cursor over 10 bytes with a 4096 chunk
yields a single 10-byte chunk; the consumer must see exactly 10
bytes, not 4096. Grounds `buf.truncate(n)`. Mutation: delete the
truncate and the chunk would carry 4086 zero bytes of padding,
failing the length assert.
- **`empty_source_yields_clean_eof_no_batches`** — EOF semantics: an
empty source (Cursor over `[]`) yields `read() == Ok(0)` on the
first call, which the producer treats as EOF and returns, dropping
tx. The consumer sees RecvError (zero batches), NOT an Err batch and
NOT a zero-length Ok batch. Grounds the `Ok(0) => return` arm.
Mutation: changing `Ok(0) => return` to `Ok(0) => continue` would
spin forever (test times out).
- **`read_error_is_propagated_as_err_batch`** — a reader that fails
mid-stream must surface the `io::Error` as an `Err` batch on the
forward channel, not swallow it. We deliver one good chunk then an
error. The consumer must see the good bytes followed by the error.
Mutation: changing `let _ = tx.send(Err(e)); return;` to a plain
`return` would drop the error silently and this fails.
- **`read_panic_surfaces_as_err_batch_not_clean_eof`** — PANIC
propagation: a reader that PANICS mid-stream must NOT be read as a
clean EOF at the demux boundary. The producer's `catch_unwind` sends
an explicit `Err` sentinel before the thread unwinds, so the
consumer sees the good bytes followed by an error batch — never a
silent truncation. Without the `catch_unwind` the panic would just
drop `tx`, the consumer would see RecvError (== clean EOF) and the
partial output would be finalized as if complete.
- **`recycled_buffer_carries_no_stale_tail`** — recycle-buffer reuse
must NOT leak stale bytes between chunks of different lengths. After
a full chunk, a short read reuses the same recycled buffer; the
regrow-then-truncate sequence must yield only fresh bytes. Verified
by reassembling the full stream. Source: 8 bytes of 0xAA + 3 bytes
of 0xBB, with chunk_bytes=8 → chunk0 = 8×0xAA, chunk1 = 3×0xBB.
- **`exact_multiple_length_no_trailing_empty_batch`** — exact-multiple
boundary: when the source length is an exact multiple of
chunk_bytes, the final non-empty chunk is followed by an `Ok(0)` EOF
read, NOT a spurious empty Ok batch. 12 bytes with chunk_bytes=4 →
three 4-byte chunks then clean EOF. Total bytes must equal 12 and no
zero-length batch may appear.
- **`drop_finite_prefetcher_joins_cleanly`** — dropping the
`BytePrefetcher` directly (without `into_channels`) must join the
producer cleanly when the source is finite. The producer reaches
EOF, drops tx, and exits; Drop's join returns. Grounds the
`BytePrefetcher` `Drop` impl. Mutation: removing the `Ok(0) =>
return` EOF exit would hang this join.
- **`drop_endless_prefetcher_joins_cleanly`** — regression: dropping a
`BytePrefetcher` directly (without `into_channels`) with an ENDLESS
source must not deadlock. Before the fix, `Drop` joined the producer
while `rx`/`recycle_tx` were still alive (sibling field drop order),
so the producer filled the depth-2 forward channel and then spun in
`send_timeout` forever (rx never drained, halt=None). The fix drops
`rx`+`recycle_tx` BEFORE the join so the producer sees
`SendTimeoutError::Disconnected` and exits.
-48
View File
@@ -1,48 +0,0 @@
# `labels::class_reader`
Hand-rolled JVM `.class` file reader, tailored to the subset needed for BD-J
label extraction (Deluxe / dbp / similar frameworks). Implements JVMS §4
(class file format) and §6 (bytecode) only as far as: constant pool,
methods, the `Code` attribute, and a non-allocating bytecode iterator.
No external deps beyond `std`. No `unsafe`. No panics on malformed input —
every parse fault is a typed `Error`. This is shared infrastructure for any
label parser that needs structured access to `.class` files inside a
`/BDMV/JAR/<x>.jar`.
## `ConstantPool::from_entries`
Test-only constructor — builds a constant pool directly from a vector of
entries. Real callers go through `ClassFile::parse`, which builds this from
class-file bytes. Used by parser unit tests (e.g. `labels::deluxe`) that
need to exercise bytecode walkers against synthetic class fixtures without
hand-rolling valid `.class` byte buffers.
Caller is responsible for: prepending a `CpInfo::Empty` at index 0 (the
spec-reserved slot), and inserting a `CpInfo::Empty` after each Long/Double
entry (the 2-slot quirk).
## `decode_modified_utf8`
Decodes JVM "modified UTF-8" (JVMS §4.4.7). Practically identical to
standard UTF-8 for the BMP-printable subset seen in label strings, but with
two notable deviations:
- U+0000 is encoded as the two-byte sequence `0xC0 0x80`, not as `0x00`.
- Supplementary characters (U+10000..) are encoded as a UTF-16 surrogate
pair, each surrogate emitted as 3-byte modified UTF-8.
For label data (mostly ASCII / Latin-1 / CJK in BMP), the simple
implementation here covers everything we'll encounter. The `0xC0 0x80` →
U+0000 case is tolerated explicitly; supplementary characters would need
surrogate-pair stitching, but no label-relevant string uses them.
## `Reader::slice` overflow test
`Reader::slice` takes an attacker-supplied length: a JVMS `u4`
`attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8 length
(§4.4.7). Adding it to `pos` without a wrap check panics on overflow in
debug and, in release, wraps to a small end offset that slips past the
bounds check and then panics inside the slice index. Both are panics
escaping a parser whose whole input is untrusted disc bytes; the contract
is an EOF error.
-18
View File
@@ -1,18 +0,0 @@
# CLPI vs MPLS cross-validation diagnostic
`src/labels/clpi_audit.rs` walks both per-clip CLPI program info and
per-playlist MPLS STN tables, normalizes their stream lists by `(PID,
language, coding_type)`, and classifies each PID into one of four
buckets:
1. **CLPI only** — a stream present in a `.clpi` ProgramInfo that no
playlist STN table references (orphan on disc).
2. **MPLS only** — a stream a playlist references that no `.clpi`
ProgramInfo lists (indicates a parser disagreement).
3. **Match** — both sources agree on coding_type and language.
4. **Divergent** — both sources see the PID but disagree on
coding_type or language.
`audit` returns a structured `ClpiVsMplsAudit` report. This is a
diagnostic surface only; it does not feed the label-selection
pipeline.
-243
View File
@@ -1,243 +0,0 @@
# CLPI Clip Information Format
## What is CLPI?
CLPI (Clip Information) files describe the structure of individual M2TS transport streams on a Blu-ray disc. Each `.clpi` file in `BDMV/CLIPINF/` corresponds to one `.m2ts` file in `BDMV/STREAM/` with the same numeric name (e.g. `00001.clpi` describes `00001.m2ts`).
The most important data in a CLPI file is the **EP (Entry Point) map**, which provides timestamp-to-sector mapping. This is essential for seeking and for extracting the specific sector ranges that correspond to a playlist's in/out time window.
## File Structure
CLPI files use big-endian byte order. The header:
```
Offset Size Field
------ ---- -----
0 4 Magic: "HDMV"
4 4 Version: "0200" (BD) or "0300" (UHD BD)
8 4 SequenceInfo start offset
12 4 ProgramInfo start offset
16 4 CPI (Characteristic Point Information) start offset
20 4 ClipMark start offset
24 4 ExtensionData start offset
```
### ClipInfo Section (offset 40)
Contains basic clip metadata. The source packet count (total number of 192-byte source packets in the M2TS file) is at offset 56:
```
Offset Size Field
------ ---- -----
40 4 ClipInfo length
44 2 Reserved
46 1 Stream type
47 1 Application type
48 4 Reserved
52 4 TS recording rate
56 4 Source packet count
```
The source packet count multiplied by 192 gives the total byte size of the M2TS file.
## EP Map
The EP map lives inside the CPI section and provides random access into the transport stream. It maps PTS timestamps to Source Packet Numbers (SPN), enabling precise seeking without scanning the stream.
### CPI Section Layout
```
Offset Size Field
------ ---- -----
0 4 CPI length
4 2 Reserved / CPI type
6 ... EP map
```
### EP Map Header
```
Offset Size Field
------ ---- -----
0 1 Reserved
1 1 Number of stream PID entries
2 ... Stream PID entry headers (one per stream)
```
Each stream PID entry header (14 bytes):
```
Offset Size Field
------ ---- -----
2 2 stream_PID (byte-aligned)
4 10 Bit-packed block, 80 bits total (see below)
```
The stream PID entry is **not** byte-aligned past `stream_PID`. Bytes 4..14 are one
80-bit packed field, read as a `u64` plus a trailing `u16`:
Bits Width Field
---- ----- -----
0-9 10 reserved
10-13 4 EP_stream_type
14-29 16 num_EP_coarse
30-47 18 num_EP_fine
48-79 32 EP_map_start_address (relative to the EP map start)
Note `num_EP_fine` is **18 bits**, not 32. See `parse_cpi` in `src/clpi.rs`.
libfreemkv parses only the first stream (primary video), which is sufficient for sector-level seeking.
### Two-Level Index
The EP map uses a two-level structure to compress what would otherwise be a very large lookup table:
- **Coarse entries**: low-resolution index covering large time/sector ranges
- **Fine entries**: high-resolution entries within each coarse range
Each coarse entry points to a range of fine entries via `ref_to_fine_id`.
### Coarse Entries (8 bytes each)
Located immediately after the fine table start offset (4 bytes) in the per-stream EP map:
```
Bits Field
------ -----
[31:14] ref_to_fine_id (18 bits) -- index of first fine entry in this range
[13:0] pts_coarse (14 bits) -- upper bits of PTS
```
Second dword:
```
Bits Field
------ -----
[31:0] spn_coarse (32 bits) -- upper bits of SPN
```
### Fine Entries (4 bytes each)
Located at the fine table start offset within the per-stream EP map:
```
Bits Field
------ -----
[31] is_angle_change_point (1 bit)
[30:28] I_end_position_offset (3 bits)
[27:17] pts_fine (11 bits) -- lower bits of PTS
[16:0] spn_fine (17 bits) -- lower bits of SPN
```
## Reconstructing Full PTS and SPN
The full timestamp and packet number are assembled by combining the coarse and fine components:
### Full PTS
```
full_pts = (pts_coarse << 19) + (pts_fine << 8)
```
The coarse component provides the upper bits, shifted left by 19. The fine component provides mid-range bits, shifted left by 8. The resulting PTS is in 45kHz ticks, matching the MPLS timestamp format.
### Full SPN
```
full_spn = (spn_coarse & 0xFFFE0000) + spn_fine
```
The coarse SPN provides the upper 15 bits (masked to clear the lower 17). The fine SPN provides the lower 17 bits. The result is a Source Packet Number -- each source packet is 192 bytes in the M2TS file.
### Resolved EP Map
The `resolved_ep_map()` method iterates through all coarse entries and their associated fine entries to produce a flat list of `(PTS, SPN)` pairs. For each coarse entry at index `ci`:
- Fine entries range from `coarse[ci].ref_to_fine_id` to `coarse[ci+1].ref_to_fine_id` (or end of fine table for the last coarse entry).
- Each fine entry is combined with its parent coarse entry to produce one full `(PTS, SPN)` pair.
## Deriving Sector Extents for Ripping
Given a playlist's in_time and out_time (from MPLS), the CLPI EP map provides the sector ranges to read from disc. The `get_extents()` method does this in three steps:
### Step 1: PTS to SPN
Binary search the resolved EP map for the in_time and out_time:
- **start_spn**: the SPN at or before in_time (seek backward to the nearest I-frame)
- **end_spn**: the SPN at or after out_time (include the full GOP)
### Step 2: SPN to Byte Offset
Each Source Packet is 192 bytes (188 bytes MPEG-TS payload + 4 bytes M2TS header):
```
byte_offset = spn * 192
```
### Step 3: Byte Offset to Sector
M2TS files are stored contiguously on disc. Sectors are 2048 bytes:
```
start_sector = start_byte / 2048
end_sector = (end_byte + 2047) / 2048
sector_count = end_sector - start_sector
```
The resulting `Extent` contains `start_lba` (relative to the M2TS file's starting sector on disc) and `sector_count`. The caller adds the file's absolute starting LBA from UDF to get disc-absolute sector numbers.
### Alignment Note
The 192-byte source packet size and 2048-byte sector size share no common factor beyond 1. A single sector contains roughly 10.67 source packets. The conversion rounds start down and end up to ensure complete coverage.
## Putting It Together
The full ripping pipeline chains three parsers:
1. **MPLS** provides clip IDs and in/out timestamps.
2. **CLPI** converts those timestamps to SPN ranges, then to sector extents.
3. **UDF** provides the file's starting LBA on disc for absolute sector addressing.
The `Disc::scan()` method in `src/disc/mod.rs` orchestrates this: for each play item in each playlist, it loads the corresponding CLPI, calls `get_extents()` with the play item's in/out times, and collects the resulting sector ranges into the title's extent list.
## ProgramInfo Section (Per-Stream Metadata)
`parse_program_info` in `src/clpi.rs` reads per-stream (pid, coding_type,
language, codec sub-fields) from the ProgramInfo section, per the Blu-ray
Disc Read-Only Format Part 3 CLIPINF (CLPI) specification:
```text
ProgramInfo:
length: 4 bytes
reserved: 1 byte
num_programs: 1 byte
for each program:
spn_program_sequence_start: 4 bytes
program_map_pid: 2 bytes
num_streams: 1 byte
num_groups: 1 byte
for each stream:
pid: 2 bytes
stream_coding_info_length: 1 byte
stream_coding_info: (varies by coding_type)
coding_type: 1 byte
per-type bytes (see match arms in parse_program_info)
```
It returns `Vec::new()` on any structural mismatch — errors aren't
propagated because the EP map is the primary CLPI output, and a corrupt
program_info shouldn't break sector-range lookups.
### Test fixture notes
- `prog_info_start_zero_does_not_parse_header_as_program_info`: the
fixture is crafted so that parsing from offset 0 WOULD yield a stream
(num_programs at `[5]`, a second program header whose num_streams
byte at `[20]` is 1, then a well-formed stream record), so an empty
result can only come from the `prog_info_start > 0` gate in `parse`.
## References
- BD-ROM Part 3, Section 5.5: Clip Information file format
- https://github.com/lw/BluRay/wiki/CLPI
-79
View File
@@ -1,79 +0,0 @@
# `src/mux/codec/mod.rs` — extended notes
Relocated prose from doc comments in `mod.rs`, trimmed there to stay within
the comment-guard's prose caps. Each `//`/`///` pointer in the source names
the section here it corresponds to.
## `Frame::discontinuity`
Carried per-FRAME (not per-PES) because buffering parsers decouple the frame
from the PES that carried the gap signal: MPEG-2 emits whole GOPs, H.264/HEVC
lag one access unit behind the PES that produced them. A clean rip leaves
every frame `false`; the flag is only ever set on the degraded/conceal path
(P3/B1), when a frame's data begins after packets the demuxer never received
— an undecryptable unit concealed as NULL-TS upstream, or a continuity break
in a damaged source.
## Drop-on-undecodable policy across codecs ("clean muxes always")
- **Audio with independent access units** (DTS, AC-3/E-AC-3, …) gates each AU
through a per-codec corruption check and drops the ones that fail, keeping
A/V sync (a drop is a silence gap, never a shift) and logging every drop
via the shared `dropgate::DropTally`. DTS validates via its core-frame
header (ETSI TS 102 114); AC-3 uses its native frame CRC.
- **LPCM is excluded on purpose**: raw PCM carries no framing or integrity
data, so a corrupt sample is indistinguishable from a quiet one — there is
nothing to detect, so nothing can be honestly dropped.
- **Video is excluded on purpose**: H.264/HEVC/MPEG-2/VC-1 are inter-frame
predicted, so dropping one frame corrupts every frame that references it
until the next keyframe. Video instead resyncs at GOP/IDR boundaries (the
ResyncGate) and lets the decoder conceal — a fundamentally different model
than per-frame audio dropping.
- TrueHD/MLP, FLAC, MP2/MP3 and AAC-ADTS also gate undecodable frames via a
`DropTally` (poison/drop-forward for MLP's inter-AU restart state on a
major-sync boundary; CRC/sync-verdict drops for the passthrough codecs).
## `codec_private()` — why "absent" is the only truthful answer for some parsers
A codec parser's `codec_private()` feeds `DiscStream::codec_private`, which
the MKV muxer turns directly into the track's `CodecPrivate` element (RFC
9559 §5.1.4.1.24): `Some(bytes)` writes an element holding exactly those
bytes, `None` omits the element entirely. The two are NOT interchangeable —
a zero-length `CodecPrivate` asserts that the codec's initialisation data IS
empty, which is not true of any codec, and a one-byte one asserts a config no
decoder can parse.
The gating parsers (ADTS, MPEG audio, FLAC) and the passthrough parser
extract no configuration at all: they validate and forward frames whose
configuration is carried in band (ADTS headers, MPEG-1 audio frame headers,
FLAC frame headers) or supplied by the source container. Having derived
nothing, the only truthful answer they can give is "absent" — any `Some`
would be a value they invented. Nothing else in the test suite distinguished
the two, so each of these impls could have returned a fabricated `Some` and
produced a malformed track header unnoticed.
## `provenance_guard` test module
Every emitted frame must carry the source byte offset of the packet it came
from. That invariant held only for video for as long as it existed: `dts`,
`ac3`, `adts`, `truehd`, `pgs`, `dvdsub`, `flac`, `lpcm`, `mpegaudio` and the
passthrough parser all built frames with `source: None`, so a multi-clip
title could not place audio or subtitles by byte and fell back to inferring
the clip from timestamps — which is what made branched titles run minutes
long.
Nothing asserted it, so nothing caught it. Finding it took a brace-balanced
scan of the tree by hand, which also turned up five sites a regex had
missed. `provenance_guard` is that scan, as a test.
### `code_only` / `frame_literals` design
`frame_literals` walks `Frame { .. }` literals with balanced braces. A regex
cannot do this — `Frame` blocks contain nested braces (`coding: Some(..)`,
closures) and a non-greedy match stops at the first `}`, which is how five
sites survived the first pass.
`code_only` strips line comments before scanning. A comment that MENTIONS
`source: None` is prose, not a construction — this guard's own doc comment
tripped it on the first run, which is the same mistake as grepping a file
and matching its commentary.
-17
View File
@@ -1,17 +0,0 @@
# `src/mux/codec/coding.rs`
## Spec references
ITU-T H.273 (CICP code points, shared elsewhere), ISO/IEC 13818-2 §6.3.10
(MPEG-2 picture coding extension: `top_field_first`, `repeat_first_field`,
`progressive_frame`), RFC 9559 §5.1.4.1.28 (Matroska `FieldOrder` element
0x9D).
## HDR10 unit scaling
`Hdr10Metadata` values are stored in their raw SEI integer units (not yet
scaled to the Matroska float domain); the muxer applies the H.265 → Matroska
unit conversion at emit time so the scaling lives in exactly one place.
Spec: Rec. ITU-T H.265 D.2.28 (Mastering Display Colour Volume, payloadType
137) and D.2.35 (Content Light Level Info, payloadType 144).
-11
View File
@@ -1,11 +0,0 @@
# `consts::coding_type` — spec citations
`coding_type` is one registry used in two places that share the same value
space: the MPEG-TS PMT `stream_type` (ISO/IEC 13818-1 Table 2-34) and the
Blu-ray STN/CLPI `stream_coding_type` (BD-ROM Part 3).
The standardized video codes (`0x02`, `0x1B`, `0x24`) are ISO assignments
(ISO/IEC 13818-1 Table 2-34); `0xEA` (VC-1) is a BD-ROM convention in the ISO
user-private range. The `0x80..=0xA2` audio/graphics codes also sit in the
user-private range and follow the Blu-ray Disc Association / ATSC A/52
convention.
-14
View File
@@ -1,14 +0,0 @@
# CRC helpers (`src/mux/codec/crc.rs`)
## `crc16_mlp` reversed byte order
MLP's checksum is the "reversed" scheme. `crc16_mlp` emits its two bytes in
the OPPOSITE order to a standard little-endian CRC readout, so the caller
swaps them back and compares against the stored trailer word read
LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
Comparing big-endian instead is precisely the bug that function was fixed
for: it could never validate a real extended major sync, so whole TrueHD
tracks were dropped silently. This comment used to prescribe exactly that,
and to point at a `truehd::mlp_major_sync_ok` that does not exist.
Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
-87
View File
@@ -1,87 +0,0 @@
# `src/labels/criterion.rs` — internal notes
## `assign_stream_numbers`
Assign a 1-based stream number per `StreamInfo`, parallel to `infos`.
A stream mapped in `playbackconfig.xml` (`stream_map`) keeps its
mapped number. Streams with no mapping (absent or incomplete
`playbackconfig.xml`, or an unmatched `StreamInfo_ID`) are numbered
1-based per type — but the fallback counter SKIPS any number already
claimed via the map, so a synthesized number can never collide with a
map-assigned one. (Both numbering domains are 1-based per type, and
`apply_labels` matches on `(type, stream_number)`, so a collision
would mislabel tracks.)
Returns `None` when the 1-based stream-number space is exhausted —
every number in `1..=u16::MAX` for that type is either already
claimed by the map or already synthesized. That is unreachable on
real media: the BD STN_table carries at most 32 primary audio and 32
PG streams per playlist, so the 65535-wide space leaves >2000x
headroom. It IS reachable from a crafted `streamproperties.xml`
listing >65535 stream entries, and the only correct answers there are
"fail the parse" or "emit colliding numbers"; we fail.
The skip search is bounded by the numbering space itself: a `u16`
`saturating_add` here parked the counter at `u16::MAX` forever
whenever the map also claimed `u16::MAX`, turning an overflow guard
into a hang that `apply()`'s `catch_unwind` cannot interrupt. The
counters are therefore widened to `u32` so the skip loop strictly
increases toward a fixed ceiling (guaranteeing termination) and
exhaustion is reported rather than absorbed.
## Test: immunity pin, section-boundary half
Covers `an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it`.
`parse_stream_infos` emits one `StreamInfo` per `*StreamInfos` element
unconditionally — no filter, no `continue` — so an element whose
fields are missing or unrecognized still occupies its position, and
`assign_stream_numbers` still spends a number on it.
That is the property that keeps this parser out of the failure mode
where a skipped entry pulls every later label one stream forward. It
is load-bearing for the fallback path specifically: with no
`playbackconfig.xml` the numbers come purely from position in this
list, so dropping an element there would shift the rest.
Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
real audio streams renumber to 1 and 2.
Each stream here is one closed XML element, and every field is read
out of `&text[start..end]` — the range `xml::find_element` returned —
so one element can never absorb the next one's fields, however the
document is malformed around it. Contrast the flat-string walk in
pixelogic, where a section whose end marker is missing keeps
consuming entries as STN slots.
The missing-boundary case fails closed. An element with no close tag
of its own ends at the NEXT close tag, so it absorbs the element
behind it — the list comes back SHORTER. It cannot come back longer:
nothing outside a returned range is ever read as a stream, and
`find_element` yields `None` rather than a range running to EOF when
no close tag exists at all. A malformed document can cost this parser
a slot; it can never invent one.
Mutation: read fields from the document rather than the element's
range, or let a close-less element run to EOF → the trailing elements
re-enter the list as extra streams.
## Test: `exhausted_numbering_terminates_instead_of_looping`
A crafted `streamproperties.xml` can drive the fallback counter to
the top of the 1-based u16 stream-number space and then present one
more unmapped stream whose successor number is also claimed by the
map.
This must TERMINATE. The bound is the numbering space itself, so the
assertion is on the spec-derived exhaustion behaviour (`None`), not
on any tunable constant. Run on a worker thread with a deadline so a
non-terminating loop fails the test in 20 s instead of hanging CI.
## Test: `full_u16_numbering_space_is_usable_and_unique`
The whole 1-based u16 space must remain usable: 65535 unmapped audio
streams get 65535 distinct numbers with no panic and no wrap. The
literals here are the JVMS-independent, spec-derived size of a u16
1-based numbering domain, not a tunable cap.
-245
View File
@@ -1,245 +0,0 @@
# CSS keyless title-key recovery (`src/css/keyless.rs`)
## The cipher this attacks
The content descrambler ([`super::lfsr::descramble_sector`]) seeds its two
LFSRs **directly** from `key = title_key XOR sector_seed` (seed =
`sector[0x54..0x59]`): LFSR1 from key/seed bytes 0-1, LFSR0 (24-bit) from
bytes 2-4 with the pre-conditioning `r0 = r0*2 + 8 - (r0 & 7)`, and each body
byte recovered as `plain = TAB1[cipher] ^ (keystream & 0xff)`. There is no
title-key mangling on the content path, so the recovery is a single inversion
of the sector cipher.
## The attack
1. **Known plaintext → keystream.** Because descramble applies TAB1 to the
ciphertext, the per-byte keystream is `TAB1[cipher[i]] ^ plain[i]`.
2. **Brute the 16-bit LFSR1 seed.** For each of 2^16 seeds, run LFSR1
forward; for the first four steps deduce the LFSR0 output bytes from the
keystream (carry-tracked), reconstructing LFSR0's state. For the next six
steps clock LFSR0 normally and check it reproduces the keystream — a wrong
LFSR1 seed fails fast.
3. **Back-clock LFSR0.** Run four backward steps (each a 256-way search for
the byte shifted in) to reach the initial state, then undo the
`r0*2 + 8 - (r0 & 7)` pre-conditioning to recover key[2..5].
4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]`.
Known plaintext for step 1 comes from the longest periodic run in the
cleartext `sec[0x00..0x80]`, assumed to continue into the encrypted region at
0x80.
## `recover_title_key_from_plain`
Core of the recovery. `crypted` is the ciphertext starting at sector byte
0x80; `decrypted` is the matching known plaintext; `seed` is
`sector[0x54..0x59]`. On success returns the recovered 5-byte title key;
`None` if no LFSR seed reproduces the keystream. At least 10 bytes of
`crypted`/`decrypted` are required (the cipher is iterated 10 times: 4 to
reconstruct LFSR0, 6 to validate).
## `attack_crib`
Scans the clear header `sec[0x00..0x80]` (never scrambled) for the longest
run that repeats with a cycle length in 2..0x2F. If the run is long enough
(`plen > 3` and at least two full cycles), the plaintext at 0x80 is taken to
be that periodic run continuing forward. Returns `None` for an unscrambled
sector or one with no usable run — such a sector can be neither cracked nor
key-validated, only descrambled with an externally-cached key.
The header is untouched by `descramble_sector`, so the crib is identical
before and after descramble: the decrypt path uses it as a per-sector
"did the cached key descramble correctly?" oracle (the predicted plaintext
must reappear at 0x80), and the cracker uses it as its known plaintext.
## Tests
### `synth_sector`
Builds a synthetic scrambled sector for a given title key and seed, with
`plain` placed as the plaintext at byte 0x80, scrambled with EXACTLY the
cipher `descramble_sector` inverts. Returns
`(scrambled_sector, full_plaintext_body)`.
### `synth_periodic_sector`
Builds a synthetic scrambled sector whose CLEARTEXT (0x00..0x80) ends in a
periodic run that continues into the encrypted region — the case
`crack_title_key` is designed to crack.
### `recover_round_trips_known_keys`
MANDATORY round-trip (Task C.1): synthesize a scrambled sector for a known
(title_key, seed), then assert `recover_title_key` returns a key that
descrambles the body back to plaintext. CSS title-key recovery is
well-defined up to keys that scramble identically; we assert the full body
round-trips (the true correctness property), and additionally that the EXACT
key is returned for the common case.
### `descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with`
`descramble_matches` is the ONLY gate between the LFSR search and a key
handed back to the caller: both `recover_title_key` and the crib-driven
`crack_title_key_inner` return a candidate only if this says the key really
descrambles the sector to the known plaintext. A body that always answered
`true` would let the first spurious LFSR-seed match through as the title
key — the ripper would then descramble the whole title with a key that opens
nothing, producing garbage rather than a "no key" error.
Pinned both directions: the genuine key is accepted, and EVERY key one bit
away from it is rejected. The one-bit neighbours are the strongest form of
wrong key — a gate that only rejects wildly different keys would still pass
a near-miss out of the 2^16 seed search.
### `descramble_matches_does_not_disturb_the_caller_s_sector`
The gate is applied to a COPY: verifying a candidate must not modify the
caller's sector. `recover_title_key` runs the gate and then hands the sector
on to be descrambled for real — if verification descrambled in place, that
second descramble would run over already-transformed bytes (and, worse, a
rejected candidate would leave the sector corrupted).
### `recover_rejects_a_sector_that_ends_inside_the_encrypted_region`
A sector buffer that ENDS inside the encrypted region must be refused, not
sliced. `recover_title_key` slices `sector[0x80..0x8A]` unconditionally
after its length guard. The existing short-sector test uses
`SECTOR_BYTES - 1`, which is still long enough for that slice to succeed —
so the guard was never the thing producing the `None`, and dropping it (or
weakening the `||` to `&&`, which a full-length crib satisfies) changed
nothing observable. On a real short read this is an out-of-bounds panic on
the rip thread.
### `a_buffer_longer_than_one_sector_still_yields_its_key`
A buffer LONGER than one sector is still one sector: both entry points read
the first `SECTOR_BYTES` and must recover the key from it. Callers read DVD
data in multi-sector blocks, so an over-long slice is the normal case, not
an exotic one. A length guard that rejected it (`len > SECTOR_BYTES` instead
of `<`) would make every block-read caller silently unable to crack
anything.
### `recover_accepts_more_than_ten_bytes_of_known_plaintext`
`recover_title_key` accepts MORE than ten bytes of known plaintext, and uses
all of it: the extra bytes tighten the `descramble_matches` gate. The
ten-byte figure is a MINIMUM (the cipher is iterated ten times), not an
exact requirement — a guard reading it as an upper bound would reject every
caller that knows a longer crib.
### `a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused`
The scramble-flag gate on a sector whose BODY really is ciphertext. Both
entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
unscrambled sector has no title key to recover, and its bytes at 0x80 are
already plaintext. Every prior test of this gate used an all-zero or
all-`0x11` sector, where the recovery would have found nothing anyway — so
widening the mask test (`&` to `|`, which makes it true for EVERY flag byte)
produced the same `None` and went unseen.
Here the sector is genuinely scrambled and its key IS recoverable; only the
cleared flag stands in the way. If the gate stops working, both functions
start returning keys for sectors the disc says are in the clear.
`attack_crib` carries its own copy of the same gate and is the one that
actually stops the crack (`crack_title_key`'s is defensive duplication); it
also doubles as the decrypt path's cached-key oracle.
### `descramble_matches_forces_the_scramble_flag_on_its_own_copy`
The gate must verify a candidate against the sector's CIPHERTEXT regardless
of what the sector's own flag byte says. `descramble_matches` forces `0x10`
on its copy precisely because `super::lfsr::descramble_sector` is a no-op
when the scramble bits are clear — without that, verifying a
scrambled-but-unflagged sector compares raw ciphertext against the crib, and
every candidate key is rejected. Nothing exercised it: every fixture already
had the flag set, where forcing the bit is a no-op.
### `descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector`
The gate compares the WHOLE supplied plaintext, clamped to the encrypted
region. Two properties in one, because they are the two halves of
`plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
- it must compare beyond the first sixteen bytes, or a key that opens only
the head of the crib is accepted; and
- it must never compare past the end of the sector — a caller that knows
more plaintext than the 1920-byte encrypted region holds otherwise
indexes off the end of the buffer and panics.
### `sector_with_trailing_run`
Builds a sector whose clear header ends in a `period`-length repeating run
of exactly `run_len` bytes immediately before 0x80. The run is anchored to
ABSOLUTE sector offset (`sec[x] = pat[x % period]`), which is what makes
"the run continues past 0x80" a statement independent of the code under
test: the byte at `0x80 + i` of the underlying plaintext is
`pat[(0x80 + i) % period]`. Everything before the run is `0x00` (the pattern
bytes are all >= 0xD0, so the run cannot be extended backwards by accident),
and the encrypted region is filled with `0xFF` — so a crib that reads past
0x80 into "ciphertext" is immediately visible.
### `attack_crib_predicts_the_periodic_run_continuing_past_0x80`
KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly at
0x80, the crib is the run continued forward — the same ten bytes for every
run length, because the prediction depends only on the pattern and the
phase, never on how many cycles happened to be visible. The short lengths
are the load-bearing ones: at `run_len = 11` the crib window starts at 0x76
and is only 10 bytes from the end of the header, so any drift in
`plain_start`, in `cycles * best_p`, or in the `i % best_p` wrap reads the
0xFF "ciphertext" instead of the run.
### `attack_crib_refuses_a_run_shorter_than_two_cycles`
A run of exactly ONE cycle (plus the trivial tail the detector counts) is
not enough to predict forward: `attack_crib` requires at least two full
cycles. Weakening that guard would let a one-off byte sequence be declared
periodic and produce a confidently wrong crib — which the decrypt path uses
as its "is my cached key still right?" oracle.
### `attack_crib_refuses_a_buffer_shorter_than_a_sector`
`attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
own length guard is the only thing between a short buffer and an
out-of-bounds read. Nothing reached it: every caller-level test used a full
sector, and the entry points' guards fire first.
### `attack_crib_survives_a_header_that_is_periodic_to_offset_zero`
A header that is periodic ALL THE WAY to offset 0 must not walk the
backward scan off the front of the sector. The detector counts backwards
from 0x7f while `j < 0x80`. On a fully periodic header the run never
breaks, so `j` reaches 0x7f and the bound is the ONLY thing that stops it —
one step further and `0x7f - j` underflows a `usize` and panics. A constant
or fully-patterned 128-byte header is ordinary DVD data (padding, a run of
zeros), not a crafted input, and every existing fixture had a
filler/run boundary well before offset 0 that stopped the scan early.
### `attack_crib_is_independent_of_the_encrypted_region`
The crib is read from the CLEAR header only. A run that reaches 0x80 must
predict from the header bytes, never from the encrypted region — the
previously-fixed bug this function's doc comment records. Pinned by
rewriting the encrypted region and requiring the crib not to move.
### `recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input`
`recover_title_key_from_plain` unconditionally builds a 10-byte keystream
buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard is
the only thing standing between a short slice and an index-out-of-bounds
PANIC. Nothing reached that guard before: `recover_title_key` rejects
`plain.len() < 10` at its own door and always hands on exactly ten
ciphertext bytes, and `crack_title_key_inner` always passes a fixed
`[u8; 10]` crib. The guard is a live contract for any future caller and was
executed by no test at either boundary.
### `recover_title_key_from_plain_xors_the_sector_seed_back_out`
The seed XOR-back (`recover_title_key_from_plain`'s last step) is what turns
the recovered LFSR key into the TITLE key: `key ^= sector_seed`. Pinned as a
known answer across seeds that differ only in one byte — the same
ciphertext/plaintext pair therefore must yield title keys differing in
exactly that byte. Without this, a body that ORed the seed in (or dropped
the step) still round-trips on any fixture whose seed is zero, and on the
non-zero ones the failure looks like "no key found" rather than a wrong
step.
-311
View File
@@ -1,311 +0,0 @@
# `src/css/mod.rs` — extended notes
Overflow detail relocated from doc comments in `src/css/mod.rs` (kept short
there per the comment-guard caps). Function/item names below match the code.
## `CSS_LOCKED_BAIL`
The bus-auth read gate is global (all-or-nothing), so a run this long of
`05/6F/03` (CSS-locked) reads means the gate is shut and nothing here is
crackable — bail instead of grinding the full 50_000-sector budget. That
grind is what made rc5 appear to hang on a wedged USB bridge.
## `crack_key`
Recovers the CSS title key with no keys, by scanning scrambled sectors and
running the known-plaintext attack (see the `keyless` module). The crib comes
from the periodic-run detector: a scrambled sector's cleartext region
(bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
constant fill); the attack assumes that run continues across the 0x80
boundary into the encrypted region, giving the known plaintext the 2^16 LFSR
recovery needs. Scans up to 50000 sectors across the extents and returns the
first sector that yields a key — no player keys, no disc-key crack. Works on
a live drive (after bus-auth unlocks reads) and on disc images alike.
This convenience form runs to completion (no cancellation) and returns just
the key; callers needing an operator-Stop / watchdog cancel, or the
three-way `CrackOutcome` (to distinguish "unencrypted" from
"encrypted-but-uncracked"), use `crack_key_outcome`, which takes a `halt`
token.
## `CrackOutcome`
Distinguishes the three cases the bare `Option<CssState>` conflated (and
which caused a silent-failure bug: scrambled-but-uncracked content was
treated as "unencrypted" and muxed as plaintext garbage at exit 0):
- `Cracked` — a scrambled sector yielded a title key.
- `Unencrypted` — NO scrambled sector was seen across the scanned extents
(`is_scrambled_pack` never true): the content is genuinely plaintext, so
proceeding without a key is correct.
- `ScrambledUncracked` — scrambled sectors WERE seen but no key could be
recovered (the recovery found no crackable crib, or the scrambled region
was unreadable). The content is encrypted; muxing it as plaintext would
emit garbage, so callers MUST surface a hard error instead of falling
through to "unencrypted" — the per-title `crate::error::Error::CssKeyMissing`
when it is ONE title's own re-crack that failed (skippable: a sibling VTS
may still crack), or the disc-level `crate::error::Error::CssNoDiscKey`
when it is the disc-wide scan (`Disc::css_error`, every title fails
identically).
## `resolve_dvd_title_key`
The SINGLE place every DVD read path obtains a title key, so the file-backed
mux highway (`crate::build_iso_pipeline`) and the live-drive single-pass
`crate::DiscStream` descramble a DVD identically ("reading is reading"). CSS
keys are per-VTS and crackable from the scrambled data itself, so a
`None`/MPEG-PS title cracks its own key here, in playback order over
`extents`. Everything else is left untouched:
- AACS keys (HD-DVD `.evo` is also MPEG-PS but arrives as `Aacs`) — no CSS.
- a title that already carries a key — nothing to resolve.
- a genuinely clear DVD (no scrambled sector) — stays `None`, a mux no-op.
A scrambled-but-uncrackable title is a hard `crate::error::Error::CssKeyMissing`,
never a silent scrambled-passthrough mux. That code is the PER-TITLE one
(`error::is_skippable_title_stub`), which is correct here: this function
cracks ONE title's own extents, and another VTS on the same disc may still
yield its key, so an all-titles rip skips this title and finishes the rest.
The whole-disc failure is `crate::error::Error::CssNoDiscKey`, raised by
`Disc::ensure_decryptable_keys` from the scan's `css_error`.
## `crack_key_scan` (private)
Tracks a `saw_scrambled` flag so a scrambled-but-uncracked disc is
distinguished from a genuinely-unencrypted one (the `crack_key` `Option`
wrapper collapses both to `None` via `CrackOutcome::into_state`).
## `descramble_sector`
A no-op unless the sector is a scrambled MPEG-2 PS PACK. The pack start code
is checked, not just the byte 0x14 flag bits, for the same reason
`descramble_region` checks it: 0x14 only means "scrambling control" inside a
pack, and in an IFO it is whatever that format stores there. A real
`VIDEO_TS.IFO` sector holds 0x15 there while starting `00 26 00 00`;
descrambling it destroyed 1912 of its 2048 bytes, and because that sector
carries TT_SRPT the disc enumerated 38 titles while an image decrypted from
it enumerated 10, silently, at exit 0.
This function has no callers inside the crate, but the module-level example
prescribes it — so the crate's own documented guidance led straight into
that defect. Making the guard part of the function, rather than something
each caller must remember, is what keeps the safe path the easy one.
## `descramble_region`
Descrambles a whole CSS buffer in place, re-cracking the title key on a VOB
region boundary. `title_key` is a CACHE of the last crack, not a fixed disc
key: it changes per VTS/VOB region, so it is validated on every scrambled
sector and re-cracked on a miss (the standard on-demand per-region rekey).
This CSS key acquisition is intrinsic to the cipher — CSS has no external key
source, the ONLY way to a title key is cracking the data — so it lives with
the CSS primitives and runs inside `decrypt::decrypt_sectors` (a public,
self-contained CSS decrypt), NOT at the post-decrypt recovery seam that AACS
key-fetch and FMTS segment-skip use (those consume external inputs).
The clear header (`<0x80`) is never scrambled, so its periodic crib predicts
the plaintext at `0x80`. Descramble with the cached key; if the crib fails to
reappear the key region changed (or the primed key was wrong) — restore the
ciphertext, re-crack from this very sector, and descramble again. A crib-less
sector (no periodic run) can be neither validated nor cracked, so it rides
the cached key — correct, because it lives in the same region as the nearby
crib sector that set the cache.
This section used to document an `Error::DecryptFailed` for the case where a
sector's crib rejects the cached key and the re-crack from that sector also
fails. That behaviour was tried and REVERTED: crib mismatch plus crack
failure is the signature of a crib FALSE POSITIVE, not of a stale key, and
failing there made real discs unrippable. The code now descrambles with the
cached key and returns `Ok`.
## `has_scramble_flag_bits`
Deliberately NOT called `is_scrambled`. Byte 0x14 only means "scrambling
control" inside an MPEG-2 Program Stream pack; in an IFO, UDF or ISO 9660
sector it is whatever that format stores there. Treating the flag alone as
proof of scrambling is what destroyed 1912 bytes of a real disc's
`VIDEO_TS.IFO` — the sector carrying TT_SRPT — so the disc enumerated 38
titles and an image decrypted from it enumerated 10, silently, at exit 0.
Callers want `is_scrambled_pack`. It asks the same question and also
requires the pack start code, which every genuinely scrambled VOB sector
carries and no IFO sector does.
It stays public only because an integration test asserts the flag
extraction directly. It has no production callers, and its previous doc
comment claimed one (`decrypt::decrypt_sectors`) that did not exist — so the
name was an invitation and the documentation was an argument for accepting
it.
## `is_scrambled_pack`
The HARDENED test the crack scan uses to set its `saw_scrambled` evidence
flag (Fix 3).
`has_scramble_flag_bits` keys solely on bits 4-5 of byte 0x14. That byte is
only meaningful inside a real DVD sector — an MPEG-2 Program Stream pack,
which ALWAYS begins with the 32-bit pack-start code `00 00 01 BA` at offset
0x00. A tiny clear / nav-only stub (a 0.5 s menu loop, an FBI-warning title)
can carry arbitrary bytes that happen to set bits 4-5 of byte 0x14; trusting
byte 0x14 alone there would flip the scan's `saw_scrambled` gate and make a
genuinely-UNENCRYPTED title report `ScrambledUncracked` — a false E7023.
Requiring the pack-start signature FIRST means only a sector that is
structurally a DVD video pack can be counted as scramble evidence. This does
NOT weaken the genuine "encrypted but uncrackable" hard-fail: a real
scrambled feature is made of valid PS packs, so its scrambled sectors still
pass this check and still drive `ScrambledUncracked` when no key cracks.
The pack-start check alone is not enough, and the sector's PES `stream_id`
(offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
(`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
stream_id even on a scrambled sector. The MPEG-PS structural packets that
are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
because on those the byte at 0x14 is NOT a PES scrambling-control field but
raw payload/structure whose bits 4-5 land set by chance.
This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
set. With no `0x11` gate those nav packs flipped the crack scan's
`saw_scrambled` flag on a disc that carries no CSS at all; the crack then
found no key (there is none) and the scan returned `ScrambledUncracked`,
hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
`0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
can act on (the recovery only recovers a key from a scrambled ES pack), so a
decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
private_stream_1 (`0xBD`) packs, none of which are excluded, so its
scrambled sectors still set `saw_scrambled` and still drive
`ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
The DESCRAMBLE path gates on this same function — `descramble_sector` and
`descramble_region` both call it, not the raw flag test — because the raw
test does not merely mis-skip a sector there: it descrambles one that was
never scrambled and destroys it. The measured case is written up above under
`descramble_region`: a `VIDEO_TS.IFO` sector holding 0x15 at offset 0x14 lost
1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38 titles
became 10 — silently, at exit 0. One gate, both paths.
## Tests (`mod tests`)
Detail relocated from test doc comments:
- `a_crib_false_positive_keeps_the_cached_key_rather_than_failing`:
`attack_crib` is a heuristic — it finds a periodic run in the unscrambled
header and predicts the run continues past 0x80. When that prediction does
not hold, the crib reports a mismatch even though the cached key is
correct, and the re-crack then fails BECAUSE the crib was never valid. So
this combination is the signature of a crib false positive, not of a stale
key, and the cached key remains the best available evidence. This test
exists because a prior round read the same code as "descrambling with a
key we just proved stale" and made it `DecryptFailed` to match the AACS
path; real DVDs hit this constantly, and the change made a real disc
unrippable (no unit test caught it — the real-media acceptance gate did).
CSS is not AACS: an AACS unit key either opens a unit or does not, whereas
a CSS title key is recovered from data whose recoverability varies sector
by sector.
- `has_scramble_flag_bits_short_buffer_is_false_no_panic`: grounding —
`sector.len() >= 2048 && (sector[0x14] >> 4) & 0x03 != 0` short-circuits so
a short buffer never reads index 0x14. Mutation: swapping operand order
would panic indexing a 20-byte slice; this test catches it.
- `has_scramble_flag_bits_uses_bits_4_5_only`: grounding —
`(sector[0x14] >> 4) & 0x03`. Mutation: widening the mask to `& 0x0F` makes
0x40 report scrambled; the 0x40 assert catches it.
- `has_scramble_flag_bits_exact_sector_length_accepted`: grounding —
`sector.len() >= 2048`. Mutation: `>= 2048` → `> 2048` makes an exact
2048-byte scrambled sector report false; this catches it.
- `is_scrambled_pack_requires_pack_start_signature`: Fix 3 hardening —
`is_scrambled_pack` requires BOTH the pack-start code and the 0x14 bits, so
a clear/nav-only stub with stray 0x14 bits isn't scramble evidence (else a
genuinely unencrypted title reports the false E7023).
- `is_scrambled_pack_excludes_nav_and_structural_stream_ids`: detection fix —
`is_scrambled_pack` must exclude system_header/padding/private_stream_2 by
`stream_id` at 0x11, since CSS never scrambles those and byte 0x14 there is
raw payload, not a scrambling-control field. This is the exact
decrypted-HD-DVD defect (an `.evo` RDI pack is 0xBF with pack-start and
0x14 bits set); the video/private_stream_1 cases prove the exclusion
doesn't weaken real-DVD detection.
- `MockSource` fields `lock_all`, `crackable`, `short_read`, `stream_id`: model
a CSS-locked drive (`05/6F/03` on every read), a designated crackable LBA
serving a full synthetic scrambled sector, a short-read count (the
`recovery: true` short-read contract, `Some(0)` being the degenerate
spin-guard case), and the PES `stream_id` written at 0x11 (used to model an
HD-DVD `.evo` RDI nav pack via `PRIVATE_STREAM_2`).
- `crackable_sector`: builds a crackable scrambled sector whose cleartext
header (0x59..0x80) carries a periodic run continuing across the 0x80
boundary, mirroring the `synth_periodic_sector` fixture in the keyless
tests but built from the crate-internal `scramble_sector`.
- `a_source_that_returns_zero_sectors_still_obeys_the_scan_budget`: `tried`
increments only per inspected sector inside `for s in 0..usable`; an
`Ok(0)` never runs that loop, so `advance` is forced to 1 to keep the
cursor moving and the extent's full declared length is walked. Mutation:
deleting the `tried` charge in the `usable == 0` arm goes red at 60,000
reads for a misbehaving/adversarial source.
- `crack_key_caps_total_tries_at_50000`: grounding — `max_tries = 50_000`,
`tried += 1` before the read, loop guard `tried < max_tries`. Mutation:
widening the cap, or removing the increment, both go red.
- `crack_key_budget_is_shared_across_extents`: grounding — `tried` is
declared outside the extent loop with a shared `if tried >= max_tries`
break. Mutation: moving the declaration inside the loop would give each
extent its own budget (80,000 total instead of 50,000).
- `crack_key_scans_from_extent_start_lba`: grounding —
`reader.read_sectors(ext.start_lba + i, ...)`. Mutation: dropping
`ext.start_lba +` would start LBAs at 0 instead of the extent's start.
- `crack_key_continues_past_read_errors`: grounding — a read `Err` falls
through to the next sector rather than aborting. Mutation: propagating the
error with `?` would stop the scan after the first failure.
- `crack_key_empty_extent_reads_nothing` / `crack_key_no_extents_is_none`:
grounding — `while i < ext.sector_count` / `for ext in extents` are no-ops
on empty input; an off-by-one (`<=`) or a pre-loop read would break these.
- `crack_outcome_reaches_cracked_with_span`: audit gap "MockSource never
yields a crackable sector" — drives the full `crack_key_scan` over a
synthetic ISO to exercise the `Cracked` branch and `crack_span` recording
end-to-end, previously untested.
- `is_scrambled_uncracked_is_true_for_that_case_and_false_for_the_other_two`:
a prior round introduced the three-way split precisely because conflating
the cases made an uncrackable disc exit 0 with garbage output; earlier
tests only asserted the TRUE direction, so a predicate answering "yes"
unconditionally was indistinguishable from a correct one. All three
outcomes here come from real `crack_key_outcome` scans, not hand-built enum
values.
- `resolve_dvd_title_key_*` tests: `resolve_dvd_title_key` is the SINGLE
shared per-title CSS step both read paths (`build_iso_pipeline` multi-pass
and `DiscStream::new` single-pass) call, so these pin its full contract at
the shared boundary — crack success, hard-fail on scrambled-uncrackable
(never leaving `keys` as `None`, which would mux scrambled bytes as
plaintext), the decrypted-HD-DVD RDI-pack regression (end to end), `--raw`
passthrough, the AACS-keys-untouched gate, the clear-DVD no-op, and the
halt-cancellation-surfaces-as-Halted fix (not a truncated-scan verdict).
- `all_locked_synthetic_iso_yields_css_key_missing_signal`: audit §2/§5 #7 —
pins that an all-locked synthetic ISO across MULTIPLE extents produces
`ScrambledUncracked`, the exact signal `disc/mod.rs` converts into
`disc.css_error = Some(Error::CssKeyMissing)`.
- `recrack_succeeds_on_other_vts_extents`: audit gap "success path missing"
— the prior re-crack test only covered the locked→None path; this proves a
key cracked for one VTS is genuinely re-derived (not reused) for another.
-32
View File
@@ -1,32 +0,0 @@
# Warner CTRM label parser — relocated rationale
Referenced from `src/labels/ctrm.rs` unit tests via `// See docs/ctrm.md` pointers.
## Immunity pin: no analog of the `paramount` forced_sub false positive
Immunity pin against the defect measured in the `paramount` parser, where a
vendor `forced_sub` cell hung off a FULL dialogue track's own slot to say
"this track also contains forced signs", and reading that cell as "this
track is forced" flagged 30 MB dialogue tracks forced.
This format cannot express that. The forced signal is not a flag beside a
track's entry — it IS the entry's stream-kind token, drawn from a closed
vocabulary in which `subtitle_production` (the full dialogue track) and
`subtitle_narrative` (the forced-narrative track) are mutually exclusive
alternatives in the same position. A row is one or the other; there is no
cell a full track can carry to acquire the qualifier, so the paramount
failure mode has no encoding here.
Mutation: give `subtitle_production` a `Forced` qualifier, or add a forced
side-flag that both kinds may carry.
## Immunity pin: stream numbers come from the row, not a counter
`language_streams.txt` states each stream's number in field 3, so a row the
parser cannot use is simply dropped — it can never renumber the rows behind
it. This is the property that keeps this parser out of the STN-slot-shifting
failure mode that bites parsers which count positionally: there, a skipped
entry silently pulls every later label one stream forward.
Mutation: replace `parts[2]` with a running per-type counter → the three
unusable rows in the corresponding test collapse the survivors onto 1/2 and 1.
-39
View File
@@ -1,39 +0,0 @@
# `labels::dbp`
## Implementation
Iterates `CpInfo::Utf8` constant-pool entries rather than raw
byte-scanning each class file. Equivalent label coverage (the literal
`TextField,...` strings live in the CP as Utf8 entries) with no
false-positive risk from method bytecode or attribute names that
happen to contain `TextField,`. Language / purpose / qualifier
classification lives in `super::vocab` so all Java-parser families
share one source of truth.
The parser ignores any prefix before the first `TextField,` occurrence
in a constant string — whatever string-pool ordering placed ahead of
it is irrelevant.
## `MAX_LABEL_BYTES` rationale
The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
constant contributes up to 65535 bytes, and the `u16` stream-number
keyspace admits 65536 of them per type.
Headroom: real dbp menu labels are short display names — "English Dolby
Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
bytes. 256 leaves >5x headroom over that, and any string past it is menu
geometry or padding, never a language name — `vocab::lang` would not
resolve it anyway.
## `MAX_LABELS_PER_TYPE` rationale
The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
type are reachable; paired with `MAX_LABEL_BYTES` this bounds the whole
scan at 2 x 512 x 256 bytes.
Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
streams per playlist, and dbp emits one menu TextField per stream. 512
leaves 16x headroom over the spec maximum.
-500
View File
@@ -1,500 +0,0 @@
# decrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/decrypt.md — <topic>` pointer
left at the original site.
## Parallel AACS decrypt
Each AACS aligned unit (6144 bytes) is decrypted INDEPENDENTLY of every
other unit — per-unit key derivation from the unit_key plus the unit's
own first-16-byte header. There is no cross-unit dependency, so a
buffer of N units can be decrypted on N threads in parallel via a
persistent rayon thread pool.
Small buffers (< `PARALLEL_MIN_UNITS` units) fall through to the serial
path to avoid pool dispatch overhead beating the per-unit AES work.
## Thread-count configuration — three layers
Resolution order (highest wins):
1. The most recent `set_decrypt_threads` call with `n > 0`. Calling
this *replaces* the live thread pool — useful for a settings-page
slider in a long-running daemon.
2. `FREEMKV_THREADS` env var, if set and `> 0`. Single knob covering
decrypt today, intended to also drive any future input-side /
output-side worker pools.
3. Default: all available cores. Algorithm optimisation comes first —
we measure single-thread performance to find serial bottlenecks
before throwing parallelism at it — but once a pool is engaged we
use the whole box. Hard cap at `MAX_THREADS` (rayon stack memory).
## DECRYPT_POOL
Current rayon pool. `RwLock<Option<Arc<...>>>` so that
`set_decrypt_threads` can swap the pool out without leaking the old
one and without blocking ongoing decrypt work (in-flight calls hold an
`Arc` clone via `decrypt_pool` and finish on the old pool; new calls
pick up the new pool).
## decrypt_pool()
Get (or lazily build) the active rayon thread pool. Returns an `Arc`
so in-flight work survives a concurrent `set_decrypt_threads` swap.
Returns `None` if the pool cannot be built (e.g. the OS refuses the
worker threads under a pid/thread limit). The caller falls back to the
serial decrypt path — library code never panics here.
## unit_is_our_phase
Does this unit belong to the phase we hold the key for?
An FMTS forensic segment interleaves two variants at the unit level:
the disc carries both, and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving
it as ciphertext is correct, because the muxer drops untouched
ciphertext cleanly.
`Phase::All` means the whole range is ours (the non-forensic case).
This lived inline inside `apply_aacs_map`'s per-unit closure, where
nothing could reach it: a mutation run flipped the `-` to `+` and the
`/` to `*` in the index arithmetic and every test still passed.
Getting either wrong silently decrypts the wrong half of a forensic
segment.
## AacsKeyMap (proactive AACS key-selection map)
Which held unit key decrypts each LBA of a title's encrypted content,
decided ONCE before mux from the disc's CPS-unit (and, later, FMTS
segment) structure — never by trial-decrypt-and-check per unit at mux
time.
This is the pivot that ends the mux "key-server storm": the old path
decrypts a unit, checks whether the plaintext looks like clean MPEG-TS,
and — because authored-bad content never reaches that bar — concludes
"wrong key, fetch a fresh one" and re-asks the key service for units it
already holds the correct key for. There is NO per-unit byte pattern
that separates "correctly decrypted but authored-bad" from "still
encrypted", so that check is unanswerable. The map removes the
question: we resolve one key per CPS unit / segment up front (see
`resolve_mux_key_map`), record which LBA ranges each covers, and at mux
time simply "decrypt this LBA with key K" and trust it — bad TS is the
muxer's concern, exactly as for a physically-read clear disc.
Ranges are `[start_lba, end_lba)` → index into the `Aacs { unit_keys }`
pool, sorted and disjoint. The map is a POSITIVE list: an LBA in no
range is passed through untouched (no default key). How a single-CPS
disc is mapped depends on the caller: the whole-disc EXTRACT path uses
one blanket range `(0, u32::MAX, 0)` so every encrypted unit — parsed
title or orphan clip — resolves to key 0; the per-title MUX/sweep path
(`resolve_mux_key_map` → `content_map`) maps only the title's own
extents, so an orphan clip outside them is left as pass-through. Either
way, clear nav/filesystem sectors (encrypted-flag off) pass through.
## AacsKeyMap::read_plan
Build the FMTS **read plan**: the title's aligned units filtered down
to only the units this rip must actually read — every default / CPS
unit, plus, inside each forensic segment, ONLY our-phase (`Phase::Even`
/ `Phase::Odd`) units. The alternate-phase units are a different device
group's variant: a licensed player never reads them, and neither do we.
They are omitted from the plan entirely, so they are never fetched,
decrypted, or handed to the demux — the demux therefore sees one
gapless our-variant stream, with no ciphertext to trip a
concealed-gap resync (the old behaviour that dropped good frames
around every segment).
`extents` are the title's clip extents (unit-aligned in the interior; a
shorter tail is ordinary content and always kept). `unit_sectors` is
the AACS aligned-unit size in sectors (3). Contiguous kept units
coalesce into as few extents as possible so the producer still issues
large sequential reads across default content; only inside a ~480 KB
forensic segment do reads become unit-granular (every other unit). A
map with no forensic (Even/Odd) range returns `extents` unchanged — the
common disc is not touched.
The parity test is byte-identical to the decrypt hot loop
(`(unit_lba - range_start) / unit_sectors`), so a unit kept here is
exactly a unit `decrypt_sectors_mapped` would open, and vice-versa.
## apply_aacs_map
AACS scheme step: apply `map`'s per-unit keys to `buf`, in-place.
`base_lba` is the absolute LBA of `buf`'s first sector; each aligned
unit (3 sectors) is decrypted with the key the map assigns to its LBA.
There is NO key trial and NO `is_clean` verdict: the map already
decided the key from disc structure, so this applies it and moves on —
a unit that decrypts to authored-bad TS passes through for the muxer to
drop, never re-fetched.
A SCHEME, not a policy. It reports what it could not open by returning
`Err(DecryptFailed)`; the decision that an unopenable unit must never
be emitted belongs to `decrypt_span`, which is the one place that
decides it for every scheme. A map index outside the held pool is
likewise a fail-loud `Error::DecryptFailed`: the resolver's job is to
guarantee every selectable index is present, so a gap here is a
resolver bug, not silent loss.
## decrypt_sectors — CSS / clear detail
For CSS: descrambles per 2048-byte sector, self-cracking the title key
from the data (no external input). For `None`: a no-op. For AACS:
**always** returns `Err(DecryptFailed)` — AACS decrypts exclusively
through the resolved key map (`decrypt_sectors_mapped`), which keys
every content unit up front and fails at RESOLVE time when a key is
missing. Reaching this arm with AACS keys means a reader was built
without installing its map (a bug), so it fails loud rather than apply
a guessed key.
`unit_key_idx` is a legacy parameter kept so the CSS / `None` wrapper
signature stays stable; it is ignored (the CSS arm self-gates on its
per-sector scramble flag). Returns `Err` if decryption was expected but
impossible; never produces silently corrupted output. The `usize`
return is a legacy unverified-byte count that is always `0` for the CSS
/ `None` arms.
## decrypt_span — orchestrator rationale
THE decrypt orchestrator. Every path into this crate's decryption goes
through here.
How a disc decrypts is one process — resolve a key for this span,
apply it, and refuse if no key can be proven. Only the resolve-and-apply
step is scheme-specific. This function owns the loop and the refusal;
the schemes below supply only what genuinely differs between AACS, CSS
and clear media.
That split exists because its absence caused six separate defects in
one release. There used to be TWO top-level paths — this one for CSS
and clear, and a wholly separate `decrypt_sectors_mapped` for AACS
whose arm here was a bare `return Err` stub — so each scheme decided
its own answer to "there is no key for these bytes" and nothing held
them to the same one. CSS drifted to descrambling with a key it had
just proven stale; the mapped path drifted to passing an unkeyable
encrypted unit through as ciphertext. Both looked like success to the
caller.
Adding a scheme means adding an arm here, which means answering the
refusal question. That is the point.
## Tests — relocated rationale
### a_forensic_read_plan_drops_units_the_full_extents_include
`DecryptKeys::None` is a no-op even with a content map + scrambled
bytes. A forensic map's read plan is NOT the extents it was given —
and that is the precondition the mux's provenance guard keys on.
A clip's feed span is measured over the FULL extents at scan time,
while the mux reads this reduced plan, so the byte offsets stamped on
frames and the offsets recorded in the spans describe different
streams. The deficit accumulates, so every frame after the first
segment resolves to an earlier clip than it came from. The spans still
tile each other, so the tiling check cannot see it; the mux compares
the plan against the full extents instead and stops trusting
provenance when they differ.
### content_gate_css_keys_is_noop
CSS ignores the content gate (it lives in the AACS arm) and always
reports `0` — confirming the gate is a no-op for CSS and the read
stays scheme-agnostic (the litmus test: adding CSS verify touches only
the CSS arm, never the read).
### content_gate_css_actually_descrambles_the_buffer
`decrypt_sectors_in_content` is the entry point `DecryptingSectorSource`
dispatches to whenever a content map is installed
(`sector/decrypting.rs` line ~211), so it is on the live read path for
every mapped rip. It must actually DECRYPT. The two `_is_noop` tests
above only assert its `usize` return is `0` — which is what a body
replaced by `Ok(0)` also returns, so neither one constrains it at all.
Here a genuinely scrambled CSS sector goes in and the CONSTRUCTED
plaintext must come out. Anything that skips `css::descramble_region` —
including a body that just reports `Ok(0)` — leaves ciphertext in the
buffer and the caller muxes scrambled MPEG at exit 0.
Expected bytes come from the plaintext this test built BEFORE
scrambling (CSS scrambles only 0x80..2048; the header stays clear), not
from re-running any descramble routine.
### content_gate_aacs_keys_fail_loud_not_ok_zero
The AACS arm of the same entry point must fail LOUD. Under the
keymap-only model AACS decrypts exclusively through
`decrypt_sectors_mapped`; reaching this wrapper with AACS keys means a
reader was built without installing its key map, and continuing would
hand the caller ciphertext under an `Ok`. `DecryptFailed` is the
correct verdict per the function's own contract — it must not be
softened into a success with a zero count.
### css_region_change_recracks_the_title_key
CHARACTERIZATION (recovery refactor safety net): the CSS arm's
per-region re-crack (the `title_key` cache is stale for a new VOB
region → restore ciphertext, `crack_title_key` this sector,
re-descramble). Two crackable sectors scrambled under DIFFERENT keys
sit back-to-back; the cache is primed to the FIRST key. Sector 0 rides
the cache (crib matches); sector 1 must trip the crib mismatch and
re-crack to its own key. Both must land correct plaintext, and the
cache must end on region 1's key.
This behaviour currently lives inline in `decrypt_sectors` (the `Css`
arm). It is the delicate logic the recovery refactor will move to the
input-stream seam, so it must stay green byte-for-byte across that
move.
### aacs_scrambled_trailing_partial_is_rejected
Whole leading unit plus a SCRAMBLED trailing partial that is FLAGGED
encrypted in its clear seed (the malformed danger case): an encrypted
unit split across an extent boundary cannot be CBC-decrypted
standalone. The mapped decrypt must fail loud with `DecryptFailed`
rather than emit the ciphertext partial as clear. Exercises the real
shipping path (`decrypt_sectors_mapped`) and its trailing-partial
guard.
### aacs_clear_trailing_partial_passes_through
A CLEAR trailing partial (encrypted flag NOT set) is a legitimate
content tail and must pass through, never trip the guard above.
"Passes through" means byte-for-byte unchanged, not merely `Ok`.
Asserting only `is_ok()` let a mutant that corrupts the clear partial
while still returning `Ok` pass — which is the whole failure this test
names. Mutation: XOR any byte of the tail before returning -> the
snapshot comparison fails.
### none_keys_is_noop
`DecryptKeys::None` is a pure no-op: the buffer must be returned
byte-for-byte unchanged with Ok, regardless of content (even content
that looks scrambled).
Grounding: the `DecryptKeys::None => {}` match arm does nothing.
Mutation: replace the empty arm with a call that mutates buf -> the
unchanged assert fails.
### is_encrypted_matches_variant
`is_encrypted` reflects the variant: None -> false, Css/Aacs -> true.
Grounding: `!matches!(self, DecryptKeys::None)`.
Mutation: invert the `!` -> None reports true, this fails.
### css_descrambles_with_title_key
The CSS path descrambles each 2048-byte sector with the title key. A
scrambled sector run through decrypt_sectors must come back to its
plaintext body (keystream XOR is involutive), proving the title key is
actually applied.
Grounding: `DecryptKeys::Css { title_key } => for chunk in
buf.chunks_mut(2048) { descramble_sector(title_key, chunk) }`.
Mutation: change `chunks_mut(2048)` to `chunks_mut(2049)` or pass a
fixed wrong key -> the body no longer matches the plaintext.
### css_processes_every_sector_in_buffer
The CSS path processes EACH 2048-byte sector independently in a
multi-sector buffer. Two scrambled sectors (with different seeds) in
one buffer must both round-trip — pinning that the loop steps by 2048
and applies the key to every sector, not just the first.
Grounding: `for chunk in buf.chunks_mut(2048)`.
Mutation: change the loop to descramble only the first chunk (e.g.
`.next()`) -> the second sector stays scrambled, assert fails.
### css_rekeys_when_title_key_region_changes
CSS title keys are per-VTS/VOB region: a real disc holds DIFFERENT keys
for different regions and the only way to get each is to crack it. The
decrypt path must re-crack when the cached key stops descrambling (its
crib no longer reappears at 0x80) instead of blindly applying one key
across a region boundary — the bug that pixelated every freemkv DVD
rip.
Two sectors scrambled under DIFFERENT keys, cache primed to ONLY the
first (exactly what the one-shot scan crack leaves). Sector 0 validates
+ descrambles with the cached key; sector 1's cached-key descramble
fails the crib, so the path re-cracks sector 1's own key and recovers
its plaintext. Before the fix (blind single-key apply) sector 1 was
garbage.
Grounding: the CSS arm's `attack_crib` → `chunk[0x80..] != crib` →
`crack_title_key` → `*title_key = fresh` rekey.
Mutation: drop the rekey branch (apply the cached key always) →
sector 1's body no longer matches its plaintext; this fails.
### css_leaves_clear_sector_unchanged
The CSS path leaves UNSCRAMBLED sectors (flag clear) byte-for-byte
untouched — descramble_sector early-returns on a zero flag. A clear
sector mixed into the buffer must not be corrupted.
Grounding: descramble_sector returns immediately when
`(sector[0x14] >> 4) & 0x03 == 0`.
Mutation: remove that early return in lfsr.rs -> a clear sector would
be XORed with a keystream and change; this fails.
### css_empty_buffer_is_ok
CSS decrypt always returns Ok (it cannot fail — descrambling is XOR,
no key validity check), even for an empty buffer.
Grounding: the CSS arm has no `return Err` path; `chunks_mut` over an
empty slice is a no-op; the function ends `Ok(())`.
Mutation: make the CSS arm return Err -> this fails.
### aacs_mapped_out_of_range_key_idx_errors
A map that selects a key index OUTSIDE the held pool must fail loud
with DecryptFailed — never silently apply a wrong key or pass
ciphertext through. This validates `decrypt_sectors_mapped`'s up-front
`key_indices()` bounds check (the real shipping AACS decrypt path).
Mutation: drop the `unit_keys.get(idx).is_none()` guard → the
out-of-range index would not error; this fails.
### aacs_via_unmapped_decrypt_sectors_fails_loud
SAFETY NET: reaching the CSS/`None` wrapper (`decrypt_sectors`) with
AACS keys means a reader was built with no map — a bug. It must fail
loud, never apply a guessed key. (AACS decrypts exclusively via
`decrypt_sectors_mapped`.)
### read_plan_forensic_reads_only_our_phase_units
FMTS: a forensic Even segment drops exactly its alternate (odd) units
from the read plan — they are never fetched — while default content on
either side stays one coalesced sequential run. The kept units are
byte-identical to the ones the decrypt hot loop opens.
### read_plan_phase_parity_is_measured_from_an_unaligned_range_start
A forensic range does NOT start on an aligned-unit boundary. Its start
LBA comes from a source-packet number — `start_spn * 192` put through
`clip_byte_to_lba` (`mux/resolve.rs`) — and 192-byte packets have no
relationship to the 3-sector aligned unit, so `range_start % 3` is
whatever the disc says.
That makes `unit_ix = (lba - range_start) / us` load-bearing in both of
its operations, and the existing coverage used a range starting exactly
on the extent's first unit, where several wrong formulas agree with the
right one by arithmetic accident.
Getting the parity wrong is not a crash. It reads and decrypts the
ALTERNATE variant's half of a forensic segment: the units this disc's
key does not open decrypt to garbage, and the units it does open are
skipped. AACS 2.1 forensic marking is exactly the mechanism that makes
the two halves different, so a phase inversion is silent — it produces
a full-length rip carrying the wrong variant.
### read_plan_gates_the_last_whole_unit_of_an_extent_not_just_the_remnant
An extent whose last whole unit is an alternate-phase unit must still
drop it. The tail guard exists for a REMNANT shorter than a unit —
bytes with no following unit to desync — and an extent ending exactly
on a unit boundary has no remnant at all.
With the guard widened to `remaining <= us`, the final unit of every
extent bypasses the phase gate and is read unconditionally. On a
forensic segment that lands at an extent end, that is one
alternate-variant unit pulled into the rip and decrypted with a key
that does not open it.
### every_scheme_gives_the_same_verdict_when_no_key_can_be_proven
Every scheme that CANNOT prove a key answers the same way.
This is the property `decrypt_span` exists to hold. There used to be
two top-level decrypt paths — one for CSS and clear media, one for
AACS — and each decided its own answer, so they drifted apart in
opposite directions within a single release: CSS descrambled with a
key it had just proven stale, and the AACS path passed an unkeyable
encrypted unit through as ciphertext. Both reported success.
Asserting one verdict across the schemes is what makes a future
divergence a test failure rather than a silent corruption. A
per-scheme test cannot do that: each would still pass while the two
disagreed.
CSS is deliberately NOT in this list. Its title key is recovered from
the data, sector by sector, by a heuristic that false-positives — a
crib mismatch whose re-crack fails means the crib was wrong, not that
the key is stale, so the cached key is kept and used. Round 9 folded
CSS in here on the reasoning that "no key" should mean one thing
everywhere; that made real DVDs unrippable, and the real-media gate
caught it. Uniform policy is right for schemes that can PROVE a key
wrong. CSS cannot.
### an_encrypted_unit_outside_every_key_range_fails_instead_of_passing_through
An ENCRYPTED unit that falls outside every key-map range must fail,
not pass through as ciphertext.
"The map has no key here" and "there is nothing to decrypt here" are
different statements, and only the second makes passing the unit
through correct. On a multi-CPS disc an orphan clip — referenced by no
playlist, so in no title extent and therefore in no range — hits the
first and was treated as the second. `extract_tree` then counted those
bytes as GOOD, dropped the `.partial` suffix, and reported `complete:
true`, exit 0: a scrambled file on disk with a clean bill of health.
A CLEAR unit outside every range is the ordinary case (filesystem and
nav on a whole-disc read) and must still pass through untouched — so
this asserts both directions.
### mapped_key_selection_is_positional_so_pool_order_matters
The mapped descramble indexes the committed key pool POSITIONALLY
(`unit_keys[key_idx].1`), so the ORDER of the `Vec<UnitKey>` a
`KeySource` returns is load-bearing — it is NOT "cosmetic, the decrypt
path strips it and tries every key", as
`keysource::resolve_and_apply_traced`'s doc used to claim. Trial-decrypt
was deliberately deleted; nothing here searches the pool. Reordering
the same two keys therefore sends each range to the WRONG key: the
range that decrypted clean now fails the correct-phase `is_clean` net
loudly (or, off a forensic phase, would decrypt a whole span under a
neighbour's key). Pins the corrected doc.
### decrypt_threads_within_valid_pool_range
The default (auto) decrypt thread count is always a usable pool size:
at least 1 (a 0-thread rayon pool is invalid) and never above
MAX_THREADS (rayon stack-memory cap). This test reads the resolved
value without mutating the process-global override, so it is safe to
run in parallel with other tests.
Grounding: `cores.clamp(1, MAX_THREADS)` in the default branch;
`env.min(MAX_THREADS)` in the env branch.
Mutation: change `.clamp(1, MAX_THREADS)` to `.clamp(0, MAX_THREADS)`
on a 0-core probe (unlikely) — more robustly, change the cap to
`MAX_THREADS * 2` -> on a many-core CI box the upper-bound assert can
fail. The lower-bound (>=1) guard is the load-bearing invariant.
### phase_gate_selects_only_our_parity_of_a_forensic_segment
The FMTS phase gate picks which half of an interleaved forensic
segment we decrypt. Both the `-` and the `/` in its index arithmetic
survived a mutation run, and getting either wrong silently decrypts the
alternate variant into garbage while reporting success.
### phase_gate_does_not_panic_on_a_malformed_map
A malformed key map must not take down a long-running service. A unit
below its own range start, or a zero unit size, are both map bugs —
they must return a defined answer rather than panicking on debug
overflow or dividing by zero.
Every case here asserts the DEFINED answer, not merely the absence of a
panic. The zero-unit-size case used to be written
`assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
accepts both answers and so pinned nothing at all: the guards could
invert and it would still pass. The answer is knowable —
`saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
is even — so pin it.
</content>
-34
View File
@@ -1,34 +0,0 @@
# `DecryptingSectorSource`
## Why this decorator is the single source of truth for decrypt-on-read
Every decrypt-on-read caller (e.g. `DiscStream`) wraps its source in this
decorator rather than calling the cipher helpers directly. The actual cipher
code lives in `crate::aacs` and `crate::css`; the decorator just calls the
existing `crate::decrypt::decrypt_sectors` helper that drives both of them
in-place after each read. Centralizing the call site means callers can wire
the decorator unconditionally and keep their pipeline shape uniform
regardless of encryption state — for `DecryptKeys::None` discs it is simply
a pass-through, so there is no branch in caller code for "is this disc
encrypted?".
## `KeyFetch`: why two explicit operations
`KeyFetch` exposes `unit_keys` and `fmts_indexes` as two separate operations
rather than one generic callback, so the "one base key vs. a whole forensic
set" contract lives in the type instead of a caller guessing at the return
length:
* `unit_keys` resolves the base Unit Key(s) for a CPS unit from real
encrypted samples drawn from it. The non-forensic path: one key per CPS
unit (the pool grows by whatever it returns). Used by the mux's base /
multi-CPS map resolution and by the sweep/patch recovery decorator.
* `fmts_indexes` resolves the disc's AACS 2.1 forensic index keys from an
index-1 single-phase anchor batch. The source hands back the COMPLETE set
(ordered index 1..N); the caller sizes the forensic map to `len()` and
never assumes a fixed N (32 is all we've seen, but the contract is
"whatever the source returns, >= 1, is all of them").
`KeyFetch` is a stateless, shared pair of `Arc<Fn>`, so one instance is built
once and cloned cheaply (two `Arc` bumps) into every read path; it is
`Send + Sync` so it can ride the mux highway's producer thread.
-227
View File
@@ -1,227 +0,0 @@
# Deluxe BD-J framework parser (`src/labels/deluxe.rs`)
Detected on discs whose `/BDMV/JAR/<x>.jar` contains a `com/bydeluxe/`
directory entry.
## What this parser reads
Deluxe-authored discs store stream labels as ordinal references into
enum classes whose names are obfuscated per-disc, so a name-based
match won't work. The label data is instead recovered by matching on
the **shape of each enum's `<clinit>`**, which is framework-stable:
| Enum | Signature |
|---|---|
| Language | 70 `ldc` operations in `<clinit>`, sequence starts `English, French, Spanish, Dutch, ...` |
| Purpose | 8 ldcs starting `Normal, Commentary, PiP, Trivia, ...` |
| VideoFormat | 7 ldcs starting `HD, HDR10 Plus, HD Dolby, ...` |
| Region | 22 ldcs starting `USA_D1, LIC1, LIC2, LIC3, ...` |
| Studio | 6 ldcs in `<clinit>` |
Matching on the shape rather than the class name keeps the parser
working across obfuscation variants. Codec strings come from the
standard BD-J `org/bluray/ti/CodingType` enum referenced directly by
the binding constructors (see `StackVal::CodingType`), not from a
Deluxe-internal enum.
## Implementation phases
- **Phase A** — master enum identification (`identify_master_enums`).
Walks every `.class`'s `<clinit>` ldc sequence and matches against
the framework-stable fingerprints. Output: `Vec<(label, MasterEnum)>`
with full ordinal → string-value tables.
- **Phase C** — binding-class identification (`find_binding_classes`).
The per-stream table is built by some class via repeated
`getstatic` references to the master enums identified in A.
That class has the highest such `getstatic` count in the jar.
Heuristic shape; precise threshold may need tuning.
- **Phase D** — binding-class bytecode decoder (`decode_binding`).
Walks the binding class's `<clinit>` with a tiny symbolic stack
machine. For each `new X / dup / ... / invokespecial X.<init>`
sequence, collects the int values and enum-reference operands
between the `dup` and the constructor call, then emits a
`DecodedStream`. The signal-to-StreamLabel mapping (which arg is
stream index? which is language? audio vs subtitle?) uses a
heuristic — see `interpret_streams` for the mapping rules.
## Confidence
`parse` returns `Some(ParseResult::high(labels))` when Phases A
through D produce at least one stream: the master enums matched their
framework-stable fingerprints (a strong ordered-prefix signature), the
binding class decoded, and at least one per-stream binding resolved to a
real (language, purpose, codec) tuple — the schema was fully recovered,
not guessed. `None` when the disc isn't Deluxe-authored or when decoding
produces zero streams (a recognized-but-broken state that the analyzer
still surfaces via `parsers_detected`).
## Size caps (Phase A candidate pool)
`MAX_CLINIT_LDC_STRINGS` (cap on `ldc` operands retained per class by
`clinit_ldc_strings`): unlike every other count cap in this crate, the
paired byte cap here cannot be the disc-file size — a `.class` entry
gated only by a `com/bydeluxe/` path prefix deflates from ~100 KB up to
the 64 MiB `MAX_CLASS_BYTES` read ceiling, so ~33M two-byte `ldc`
instructions — one retained `String` each — are reachable from a small
crafted disc. The bound has to be on the decompressed work, so it is
applied here. Headroom: the largest framework-stable enum is `Language`
at 70 values (`FINGERPRINTS`), and no fingerprint matches a count more
than `LDC_COUNT_TOLERANCE` away from its expected size, so anything past
~74 can never identify a master enum. 4096 leaves ~55x headroom over the
largest real enum for framework drift.
`MAX_CLINIT_LDC_BYTES` (companion byte cap): the count cap alone still
admits 4096 x 64 KiB of `Utf8` (a JVMS `CONSTANT_Utf8_info` length is a
u16), i.e. ~268 MB per class from repeated `ldc` of one huge constant.
Headroom: master-enum values are short display names ("English",
"HDR10 Plus", "USA_D1") well under 32 bytes, so a real Language enum
retains ~1 KB. 256 KiB admits 4096 values averaging 64 bytes each.
`MAX_CANDIDATE_TOTAL_BYTES` (aggregate companion, bounds retention across
ALL classes at once): `identify_master_enums` holds every class's
retained strings in one map simultaneously, so the per-class cap alone
still admits `classes x 256 KiB`: a 64 MiB jar of minimal `.class`
entries reaches tens of GiB. This is the same bound one level up.
Headroom: the five `FINGERPRINTS` enums together hold ~113 short values
(~1.2 KB). Every other class in a real BD-J jar contributes only
whatever string constants its own `<clinit>` loads — resource paths,
config keys — so a large authored jar lands in the low hundreds of KB.
16 MiB leaves ~40x headroom over a deliberately generous 400 KB estimate
for real media.
`MAX_CANDIDATE_CLASSES` (entry-count companion to
`MAX_CANDIDATE_TOTAL_BYTES`): the byte budget alone still admits ~16M
map entries when every class retains a single one-byte `ldc`, and the
per-entry `HashMap` + `String` overhead is not counted by that budget.
Headroom: a large retail BD-J title ships on the order of 1-3k classes,
and only those with a non-empty `<clinit>` ldc sequence become
candidates. 65536 leaves >20x headroom over the class count of any real
jar.
## Internal helpers
- `clinit_enum_field_names`: collects the static-field names an enum
class's `<clinit>` stores its own instances into, in declaration order
— the ordinal -> field-name mapping that mirrors `clinit_ldc_strings`'s
ordinal -> value mapping. An obfuscated Deluxe enum constant compiles
to `new E; dup; ldc "Value"; invokespecial E.<init>(…); putstatic
E.<field>`, so the `putstatic` whose owning class AND field descriptor
are both this class's own type names the field for that ordinal. The
binding class then references that constant as `getstatic E.<field>`;
without this mapping `MasterEnumTable` cannot turn the obfuscated field
name back into an ordinal. Bounded by `MAX_CLINIT_LDC_STRINGS` like the
value walk.
- `clinit_ldc_strings`: walks `<clinit>` and collects every `ldc`/`ldc_w`
operand that resolves to a `String` or `Utf8` constant, in declaration
order; `None` if the class has no `<clinit>`. Collection stops at
`MAX_CLINIT_LDC_STRINGS` operands or `MAX_CLINIT_LDC_BYTES` of retained
text, whichever comes first: the walk is driven by the decompressed
class, so it isn't bounded by the disc-file size cap the way the rest
of this module's counts are. Truncation cannot lose a real match — a
truncated sequence is far longer than any `FINGERPRINTS` entry's
`expected_count + LDC_COUNT_TOLERANCE`, so it would have been rejected
on count anyway.
- `find_binding_classes`: identifies all binding-class candidates by
getstatic-count to the master enums, using `MIN_GETSTATIC = 4` — the
binding class on a typical disc has 50+ such getstatic references (one
per slot x arity); the low floor lets a small disc with few streams
still qualify, while still filtering out classes that just reference
the language enum once for a config string. Some Deluxe discs split
the per-stream table across two binding classes (one for audio, one
for subtitle — the audio class commonly has the most getstatic refs,
the subtitle class somewhat fewer, both sharing the master Language +
Purpose enums), so this returns top-K candidates ordered by descending
getstatic count, filtered to a minimum concentration of references.
`MAX_CONSTRUCTIONS` (Phase D, cap on `Construction`s retained from a
binding `<clinit>` walk): one entry is appended per matched `new X / dup
/ invokespecial X.<init>` with no other bound — a `.class` gated only by
a `com/bydeluxe/` path prefix deflates to the 64 MiB `MAX_CLASS_BYTES`
ceiling, and the shortest matching sequence is a handful of bytes, so
millions of `Construction`s (each a `String` plus an arg `Vec`) are
reachable from a small crafted disc (~1 GiB). The same cap bounds the
per-class union in `decode_binding_class` (a crafted class may repeat
`<clinit>`, which JVMS §4.6 forbids but this reader tolerates) and the
cross-class union in `parse`, so the whole phase retains at most this
many. Headroom: the BD `STN_table` (BDAV stream-entry counts) admits at
most 32 primary audio and 32 PG streams per playlist, and a Deluxe
binding table covers the disc's playlists — low hundreds of entries on
the largest retail titles. 4096 leaves >20x headroom.
`Decoder::push` (symbolic-stack push during binding `<clinit>` walking):
honours the Code attribute's declared `max_stack`. JVMS 4.7.3 requires
that a method's operand stack never exceed `max_stack` at any point, so
a push past it can only come from bytecode that would fail JVM
verification. Dropping it costs nothing on real bytecode and bounds the
decoder: a `.class` gated only by a `com/bydeluxe/` path prefix deflates
from ~100 KB to the 64 MiB `MAX_CLASS_BYTES` ceiling, i.e. ~67M
single-byte `iconst_0` (~2 GiB of `StackVal`) on an unbounded `Vec`.
Headroom: `max_stack` is exactly what javac computed for the real
binding `<clinit>`, so no real construction can be clipped. The symbolic
stack counts `long`/`double` as one slot where the JVM counts two, so
its depth is never greater than the verified depth.
## interpret_streams: Constructions -> StreamLabels
Converts the per-construction tuples from Phase D into `StreamLabel`s.
Two binding-constructor shapes are handled:
- 5-arg: `BindingType.<init>(I, Lang;, Lpurpose;, I, LCodingType;)V`
- 4-arg: `BindingType.<init>(I, Lang;, Lpurpose;, LCodingType;)V`
Args are identified by **TYPE**, not position:
- First `EnumRef{kind: "Language"}` -> audio/subtitle language
- First `EnumRef{kind: "Purpose"}` -> Deluxe purpose ordinal
- First `CodingType(name)` -> codec field name (translated via
`coding_type_to_codec_hint`)
- First `Int(n)` -> stream index (preserved as ordering hint;
per-type sequential stream_number is what actually goes into the
StreamLabel, since BD spec stream-numbering is anchored on MPLS
data, not the binding code)
Stream type inference:
- Construction has a `CodingType` arg -> audio stream (subtitles on
Deluxe don't carry a CodingType; their codec is implicit PGS via
the BD spec).
- Construction has Language but no CodingType -> subtitle stream.
- Neither, and its binding type never yielded a stream -> not a
stream (skip). See `slot_kind` for why the binding type is
consulted rather than the language alone.
`slot_kinds`: which stream list each binding type enumerates, learned
from the constructions that DID resolve a language. A `<clinit>` walk
emits a `Construction` for every `new X; ...; invokespecial X.<init>`
it sees, so the list mixes real stream bindings with whatever else the
class initializer builds. `binding_type` is the constructed class name,
which is how the two are told apart: the stream bindings all share one
class (Deluxe splits audio and subtitle across two), and that class is
identifiable from the slots that resolved. A binding type that resolved
as both kinds is left out — with no consistent answer, guessing a list
to advance would be worse than not advancing.
`slot_kind`: the stream list an unresolved construction occupies a slot
in, or `None` when it is not a stream binding. A
`org.bluray.ti.CodingType` argument is decisive on its own: nothing but
an audio stream binding is handed one. Otherwise falls back to what the
binding type's resolved siblings showed (`slot_kinds`).
## Confirmed studio variants
- **Universal** (`studio="uni"`): Language enum `pd` (65 values,
`English, French, Spanish, Dutch, …`), Purpose enum `lp`
(`Normal, Commentary, PiP, Trivia, Descriptive, Score`), audio binding
`np.<init>(I, Lpd;, Llp;, Lorg/bluray/ti/CodingType;)`, subtitle binding
`wb.<init>(I, Lpd;, Llp;, Lmi;)`, all built in one binding class's
`<clinit>` alongside a title-wrapper object whose constructor takes the
per-stream arrays (filtered out by the array-parameter guard in the
decoder). SDH/RNIB is encoded as a distinct Language VALUE
("English SDH", "English RNIB"), recovered into the qualifier from the
name. Grounded on fixtures captured from a real Universal Blu-ray release.
- **Disney/Warner**: 70-value Language enum, same binding shape; the
original corpus this parser was written against.
-74
View File
@@ -1,74 +0,0 @@
# `demux://` sink — per-codec ES output
The `demux://` sink is a write-only `crate::pes::Stream`. Instead of muxing
the per-track `PesFrame` stream into a single container (as `MkvStream`
does), it routes each frame's payload to the file for `frame.track`,
post-processing where the internal codec `Frame` form differs from the
standalone on-disk ES form:
- **HEVC / H.264**: the codec parsers emit hvcC/avcC-style 4-byte
length-prefixed NALs (start codes stripped) and carry the parameter sets
out-of-band in `codec_private`. A standalone `.hevc`/`.h264` needs Annex-B
framing with the parameter sets prepended — see `AnnexBWriter`.
- **PGS**: the parser collapses a display/clear PCS pair into one
duration-bearing `Frame` with the raw segment payload but no `PG` magic /
timestamp header. A `.sup` needs the HDMV segment framing rebuilt — see
`PgsSupWriter`.
- **VobSub**: the `.sub` is the raw SPU stream; the `.idx` sidecar is
synthesized from per-SPU PTS + byte offsets — see `VobSubWriter`.
Every other codec writes `frame.data` verbatim (`PassthroughWriter`).
The sink does NOT touch the MKV mux path; it is purely additive.
## `PgsSupWriter`: rebuilding HDMV segment framing
The parser hands the demux sink the concatenated PGS segments of a display
set in `frame.data` (segment_type + segment_size + payload, repeated), with
no `PG` magic and no PTS/DTS. A `.sup` prefixes each segment with a 13-byte
header: `0x50 0x47` ("PG") | PTS u32 BE | DTS u32 BE | (segment_type|size
already present in the payload). When the parser folded a trailing clear
(`duration_ns` set), the writer re-emits it as an empty composition at
`pts + duration` so players time the subtitle out.
## `synthetic_clear_display_set`: synthesizing the clear PCS
Builds a synthetic "clear" display set: an empty PCS (0 composition objects)
followed by an END segment. The parser folds the original clear/end PCS
pair's wipe time into the display frame's `duration_ns` and drops the clear
bytes, so a faithful `.sup` re-emits one here at `display_pts + duration`;
without it every subtitle lingers to EOF. `width`/`height` are carried from
the display set's PCS so the clear PCS advertises the same video geometry —
they don't affect the wipe but keep the segment well-formed. Returned bytes
are concatenated `type(1)+size(2 BE)+payload` segments, the same shape
`emit_segments` consumes.
## Test: `epoch_driver_follows_ref_video_not_track_zero`
Regression for the hardcoded `frame.track == 0` epoch driver: on an M2TS/PMT
title the PMT can list an AUDIO ES before the VIDEO ES, so the video lands
at stream index 1. The sink already resolves the video reference
dynamically (`ref_video_track`); the epoch driver must use that SAME
reference, not the literal 0. In the test, track 0 is audio and track 1 is
video; a video-only clip boundary (track 1) must open a new epoch (bump
`offset_ns`). With the bug (only `frame.track == 0` drives epochs) the
video back-jump would be treated as a passive rider and `offset_ns` would
stay 0 — corrupting every track's rebased timeline.
## Test: `a_demux_export_the_seam_plan_emptied_fails`
Audit finding: this sink builds its timeline with
`TimelineContinuity::with_clips` but no test ever gave it a title with
usable PlayItem marks, so `frames_mapped` and `dropped_total()` were always
zero and BOTH gates in `finish` were unreachable. Deleting either left the
whole suite green while a `demux://` export of a seamless-branching title
wrote zero-byte track files beside a populated chapters document, at exit 0.
## `sanitize`: path-hostile filename characters
Control characters are replaced, NUL above all. Every string this touches is
disc bytes, and a language code is three raw STN bytes run through
`from_utf8_lossy` with no validation — `00 00 00` is the ordinary
"undefined" encoding on real Blu-rays. A NUL in a path aborts
`File::create` with `InvalidInput`, which took the whole demux export down
before a single track file was opened.
-65
View File
@@ -1,65 +0,0 @@
# `DemuxThread`
## Why a second worker thread
With `crate::sector::PrefetchedSectorSource` alone, read+decrypt already
runs on a producer thread; the *consumer* (main) thread still serialises
`ts_demuxer.feed` (M2TS parsing) with the codec parsers. Profiling showed
feed at ~37% and codec parse at ~44% of consumer wall time — i.e. feed is
heavy enough that pipelining it with parse pays for itself.
Splitting them: feed runs in `DemuxThread`; the consumer thread receives
`Vec<PesPacket>` batches and runs codec parse + frame emission only. Total
throughput becomes `1/max(feed, parse)` instead of `1/(feed + parse)`.
## Lifecycle
`DemuxThread::spawn_zero_copy` consumes the prefetch channels and the
demuxer state, returning a handle plus a `Receiver<DemuxBatch>`. Dropping
the handle closes the channel which signals the worker to exit; the join
in `Drop::drop` blocks until the worker observes channel closure and
returns (no timeout — a wedged downstream would block the drop until it
releases the channel).
## `spawn_zero_copy` rationale
Instead of taking a `SectorSource` and memcpy-ing through its
`read_sectors` API, this constructor consumes the prefetch channels
directly: filled buffers come in via `prefetch_rx`, the demux thread
feeds them, then returns them to `recycle_tx` for the producer to
re-fill. Eliminates the 16 MiB memcpy per batch that the SectorSource
adapter incurred (and, with the producer-side recycling pool, also
eliminates the per-batch heap alloc / cross-thread free that was costing
40%+ of demux-thread time before).
## Test: empty-batch disconnect fix (`worker_exits_promptly_on_consumer_drop_during_empty_batches`)
Regression: worker must detect consumer disconnect even when every demux
batch is empty (no matching PIDs / null packets).
Before the fix, `tx.send()` was never called for empty batches so the
worker never observed the consumer drop — it would spin through ALL
remaining extents before exiting, causing `DemuxThread::drop`'s `join()`
to block for minutes on a mostly-untracked disc region.
The watchdog: if the worker doesn't exit within 1s of the consumer drop
the test fails (rather than hanging forever as the bug would).
## Test: spawn-failure drop-order (`channels_disconnected_before_producer_join_on_spawn_failure`)
Regression: on thread-spawn failure the channels must be dropped BEFORE
the producer shell so the upstream producer observes disconnection and
exits, allowing `join()` to complete without hanging.
A true EAGAIN/pids-limit spawn failure cannot be reliably forced in a
unit test without root or ulimit co-operation, so the test exercises the
drop-order contract directly: a mock shell that panics if `join()` is
called while either channel end is still open.
The test constructs a `(prefetch_tx, prefetch_rx)` pair where the tx side
is held by a sentinel that stays alive as long as either channel end is
open, then asserts that the sentinel is gone by the time
`producer_shell`'s join logic would run. Because a real spawn failure
can't be forced, the test instead verifies the helper logic in isolation:
drop `prefetch_rx` and `recycle_tx` first, then observe the producer-side
sender is disconnected, which is the property the fix relies on.
-65
View File
@@ -1,65 +0,0 @@
# diag.rs — scan diagnostics rationale
## Why this module exists
A bug report log must be self-diagnosing: everything needed to explain *why*
freemkv made the choices it did at scan must be in the log, in a compact,
machine-parseable form. `diag.rs` emits one terse line per row (title, cell,
stream, decision) under the `tracing` target `freemkv::diag`, which the CLI
routes to `log.txt` when `--log-level 3` (debug) is set.
## Format conventions (stable, greppable)
- Every line is prefixed by a `tag=` so a log scraper can filter (`disc`,
`title`, `dvd.cell`, `dvd.vattr`, `dvd.aattr`, `bd.clip`, `bd.mark`, `aacs`,
`stream`, `decision`).
- Raw bytes are shown as `0xNN` next to their decode so a wrong decode is
obvious against the raw value.
- This module only READS already-parsed scan state — it never re-reads the
disc and never mutates anything.
## DVD per-cell table vs. Disc-level dump
The DVD per-cell table (with the raw cell-category byte) is emitted from the
IFO scan itself (`dump_dvd_cells`), because the per-cell `ifo::DvdCell` detail
is lowered away before the `Disc` is built. The `Disc`-level dump
(`dump_disc`) covers everything that survives lowering: titles, streams, the
picked main feature, and AACS state.
## `dump_dvd_substream_probe`
Emits the ACTUAL per-physical-sub-stream AC-3 channel counts read off the VOB
during the mux-time sub-stream probe (the Silence-of-the-Lambs wrong-stream
fix). This is the ground truth the IFO nibble is compared against: each row is
`sub_id=0x8x channels=N` for a physical `private_stream_1` AC-3 sub-stream
whose first frame was decoded. An empty probe (scrambled / unreadable / short
VOB) logs a single `probed=0` line so the absence is explicit in a bug log.
Self-sufficiency: with `tag=dvd.aattr` (the IFO's declared sub_id + claimed
channels) and these `tag=dvd.substream` rows (the physical reality), a bug log
alone shows whether the ordinal `0x80` actually carries the declared channel
layout — no disc needed to diagnose a wrong-substream rip.
## `dump_mkv_track`
Emits the MKV `TrackEntry` elements the muxer is about to WRITE for one track
— the Windows-fps-class metadata (FlagInterlaced, FieldOrder,
DefaultDuration, DefaultDecodedFieldDuration, Display dims) plus the
codecPrivate as hex. With this row a bug log alone is enough to verify why
Windows Explorer reports a given frame rate for an interlaced SD track: the
container values that drive its fps derivation are all present, no disc and
no MediaInfo needed.
`track_number` is the 1-based MKV track number; `track` is the built
`crate::mux::mkv::MkvTrack` whose fields map one-to-one onto the emitted
elements (see `MkvMuxer::new`).
## `main_feature_reason`
The `reason=` token on the main-feature decision row is DERIVED from
`Disc::CANONICAL_TITLE_ORDER_KEYS`, which lives beside the comparator that
actually implements them — never restated here. The previous hand-written
copy drifted (it advertised a `fewest-clips` key the comparator had replaced
with largest-physical-size), which made the self-diagnosing log explain the
pick with a rule the code does not apply. A diagnostic that disagrees with the
decision it documents is worse than no diagnostic.
-132
View File
@@ -1,132 +0,0 @@
# Rationale for `src/dirimage/tests.rs`
Longer-form notes for individual tests, referenced from `// See docs/dirimage-tests.md`
pointers in the source so the inline comments stay within the comment-guard caps.
## `Scratch::unique_path`
The path a scratch dir takes is kept separate from directory creation so its
uniqueness is testable without a syscall between draws. A monotonic counter,
NOT a timestamp, is what guarantees it: `SystemTime::now()` resolves to only
a MICROSECOND on macOS, so two of the seven parallel tests that share the
"bdmv" tag routinely read the same value within one microsecond, collide on
the same directory, and the first to finish `remove_dir_all`s it out from
under the others' reads — an intermittent `read_sectors` ENOENT. The counter
is unique regardless of clock granularity; pairing it with the pid keeps it
unique across test-binary processes.
## `scratch_paths_are_unique_even_at_clock_resolution`
Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
tag and run in parallel; if two land the same path, the first to drop
`remove_dir_all`s it out from under the other's reads, which surfaced as an
intermittent `read_sectors` failure. Regression: with the old
`SystemTime::now()` name this is RED, because macOS's clock advances only
per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
between them to nudge the clock, mirroring the real cross-thread collision —
hands back the same value many times over. The counter-based name is unique
regardless of clock granularity.
## `a_directory_spanning_many_blocks_reads_back_whole`
A directory with enough children that its FID list spans several 2048-byte
blocks, read back through the production parser.
This is the case the block-alignment question turns on: FIDs are packed
contiguously and a descriptor may straddle a block boundary, because
`read_directory` (`udf.rs:1312`) walks the directory extent as one flat byte
run and BREAKS at the first non-257 tag. Padding each block would truncate
the directory at the first pad. 200 entries also exceeds the 16-handle LRU
several times over, so it exercises handle eviction on the read path.
## `a_file_below_video_ts_is_placed_not_just_declared`
A file inside a SUBDIRECTORY of `VIDEO_TS` must still have its data placed.
Audit finding. The DVD branch places only the files directly inside
`VIDEO_TS`, because only those carry the IFO-relative constraints — and the
follow-up loop skipped that directory entirely, so anything one level deeper
got a File Entry declaring the file's real size with no extents behind it.
It appeared in the tree at full length and read back as nothing, at exit 0.
The identical folder under `BDMV/` was always placed correctly, which is
what made the gap easy to miss.
## `a_vob_offset_past_the_image_cap_is_refused`
A VOBS offset far past the content must be REFUSED, not honoured.
Audit finding. The planner honours a title set's declared VOBS offset
because honouring it is what makes a real backup readable, and nothing
bounded it: a regenerated `.BUP`, a rewritten IFO or a hand-assembled folder
naming a huge offset grew the image to wherever it pointed. A `u32` sector
count reaches roughly 8.8 TB, and writing that to an `iso://` destination
would fill a disk with zeros before anything noticed.
The companion (`an_oversized_vob_offset_leaves_a_gap_rather_than_failing`)
pins that a MODEST oversize is still honoured as a gap, so this cap refuses
only what it must.
## `playable_bdmv`
A BD folder that really enumerates a title, so the AACS content probe has
an extent to sample.
The `.m2ts` is built as real 192-byte BD source packets, because that is
what the probe judges: byte 0 carries the AACS Copy Permission Indicator in
its top two bits, and byte 4 is the MPEG-TS sync. `scrambled` sets the CPI
and withholds the sync — "flagged and not structurally clean", which is
exactly `aacs_unit_needs_decrypt`. An all-zero payload would prove nothing
either way: `is_clean_ts` skips zero payloads as padding.
## `both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory`
The two doors must AGREE. Same folder, same verdict, same extents.
Stated as a differential rather than a bare `is_ok()` so it cannot go
vacuous: if the fixture ever stops producing an AACS state, a one-sided
`Ok` assertion would still pass while guarding nothing, whereas "both doors
see the same title" is the invariant the shared function actually exists to
hold.
## `a_scrambled_folder_is_refused_through_the_dir_url_door_too`
The other verdict, through the same door: a folder whose content units are
really scrambled is refused with the TYPED code, not muxed into garbage.
This is the load-bearing half. `E_DIR_IMAGE_ENCRYPTED` is produced at
exactly one site in the crate (`session::apply_folder_encryption_verdict`),
reachable from here only through the `is_folder` argument — so this test
cannot pass if that argument is dropped, whatever else changes.
## `write_and_mount_externally`
Write a synthesized image to a real file and ask the OS to mount it.
This is the only check here that is not circular: `read_filesystem` shares
every assumption with the encoder, an operating system's UDF driver shares
none of them. Ignored by default because it shells out to `hdiutil` and
needs a host that can attach an image; run with
`cargo test -- --ignored write_and_mount_externally --nocapture`.
## `dvd_placement_invariant_on_a_real_folder`
Diagnostic (opt-in): verify the DVD placement invariant for every title set
in a REAL folder. `ifo.rs` derives a title's extents as
`file_start_lba(VTS_nn_0.IFO) + vtstt_vobs`, so that sum must land exactly on
`VTS_nn_1.VOB` or the mux reads the wrong sectors for that title.
Run with: `FMKV_DVD_FOLDER=/path/to/tree cargo test --lib
dvd_placement_invariant_on_a_real_folder -- --ignored --nocapture`
## `a_multi_clip_playlist_produces_feed_spans_that_tile_the_feed`
`Clip::feed_span` is the INPUT to the whole provenance feature — it is what
tells the muxer which clip a byte offset belongs to — and it is produced in
exactly one place, `disc/bluray.rs`. Every `SeamPlan` test synthesizes spans
by hand, so the consumer is thoroughly tested against fixtures written from
the same assumptions as the producer, and nothing checks the producer at all.
That is the shape of the original defect: the placement logic was tested and
correct, and the thing feeding it was wrong. `SeamPlan` only trusts spans
that TILE the feed contiguously from zero, so this asserts exactly that,
against the real scanner reading a real synthesized filesystem.
-185
View File
@@ -1,185 +0,0 @@
# `dirimage` — `dir://` as an image-level source
`dirimage` synthesizes a real, minimal, valid UDF 1.02 volume over a host
folder (a DVD `VIDEO_TS/` or Blu-ray `BDMV/` extracted by a MakeMKV-style
backup). Nothing is emulated:
* **Metadata sectors** (anchors, the volume descriptor sequences, the File
Set Descriptor, every File Entry, every directory's FID list) are encoded
into RAM by `encode` — a few MiB even for a large Blu-ray.
* **Data sectors** are not materialized at all. Each one maps to a byte
range of a real file, read on demand.
So `udf::read_filesystem` parses this image by exactly the same code path it
parses a real disc with, and every consumer above it is unchanged. The cost
is that a single-partition synthetic volume never exercises the UDF 2.50
Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
this module's tests do not cover that block and must not be read as if they
did.
## What this module deliberately does NOT do
* **3D / SSIF** — rejected up front (`Error::DirImageSsifUnsupported`). An
SSIF aliases the same sectors as its base and dependent `.m2ts`; the
planner allocates disjoint extents, so a 3D folder would produce silently
wrong output.
* **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
* **Encrypted folders** — a folder whose content is still AACS-scrambled is
rejected by the caller-side probe, not decrypted here.
## `layout.rs` — rationale detail
Full detail for the comments trimmed in `src/dirimage/layout.rs`, in file order.
**Module overview.** Two phases, kept apart because they fail for different
reasons: walk rejects what the image model cannot represent (3D SSIF, an
unrecognized tree, a case collision inside one directory); assign places
metadata first (File Set Descriptor, one File Entry per node, then the
directory FID lists), then file data. Data placement is the part with real
constraints. For a Blu-ray there are none — `.mpls`/`.clpi` address clips by
name, never by LBA — so files are packed sequentially. For a DVD the IFOs
record VOB positions as sector offsets from the IFO's own start, and `ifo.rs`
re-derives every title extent from them, so the placement must reproduce
those offsets exactly or the rip reads the wrong sectors (`place_video_ts`).
**`DATA_FLOOR`.** Well clear of the volume-space descriptors, and — the
load-bearing part — far from zero: `disc/bluray.rs:137` drops any clip extent
whose LBA is 0, so a file that landed at block 0 would vanish from the title
with no error.
**`MAX_AD_BYTES`.** The AD length field is 30 bits (the top two are the
ECMA-167 4/14.14.1.1 extent type), so 0x3FFF_FFFF is the arithmetic ceiling —
but a non-final extent must be a whole number of blocks, so the usable
ceiling is the largest multiple of 2048 below it. Files larger than this are
split across several ADs; `udf.rs` reads them back as a multi-extent file,
the same shape a dual-layer disc produces.
**`MAX_DEPTH`.** Matches `udf.rs`'s `MAX_DIR_DEPTH`: anything deeper would be
recorded but never descended into, so the files under it would be invisible
to every consumer. Refuse instead of silently dropping.
**`MAX_CS0_NAME_BYTES`.** The FID's name-length field is one byte, so 255 is
the ceiling; 254 leaves the encoder no way to produce a value that wraps to
zero.
**`MAX_SUBDIRS`.** A directory File Entry's link count is `u16` and counts
one per child directory plus one for its own entry in the parent, so the
last usable value is `u16::MAX - 1`. Lowered under `cfg(test)` ONLY so the
guard can actually be executed: building a folder with 65534 subdirectories
to reach the real cap is not a test anyone can run, so an earlier test
asserted arithmetic about the constant instead and would have passed with
the guard deleted. The production value is unchanged.
**Module-scope link-count assert.** Must live at module scope, not inside
`mod tests`: it previously sat inside `#[cfg(test)] mod tests` while also
carrying its own `#[cfg(not(test))]`, so it was compiled in NO configuration
and could never fire — exactly the dead gate the test above it was written
to replace.
**`MAX_IMAGE_SECTORS`.** A DVD title set records where its VOBS begins as an
offset in its own IFO, read verbatim out of a file in the folder. A
regenerated `.BUP`, a tool that rewrote an IFO, or a hand-assembled folder
can therefore name an offset far beyond the content — and the planner
honours it, because honouring it is what makes a real backup readable.
Without a ceiling the image grows to wherever that offset points: a `u32`
sector count reaches ~8.8 TB, and writing one to an `iso://` destination
would fill a disk with zeros before anything noticed. 128 GiB clears the
largest real medium (BD-100) with room to spare, so a genuine disc folder
never meets it.
**`MAX_META_BYTES`.** Every node costs a 2 KiB File Entry sector held for the
life of the image, so the entry cap alone permits ~205 MB of metadata for
content of no size at all — and the mux holds two of these at once while
probing. The module documents a budget of "a few MiB even for a large
Blu-ray"; this is what enforces it rather than merely asserting it.
**`is_excluded`.** Dot-files are host artefacts, never disc content: macOS
sprays `.DS_Store` and `._*` resource forks through any folder a Finder
window has touched, and including them would put files on the "disc" that
were never on the disc. `.partial` is freemkv's own in-flight extraction
suffix (see `disc/extract.rs`) — picking one up would mean planning an
extent over a file that is still being written.
**`read_head`.** Reads the first `n` bytes of an IFO so its placement
offsets can be resolved. Errors propagate: returning an empty buffer instead
would send every offset through `unwrap_or(0)`, recording NO placement
constraint, and a VOB placed without its constraint yields an image that
reads at the wrong offset with nothing reported. An IFO that cannot be read
is a folder that cannot be planned.
**`place_video_ts`.** DVD-Video records VOB positions INSIDE the IFOs, as
sector offsets from the IFO file's own first sector: `VIDEO_TS.IFO` + 0xC0
(`VMGM_VOBS`) → `VIDEO_TS.VOB`; `VTS_nn_0.IFO` + 0xC0 (`vtsm_vobs`) →
`VTS_nn_0.VOB` (the VTS menu); `VTS_nn_0.IFO` + 0xC4 (`vtstt_vobs`) →
`VTS_nn_1.VOB` (the title stream). `ifo.rs:554-556` re-derives every title
extent as `file_start_lba(IFO) + vtstt_vobs + cell.first_sector`, so only the
third of those is load-bearing for freemkv itself — but the other two are
load-bearing for DVD PLAYERS, which read the menus freemkv ignores. All
three are honoured. `VTS_nn_1.VOB … VTS_nn_9.VOB` are one logical stream
split at the 1 GB file-size limit, and the cell sector addresses run
continuously across the split, so they are placed back-to-back with no gap;
laying the group out in its canonical on-disc order gives exactly that for
free, so the constraint becomes a CHECK, not a search. The check can fail —
a regenerated `.BUP`, a tool that rewrote an IFO, a folder assembled by hand
— and when it does the required position lies below the end of the file
that must precede it. There is no placement that satisfies it, so it is a
typed error naming the file rather than a silent misplacement.
**`reject_ssif`.** `disc/bluray.rs:124-130` probes
`/BDMV/STREAM/SSIF/{clip}.ssif` and sets `is_3d` the moment one resolves —
unconditionally, before any capability check. On a real 3D disc the `.ssif`
and the base/dependent `.m2ts` files ALIAS the same sectors (the SSIF
interleave IS the two m2ts streams), which this planner cannot express: it
would allocate three disjoint copies, so the clip's extents and the m2ts's
extents would disagree and the rip would read the wrong bytes at exit 0.
Reject the folder instead.
**`plan` — capacity, and what it changes.** `total_sectors` becomes the
`Disc`'s `capacity_sectors` / `capacity_bytes`, and `capacity_bytes` is not
inert: `canonical_title_order`'s oversize gate (`disc/mod.rs:2039-2040`, body
`:2209-2210`) demotes any title whose `size_bytes` exceeds it, which decides
which title sorts first and therefore what `-t 1` selects. The capacity
reported here is **the synthesized image's own size** — what an ISO built
from this folder would report — and nothing else. It is NOT padded up to a
media tier (BD25/BD50/…): that was considered and does not work, since a
folder does not record the tier of the disc it came from, so a 22 GB folder
off a BD50 would pad to BD25 and diverge anyway, and the padding would
inflate any pre-sized output written from the image. The consequence, stated
plainly: **for a folder that is missing files the source disc had — a
selective MakeMKV backup being the normal case — the capacity is smaller
than the source disc's, the oversize gate is therefore stricter, and a
borderline "play-all" composite title that the ISO kept can be demoted here.
`-t 1` can select a different title from `dir://FOLDER` than from an
`iso://` of the same disc.** For a complete folder the two agree, because
the capacities agree. This is chosen over the alternatives because it is the
only one that is self-consistent: `dir://X` and an `iso://` built from `X`
describe the same image and must answer the same way.
**Tests.**
* `a_file_past_the_ad_ceiling_splits_on_a_block_boundary` — a file above the
30-bit AD ceiling must split into MULTIPLE descriptors, and every
non-final one must be a whole number of blocks, or the next extent's bytes
start mid-sector and the file reassembles wrong. Kills a mutant that emits
one oversized AD (whose length would collide with the extent-TYPE bits
`udf.rs:642` reads).
* `two_names_the_reader_cannot_tell_apart_are_refused` — audit finding: the
uniqueness check compared raw host names, while `parse_udf_name` trims
whitespace and drops code units `char::from_u32` rejects. So
" 00000.m2ts" and "00000.m2ts" were two distinct entries at plan time and
ONE name at read time; `find`/`read_file` take the first match, so a title
resolved to the wrong file's extents and muxed the wrong bytes at exit 0.
Calls `plan` on a real folder and fails if the round-trip key is reverted
to comparing host names.
* `an_over_long_name_is_refused_by_the_planner` — audit finding: the length
was narrowed with `as u8`, so a 255-byte ASCII name (POSIX NAME_MAX, legal
on ext4/APFS/NTFS) encoded to 256 bytes with the CS0 compression byte and
wrote a length of ZERO, making every later entry in that directory read
from the wrong offset. An earlier version of this test asserted arithmetic
about the constants and never called `plan`, so it would have passed with
the guard deleted.
* `the_subdir_cap_refuses_a_folder_with_too_many_subdirectories` — pins the
arithmetic relationship the guard relies on and executes the guard rather
than restating the constant: link count (child dirs + 1) is 16 bits, so
exceeding it wraps and the image lies about its own directory structure.
Creating 65,535 directories in a test is not reasonable, so this does NOT
exercise `walk`.
-492
View File
@@ -1,492 +0,0 @@
# src/disc/mod.rs — internal notes
Overflow rationale for internal (non-`pub`) items in `src/disc/mod.rs`,
relocated here by the comment-guard so the in-file comment can stay short.
Each section is pointed to by a one-line `// See docs/disc-mod.md — <topic>`
comment at the corresponding call site.
## aacs_dir_present
THE ONE definition of "is this disc structurally AACS-encrypted". Both
`Disc::identify` (fast path, filesystem only) and `Disc::scan_with` (full
scan) need this, and both used to spell out the same two `find_dir` calls by
hand. They agreed today; nothing made them agree tomorrow — adding a third
AACS location, or excluding an empty placeholder directory, to one copy and
not the other would silently desync the fast identify from the full scan (the
same disc reported encrypted by one and clear by the other).
This is STRUCTURAL, not cryptographic: it says the tree looks like an
encrypted disc, not that any sector actually is. A folder copied verbatim
from a decrypted disc keeps its `AACS/` and answers true here (see
`session::scan_dir`, which corrects for exactly that).
## correct_truehd_channels
Corrects a title's TrueHD audio-stream metadata by probing the first
decrypted access units — channel count, real sample rate, and Atmos detection
in a single major-sync read. The MPLS descriptors declare the BASE layout
(often 5.1, or a container-guessed rate) even for a 7.1/Atmos TrueHD track;
the truth is in the MLP major sync. `reader` must yield DECRYPTED sectors
(the m2ts is AACS-encrypted, so this can only run at mux time, not scan).
Reads a bounded window of the title's first extent.
Corrections, each individually guarded so a malformed field never writes a
wrong header:
- **Channels**: from the presentation channel masks (as before).
- **Sample rate**: from the whitelisted rate nibble; left untouched on an
unknown rate or no major sync.
- **Atmos**: when a 4th substream is detected AND the stream still carries
the basic descriptor label, the label is promoted to the Atmos form;
richer editorial labels (e.g. an existing "Dolby Atmos") are left intact.
## Resolution::pixels
`Unknown` deliberately does NOT fabricate a plausible 1920x1080. This is the
same trap already closed on `AudioChannels::count` (which used to return 6)
and `SampleRate::hz` (48000.0): a plausible wrong answer is indistinguishable
from a real one at every call site, so a caller can forget to check the
variant first — the `json://` sink had already walked into exactly that,
reporting confident dimensions for a stream whose neighbouring `resolution`
field said "unknown".
It also does not return `(0, 0)`: that shape looks like a usable pair, so the
MP4 sink stored it and serialised a 0x0 video track — a structurally complete
file no player can render, written with no error. Before that it returned a
fabricated 1920x1080, which was wrong but at least playable, so the sink had
never needed a guard and the absence of one was invisible. A doc comment
listing which callers are safe cannot hold this: one did not check, and a
third kept its own duplicate `Unknown` test long after the accessor took the
job over. `Option` forces every caller to make its OWN decision: Matroska
omits the optional PixelWidth/PixelHeight elements, the metadata sinks report
the dimensions as absent, and MP4 must refuse per ISO/IEC 14496-12 (width and
height are mandatory in both `tkhd` 8.3.2 and VisualSampleEntry 12.1.3).
## has_probable_video
The exposed `streams` come from the title's FIRST PlayItem only, so a
feature whose first PlayItem is a non-video bumper reports
`has_video() == false` and would be wrongly disqualified (audit R6). This
predicate is deliberately MORE permissive than `has_video` — it only ever
ADMITS a title the stricter gate would reject — so it cannot reopen the
decoy hole: a size-inflated streamless decoy is demoted STRUCTURALLY by
composite detection in `Disc::rank_titles`, not by this gate, while it still
rejects tiny streamless menu/metadata playlists. The size fallback requires
at least one clip, so a clip-less pure-metadata decoy (which cannot be a
real bumper-lead feature — that has the feature clip) is still rejected.
## image_crack_extents
The extents an image-time CSS crack scans, in the crate's CANONICAL order:
the main feature's own extents, in natural playback order. Both halves are
deferred to logic the crate already owns rather than re-derived here:
- WHICH title — `scan_with` has already sorted `titles` with
`Self::canonical_title_order`, so the canonical main feature is the first
title that actually has extents. No local pick.
- WHAT ORDER — playback order, i.e. the extent vector untouched, exactly as
`Self::decrypt_keys_for_title` hands `&title.extents` to
`css::crack_key_outcome`.
**Why this function exists (do not let the copy grow back):** `Disc::scan_image`
used to re-implement both halves inline — it picked the title with the
largest total sector count, then re-sorted that title's extents
LARGEST-CELL-FIRST before handing them to `crack_key_outcome`. Both had
drifted from the canonical rules:
- Largest-cell-first is the 1.5.1 garbage bug. A CSS DVD's biggest cell opens
with a long CLEAR run, and the crack's 50_000-sector budget is shared
across the whole extent list, so starting there can exhaust the budget
without ever reaching a scrambled sector — the crack reports `Unencrypted`
and the mux emits scrambled MPEG as plaintext. Playback order reaches the
scrambled feature body after only the small clear front matter that
precedes it on a real disc (CSS: the title key is recovered from the
scrambled data itself, so the scan must actually MEET scrambled data).
- The sector-count pick ignores `canonical_title_order`'s capacity gate, so
it selects the oversize "play-all" composite (whose declared cells
double-count data shared with other playlists) instead of the real feature
— i.e. a DIFFERENT title, and on a multi-VTS disc a different VTS, whose
CSS title key does not descramble the feature at all.
The result: the same disc cracked as an ISO could disagree with the same
disc cracked from the drive, which is precisely what the duplicate was free
to do. Keep the derivation here, shared, so it cannot recur.
## read_aacs_inputs_from_reader
Reads a disc's AACS key-input files from a sector source: returns
`(Unit_Key_RO.inf, MKB)` raw bytes. Shared body for `Disc::read_aacs_inputs`
(ISO) and `Disc::read_aacs_inputs_from_drive` (live drive). Prefers MKB_RO,
falls back to MKB_RW, then TRIMS to the real record length. Both files are
allocated to a fixed ~128 MiB and zero-padded, so reading either ships up to
~124 MiB of nothing — trim to the record stream so callers send/store a few
MB, not 128 MiB.
## read_aacs_version
AACS major version (`crate::aacs::mkb::AACS_MAJOR_BD` /
`crate::aacs::mkb::AACS_MAJOR_UHD`) from the content certificate. Drives the
`Unit_Key_RO.inf` parse stride (48-byte V10 vs 64-byte V20/V21), so the
out-of-band key-fetch path parses `enc_title_keys` at the right stride (a
server VUK then derives the correct unit keys).
When no content certificate is readable/parseable, defaults to **UHD (V20,
64-byte stride)** — the conservative choice the pre-1.2.0 fetch path
hardcoded — and logs it: a wrong stride here folds a server VUK against
mis-strided title keys (silent wrong unit keys), so a missing cert must not
quietly pick the V10 stride for a UHD disc.
## read_mkb_content
Reads the AACS MKB's real record stream — NOT its zero padding. `MKB_RO.inf`
/ `MKB_RW.inf` are allocated to a fixed ~128 MiB and zero-padded; the actual
record stream is a few MiB. Reads a bounded prefix, finds the record-stream
length via `crate::aacs::mkb::mkb_content_len` and returns exactly that,
growing the prefix if the records run past it. This avoids reading 100+ MiB
of padding on every scan AND avoids the `read_file` `MAX_FILE_BYTES` cap that
(since 0.31.0) rejected the padded 128 MiB MKB outright — which made
`read_aacs_inputs` fail and autorip report "could not read this disc's key
files" without ever contacting the keyserver.
## CANONICAL_TITLE_ORDER_KEYS
**Why not just sort by duration descending?** Branching UHDs (and some BD
authoring) ship a "play-all" virtual playlist that references the same
source clips multiple times for seamless alternate-angle / alternate-ending
playback. Those playlists report an inflated `duration_secs` (often 4+
hours) and an inflated `size_bytes` greater than the disc's physical
capacity. Example seen in the wild — *The Amateur (2025)* UHD, 58.5 GB
BD-100 disc:
| Title | Playlist | Duration | Size | Clips |
|-------|------------|----------|---------|-------|
| 1 | 00020.mpls | 4h 13m | 92.4 GB | 253 |
| 2 | 00800.mpls | 2h 02m | 57.2 GB | 1 |
Title 1's 92.4 GB cannot fit on a 58.5 GB disc unless the same clip data is
referenced multiple times — proof it's a virtual composite. A duration-only
sort would put it at `titles[0]`, so `freemkv -t 1`, `disc.titles.first()`,
and autorip's main-feature picker all grab the 4-hour composite instead of
the 2-hour movie that actually matches TMDB.
**Effect on non-branching discs:** unchanged — the main movie is already the
longest 1-clip title. **Effect on branching UHDs:** the virtual play-all
playlist is pushed to the back, the actual movie surfaces at index 0.
Defined immediately beside the comparator, so a diagnostic can NAME the
ordering instead of restating it. The `freemkv::diag` main-feature decision
row used to carry its own hand-written copy of this list, and it drifted: it
still advertised a `fewest-clips` key long after the comparator replaced
clip-count with largest-physical-size, so the `--log-level 3` bug-report log
explained freemkv's top-level pick with a rule freemkv does not apply. Any
change to the keys must change this list in the same edit.
## probe_forced_subtitles_for_bdts_titles
Pulled out of `scan_with` as its own callable predicate: the decision is
otherwise reachable only by driving the full scan pipeline (tree dispatch ->
title parsing -> this loop), so a test could not pin the gate without also
authoring an entire synthetic disc image. Running it against an HD-DVD/DVD
(`MpegPs`) title's extents would demux the wrong container, since only
`BdTs` titles carry PES-wrapped PGS the shared classifier
(`pgs_forced_probe`) understands.
## MAIN_FEATURE_ORDER_KEYS
- `nav-feature`: the title the disc's own HDMV navigation plays as the
feature, resolved by running First-Play in the `crate::bdnav` VM the way a
real player would. This is the authoritative, size-independent pick; it is
video-gated (a mis-resolve cannot select a streamless playlist) and simply
absent when the VM abstains (BD-J discs, or non-convergence), in which
case the lower keys decide.
- `authoring-feature`: a title the disc's own menu authoring designates as
the feature (has video, is not itself a composite, and is within a sane
fraction of the longest title's duration) outranks everything else — a
size-independent signal an obfuscation decoy cannot forge.
- `standalone`: a title that is a play-all / wrapper COMPOSITE — its clip
set properly contains another substantial video title's whole clip set (a
real UHD title's decoy playlist `00245` = `[bumper][00001 feature
clip][outro]` case) — is demoted below the standalone title it wraps. This
is the STRUCTURAL anti-decoy signal; it does not depend on stream counts,
physical size, or disc capacity, so it survives where those are spoofed or
unknown.
- `has-video`: a title with no plausible video content (see
`DiscTitle::has_probable_video`) is demoted below every title that has
some — a floor that drops streamless menu/metadata playlists.
## rank_titles
This catches both the wrapper decoy (`00245 ⊋ 00001`) and the oversize
play-all — independent of capacity, which the `fits-disc` gate needs and
which is `0`/unknown on drives that fail READ CAPACITY.
### Wrapper-shape gate (issue #45)
A proper-subset relation with `Q.size ≥ 0.5·P.size` is NOT by itself proof
that `P` is a wrapper. A genuine **seamless-branch feature** `P` (body clip
plus intro/credits/alternate-branch segments) is a *superset* of its
single-clip body decoy `Q`: `Q` runs almost as long as `P` (e.g. 1h50 vs
2h03, ~0.89 of the duration) yet is markedly smaller, because `P`'s branch
segments add bytes without adding much played runtime. The old size-only
gate mis-flagged such a `P` as a "play-all wrapper" and demoted the real
2h03 feature below the shorter 1h50 body — a SAFETY bug (wrong title ripped).
So the composite flag additionally requires `P` to *have* a wrapper shape,
via one of two capacity-independent, per-title (`duration_secs`, `size_bytes`)
signals on the subset `Q`:
- **concat / play-all:** `Q.duration < 0.85·P.duration` — `Q` is a much
shorter cut and `P` pads its runtime with other segments (the true
play-all runs ~the SUM of its parts, so each part is well under half).
- **bumper / outro wrapper:** `Q.size ≥ 0.90·P.size` — `Q` already holds
almost all of `P`'s bytes and `P` merely prepends/appends a short
non-feature lead/tail (the `00245 ⊋ 00001` decoy: `Q` is ~0.96 of both the
duration and the size, so the size limb keeps it demoted).
Otherwise — `Q` runs almost as long as `P` (≥0.85) yet is well under 0.90 of
its size — `P` is the fuller branch set and is kept. `0.85` sits above a real
concat's per-part ratio (~0.5) and below a seamless body's (~0.89); `0.90`
sits above a bumper wrapper's near-1.0 size share and below a branch body's.
**Known residual:** a wrapper that pads runtime only modestly AND withholds
most of its bytes from the body clip is ambiguous by shape alone. The corpus
title-selection gate is the backstop across the disc hoard, and
higher-precedence nav/authoring signals override the composite key.
## aligned_unit_keys_validate
Conservative: a sample that is not AACS-scrambled proves nothing, and with
no scrambled sample at all there is nothing to disprove against, so it
returns `true` (accept). It returns `false` when ANY scrambled sample cannot
be restored to clear MPEG-TS by ANY unit key in the set. That covers two
distinct failure shapes: (1) a wholly wrong key (a keydb VK that does not
match this disc) — no sample decrypts; and (2) a *partially* applicable key
set on a multi-CPS-unit disc — the resolved keys cover CPS unit 0 but not
CPS unit 1. Accepting on the first sample that decrypts (the old behaviour)
would commit such a set, after which CPS-unit-1 sectors pass through as raw
encrypted bytes into the ISO/MKV with no error surfaced anywhere. Requiring
every scrambled sample to decrypt rejects the incomplete set so the caller
falls through to the next candidate. Reuses the ecosystem's single
`is_clean` content-clarity predicate and the full (bus + AACS) unit decrypt,
so it agrees with the actual mux decrypt.
## ensure_decryptable
The verdict, in order:
- `raw == true` → `Ok(())`. `--raw` intentionally skips decryption and needs
no key (the caller wants an encrypted image).
- `self.css_error.is_some()` → `Err(Error::CssNoDiscKey)`. The scan saw
scrambled CSS sectors but recovered no title key (`self.css` is `None` yet
the content IS encrypted). Treating `css.is_none()` as "unencrypted" would
mux scrambled MPEG as plaintext garbage. A DISC-LEVEL verdict
(`error::is_disc_level_no_key`) — the main feature's crack failed, so
every title fails the same way and the rip loop must stop rather than
skip each title in turn.
- AACS-encrypted (`self.aacs.is_some()`) with no usable key
(`decrypt_keys()` is `None`) → `Err(Error::NoDiscKey { .. })`, naming the
disc by hash.
- CSS-encrypted (`self.css.is_some()`) with no usable key →
`Err(Error::CssKeyMissing)`. (The disc-wide `decrypt_keys()` yields
`Css{..}` whenever `css.is_some()`, so this is defensive; the live
multi-VTS case is gated by `Self::ensure_decryptable_keys`.)
- otherwise → `Ok(())`. A genuinely unencrypted disc has `None` keys
legitimately, and a CSS disc whose keyless crack succeeded has a key.
## decrypt_keys_for_title
For a DVD the CSS title key MUST be recovered before descrambling: a
scrambled sector without a recoverable crib cannot self-crack, and CSS
leaves the pack/PES header clear, so a sector left un-descrambled would mux
as a structurally-valid but corrupt PES packet with no loss reported. Two
ways to get it:
- **Fast path** — the scan already cracked a key whose LBA span
(`crate::css::CssState::crack_span`) covers this title's VTS: reuse it. No
re-read, and on a live drive no second CSS bus-auth round-trip. CSS title
keys are per-VTS, so an overlapping span is the same key.
- **Crack** — when up-front detection missed (`self.css == None`) or the
title lives in a different VTS: crack the key from this title's OWN
extents in a SINGLE scan, in natural PLAYBACK ORDER (never
largest-cell-first, which starved the crack in a big cell's clear prefix —
the 1.5.1 bug). Playback order reaches the scrambled feature body after
only the small clear front matter (logo / rating card) that precedes it.
One scan = one CSS-locked early-bail, so a locked title is not
re-hammered per cell against a live drive (hard rule #2); its 50k-sector
budget is the same accepted bound the disc-wide scan uses.
Outcomes (`batch_sectors` sizes the crack's batched reads):
- `(Css{title_key}, false)` — descramble with the recovered/reused key.
- `(None, true)` — a genuinely-clear title needs no key; the gate passes it.
- `(None, false)` — scrambled but no key recoverable → hard failure via
`Self::ensure_title_decryptable`, never a silent garbage mux.
## inject_unit_keys
Injects pre-resolved AACS unit keys into a scanned disc — the deferred-mux /
resume path. The keys come from the mapfile's `# freemkv-uk:` header
(persisted at sweep time when the disc was keyed), so the mux decrypts
directly with NO key-service round-trip. Populates `self.aacs.unit_keys` so
`Self::decrypt_keys` returns them and marks the source `ExternalUk`.
If the scan built no AACS state (`self.aacs == None`) — which happens when
the keydb was absent at scan time (`scan_aacs_no_keydb` ->
`aacs_error = KeydbLoad`) — this synthesizes a minimal `ExternalUk` state
for an encrypted AACS disc. A Unit Key is the FINAL per-title decryption
key; the keydb is only needed to *derive* it, and that derivation already
happened at sweep (the UK is in the mapfile). So a UK alone is sufficient to
decrypt the on-disk ISO — AACS 2.0 bus decryption was applied by the drive
at read time, so `read_data_key` is unused for file-backed mux. Without
this, a keyed disc swept without a keydb would recover its UK yet still
report E8005 (no usable `decrypt_keys`) at remux. No-op for an unencrypted
or CSS (DVD) disc.
## scan_with_passes_the_halt_flag_to_the_bluray_enumerator
Every BD and DVD cancellation test in this crate calls
`scan_bluray_titles` / `scan_dvd_titles` DIRECTLY, so the three-line wiring
in `scan_with` that connects the operator's Stop to them was covered by
nothing at all: replacing `opts.halt.as_ref()` with `None` on either branch
left the whole suite green while a Stop during a BD or DVD scan silently did
nothing. Only the HD-DVD branch was pinned.
The fixture is deliberately the smallest disc that takes the BD branch — a
bare `/BDMV` with no PLAYLIST — because the point under test is the
ARGUMENT, not the enumerator's own (separately tested) halt polling. Nothing
between `scan_with`'s entry and the enumerator reads the flag (the disc is
unencrypted, so the AACS path is skipped), so a green here can only mean the
flag arrived. Mutation: `Self::scan_bluray_titles(reader, &udf_fs, None)?`
fails here.
## halted_reads_do_not_report_the_hddvd_scan_as_successful
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The HD-DVD enumerator must not swallow that into a
successful scan. Measured before this was fixed: the scan returned `Ok`
with both titles present and ZERO streams on each — a cancelled scan
wearing the shape of a disc whose clips carry no video or audio. Downstream
that is a title list to cache, display and rip from.
## scan_with_capacity_bytes tests
`scan_with`'s `capacity_bytes = capacity as u64 * 2048` feeds
`canonical_title_order`'s "bigger than the whole disc = play-all composite"
threshold.
`..._not_addition`: chosen so `capacity * 2048` clears both titles (neither
is oversize, so the bigger one — OTHER — ranks first), but
`capacity + 2048` lands BETWEEN the two sizes: MAIN stays non-oversize while
OTHER flips to oversize and gets demoted, changing `titles[0]`. A `*` -> `+`
mutation is caught by this ranking flip, not merely a wrong numeric
threshold value.
`..._not_division`: same mechanism, tuned to catch a `*` -> `/` mutation
instead: `capacity * 2048` clears both titles, but `capacity / 2048`
collapses to a threshold far below MAIN's declared size while still leaving
MAIN under it, so only OTHER flips to oversize.
## read_mkb_content_grows_prefix_past_16mib_when_records_run_longer
This fixture's record stream is deliberately built so the first 16 records
land EXACTLY on the 16 MiB boundary (so the 16 MiB prefix alone parses as a
clean, complete record stream — the case a naive "n < buf.len() means done"
check would wrongly accept) and 4 more MiB of records follow, terminated by
the explicit `00 000000` end marker. Only a caller that actually grows past
16 MiB recovers the true 20 MiB content length.
## main_feature_order_demotes_wrapper_composite_decoy
The decoy `00245.mpls` is a WRAPPER COMPOSITE — `[00339 2s bumper][00001 the
whole feature clip][00336 outro]` — so its first-PlayItem STN reports v=0,
and it is the LONGEST and LARGEST title (8350s / 78.4 GB vs the feature's
8038s / 76.5 GB). Muxing its zero frames produced E6008 forever. It must be
demoted STRUCTURALLY (its clip set properly contains the feature's), not by
the fragile first-PlayItem stream count — this is what makes the fix robust
to the video-bearing-composite variant (see
`main_feature_order_demotes_video_composite`).
## key_source_failure_is_not_reported_as_a_missing_disc_key
Drives the real chain end to end — `KeySource` -> `resolve_and_apply_traced`
-> `Disc::aacs_error` -> `ensure_decryptable` — twice over the SAME disc,
and asserts the two operator-visible verdicts differ:
- source returns `Err(KeyServiceUnavailable)` (the HTTP-502 outage) → E7028
"the key service could not answer; retry later",
- source returns `Ok(vec![])` (the service answered, no entry) → E7022 "no
key source has a decryption key for this disc".
The credential and rate-limit verdicts are asserted alongside, since each is
a different operator action (fix the token / back off).
## css_disc_wide_no_key_is_disc_level_while_per_title_stays_skippable
The two CSS no-key conditions are NOT the same verdict and must classify
oppositely through the public predicates:
- **disc-wide** — `css_error` is set: the MAIN feature's crack failed, so
every title of this disc fails identically. Must be
`crate::error::is_disc_level_no_key` (the rip loop fail-fasts) and must
NOT be `crate::error::is_skippable_title_stub`. While both conditions
shared `E_CSS_KEY_MISSING`, an uncrackable CSS disc iterated all N titles
logging "title skipped" and exited 0 — a total failure reported as
success.
- **per-title** — one title's own re-crack failed on a multi-VTS disc
(`title_is_clear == false`, no key): skipping it and finishing the rest is
correct policy, so it must STAY skippable and must NOT be disc-level.
Pinned in both directions so a future change cannot silently flip either.
## correct_truehd_channels_full_correction_and_atmos_promotion
Kills the `was_basic ==`, the channels/rate `!=`/`&&` guards' "already
correct" branch, the `!matches!` per-stream skip, the `is_atmos ==
Some(true)` branch, and the whole-function no-op mutant.
## byte_offset_in_title_accumulates_across_extents
Kills: `lba >= start && lba < end` flipped to `||` (the first extent's
disjunction would trivially match almost any lba and return the wrong,
too-early offset); `cumulative +=` flipped to `*=` (cumulative is seeded at
0, so `*=` freezes it at 0 forever); and `sector_count * SECTOR_BYTES_U64`
flipped to `+` or `/` (wrong per-extent byte length folded into cumulative).
## chapter_at_offset_concrete_arithmetic
Kills: the `/` (time-fraction) flipped to `%` or `*`; the `*` (duration
scale) flipped to `+`; the `<=` chapter-scan comparison flipped to `>`; the
final `chapter_idx + 1` flipped to `-` or `*`; and every whole-function
fixed-tuple replacement (none produce `(2, 60.0)`).
## locate_ranges_positive_bps_duration_and_at_risk
Kills `bps > 0.0` flipped to `==`/`<` (both would take the `else 0.0`
branch here, wrongly reporting 0); `(*size as f64) / bps` flipped to `%` or
`*`; and the trailing `* MILLIS_PER_SEC` flipped to `+` or `/`.
## decrypt_keys_for_title_css_span_reuse_is_half_open
A CSS title key is per-VTS: reusing the scan's cracked key for a title that
lives OUTSIDE the cracked span descrambles that title with the wrong key,
and the mux emits garbage at exit 0. `crack_span` is documented as the
half-open LBA span `[start, end)`, so overlap is `extent.start < span.end &&
span.start < extent.end` — both comparisons STRICT. A title that merely
ABUTS the span (ends exactly where it begins, or begins exactly where it
ends) shares no sector with it and must NOT reuse the key.
The reader serves only clear (all-zero) sectors, so a title that falls
through to its own crack is reported unencrypted — distinguishable from the
reused-key answer both in the key and in the is-clear flag.
## merge_content_key_ranges
Merges per-title AACS key ranges into the sorted, disjoint set the
whole-disc map needs (`crate::decrypt::AacsKeyMap::entry_for` requires
disjoint ranges). Titles that share a clip resolve the SAME physical span
(same LBAs -> same CPS unit -> same key). When a later range overlaps a kept
one that carries the SAME key index and phase, the two are UNIONED (end
extended to the max) — this covers both the exact-duplicate (shared clip)
case and any partial overlap without ever dropping coverage, so no encrypted
LBA is left in no range (which would pass through as ciphertext). A real disc
never produces two DIFFERENT keys for one LBA; if that malformed case ever
appeared, the later range is dropped to keep the set disjoint rather than
extend one key over another key's LBAs.
-9
View File
@@ -1,9 +0,0 @@
# `disc::profile`
Everything in `DiscProfile` is DERIVED from the `Disc` model at construction:
the `Stream` enum is split into three typed vectors (audio/video/subtitle),
the `qualifier` / `purpose` enums are decomposed into booleans, and the
"default track" selection is precomputed so downstream never recomputes it.
Every field is always populated — a sensible default (`"und"` language, empty
`name`, `false` flag) stands in where the model has nothing, so consumers
never handle a bare `Option`.
-17
View File
@@ -1,17 +0,0 @@
# disc_tests: capacity/layer derivation
`scan_image_reports_capacity_in_bytes_and_the_layer_count` covers the scan
reporting the medium's size on both axes it exposes, derived from the one
sector count the caller hands in:
* `capacity_bytes` is that sector count times the 2048-byte logical sector
(ECMA-167 / BD-ROM logical block size). It is what sizes a full-disc image
read and what the progress percentage divides by, so a wrong scale is a
wrong ISO length, not a cosmetic number.
* `layers` distinguishes single- from dual-layer media. The threshold sits
between the two real capacities: a single-layer BD-25 is 12,219,392
sectors (25,025,314,816 bytes / 2048) and a dual-layer BD-50 is 24,438,784
sectors, so BD-25 must report 1 layer and BD-50 must report 2.
`scan_image` takes the sector count as a parameter, so this exercises the
real derivation without a 50 GB fixture.
-135
View File
@@ -1,135 +0,0 @@
# Disc to Rip: End-to-End Flow
How libfreemkv goes from a disc in the drive to decrypted content ready for backup.
This is the starting point for understanding the library.
## The Pipeline
```
Insert disc
│
▼
1. Open drive (drive/mod.rs)
│ INQUIRY → identify drive (DriveId)
│
▼
2. Init drive (drive/mod.rs → unlock seam)
│ Walk the registered-unlocker registry; first match unlocks the drive
│ (firmware/vendor handshakes are the unlocker's own business)
│ No match → drive untouched; host-cert AACS handshake carries the disc
│ Speed control → probe_disc()
│
▼
3. AACS handshake (aacs/handshake.rs) — optional
│ Allocate AGID
│ Exchange certificates + nonces (ECDH)
│ Derive bus key
│ Read Volume ID + read_data_key
│ (fails gracefully if drive doesn't support AACS for this disc)
│
▼
4. Read UDF filesystem (udf.rs)
│ Sector 256: AVDP → find Volume Descriptor Sequence
│ VDS: Partition Descriptor (physical start) + Logical Volume (metadata start)
│ Metadata partition → File Set Descriptor → Root directory
│ Walk directory tree: BDMV/, AACS/, CERTIFICATE/
│ → docs/udf.md
│
▼
5. Read AACS files from disc (aacs/mod.rs)
│ AACS/Unit_Key_RO.inf → SHA1 = disc hash
│ AACS/Content000.cer → AACS version (1.0 or 2.0), bus encryption flag
│ MKB via SCSI → for key derivation fallback
│
▼
6. Resolve encryption keys (decrypt.rs → resolve_encryption)
│ BD AACS:
│ Path 1: disc hash → KEYDB.cfg → VUK (fast, 99% of discs)
│ Path 2: KEYDB media key + Volume ID → VUK
│ Path 3: MKB + processing keys → media key → VUK
│ Path 4: MKB + device keys → subset-difference tree → VUK
│ VUK → decrypt unit keys from Unit_Key_RO.inf
│ DVD CSS:
│ Table-driven cipher — no KEYDB needed
│ → docs/aacs.md
│
▼
7. Parse playlists (mpls.rs) — BD/UHD only
│ BDMV/PLAYLIST/*.mpls → titles with play items
│ Each play item: clip ID, in/out timestamps
│ STN table: video, audio, subtitle streams with codec + language
│ → docs/mpls.md
│
▼
8. Parse clip info (clpi.rs) — BD/UHD only
│ BDMV/CLIPINF/*.clpi → EP map (timestamp → sector mapping)
│ Coarse + fine entries → full PTS and SPN
│ SPN → byte offset → sector extents for reading
│ → docs/clpi.md
│
▼
9. Parse BD-J labels (labels/) — optional
│ BDMV/JAR/*.jar → Java class constant pool strings
│ 5 format parsers: Paramount, Criterion, Pixelogic, CTRM, Deluxe
│ Audio track labels: "English Descriptive Audio", "French 5.1", etc.
│
▼
10. Stream content (mux/disc.rs → DiscStream)
│ Read sectors → decrypt → TS demux → PES frames
│ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — freemkv_engine::recovery::patch re-runs against the mapfile.
│
▼
PES frames → output stream (MKV, M2TS, network, etc.)
```
## API Summary
```rust
// Open + init drive
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?;
drive.init()?;
drive.probe_disc()?;
// Scan disc (UDF + playlists + AACS — all automatic)
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
// Stream pipeline — PES frames from any source to any output.
// input() returns Box<dyn FrameSource>, output() returns Box<dyn FrameSink>;
// direction is type-checked, so calling .write() on an input is a compile error.
let opts = InputOptions::default();
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
output.write(&frame)?;
}
output.finish()?;
```
## Module Reference
| Module | Doc | Purpose |
|--------|-----|---------|
| drive/ | [drive-access.md](drive-access.md) | Open, identify, init, unlock, single-shot read |
| scsi/ | [drive-access.md](drive-access.md) | Platform SCSI transport (Linux, macOS, Windows) |
| udf.rs | [udf.md](udf.md) | UDF 2.50 filesystem |
| mpls.rs | [mpls.md](mpls.md) | MPLS playlists + STN streams |
| clpi.rs | [clpi.md](clpi.md) | CLPI clip info + EP map |
| ifo.rs | -- | DVD IFO parser |
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan (sweep/patch/mapfile moved to freemkv-engine in 1.6.0) |
| labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
| sector/ | -- | SectorSource / SectorSink + DecryptingSectorSource decorator |
| io/ | -- | Pipeline<I, R> + Sink trait + WritebackFile |
| halt.rs | -- | Halt cancellation token |
| keydb.rs | -- | KEYDB download, parse, save |
| error.rs | -- | Error codes (E1xxx-E8xxx) |
| event.rs | -- | Drive event system |
-251
View File
@@ -1,251 +0,0 @@
# Drive Access and Unlock
Technical reference for how libfreemkv opens, identifies, unlocks, and reads
optical drives.
---
## Drive
`Drive` is the primary API. It owns the SCSI transport and the drive
identity (`DriveId`); any drive-specific unlock logic lives behind the
pluggable [unlock seam](#drive-unlock-seam), not in `Drive` itself.
### Opening a Drive
```rust
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
```
`open()` performs: open device → send INQUIRY → build `DriveId`. The drive
is ready for `wait_ready()` and `init()` (which routes through the unlock
seam).
### Drive Operations
| Method | Description |
|--------|-------------|
| `wait_ready()` | Wait for disc insertion (30s timeout, TUR polling) |
| `init()` | Route to the matching registered unlocker (if any), then prepare for reads |
| `probe_disc()` | Probe disc surface for optimal speeds |
| `read(lba, count, buf, recovery)` | Read sectors. Single-shot — no inline retries or reset. |
| `lock_tray()` | Prevent tray ejection during rip |
| `unlock_tray()` | Allow tray ejection (also runs on Drop) |
| `eject()` | Eject disc tray |
| `drive_status()` | Query physical state (disc present, tray open, etc.) |
| `has_profile()` | Whether a registered unlocker matches this drive |
| `close()` | Consume Drive, cleanup (also runs via Drop) |
### init() Sequence
`init()` routes drive preparation through the unlock seam:
1. Walk the registered-unlocker registry; the first whose `matches()` is true
is asked to `unlock_drive()` over the raw transport.
2. Whatever that unlocker needs (firmware upload, vendor handshakes, retries)
is the unlocker's own business — libfreemkv only forwards the transport.
3. If no unlocker matches, the drive is left untouched and the library uses
the host-certificate AACS handshake.
See [Drive Unlock Seam](#drive-unlock-seam) for the trait and registry.
### read() — single-shot
`Drive::read(lba, count, buf, recovery)` is the single read method. It issues
exactly one READ(10) CDB and returns the result. The `recovery` parameter only
selects the per-CDB timeout:
| `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------|
| `false` | 10 s | `freemkv_engine::recovery::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 60 s | `freemkv_engine::recovery::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`:
- **Layer 1 — `freemkv_engine::recovery::patch`** (in the engine crate, not
here) loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read
streak.
Inline recovery (5× gentle retry → close + reset + reopen → 5× more) was
removed in 0.13.6. See the stop-wedge postmortem (2026-04-25)
for rationale: the inline reset wedged drive firmware on the LG BU40N (Initio
USB-SATA bridge) without ever recovering a sector. See
[`rip-recovery.md`](rip-recovery.md) for the full three-layer model.
---
## SCSI Transport
### Trait
```rust
pub trait ScsiTransport: Send {
fn execute(
&mut self,
cdb: &[u8],
direction: DataDirection,
data: &mut [u8],
timeout_ms: u32,
) -> Result<ScsiResult>;
}
```
All drive communication goes through this trait. The library never opens file
descriptors or calls ioctls outside of a `ScsiTransport` implementation.
### Platform Backends
| Platform | Implementation | Device |
|----------|---------------|--------|
| Linux | `SgIoTransport` — synchronous `ioctl(fd, SG_IO, &hdr)` on `/dev/sg*` | `/dev/sg*` |
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
The Linux backend issues each command as one synchronous `ioctl(fd, SG_IO, &hdr)`
(`linux.rs`); the timeout is the kernel's, set via `hdr.timeout`, not a userspace
`poll()`. Opens with `O_RDWR | O_NONBLOCK`. (Pre-0.13.20 this backend ran an async
`write()` + `poll(1.5s)` + close-on-timeout + background-reopen pattern; it was
reverted in 0.13.20 — see the note at the top of `linux.rs`.)
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
The shim handles:
1. `shim_open_exclusive(bsd_name)` — unmounts the target device via `diskutil`,
then walks the IOKit registry to find the `IOBDServices` matching the
requested BSD name (IOBDServices → IOBDBlockStorageDriver → IOMedia → "BSD Name"),
then creates MMCDeviceInterface → SCSITaskDeviceInterface → ObtainExclusiveAccess.
2. `shim_list_drives()` — registry-based enumeration with zero SCSI, zero exclusive
access, zero unmounts. Reads IOBDServices "Device Characteristics" for
vendor/model/firmware and child IOMedia "BSD Name" for the device path.
3. `shim_execute()` / `shim_close()` — raw CDB dispatch and cleanup.
On non-zero SCSI status, the transport parses sense key from the sense buffer
and returns `Error::ScsiError`.
The Linux backend has no `reset`. `SG_SCSI_RESET` and STOP+START UNIT
escalation were tried in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA
bridge); both failed to recover wedged drives and made the wedge worse, so no
Linux transport-reset path exists today — recovery is just a fresh fd opened on
the next command. The only `reset` on a transport in the crate is
`SptiTransport::reset` on Windows (`windows.rs`). The macOS reset (which had
been a no-op) was removed entirely in 0.13.6, and the top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
removed at the same time (no callers).
### CDB Builders
The `scsi` module provides platform-agnostic CDB constructors:
| Function | CDB | Use |
|----------|-----|-----|
| `inquiry()` | INQUIRY (0x12) | Drive identification |
| `get_config_010c()` | GET CONFIGURATION (0x46) | Feature 010C firmware date |
| `build_read_buffer()` | READ BUFFER (0x3C) | All platform commands |
| `build_set_cd_speed()` | SET CD SPEED (0xBB) | Speed control |
| `build_read10_raw()` | READ(10) (0x28) with flag 0x08 | Raw sector reads |
---
## Drive Identification
`DriveId::from_drive()` sends two standard SCSI commands and extracts identity
fields:
| Field | Source | SCSI Reference |
|-------|--------|----------------|
| `vendor_id` | INQUIRY bytes [8:16] | SPC-4 section 6.4.2 |
| `product_id` | INQUIRY bytes [16:32] | SPC-4 section 6.4.2 |
| `product_revision` | INQUIRY bytes [32:36] | SPC-4 section 6.4.2 |
| `vendor_specific` | INQUIRY bytes [36:43] | SPC-4 section 6.4.2 |
| `firmware_date` | GET CONFIGURATION Feature 010C | MMC-6 section 5.3.10 |
The match key is `"VENDOR|PRODUCT|REVISION|VENDOR_SPECIFIC"`. Profile matching
tries all four fields first, then falls back to matching without the firmware
date for drives where Feature 010C is unavailable.
---
## Drive Unlock Seam
libfreemkv ships **no firmware, no unlock CDBs, and no drive profiles.** It
knows only the *seam*, never the *mechanism*. The seam is the `Unlocker`
trait plus a small process-wide registry (`src/unlock.rs`):
```rust
pub trait Unlocker: Send + Sync {
/// Stable, language-neutral identifier (logged).
fn name(&self) -> &str;
/// True if this unlocker handles the given drive.
fn matches(&self, id: &DriveId) -> bool;
/// Put the drive into extended-access mode. The one required capability.
fn unlock_drive(&self, scsi: &mut dyn ScsiTransport, id: &DriveId) -> Result<()>;
/// Read the disc Volume ID via the drive's OEM path. Default: no-op.
fn read_volume_id(&self, _scsi: &mut dyn ScsiTransport, _id: &DriveId)
-> Result<Option<[u8; 16]>> { Ok(None) }
/// Raise the drive to its maximum read speed. Default: no-op.
fn set_max_read_speed(&self, _scsi: &mut dyn ScsiTransport, _id: &DriveId)
-> Result<()> { Ok(()) }
}
```
An unlocker is supplied by an **external crate** and registered once at
process start:
```rust
libfreemkv::register_unlocker(Box::new(some_unlocker::Plugin::new()));
```
The implementor owns everything about *how* a particular drive family is
driven — drive identification against its own profile database, firmware
upload, vendor CDBs, variant logic. libfreemkv only hands over the raw
`ScsiTransport` and the `DriveId`.
### Routing
At drive-prep the registry is walked in registration order; the first
unlocker whose `matches()` returns true is asked to `unlock_drive()` (and,
when needed, `read_volume_id()` / `set_max_read_speed()`). If no unlocker
matches, the drive is left untouched and the library falls back to the
standard host-certificate AACS handshake (the "OEM route"). The
`register_unlocker(...)` line is the entire plug: drop it (and the unlocker
crate) and libfreemkv still compiles and rips via the cert handshake.
Concrete unlockers — including the firmware-unlock profile databases,
variant logic, and vendor CDBs that used to live in-tree — are maintained
in the separate **[freemkv-unlock](https://github.com/freemkv/freemkv-unlock)**
repository, never here.
---
## Speed Control
A matching unlocker may raise the drive to its maximum read speed via
`set_max_read_speed()` (a no-op when no unlocker matches or the unlocker
declines). The library issues SET CD SPEED (0xBB) through the generic CDB
builder; the concrete speed policy lives in the unlocker.
Available speeds:
| Format | Speeds |
|--------|--------|
| Blu-ray | 1x (4,500 KB/s) through 12x (54,000 KB/s) |
| DVD | 1x (1,385 KB/s) through 16x (22,160 KB/s) |
| Max | 0xFFFF (drive decides) |
---
## Windows Path Normalization
`normalize_path` in `src/drive/windows.rs` converts a device path ("D:",
"D:\\", "\\.\D:", "\\.\CdRom0") to Windows `\\.\X:` form. A near-identical
`normalize_device_path` exists in `scsi::windows`; both are kept because
they live in separate `cfg(windows)` modules that cannot easily share a
helper without introducing cross-module coupling.
-15
View File
@@ -1,15 +0,0 @@
# Linux `sg*` device enumeration
`enumerate_sg_names` (in `src/drive/linux.rs`) lists `/dev/sgN` SCSI-generic
device names to probe for optical drives.
Linux assigns `/dev/sgN` sequentially across *all* SCSI-generic devices
(disks, tape, HBAs, optical), not just optical drives. That means a fixed
`sg0..15` range can miss an optical drive on a host with many other SCSI
targets ahead of it in enumeration order.
To avoid that, the function prefers the exact present-device list from
`/sys/class/scsi_generic/`, which enumerates only the `sg*` nodes that
actually exist. It falls back to a bounded `sg0..15` probe only when sysfs
itself is unreadable, which happens in some minimal containers that lack a
mounted `/sys`.
-142
View File
@@ -1,142 +0,0 @@
# src/drive/mod.rs — internal notes
Long-form rationale relocated from `src/drive/mod.rs` comments that exceeded
the comment-guard's internal-comment cap. Each section is pointed to by a
short `// See docs/drive-mod.md — <topic>` comment at its original site.
## Error-recovery mode page (PER bit)
`MODE_PAGE_ERROR_RECOVERY` (0x01) is the SBC/MMC Read-Write Error Recovery
mode page. We flip the `PER` bit to make the drive REPORT a recovered read
(via CHECK CONDITION + sense key RECOVERED ERROR) instead of silently
returning best-effort data as GOOD status. On marginal/dirty media that
silent-GOOD data can be mis-corrected — a rip that "passed clean" but decoded
with errors. With PER on, freemkv sees the marginal read and re-reads it in
Pass N (a loud miss, never a silent commit). See
`build_error_recovery_select_payload`.
## `sleep_until_halted` — why it exists and why it was dormant
This was `#[cfg(test)]` for two releases, kept alive only by its unit tests,
while the two production paths that actually sleep — the `wait_ready` poll
backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
`std::thread::sleep` and so were deaf to the operator's Stop for ~30 s and
~15 s respectively. Every other drive path returns `Halted` at the next
`checked_exec` boundary; a Stop pressed during spin-up simply did nothing
visible until the poll ran out. The primitive existed; the sleeping code just
did not call it. It is production code again.
## `Drive::read` — single-shot contract and the removed inline retries
`recovery=true` uses `crate::scsi::READ_RECOVERY_TIMEOUT_MS` (60 s, matches
sg_dd) for the `freemkv_engine::recovery::patch` pass; `recovery=false` uses
`crate::scsi::READ_TIMEOUT_MS` (10 s) for `freemkv_engine::recovery::copy`'s
fast skip-forward sweep. Both budgets are generous enough that the drive can
finish ECC recovery on a marginal sector — pre-0.13.21 this was 1.5 s on the
fast path which forced the kernel mid-layer to time out and escalate while we
waited anyway. On any failure this returns `Err(DiscRead)` immediately;
orchestration (`freemkv_engine::recovery::patch` multi-pass, `DiscStream`
adaptive batch halving) handles retry policy.
Inline retry phases (5x gentle + reset+reopen + 5x more) were removed in
0.13.6: on some USB-SATA bridges the inline reset wedged drive firmware
without ever recovering a sector. The remaining recovery layers
(`freemkv_engine::recovery::patch` multi-pass, `DiscStream` batch halving) do
not touch the wedge-prone reset path.
## `find_drive` — media-preference selection policy
On a multi-drive system (common on Windows, where an empty/not-ready drive
can enumerate first) returning the first drive blindly can pick a drive with
no disc, dooming the operation. So `find_drive` opens each candidate in
enumeration order, queries `Drive::drive_status` (GET EVENT STATUS, which
works regardless of firmware state), and returns the first drive reporting
`DriveStatus::DiscPresent`. If no drive reports a disc — or `drive_status()`
is unavailable/returns `Unknown` everywhere (single-drive or quirky bridges)
— it falls back to the first drive that opened, preserving historical
behavior so those setups don't regress. For just listing drives without
opening (e.g. UI sidebar), use `scsi::list_drives()` instead — it returns
`DriveInfo` (path + identity) without the cost of running every drive's
profile + identity probe.
## `open_block_device_for_sg` — why no `O_DIRECT`
Resolves a `/dev/sg*` path to the corresponding `/dev/sr*` block device by
walking sysfs, then opens it for read. No `O_DIRECT`:
`posix_fadvise(POSIX_FADV_DONTNEED)` flushes the cache before each pread,
which avoids buffer-alignment requirements while still forcing fresh device
reads. Returns `None` on any error (sysfs not present, no matching block
device, open failed); callers treat that as "no fallback available" and
propagate the original SCSI READ error.
## `resolve_device` — staged, not yet wired
The cross-platform dispatch is kept ready for the caller that will consume
it, so the per-platform implementations below it (and their tests) stay
live. `allow(dead_code)` marks that deliberately — this is not an accidental
orphan.
## Test rationale: recovery/halt/chunking coverage
The `halt_tests` and `command_tests` modules exercise mutation-sensitive
edges (halt-aware sleep timing, chunked-read offset arithmetic, drive-status
byte decoding, spin-cycle/wait-ready halt responsiveness). The following
notes preserve the "why this exact scenario" reasoning that the inline test
doc comments were trimmed from:
- **`disc_is_dvd_matches_only_dvd_profile_family`**: must match the DVD
profile family (0x0010..=0x001F) and ONLY that family. A false positive on
a BD/UHD profile (0x0040+) would skip the drive unlock UHD reads require; a
false negative on a DVD would re-introduce the CSS read failure. Widening
the range to `..=0x0040` makes the BD-ROM assert fire; a failed/short GET
CONFIGURATION must default to NOT-DVD so the unlock still runs.
- **`media_event_reply` / media-status decoding**: MMC-6 §6.7 — byte 5 of the
GET EVENT STATUS reply is a Media Status only when the Event Header says a
media event descriptor follows (NEA clear AND Notification Class == 4).
Decoding byte 5 regardless of those preconditions reads a reserved/zero
byte as Media Status 0 and reports NoDisc on a drive that has a disc loaded
— the classic "works on my drive, not theirs" firmware split. The drive is
untrusted input, so an event-less reply must fall back to the TEST UNIT
READY status instead.
- **Chunked-read tests** (`read_chunks_write_into_correctly_offset_buffer_regions`,
`an_undersized_buffer_errors_on_the_chunked_path_just_like_the_single_one`,
`chunk_lba_near_u32_max_errors_not_overflows`): the multi-chunk path slices
the caller's buffer by `count * 2048` and advances the per-chunk LBA with
`lba + done`, both unchecked. An undersized buffer PANICKED
("range end index out of range") out of the public `read`/`read_fua`
instead of returning `Err(DiscRead)` like the single-chunk path; an LBA
range crossing `u32::MAX` overflowed (debug panic, or release wrap to a low
LBA silently read). These tests pin both to error-not-panic/overflow, and
one writes a distinct marker per chunk to confirm the byte-offset
arithmetic (`done * 2048`, not `done + 2048`) is correct — no other mock
touches the buffer contents, so a `*` -> `+`/`/` mutation would otherwise go
unasserted.
- **`wait_ready`/`spin_cycle` halt tests**: `wait_ready` was the one drive
path that called `self.scsi.as_mut().execute(..)` instead of
`checked_exec`, and its 60 x 500 ms loop never read `self.halt` — a cancel
during spin-up was ignored for ~30 s. `spin_cycle` had the same gap across
two bare `execute` calls and two plain `std::thread::sleep` pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`), and it runs from the
recovery path — precisely when an operator is most likely to press Stop.
The tests pin: a Stop set before the call returns `Halted` immediately with
no command issued; a Stop set mid-poll (on a deterministic Nth call, no
wall-clock race) exits at the next `checked_exec` boundary; a Stop that
lands during the spin-down pause wakes the halt-aware sleep within
~100 ms instead of blocking for the full 5 s.
- **`read_logs_a_good_status_short_transfer`**: a READ(10) that completes
with GOOD status but a residual underrun is correctly refused, but the
refusal used to log nothing — so a drive that residual-underruns on GOOD
status produced the same silent journal as a scratched disc, and the
operator was sent after the wrong remedy (replace the drive vs. clean the
disc). The test pins that the refusal now warns with lba/transferred/
expected/code.
## `open_block_device_for_sg` doc contract (kept short in code)
See the section above for the `O_DIRECT` rationale; the pointer comment near
the function keeps only the API contract.
-69
View File
@@ -1,69 +0,0 @@
# `dropgate` — audio drop bookkeeping rationale
## Keep/drop rule
A clean mux keeps every frame it can and drops the ones it can't — video
always survives (it's inter-frame predicted; a per-frame drop would cascade,
so video resyncs/conceals instead), audio keeps every decodable access unit,
and a damaged audio AU is dropped rather than shipped as a decoder-choking
glitch.
## Detection is per-codec
The DETECTION is inherently per-codec — each format carries its own
authoritative corruption check (DTS: the core sync/header parse per ETSI TS
102 114; AC-3: the header CRC per ETSI TS 102 366; FLAC: the frame CRC-16;
…). `DropTally` only carries the UNIFORM response so every audio parser
behaves identically:
1. **Count** kept vs dropped AUs and the dropped duration.
2. **Log** every drop (fail-loud, never silent) — a per-drop trace plus a
once-per-track aggregate at `warn` so it surfaces without debug logging.
3. **Whole-track fallback**: once a track is judged mostly undecodable, latch
a poison flag so the remainder is dropped too (a track that damaged isn't
worth muxing).
## Sync preservation
**Sync preservation is the caller's responsibility**, not this type's: the
parser must advance its PTS clock across a dropped AU exactly as it would for
an emitted one, so a drop becomes a silence gap and never a shift of the
following audio. See `DtsParser`'s `stamp_pts` call ordering for the pattern.
## `verified_dropped` vs `dropped`
`verified_dropped` counts AUs dropped because they were INDIVIDUALLY verified
undecodable (a failed CRC/header/parity check); only these feed the
whole-track poison verdict. `dropped` also counts *collateral* drops — AUs
discarded as a consequence of one corruption (TrueHD's resync-forward run, or
a poisoned track) — which must NOT amplify a few real errors into a false
whole-track loss.
## `record_collateral_drop`
A collateral drop is caused by another AU's corruption (TrueHD's
resync-forward run to the next major sync, or an already-poisoned track), not
by being individually verified undecodable. It is counted and logged for the
drop report, but deliberately does NOT feed the poison verdict, so one
corruption event can't amplify into a false whole-track loss.
## `maybe_poison`
Whole-track fallback: after enough AUs to judge, if more than half were
dropped the track is too damaged to be worth muxing — latch `poisoned` and
log it loudly once. The minimum-sample gate keeps a short damaged burst from
poisoning an otherwise-good track. Judge on VERIFIED drops vs all AUs seen: a
track is only poisoned when a majority of its access units are individually
undecodable — not when a couple of corruption events forced long collateral
resync runs.
## Test: `interleaved_keeps_are_in_the_poison_denominator`
The poison verdict is a RATIO — verified drops against every AU seen — so
the kept count is half of it. `does_not_poison_a_mostly_good_track` records
its keeps AFTER the single drop, and `maybe_poison` only runs inside
`record_drop`, so the keeps are never in the denominator when the verdict is
actually computed: that test passes even with the kept count never
incremented. Interleaving them puts the kept count on the critical path,
where losing it turns the ratio into "verified drops vs verified drops" —
always >50% — and silently discards a healthy track.
-309
View File
@@ -1,309 +0,0 @@
# DTS / DTS-HD elementary stream parser (`src/mux/codec/dts.rs`)
Internal design notes moved out of source comments (see `ci/comment-guard.py`
prose caps). Each section is pointed to by a short `//` comment at the
corresponding site in `dts.rs`.
## `DTS_HD_EXT_SYNC`
An access unit is delimited by the next CORE sync; the parser locates and
exactly sizes each extension substream (via `exss_frame_size`) so a false
core sync inside the EXSS payload can't split the AU and truncate the
lossless extension.
## `emit_or_drop`
Gates an assembled access unit through the decodability check and either
pushes it or drops it. `au_pts`/`dur_ns` are already stamped on the shared
PTS clock (which the caller advances whether or not the AU survives), so a
drop leaves the following audio on its true timeline — a gap, not a shift.
Every drop is logged (fail-loud, never silent).
## `stamp_pts`
`front` is the AU's own core-PES PTS (from `front_pts`); `dur_ns` its
decoded duration.
The model matches the (correct) AC-3 path: **re-base to each PES's own
container timestamp, and advance by one frame duration ONLY within a run of
AUs that share the same PES.** A new PES (its `front` differs from the
previous AU's) trusts its own timestamp — so the emitted timeline tracks the
container and never drifts. Advancing within a PES is what fixes the DVD
case (several DTS core frames packed in one PES, which otherwise all
collided on that single PES timestamp → "non monotonically increasing dts").
A global running clock was WRONG here: once accumulated frame durations
exceeded the PES spacing it never re-based, drifting the track minutes past
the real length over a feature-long title.
## `CORE_HEADER_MIN_BYTES`
Number of leading bytes that must be buffered before the core `fsize` field
(bytes 5-7) can be decoded. This is a HEADER-LAYOUT minimum — "enough bytes
to read the size field" — and is deliberately distinct from
`MIN_CORE_FRAME_BYTES` (the decoded-frame-size validity floor). They must
not be conflated: this one gates buffer reads of the header, the other
rejects implausible decoded sizes.
## `MIN_CORE_FRAME_BYTES`
Minimum plausible decoded DTS core frame size, per ETSI TS 102 114: the
on-wire FSIZE floor is 95, so a real core frame is at least 96 bytes. A
decoded `core_size` below this means we matched a false/corrupt core sync
(a lucky 0x7FFE8001 in extension-substream payload whose 14-bit `fsize`
decoded to a tiny value) rather than a real frame, so we resync instead of
closing an access unit at a junk boundary and dropping the DTS-HD extension
tail.
## `PTS_UNSET`
Sentinel for "no valid PTS base captured yet". Real PTS-in-ns values are
non-negative (derived from the unsigned 90 kHz PES timestamp), so a
negative value can never collide with a genuine timestamp. Used to mark the
PTS base invalid after a forced flush so the next PES sets it regardless of
buffer state.
## `flush_tail`
Emits the final buffered access unit (the last core + its extension
substreams, which had no following core sync to close it during
streaming), gated through the decodability check. Requires a complete core
frame; drops a bare partial sync tail.
## `SYNCWORD_BYTES`
Byte length shared by the DTS core sync and the DTS-HD EXSS sync (both
32-bit words). `next_core_boundary` finds the next *valid* core sync after
the current core frame, to delimit the access unit: extension-substream
payload can contain byte sequences that match the core syncword, so each
candidate is validated by decoding its core size — a match whose decoded
size is implausible (`< MIN_CORE_FRAME_BYTES` or `> MAX_AU_BYTES`) is a
false sync and is skipped, continuing the search.
## EXSS header field bit widths
`EXSS_USER_DEFINED_BITS`, `EXSS_INDEX_BITS`, `EXSS_HEADER_SIZE_TYPE_BITS`,
etc. are the DTS-HD extension-substream (EXSS) header field bit widths
(ETSI TS 102 114, ExtSS header). `bHeaderSizeType` selects the short form
(`nuExtSSHeaderSize` 8 bits, `nuExtSSFsize` 16 bits) or, for larger
substreams, the long form (12 / 20 bits).
## `exss_frame_size`
DTS-HD extension substream (EXSS) total byte size — INCLUDING the
`0x64582025` syncword — read precisely from its header. `buf` must begin
with `DTS_HD_EXT_SYNC`. Returns `None` when the size fields aren't fully
buffered.
`nuExtSSFsize` is the total frame size in bytes minus one. Parsing it lets
the AU framer skip the extension by its exact length instead of scanning
its (arbitrary) payload for a core sync.
## `next_core_boundary`
Offset where the current access unit ends (the start of the next core
frame). The AU is the core frame plus its trailing DTS-HD extension
substreams, which are skipped PRECISELY by their declared size — so a
chance core syncword inside the XLL lossless payload can never be mistaken
for the next AU boundary (the bug that truncated the extension and produced
the "Failed to decode block code(s)" class). Falls back to the heuristic
core-sync scan only when an extension can't be sized (malformed / truncated
input).
## `scan_for_next_core`
Heuristic fallback (the pre-fix behaviour): scan forward for the next core
syncword whose decoded size is plausible. Used only when precise extension
skipping can't proceed; a chance core syncword in extension payload usually
decodes to an implausible size and is skipped.
## `dts_core_frame_size`
`fsize` is the 14-bit field at bits 46-59 of the header (bytes 5-7). On the
wire `fsize` is the frame length minus one, so this returns `fsize + 1`,
i.e. the core frame length in bytes (range 1..=16384). Callers treat the
result as the actual byte length and the MIN..=MAX range checks assume so.
Returns `0` when `data` is shorter than `CORE_HEADER_MIN_BYTES` — every call
site rejects that via the minimum-frame lower bound, so a `0` is never
mistaken for a valid tiny frame.
## `dts_core_samples`
Samples in one DTS core frame: `(NBLKS + 1) * 32`. `NBLKS` (7 bits) is the
core-header PCM-sample-block count (ETSI TS 102 114) that fixes the frame's
decoded sample count. Bit layout after the 32-bit sync: FTYPE(1) SHORT(5)
CPF(1) **NBLKS(7)** FSIZE(14) …, so NBLKS = byte4 bit0 + byte5 bits7-2.
## `DTS_PCMBLOCK_SAMPLES` / `DTS_SUBBAND_SAMPLES` (core-header validity)
DTS core-header validity constants (ETSI TS 102 114). For a NORMAL frame
`deficit_samples` must equal `DTS_PCMBLOCK_SAMPLES` — a termination frame
may carry fewer; `npcmblocks` must be a multiple of `DTS_SUBBAND_SAMPLES`;
`audio_mode` must be below `DTS_AMODE_COUNT`; `lfe_present ==
DTS_LFE_FLAG_INVALID` is rejected.
## `DTS_AMODE_COUNT`
Number of LEGAL `AMODE` (channel-arrangement) codes. The 6-bit AMODE field
(ETSI TS 102 114 §5.3.1) has 16 defined channel arrangements, codes 0-15;
only 16-63 are reserved/user-defined and undecodable. The per-AMODE channel
counts in ETSI TS 102 114 §5.3.1 cover all 16, confirming they are
decodable — codes 10-15 are the 6/7/8-channel layouts. A frame is dropped
only when `audio_mode >= DTS_AMODE_COUNT` (i.e. a truly reserved 16-63
code); dropping a legal 10-15 multichannel core would silence recoverable
audio.
## `DTS_CORE_SR_VALID`
Sample rate (Hz) per core `SFREQ` code (ETSI TS 102 114 Table 6-4); a `0`
entry marks a reserved code that fails header validation as an invalid
sample rate. Valid entries are locked to the spec by
`dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
{0, 4, 5, 9, 10}.
## `core_header_drop_reason`
Decodability gate: the core-frame header validity checks from ETSI TS 102
114. Returns `Some(reason)` when the DTS core-frame header is invalid — in
which case the packet is undecodable ("Invalid data found") and dropping it
loses nothing a decoder could have used. Returns `None` (keep) for a
decodable header OR if the header can't be fully read (never false-drop on
our own buffer underrun; the framer only emits AUs whose core is fully
buffered and ≥ 96 bytes).
The 4-byte core sync is already validated by the framer, so this reads the
header fields that follow it. The 16-bit CPF header CRC is not verified (we
skip past it) and the audio-header/side-info CRCs are likewise not checked,
because decoders treat those bytes as optional/ignored — verifying them
would drop frames that decode fine (false positives).
## Test: `stamp_pts_reuses_front_when_no_running_cursor_yet`
`stamp_pts`'s third arm (`else if front != PTS_UNSET`) is only reached once
the first arm's `front != self.last_front_pts` has already failed (i.e.
`front == self.last_front_pts`) and no within-PES running cursor is
available yet (`next_pts_ns == PTS_UNSET`). In that state the AU must still
be stamped with its own (repeated) front timestamp, not silently collapsed
to 0 — a `!=` -> `==` typo on the guard would only take this arm when
`front` IS the unset sentinel, producing 0 instead here and the sentinel
value itself if front really were unset.
## Test: `drain_front_collapses_offset_zero_markers_instead_of_leaking`
Every drained access unit that shares its PES with the previous one pushes
a new `(0, pts)` marker once rebased; `drain_front` must collapse those down
to the single most-recent one each time, or `pts_marks` grows once per
access unit for the life of the track — an unbounded allocation on a
multi-hour disc. `front_pts()` alone can't observe this: it always finds the
last offset-0 entry regardless of how many duplicates precede it, so the
leak is invisible unless something checks that the collapse actually ran.
## Test: `make_exss`
Builds a real DTS-HD EXSS substream of `total` bytes (short header form),
with an optional false DTS core syncword embedded in its payload (decoding
to a plausible core size) — to prove precise sizing, not a payload scan,
bounds the extension.
## Test: `make_dts_ext`
Builds a minimal DTS-HD extension substream of `size` bytes (just the sync
+ zero-padding). The parser delimits extensions by the next CORE sync, not
by the extension's own size header, so a valid header isn't required — only
that the bytes carry no spurious core sync.
## Test: `bogus_tiny_core_sync`
Builds 4 bytes that look like a DTS core sync but whose `fsize` field
decodes to a tiny `core_size` (< MIN_CORE_FRAME_BYTES). With the dead-code
guards this passed validation and could close an access unit at a junk
boundary; with the fix it must be drained and resynced past.
## Test: `next_core_boundary_exact_syncword_length_is_not_need_more`
Only the 4-byte EXSS syncword is buffered after the core frame — no header
fields at all. The guard `buf.len() < pos + SYNCWORD_BYTES` must be strict
`<`: at exactly `pos + SYNCWORD_BYTES` the sync bytes ARE fully present, so
the function proceeds to identify them as an extension sync (then fails to
size the header and falls back, ending in `NextCore::None` here since
nothing after it looks like a core sync either). A `<=` typo would instead
return `NeedMore` at this exact length without ever inspecting what the 4
buffered bytes are.
## Test: `make_bad_dts_core`
Builds a structurally-framed but UNDECODABLE core: a valid `make_dts_core`
whose LFE flag is set to the reserved value 3 (`DTS_LFE_FLAG_INVALID`). It
still sizes and syncs correctly (so the framer delimits it normally), but
the core-frame header validity check rejects it as an invalid LFE flag
(ETSI TS 102 114 §5.3.1: an LFF value of 3 is invalid). LFE is byte10
bits2-1, and does NOT feed the frame duration (NBLKS + SFREQ only), so a
dropped bad core still carries the same `DTS_CORE_DUR_NS` as its good peers.
## Test: `reparse_real_dts_file`
Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
through `DtsParser` and writes the emitted access units back out, so the
garbage-extension → core-only drop can be validated against an actual
damaged stream (e.g. an extracted DTS-HD MA track) end-to-end with an
external DTS decoder. Env: `DTS_IN` (input), `DTS_OUT` (output).
cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture
## Test: `find_sync_matches_a_buffer_that_is_exactly_the_syncword`
`find_sync` scans `0..=len-4`, so a buffer that is EXACTLY the syncword
still matches. The existing tests cover 0, 3 and "longer than 4", which
leaves the `len == 4` boundary open — and that is the case a syncword split
across PES packets lands on the moment its last byte arrives.
## Test: `a_core_header_of_exactly_the_minimum_length_is_decoded_not_deferred`
`CORE_HEADER_MIN_BYTES` is "enough bytes to DECODE the size field", so a
buffer holding exactly that many must be decoded, not deferred. The
distinction is visible precisely at the boundary: with 10 bytes of a core
sync whose decoded size is sub-spec, the parser must recognise the false
sync, drain past it and resync down to the 3-byte carry-over tail. Waiting
instead leaves the false sync sitting at the front of the buffer, where it
blocks every later real core behind it.
## Test: `flush_emits_a_core_that_exactly_fills_the_buffer`
The end-of-stream flush must emit a final access unit whose core is exactly
as long as the buffer — the ordinary case, since the last AU is closed by
end-of-stream rather than by a following core sync. Rejecting it at the
boundary silently drops the last frame of every DTS track.
## Test: `flush_discards_a_long_buffer_that_does_not_begin_with_a_core_sync`
The flush guard is a DISJUNCTION: a buffer that does not BEGIN with a core
sync is discarded whatever its length. Requiring both conditions instead
lets a long run of junk through to `dts_core_frame_size`, which happily
decodes a 14-bit size out of arbitrary bytes — and the flush then emits an
"access unit" that is not DTS at all.
The junk here is sized so that mis-decoding it yields a plausible core size
that the buffer fully covers, which is exactly when the wrong answer looks
like a right one.
## Test: `exss_frame_size_needs_the_worst_case_header_and_no_more`
`EXSS_HEADER_MIN_BYTES` is the WORST-CASE header length — the long form's
43 bits after the syncword, rounded up to 6 bytes, plus the 4-byte sync. It
gates whether an extension substream can be sized precisely, and that is
what keeps a chance core syncword inside XLL payload from being mistaken
for the next AU boundary; too large and every extension falls back to the
payload scan, too small and the reader runs off a truncated header.
Pinned at both sides of the boundary rather than by value, so the field
widths it is summed from stay honest.
## Test: `an_access_unit_carries_the_source_of_the_packet_its_core_arrived_in`
The whole point of the shared buffer: an access unit whose core arrived in
an EARLIER packet keeps that packet's source offset, not the offset of
whichever packet completed it. At a clip boundary the two belong to
different clips, and taking the later one puts the unit in the wrong clip —
which is what left nine audio and subtitle tracks unplaceable.
</content>
</invoke>
-149
View File
@@ -1,149 +0,0 @@
# DVD AC-3 physical sub-stream probing
## Why this exists (a real-disc wrong-substream bug)
A DVD VTS IFO declares its audio streams in a fixed table, and freemkv's
scan assigns each declared stream a `private_stream_1` sub-stream id purely
by per-codec ordinal — the first AC-3 stream becomes `0x80`, the second
`0x81`, and so on (`ifo::assign_audio_sub_stream_ids`). That assumes the
physical sub-stream order on the wire matches the IFO declaration order.
On some discs it does NOT. A real R2 PAL DVD release's feature declares
ONE AC-3 audio stream the IFO nibble marks as 5.1 (6 channels), but the
physical VOB carries the 5.1 main mix and a 2.0 down-mix on DIFFERENT
`0x8x` sub-stream ids, and the 2.0 is the one that happens to land at the
ordinal `0x80` slot. Routing the declared 5.1 stream to `0x80` by ordinal
therefore muxes the 2.0 down-mix while labelling it 5.1 — the wrong
physical track.
The robust fix is data-driven and codec/disc agnostic: read each physical
AC-3 sub-stream's REAL channel count from the VOB (the `acmod`/`lfeon` of
its first frame after the `0x0B77` sync) and route each IFO-declared AC-3
stream to the physical sub-stream whose actual channel count matches the
IFO's declared count — instead of trusting the ordinal. This never
re-reads the disc beyond a bounded head-of-feature probe and degrades to
the original ordinal mapping when the probe yields nothing
(unreadable/short VOB).
## `PROBE_SECTORS` sizing
How many 2048-byte sectors of the first feature extent to probe. The head
of a DVD feature opens with logos/warnings whose audio is frequently a
thin 2.0 bed on the FIRST sub-stream only — the other physical `0x8x`
sub-streams and the main 5.1 mix do not appear until a sector or two
further in. 512 sectors (1 MiB) was too short: on one real disc it saw
ONLY `0x80`, and only its opening 2.0 frames. 1024 sectors (2 MiB)
reliably contains at least one frame of every physical AC-3 sub-stream
AND enough of `0x80` to reach its 5.1 frames. Still bounded so a live
drive is never hammered (see the project "don't hammer the live drive"
rule).
## Why the maximum channel count, not the first frame
The first frame of a sub-stream at the head of a feature is NOT
representative. A DVD opens with logos/warnings, and the main `0x80`
sub-stream there frequently carries a thin 2.0 bed before transitioning
to its real 5.1 main mix a fraction of a second later (observed on a real
disc: `0x80`'s first frames are acmod=2 → 2 channels, then it becomes
acmod=7+lfe → 6 channels within the same 2 MiB window). Recording only
the FIRST frame read `0x80=2` and missed the 5.1 entirely, defeating the
channel-match routing. The 5.1 capability of a sub-stream is the
*maximum* channel count any of its frames carries, so
`probe_ac3_substream_channels` scans them all and keeps the max.
## `remap_audio_pids` conservatism
Conservative — it only ever REASSIGNS among the physical sub-streams the
probe actually saw, and only when a better (exact-channel) match exists
than the stream's current assignment. A stream whose current sub-stream
already matches is left alone; a stream with no matching physical
sub-stream keeps its ordinal assignment. So a normal disc (physical order
== IFO order) is a no-op.
## Test helper notes (`dvd_audio_probe::tests`)
- `ac3_frame`: builds a single, correctly-SIZED AC-3 frame whose
`acmod`/`lfeon` encode a known channel count. `byte4` is
`fscod=0 | frmsizecod=0`, so `ac3_frame_size` reports 128 bytes and the
frame is zero-padded to exactly that — this lets `max_substream_channels`
advance frame-by-frame over a multi-frame payload exactly as it does on
real VOB data. The BSI bits are laid down with a writer so the test
never hand-miscomputes the lfeon offset, matching `acmod_channels`'
reader.
- `ps_ac3_frames`: builds a minimal `private_stream_1` PES carrying
`frames` for `sub_id`, each preceded only by the 4-byte AC-3 sub-header
at the PES head. Mirrors the on-disc layout the PS demux expects: PES
start `0x000001BD`, length, PES header (no PTS), sub-header
`[sub_id, frame_count, ptr_hi, ptr_lo]`, then the concatenated AC-3
frames.
### `probe_reads_max_channels_no_cross_contamination`
Real-disc regression — the probe must read each sub-stream's TRUE
(max-mix) channel count, not be poisoned by an unrepresentative head
frame, and must NOT cross-contaminate between sub-streams. Mirrors the
real on-disc layout that caused the mis-read: the feature head carries
`0x80` opening with a 2.0 frame and THEN a 5.1 frame (its real main mix),
interleaved with `0x81` carrying only 2.0. The old first-frame probe read
`0x80=2` (the logo bed) and missed the 5.1; the max-over-frames probe
must report `0x80=6` and `0x81=2`.
### `max_substream_channels_locates_sync_after_leading_non_sync_bytes`
Regression guard for a hand-checked mutation (`+` → `-` at the
`pos + rel` offset computation): with `pos` starting at 0 and the first
sync found 3 bytes in, `pos - rel` would underflow a `usize` and panic,
or (if it somehow didn't) index the wrong start entirely. `pos + rel` is
the only computation that is always in-bounds, since `rel` is itself
bounded by the length of the slice searched from `pos`.
### Unmappable-size fallback tests
When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
`fscod == 3`), `max_substream_channels` must fall back to stepping
`start + 2` bytes past the sync to re-lock onto the next genuine sync,
and must keep making forward progress doing so (never revisit the same
sync, which would loop forever, and never jump so far that it skips the
very next real frame).
- `max_substream_channels_unmappable_size_steps_forward_by_two` lays a
bogus-sized header at absolute offset 4 (so `start == 4`,
`start + 2 == 6`) immediately followed, at offset 6, by a real, fully
decodable 2.0 frame — the position the `+ 2` fallback must land on
exactly.
- `max_substream_channels_unmappable_size_at_start_steps_forward_not_back`
puts the unmappable-size sync at absolute offset 0 (`start == 0`) so
that stepping backward instead of forward (`start - 2`) would underflow
rather than merely land on the wrong byte. Also proves the real frame is
still found 6 bytes further in, confirming forward progress past the
bogus header.
### `remap_reads_current_substream_via_and_not_or_or_xor`
`remap_audio_pids` must read a stream's CURRENT physical sub-stream id
from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
`0x00FF`, both of which force the low byte to `0xFF` regardless of the
real PID and so always miss the "already matches" shortcut. That matters
observably when TWO physical sub-streams share the same probed channel
count: with a correct read, a stream already sitting on a matching
sub-stream is left alone (conservative, per the module's documented
behaviour); with the low byte forced to `0xFF`, `probed.get(&0xFF)` is
always `None`, so the code falls through to the "find any unclaimed
match" path and picks the FIRST (lowest-keyed, BTreeMap-ordered) matching
physical sub-stream instead — which here is a *different* sub-stream
(0x80) than the one the PID already correctly names (0x81), producing a
spurious PID change.
### `probe_and_remap_reroutes_swapped_substream_scenario_end_to_end`
End-to-end `probe_and_remap`: a real-disc-shaped MpegPs title (one
declared 5.1 AC-3 stream ordinally assigned 0x80) whose physical VOB
bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on 0x81. This must
reach the `remap_audio_pids` call and re-route the stream to 0xBD81. It
also, by construction, proves each of the guards along the way lets a
real, positive case through: the content-format check must NOT bail on
`MpegPs` (only on non-`MpegPs`), the AC-3 presence check must NOT bail
when AC-3 IS present, and the sector-count check must NOT bail when the
count is nonzero — any one of those inverted would skip the probe
entirely and leave the PID at its untouched ordinal value (0xBD80), which
the test's assertion would catch.
-177
View File
@@ -1,177 +0,0 @@
# DVD Title Scan (src/disc/dvd.rs) — Internal Notes
Rationale, regression history, and test-fixture layout notes that were
originally inline comments/doc-comments on private items in
`src/disc/dvd.rs`, relocated here to satisfy the comment-guard's internal-
comment cap. Each pointed-to site in the source carries a short `// See
docs/dvd.md — <topic>` line.
## `Disc::scan_dvd_titles` halt handling
Cancellation: `halt` is polled before the IFO tree is read, and an IFO read
that fails with `Error::Halted` — how a live drive reports a Stop, since
`Drive::checked_exec` fails every command once its flag is set — is
propagated rather than swallowed. Every other IFO failure keeps its
best-effort `Ok(vec![])`.
It has to be an error and not an empty title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles. This one was the worst of the three enumerators — a bare
`Err(_) => return Vec::new()` turned an operator Stop into ZERO titles at
rc=0, a disc reported as carrying no video at all.
`scan_dvd_titles` returns the scanned titles plus, when the disc's own
First-Play navigation deterministically dispatches to a title, that title's
`playlist_id` — the nav-resolved main feature. The caller feeds it into
`sort_titles_by_main_feature` as the DVD `nav_feature`, the same seam the BD
path uses. It is `None` (existing ranking heuristics stand) whenever the
navigation is menu-only, malformed, or does not converge.
## DefaultDuration / cadence TODO
`measured_cicp` TODO(spec): populate `top_field_first` + measured colour
from `Mpeg2Parser` once tracks are built after frame parsing — needs a
`CodecParser`->title channel like `codec_private`. Until wired, `None`
falls back to TFF / PAL-NTSC guess.
DefaultDuration TODO(spec): comes from IFO `frame_rate`, so a
soft-telecined 23.976-in-29.97 DVD reports 29.97 instead of true film
rate. Cadence is detectable via `nb_fields` (2:3 pulldown); needs the same
channel as `top_field_first` above.
## Test fixture builders: `build_vmg` / `build_vts`
`build_vmg` builds a VIDEO_TS.IFO: magic "DVDVIDEO-VMG"@0, TT_SRPT sector
ptr@0xC4. TT_SRPT lives at `tt_srpt_sector*2048`: `num_titles(u16)`@0, then
12-byte entries from +8. Each entry: `num_chapters(u16)`@+2, `vts_number`@+6,
`vts_title_num`@+7.
`build_vts` builds a VTS_XX_0.IFO. Layout per ifo.rs:
- magic "DVDVIDEO-VTS"@0
- `vtstt_vobs` (Title VOBS start sector, u32 BE)@0xC4 — the production
`vob_start_sector` the cell sectors are relative to. (0xC0 is
`vtsm_vobs`, the menu VOBS, which the scan must NOT use.)
- VTS_PGCIT sector ptr(u32 BE)@0xCC
- video attr byte@0x200
- `num_audio(u16 BE)`@0x202, audio blocks (8B) @0x204
- `num_subs(u16 BE)`@0x254, subtitle blocks (6B) @0x256
PGCIT (at `pgcit_sector*2048`): `num_pgcs(u16)`@0, PGC info entries (8B)
from +8 with PGC byte offset(u32 BE)@+4. PGC: `nr_programs`@0x02,
`nr_cells`@0x03, BCD time@0x04, `pgm_map` ptr@0xE6, `cell_playback` ptr@0xE8
(both u16 BE rel to PGC start).
## `HaltingReader` test double
A `SectorSource` that fails every read at or above `halt_at` with
`Error::Halted` — how a LIVE DRIVE behaves once the operator presses Stop:
`Drive::checked_exec` fails every SCSI command with `Halted` from then on,
and `Drive::read` deliberately preserves the variant. Reads below the
threshold still succeed, so the scan gets far enough to have something to
truncate.
## Test: `halted_ifo_read_is_not_reported_as_a_shorter_disc`
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The DVD enumerator must not swallow that into a
successful scan.
This was the worst of the three enumerators. RED BEFORE GREEN, two
distinct swallows, both measured with the fix reverted:
- `ifo::parse_vmg` treats a failed title set as a placeholder entry and
continues, so a cancel landing on VTS_02's IFO returned
`Ok([VTS_01_1.VOB])` — one title from a two-title disc.
- `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel landing
on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc reported as
holding no video at all.
Both are indistinguishable from a real disc, and both are now
`Err(Error::Halted)`.
Fixture: two title sets, VTS_01's IFO at `PART_START+6000`, VTS_02's at
`PART_START+7000`. Both ICBs sit far below, so filesystem metadata
resolves fine and only the second title set's CONTENT read is cancelled —
the truncation case.
## Test: `scan_dvd_titles_uses_title_vobs_not_menu_vobs`
Regression (first-play menu prepended to the feature): `vob_start` must
come from the **Title** VOBS pointer `vtstt_vobs` (VTS_IFO 0xC4), NOT the
**menu** VOBS pointer `vtsm_vobs` (0xC0). On discs with a per-title menu —
e.g. the Universal "the parental level has been set, press yes" first-play
still — `vtsm_vobs` points at that menu VOB, which sits just before the
title VOB. Cell `first_sector` values are relative to `vtstt_vobs`; reading
0xC0 prepended the menu and shifted every extent back by
`vtstt_vobs - vtsm_vobs`, so the rip opened on the parental prompt instead
of the movie (one real NTSC R1 disc: vtsm=44, vtstt=3640).
Here `build_vts` stamps `vtstt_vobs = 3640` (0xC4); the test additionally
stamps a *different* `vtsm_vobs = 44` (0xC0). The extent must resolve
from 3640.
## Test: `scan_dvd_titles_extent_is_absolute_three_term_sum`
ABSOLUTE-REBASE regression (real-disc menu-at-start fix): `ifo::parse_vts`
now sets `vob_start_sector = file_start_lba(IFO) + vtstt_vobs`, so an
extent's `start_lba` must equal the sum of THREE independent terms — the
IFO file's absolute on-disc LBA, the IFO-relative `vtstt_vobs` (0xC4), and
the cell's `first_sector` — none of which may be dropped. The earlier code
used the bare relative `vtstt_vobs`, placing every extent `ifo_lba`
sectors too early (the rip opened in the VMGI/menu region before drifting
into the movie). The other tests fold two of the three terms together
(zero cell offset, or a single combined expectation); this one keeps all
three distinct and non-overlapping so a regression to ANY two-term
combination is caught.
## Test: `scan_dvd_titles_lpcm_routes_to_a0_pid_range`
LPCM SCAN ROUTING (audit §2 / §5 #6): the 0xA0..=0xA7 PID range was never
exercised in the dvd.rs scan. An LPCM stream (coding_mode 4) at audio
position 1 must get `sub_stream_id 0xA1` → PID `0xBDA1` via
`dvd_audio_pid`, distinct from the AC-3 `0xBD80` space, with its real
channel count preserved.
## Test: `scan_dvd_titles_multiple_vobsub_tracks_distinct_pids`
MULTI-VOBSUB SCAN (audit §2 / §5 #6): the single-subtitle test covered one
track; a multi-subtitle VTS must emit one `Stream::Subtitle` per entry
with distinct PIDs (`0x20 + ordinal`) and per-language tags, all sharing
the PGC palette `codec_data`.
## `stamp_first_play_jumptt` test helper
Stamp a First-Play PGC into a VMG whose pre-command list is a single
unconditional `JumpTT ttn`, so `dvdnav::resolve_main_title` resolves that
title through TT_SRPT. Placed in sector 0, clear of the magic / FP_PGC /
TT_SRPT pointers `build_vmg` already writes.
## Test: `scan_dvd_titles_nav_promotes_first_play_target`
The DVD First-Play nav promotion branch (issue #40): when
`resolve_main_title` returns a `(vtsn, vts_ttn)` target, `scan_dvd_titles`
must map it — via the 1-based `vts_ttn == vts_title_idx + 1` join within
the matching `vts_number` — to the running `title_number`, and surface it
as the `nav_feature`. TT_SRPT here maps title 2 → (VTS 2, title-in-set 1);
the First-Play unconditionally `JumpTT 2`, so the promoted feature must be
the SECOND scanned title (VTS_02 → title_number 2). An off-by-one on the
join or a `vtsn`/`vts_ttn` field swap would promote the wrong title or
none.
## Test: `scan_dvd_titles_drops_leading_scene_index_cell`
End-to-end bug-4 fix: a feature PGC that opens with a leading interleaved-
angle sub-block cell (category 0x90 = in-block cell of an angle block)
must have that cell DROPPED from the muxed extents, so the rip starts at
the real feature. Chapters shift earlier by the dropped duration.
## Test: `scan_dvd_titles_mp2_audio_pid_fallback_is_additive`
Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio stream
has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio, per
`ifo::assign_audio_sub_stream_ids` — the PID falls back to `0xBD00 + i`
where `i` is the stream's positional index in the IFO audio-attribute
table. Two MPEG-audio (coding_mode 2) streams must land on two DISTINCT,
correctly-offset PIDs: 0xBD00 and 0xBD01. This pins the `+` (not `-`/`*`)
so the second stream doesn't collide with, or wrap under, the first.
-23
View File
@@ -1,23 +0,0 @@
# src/dvdnav/mod.rs
## Module rationale
Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
PGC/cell tables, PCI/HLI button packets); the VM command decoder is
verified against real discs.
Current contents: `vmcmd` — the VM command decoder; `nav` — the
First-Play navigation executor that resolves which title the disc's own
navigation selects as the feature.
## `resolve_feature_start` — parked status (issue #40)
PARKED (issue #40, `USE_NAV_RESOLVER=false`): this is intentionally a stub,
not an accidental dead one. It unconditionally returns `None` and ignores
`reader`/`udf` on purpose — the IFO/PCI parsing + nav executor (built on
`vmcmd`) land incrementally behind that flag. Until wired in, calling this
is behaviour-neutral (the caller always falls back); improvements to the
resolver take effect here without touching the call site. Do not delete.
`reader`/`udf` are the seam inputs the nav executor will consume to read
VIDEO_TS.IFO + the VTS IFOs/menu VOBs. Reserved until that lands.
-28
View File
@@ -1,28 +0,0 @@
# dvdsub — VobSub SPU boundary rule
The presence of a PTS — not merely an open `pending` — is the authoritative
SPU-boundary signal, so a lost continuation or a corrupt SPU_size can't merge
the next subtitle into the stuck unit.
## ycbcr_to_rgb range convention (deliberate)
This uses the **full-range (JFIF) BT.601** coefficients with no 16/235 luma
scaling. DVD IFO palette YCbCr is nominally studio-swing BT.601, so
studio-swing math would be more colorimetrically "correct" in isolation. But
the output here is a VobSub `.idx` `palette:` line, and the entire VobSub
ecosystem (the original tooling, mkvtoolnix, players that read the `.idx`
palette) is built around this full-range formula — it is the de-facto
on-disk convention. Emitting studio-swing-scaled RGB here would make
freemkv's palettes inconsistent with every other tool and wrong in players
that assume the VobSub convention. Do NOT "fix" this to studio-swing without
changing the consuming side in lockstep.
## ycbcr_to_rgb_reads_byte2_as_cr_and_byte3_as_cb test fixture
A saturated RED entry appears on disc as Y=76, Cr=255, Cb=85 (full-range
BT.601 encoding of RGB #FF0000), i.e. bytes `[0x00, 76, 255, 85]`. Reading
byte 2 as Cb and byte 3 as Cr instead turns this entry BLUE, which is the
exact user-visible symptom. The fixture is deliberately NOT built from this
crate's own doc comments: those described the order wrongly for a long time,
and the previous version of this test inherited the error from them and
therefore could not detect it.
-62
View File
@@ -1,62 +0,0 @@
# src/mux/ebml.rs
## `fixed_width_vint8`
Encodes `data_size` as the FIXED-WIDTH 8-octet EBML VINT used to back-patch
a master element's size field: the `0x01` VINT_MARKER octet followed by the
56-bit VINT_DATA payload, big-endian (RFC 8794 section 4.4 — an 8-octet
VINT carries 7 octets of VINT_DATA).
Extracted so the full 56-bit payload can be exercised directly: reaching
the high payload bytes through `end_master` / `end_master_buf` would take
a multi-terabyte buffer, leaving them unconstrained by any test.
`data_size` must be below 2^56; both callers check that first.
## `start_master_buf` / `end_master_buf`
This is the seek-free twin of `start_master`/`end_master` for elements
small enough to assemble whole before hitting the file (a BlockGroup: one
Block plus a couple of tiny elements). Because `end_master` always
back-patches a FIXED-WIDTH 8-byte VINT (`0x01` + 7 payload bytes) rather
than a minimal-width one, the bytes produced by this pair are byte-for-byte
identical to the seek-and-back-patch pair — assembling in memory cannot
change the emitted Matroska.
## `read_exact_bounded`
Reads exactly `len` bytes WITHOUT trusting `len` to size the allocation.
`vec![0u8; len]` on an attacker-controlled EBML size would allocate
gigabytes before the read fails. Instead the reader is capped to `len`
and the buffer grows as bytes actually arrive: a malformed element that
claims a huge length but supplies few bytes allocates only what it
delivers, then errors on the short read.
## Test: `buffered_master_matches_seeking_master_byte_for_byte`
The buffer-based master helpers must produce byte-for-byte what the
seek-based ones produce. `write_block_group` was rewritten onto
`start_master_buf`/`end_master_buf` to eliminate ~4 seeks (and 4 MiB
BufWriter flushes) PER FRAME, which on an MPEG-2 title is ~350k frames — a
change only safe because the two encodings are identical.
Both write a FIXED-width 8-byte VINT placeholder (0x01 + 7 payload bytes)
and patch it in place; neither ever emits a minimal-width size. This test
pins that, so a future "optimisation" of either one to minimal-width sizes
cannot silently desync the two and change emitted Matroska.
## Test: `buffered_master_nests_correctly`
Nested masters must patch correctly too — the MVC BlockGroup nests
BlockAdditions > BlockMore inside the BlockGroup, and in-memory patching
works by index rather than by file offset, so nesting is where an
index-arithmetic slip would show up.
## Test: `fixed_width_vint8_is_big_endian_over_the_full_payload`
The fixed-width 8-octet VINT is `0x01` followed by the 56-bit VINT_DATA
payload in BIG-ENDIAN order (RFC 8794 section 4.4). Every payload octet is
distinct here, so a shifted, reversed or dropped octet is visible; the top
payload octets are unreachable through `end_master` without a
multi-terabyte buffer, which is why this is tested at the encoder.
-51
View File
@@ -1,51 +0,0 @@
# `dirimage::encode` — descriptor encoder notes
Overflow rationale for comments in `src/dirimage/encode.rs` that would
otherwise exceed the comment-guard caps. Each section below is pointed to
by a short `//` comment at the relevant spot in the code.
## Why a pure function
`encode` is a pure function from `Layout` to sectors — nothing here touches
the filesystem. That is what makes it testable against the production
parser in `udf.rs`: the same bytes this module writes are read back and
checked by the real UDF reader, so the encoder and the parser can't drift
apart silently.
## Why UDF 1.02 with a single Type-1 partition map
UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
the DVD-Video profile, it is the shape `read_filesystem` takes when
`num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
entirely. That also means a synthetic image never exercises the Metadata
Partition path in `udf.rs` (`:946-991`).
## `finish_tag`: why `tag_loc` correctness matters
Getting `tag_loc` wrong (absolute vs. partition-relative) is the classic
reason a hand-built volume mounts nowhere: a driver that validates the tag
location rejects the descriptor outright. `desc_len` is the descriptor's
total length including the tag; the CRC covers `buf[16..desc_len]`.
## `encode_cs0`: why ASCII, not Latin-1, for the 8-bit form
`parse_udf_name` (`udf.rs:1467`) decodes a compression-8 name with
`from_utf8_lossy`, so a 0x80-0xFF byte — legal CS0 under Latin-1 — would
come back as U+FFFD. Every character above 0x7F therefore takes the 16-bit
(UTF-16BE) form, which that parser decodes correctly.
## `file_entry`: why tag 261, not the Extended File Entry (266)
Real BD-ROMs use the Extended File Entry (tag 266), but an EFE requires
UDF 2.00+, and this image declares 1.02, so `file_entry` writes the plain
File Entry (tag 261) instead. `udf.rs` reads both — the 261 field offsets
it uses (`l_ea` 168, `l_ad` 172, ADs at `176 + l_ea`) are the ones written
here.
## `push_fid`: why FIDs may span logical blocks
FIDs are packed with no inter-descriptor padding beyond the 4-byte
alignment the spec mandates, and they are allowed to span logical blocks —
which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
directory extent as one flat byte run and stops at the first non-257 tag,
so any block-alignment gap would truncate the directory.
-151
View File
@@ -1,151 +0,0 @@
# encrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/encrypt.md — <topic>` pointer
left at the original site.
## HandshakeResult::drive_unlocked
True when the VID came from an unlocker (in `freemkv-unlock`) that
unlocked the drive. Such a drive serves CLEAR content, so AACS bus
encryption is already removed AT THE DRIVE — the same end state a
successful cert handshake's `read_data_key` provides, just via
firmware instead of the AKE. The bus-key gate MUST credit this as a
valid bus-removal: bus encryption is unremovable only when NEITHER the
firmware unlocked the drive NOR the cert handshake yielded a bus key.
Without this, a SUCCESSFUL unlock (VID present, `read_data_key: None`)
paradoxically trips the gate and blocks ALL key resolution (incl. the
online source).
## bus_encryption_removed
Single source of truth for "is AACS bus encryption gone for this
scan?". The gate asks ONLY this — `if !removed { error }` — never
enumerating cases. Bus encryption is gone when ANY of these holds:
the disc never had it; file/ISO reads (content already clear at read
time); an unlocker unlocked the drive (it serves clear content); or
the cert handshake produced the bus key. Add a NEW removal mechanism
HERE, never in the gate.
## AacsCertUnlocker
libfreemkv-side driver for the AACS cert route. It owns the host-cert
collection (a keysource concern that stays in libfreemkv) and then
dispatches the actual mutual-auth to the `freemkv-unlock` AACS
unlocker via the `crate::unlock_bridge`. The firmware (drive-prep) and
CSS routes dispatch the same way at their own call sites; this one
carries the host certs.
## AacsCertUnlocker::authenticate
Runs the host-certificate mutual-auth handshake: collect
non-compiled-in host certs from the key sources + credentials, then
hand them to the AACS unlocker (via the `freemkv-unlock` dispatch),
which tries each cert (wedge-guarded) and on success yields the
Volume ID + `read_data_key` (the AACS 2.0 bus key). Returns a
`CertUnlockFailure` on every no-VID outcome.
## unlock_error_to_error
Maps a `CertUnlockFailure` back to the `Error` variant
`do_handshake_cert` has always surfaced, so `scan_with`'s rendering
and the path-1 disc-hash → VUK fallback are byte-for-byte unchanged.
(`NoHostCert` keeps the `<no host cert>` sentinel.)
## handshake_has_volume_id
Did the cert handshake actually carry a Volume ID? Extracted so it can
be tested as a VALUE. It only ever reaches an operator as the
`has_volume_id` field of the `bus_key_unavailable` warn, and asserting
on a `tracing` field means installing a capturing subscriber — which
is thread-local, while `tracing`'s callsite-interest cache is global.
Those two facts race: the test failed roughly one run in ten under the
full parallel suite while passing every time in isolation, and
serialising the captures was not enough because the cache can be
re-evaluated against the process default rather than the thread-local
dispatch.
A predicate this small does not need a subscriber to verify. The
polarity is the whole point: an `==` here would tell an operator a VID
was absent on exactly the discs where one was present.
## Disc::do_handshake
SCSI handshake — drives the VID-acquisition flow and returns a
structured `HandshakeResult` for downstream key resolution.
VID acquisition runs through `do_handshake_cert`, which first uses the
OEM VID a firmware unlocker may have stashed at drive `init()` (a
drive-functionality capability decoupled from the host cert + HRL) and
falls back to the cert-based mutual-auth handshake (dispatched to the
`freemkv-unlock` AACS unlocker) when none is present. The cert path
also yields `read_data_key`, required for AACS 2.0 bus decryption.
Returns `(handshake, error)`:
- `(Some(_), None)` — VID acquired
- `(None, Some(_))` — specific failure mode (`AacsHostCertRejected` or
`AacsVidUnavailable`)
- `(None, None)` — handshake not attempted (no keydb; resolution will
proceed with VID=zero and rely on path 1 disc-hash → VUK lookup)
## Disc::do_handshake_cert
Cert-based AACS handshake — the cert route for VID acquisition.
Before running the cert mutual-auth, this checks for an OEM Volume ID
a firmware unlocker stashed at drive `init()`. Such an unlocker
unlocks *drive functionality*, not just the disc: VID retrieval via
the drive's OEM CDB is a capability separate from `unlock`. When one
served a VID, we use it and SKIP the cert handshake entirely — the
OEM path gets the VID *without* the host certificate + HRL, decoupling
VID from the cert chain. The OEM path yields no `read_data_key` (no
bus-key is derived); AACS 2.0 content needing read_data_key for bus
decryption must still use the cert path, so an unlocker with no OEM
VID capability returns `None` and we fall through to cert auth
unchanged.
## Disc::collect_host_certs
Collect every AACS host cert the caller carries, from BOTH the
explicit `DriveCredentials` and the key-source layer
(`crate::KeySource::host_certs` across each source), unioned. Host
certs are keysource-served, never compiled in; this is the one place
the OEM cert route gathers them. An empty result is the graceful
no-cert signal (the caller turns it into `Error::AacsNoHostCert`).
`mkb` is the disc's MKB generation when known, forwarded to each
source's `crate::KeySource::host_certs` so a source MAY return only
generation-appropriate certs (the default ignores it).
## Disc::resolve_vid_only
Build a keys-free AACS state that carries only the Volume ID (+
version metadata), for callers that resolve Unit Keys out-of-band and
have disabled the local keydb. The VID is on-disc content read during
the handshake; preserving it here lets the out-of-band path use it. No
keys are present (`unit_keys` empty, `vuk` None), so the disc reports
as "encrypted, no keys" until the caller re-scans with a resolved Unit
Key.
## build_mkb (test helper)
An MKB with one Type-and-Version record (type 0x10) carrying the
version as BE u32 at record offset 8, followed by a recorded EOF
record then trailing zero padding. mkb_content_len walks records and
stops at the first padding (type 0) byte (aacs/inf.rs).
## resolve_vid_only_no_cert_defaults_version_uhd (test)
No content cert at all → version defaults to UHD (major 2), matching
`read_aacs_version` so the scanned `AacsState.version` and the
out-of-band fetch agree on the Unit_Key_RO stride (audit #4: a wrong
BD-vs-UHD guess mis-parses unit keys). bus_encryption false
(unreadable → off).
## Skipped/deferred test coverage
Tests for `read_vid_oem` (response parsing) are skipped in this file
since `Drive` construction requires a live transport; parsing branches
are exercised through `read_vid_oem`'s callers in integration. The
`collect_host_certs` tests union `DriveCredentials` with the
key-source layer; empty means the route fails gracefully
(`AacsNoHostCert`).
-181
View File
@@ -1,181 +0,0 @@
# Error Codes
Overflow detail for `src/error.rs` doc comments that would otherwise exceed
the comment-guard caps. Each section below is pointed to by a short `//` or
`///` comment at the corresponding definition.
## Error Code Ranges
| Range | Category |
|-------|----------|
| E1xxx | Device errors |
| E2xxx | Profile errors |
| E3xxx | Unlock errors |
| E4xxx | SCSI errors |
| E5xxx | I/O errors |
| E6xxx | Disc format errors |
| E7xxx | AACS errors |
| E8xxx | Keydb errors |
| E9xxx | Stream/mux errors |
## E_UDF_NO_USABLE_EXTENT
A file that EXISTS and whose allocation descriptors resolved without error,
yet yields not one usable extent: an empty AD list, or a list every entry of
which is zero-length or points at LBA 0. Reported by the HD-DVD clip resolver
(`disc::hddvd`).
It has no `Error` variant on purpose. `UdfFs::file_extents` returns
`Ok(vec![])` here rather than failing — the emptiness is only a defect in the
eye of a caller that needs bytes — so the condition is DETECTED by the
caller, not returned to it. The code exists so that detection can be
accounted in the log with something other than a neighbouring error's code:
logging it as E6017 would file a zero-length AD list as an authoring hole
and send whoever triages it at the wrong population.
## E_CSS_NO_DISC_KEY / CssNoDiscKey history
While the disc-wide CSS failure and the per-title CSS failure both used
`E_CSS_KEY_MISSING`, an uncrackable CSS disc was iterated title by title,
each one logged as skipped, and the run exited reporting success — a total
failure reported as success. `E_CSS_NO_DISC_KEY` / `Error::CssNoDiscKey` was
split out so `is_disc_level_no_key` classifies it and a multi-title rip loop
fails fast on it instead.
## E_KEY_SERVICE_UNAVAILABLE
A seven-hour run of HTTP 502s reported as `E_NO_DISC_KEY` told operators
their disc was not in the key database and sent them hunting for a VUK that
was never missing; the correct action was to wait. `E_KEY_SERVICE_UNAVAILABLE`
exists so a key-source outage (transport error, DNS failure, timeout, TLS
failure, HTTP 5xx, unreadable reply) is distinguishable from a source that
answered and simply has no key.
## E_MKV_SOURCE_INVALID vs E_MKV_INVALID
`is_skippable_title_stub` classifies `E_MKV_INVALID` as a title that yielded
no muxable frames, which an all-titles rip may skip while finishing the
rest. A corrupt or truncated `mkv://` source file is a FAILURE, not a stub —
reporting it as skippable would let a broken source be silently passed over
by a run that then exits successfully. `E_MKV_SOURCE_INVALID` (and the
sibling codes `E_MUX_HEADER_BUFFER_EXCEEDED`, `E_MKV_LACING_INVALID`) exist
to keep that distinction, and are fatal.
## UdfEmbeddedData
A file that legitimately stores its data this way is tiny (an ICB caps it at
well under 2 KiB — the AACS `*.inf` key files are the usual case), and the
callers that expect one read it via `read_inline_data` long before extents
are ever requested. A stream file cannot be one. `read_directory` already
refuses the same shape for directories; `UdfEmbeddedData` is the file half
of that decision.
## MkvInvalid / MuxHeaderBufferExceeded history
`MuxHeaderBufferExceeded` exists because hundreds of megabytes of real
frames with unresolvable codec-init headers is NOT the same thing as
`MkvInvalid`'s canonical case (an empty nav/menu PGC stub). Reporting the
former as the latter would have `is_skippable_title_stub` silently drop a
main feature from a rip that then exits successfully.
## AacsBusKeyUnavailable
NOT raised for AACS 1.0 BD (no bus encryption, `read_data_key` legitimately
absent) nor for file-backed (ISO) scans, where bus encryption was already
removed at read time and no handshake runs. Only a live-drive AACS 2.0/UHD
scan that obtained the Volume ID but produced no bus key raises this.
## SourceTerminated
The alternative answer is a lie: a dead source that reports `Ok(0)` is
indistinguishable from end-of-stream, and `DiscStream::fill_extents`
legitimately reads a short count as a skippable hole — zero-filling and
advancing over every remaining sector of the title and still returning
success. Unlike a bad sector, this condition cannot be retried at a smaller
size or skipped past.
## error_code / From<Error> for io::Error round-trip
`mux_stream` hands back an `io::Error`, and a front-end reporting *why* a
title failed had no way to recover the code from it — the typed `Error` is
gone by then and only the `E<code>` string prefix survives. Parsing that
prefix is `error_code`'s job; every consumer re-implementing the parse is how
the string-matching this crate spent 1.5.x removing comes back.
`From<Error> for io::Error` boxes the typed value as the payload
(`io::Error::new(kind, e)`), whose `Display` is the same `E<code>[: …]`
string the stringifying version produced — so `error_code`'s parse is
unaffected, and `From<io::Error> for Error` can additionally `downcast` the
payload back to the exact typed error. Errors that did NOT come from this
crate carry no `E<code>` prefix and yield `None`.
## is_skippable_title_stub
This replaces the CLI's `E7023`/`E6008` string-match with a typed check on
the `io::Error` `mux_stream` returns.
`Error::MkvSourceInvalid`, `Error::MkvLacingInvalid`, `Error::MkvUnencodable`
all used to be raised as `Error::MkvInvalid` and therefore landed in the
skippable set, so a corrupt source was reported as a title worth silently
passing over by a run that then exited successfully. They now carry their
own codes and are fatal, as is `Error::MuxHeaderBufferExceeded`.
`Error::CssNoDiscKey` (the disc's CSS crack recovered no key at all) is the
same conflation on the decrypt axis: while it too was raised as
`Error::CssKeyMissing`, every title of an undecryptable disc classified as a
skippable stub, so the rip loop skipped all of them and exited successfully.
It belongs to `is_disc_level_no_key` instead, and is fatal here.
## is_disc_level_no_key
`E_CSS_NO_DISC_KEY` is the CSS side of the disc-level/per-title split, and it
exists because the disc-wide CSS failure used to be raised as the per-title
`E_CSS_KEY_MISSING`: an undecryptable CSS disc landed in
`is_skippable_title_stub`, so the rip loop skipped all N titles with an
"empty stub" notice and exited successfully.
The key-SOURCE failures (`E_KEY_SERVICE_UNAVAILABLE`,
`E_KEY_SERVICE_UNAUTHORIZED`, `E_KEY_SERVICE_RATE_LIMITED`) are classified
here for the same fail-fast reason and NOT because they mean "no key": a
service that is down, refusing the token, or throttling is down for every
title on the disc, so iterating N titles re-issues N doomed requests (and,
on 429, digs the rate-limit hole deeper). They are separate CODES precisely
so the front-end can say "retry later" / "fix the token" instead of
`E_NO_DISC_KEY`'s "no key source has a key for this disc".
## is_marginal_read
- MEDIUM ERROR (sense key 3) — canonical bad-sector signal.
- ABORTED COMMAND (sense key B) — transient; retry usually works.
- NOT READY (sense key 2) — the dominant bad-sector response on the BU40N
(ASC 0x04/ASCQ 0x3E); a pause + retry often recovers.
- RECOVERED ERROR (sense key 1) / NO SENSE (sense key 0) — not classified as
fatal; treat as recoverable.
Caller-agnostic predicate — describes a property of the *error*, not what
one specific call site should do with it. Used by
`freemkv_engine::recovery::copy`'s hysteresis dispatch.
## Test rationale: declared_error_codes / uniqueness
The uniqueness test used to carry a hand-maintained `vec![]` of the error
constants, with a doc comment claiming it "pins all code assignments". It did
not. At the time this was written the file declared 127 `pub const E_*` and
the vector named 109 of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
`E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`, `E_DIR_NAME_COLLISION`,
`E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`, `E_DIR_RAW_REJECTED`,
`E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`, `E_DRIVE_INQUIRY_SHORT`,
`E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`, `E_SEAM_PLAN_DROPPED_MOST`,
`E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`, `E_SOURCE_TERMINATED`,
`E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were outside the guarantee entirely,
so a new variant colliding with any of them passed green. Worse, an earlier
audit READ that doc comment and trusted it while assigning new codes.
The defect is not the eighteen omissions, it is that the list is
hand-maintained at all: adding a constant and forgetting the vector is a
silent no-op, which is the definition of a guarantee that decays. Deriving
the list from the declarations makes forgetting impossible — the only way to
escape the check is to stop declaring the constant. `include_str!` of the
same file is the cheapest seam that does that: no `build.rs`, no proc macro,
no dependency, and — being `#[cfg(test)]` — not one byte of the source
embedded in a release build.
-150
View File
@@ -1,150 +0,0 @@
# `src/disc/extract.rs` — decrypted file-tree extraction
## Relation to the ISO sweep/patch recovery passes
`extract_tree` is a sibling of the disc→ISO sector dump (the sweep/patch
recovery passes, which now live in the `freemkv-engine` crate), specialized
to write **per file** rather than a whole image, applying decryption on the
way out, and **without** any multipass / recovery orchestration — it is
1-shot, decrypt-only.
No new error codes are introduced: it reuses the existing `Error` set
(`error.rs` is numeric-only, no English).
## `extract_tree` contract details
`reader` is consumed for content reads (it is moved into a
`DecryptingSectorSource`); the structure + extent metadata are read first
with the raw reader. `dest` receives the tree STRAIGHT IN (no auto-named
subfolder). The caller must have run the pre-flight decrypt gate
(`Disc::ensure_decryptable`); this method resolves per-VTS CSS keys itself
for DVD.
Bad sectors become recorded zero-filled holes in the affected file (the run
does not abort); undecryptable units are counted as loss. Files are written
`<name>.partial` and renamed on success, so an interrupted run never leaves
a half-written file that looks complete.
## `resolve_vts_key` — why a failed crack must be a hard error
FALLIBLE, because the three outcomes of a crack are not two.
[`crate::css::CrackOutcome`] exists precisely to separate "no scrambled
sector was seen, so there is nothing to decrypt" from "scrambled sectors
were seen and no key came out", and its doc says callers MUST surface the
second as a hard error. An earlier version of this function used the
`Option`-returning `crack_key`, which collapses both into `None`, and then
fell back to the disc-wide key — descrambling this VTS with ANOTHER VTS's
title key. The hazard is described in the ordering note in the function
body, which was written about the same fallback; ordering makes the crack
far more likely to succeed, but it cannot make a failed crack safe.
## `Borrowed` — why a borrowing `SectorSource` wrapper
Lets the decrypting decorator "own" an inner source for its lifetime while
the caller keeps the underlying `&mut dyn SectorSource` (the decorator is a
`DecryptingSectorSource<S>` generic over `S`, so it does NOT require a
`'static` boxed inner — unlike the mux highway, which takes the reader by
value). The decorator is dropped before `extract_tree` returns, so the
borrow never escapes.
## `whole_unit_batch` — the trailing-partial rule
`remaining` is the sectors left in the current extent; the batch is capped
at `READ_BATCH_SECTORS` and rounded DOWN to a whole number of 3-sector units
UNLESS it is the extent's final (possibly short) tail — the tail always
begins on a unit boundary, so a 1-2 sector partial there is handled by
`decrypt_sectors`' trailing-partial contract. Used by `extract_one_file` so
this rounding rule lives in exactly one place.
## `available_space` (windows) — why the free-space gate matters
Windows has no `statvfs`. This once returned `None` unconditionally, which
does not merely skip a test — it skipped the free-space GATE, so a Windows
user extracting a disc to a full volume got a confusing failure part-way
through instead of a clear refusal up front. libfreemkv ships a Windows
GUI, so that is the platform where the friendly error matters most.
`FreeBytesAvailableToCaller` (not `TotalNumberOfFreeBytes`) is used because
it accounts for per-user quotas, matching what `statvfs`'s `f_bavail` gives
on the unix side.
## `sanitize_component` — why reserved names are substituted, not rejected
A name whose base matches a Windows reserved device name (`NUL`, `CON`,
`COM1`..) is **substituted** (prefixed with `_`) rather than rejected — a
Linux-authored disc may legally carry such a file and a single reserved
name must not abort the whole tree walk. An empty result after stripping
is an error.
## `names_differing_only_by_case_are_a_collision` — why this must be an error
Two disc names differing only by CASE are one host file on macOS APFS and
Windows NTFS, both case-insensitive by default. Keyed by the case-preserving
path, the collision map would see two entries, raise nothing, and the
second file extracted would overwrite the first — while both `PlannedFile`s
report `complete: true`. Silent data loss reported as a clean extract is
the one outcome this crate must never produce, so the host-equivalence key
has to model the host's namespace, not the disc's.
## `a_files_partial_path_colliding_with_another_files_final_name_is_an_error`
A disc carrying both `X` and `X.partial` plans two distinct final names, so
nothing collides in the final-name check — but extracting `X` writes
through `X.partial`, the same host path the other file owns. Whichever
lands second truncates the other, and both entries would still be reported
complete without this guard.
## `multi_extent_aacs_anchors_unit_base_per_extent` — the alignment bug
Regression (rc.6 audit, finding #449): a MULTI-EXTENT AACS file must
re-anchor the unit-alignment base PER extent, not once at the first extent.
The two extents in this test start at absolute LBAs whose difference is NOT
a multiple of 3 (`PART_START+5000` vs `PART_START+5004` — a 4-sector delta).
With a single first-extent unit base the second extent's first read is off
the unit grid (offset 4, `4 % 3 == 1`) → `is_unit_aligned` fails →
`DecryptFailed` → the whole extent becomes a zero-filled (false) hole. With
per-extent anchoring both extents read clean: `bytes_unreadable == 0` and
the file content survives. The fixture content is clear (no TS syncs) so
the decrypt step restores each unit verbatim — this isolates the GATE from
the cipher math.
## `extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it`
An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED: the
space belongs to the file and occupies its byte range, but nothing was ever
written there, and the standard defines its contents as zeros.
`read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
`read_file_limited` honours it by emitting zeros WITHOUT touching the
media. The tree extractor reads the same ICBs and must agree: reading those
sectors returns whatever the media happens to hold there — on an AACS disc,
ciphertext that decrypts to noise — and writes it into the extracted file
as if the disc had recorded it.
## `per_extent_base_is_aligned_first_extent_base_is_not`
Focused alignment-computation check underpinning the per-extent fix: when
each extent anchors its OWN start as the unit base, the extent's own batch
starts are always unit-aligned; anchoring a later extent against the FIRST
extent's base mis-aligns whenever the extents' starts differ by a
non-multiple of 3 sectors. This is the exact arithmetic the decrypt-on-read
gate (`aacs::content::is_unit_aligned`) performs.
## `css_two_vts_groups_do_not_cross_contaminate_keys`
Regression: the per-VTS key crack in `resolve_vts_key` must gather ONLY
this VTS's own title-VOB extents. Two VTS groups here carry DISTINCT
scrambling keys; if the group filter (`vts_group_of(..) == Some(vts) &&
is_title_vob(..)`) is loosened (`!=`, or `&&` -> `||`), one VTS's resolve
gathers the OTHER VTS's extents, cracks the wrong key, and that VTS's own
content silently fails to descramble to the expected plaintext.
## `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`
A VTS that IS scrambled but whose key could not be recovered must FAIL, not
borrow another VTS's key. `crack_key` returns `Option`, which collapses "no
scrambled sector was seen" with "scrambled sectors were seen and no key
came out"; the fallback then descrambled this VTS under the disc-wide key
and `extract_tree` reported `complete = true` at exit 0, because no read
had failed. `CrackOutcome` exists to keep those two apart, and every
sibling path in the crate already honours it. The fixture: VTS_01 is
crackable, VTS_02 is scrambled with NO periodic crib, so its crack
genuinely fails.
-36
View File
@@ -1,36 +0,0 @@
# `file_sector_source::linux` — read-side platform hints
## Why both `hint_sequential` and `drop_window`
`POSIX_FADV_SEQUENTIAL` at open widens the kernel's readahead window so
each pread aggregates into fewer NFS round-trips. `DONTNEED` on the
consumed window (called periodically by the caller) drops the
already-read pages from the page cache so an 85 GB streaming ISO read
doesn't fill memory and starve concurrent writes (the MKV output
during mux). Together they mirror the write-side WritebackPipeline's
policy.
## History
Pre-Phase-1 (0.20.7 baseline) had both. Phase 1's introduction of
`FileSectorSource` silently dropped the read-side DONTNEED, and
0.21.2's revert of `SEQUENTIAL` (mistakenly attributing a regression
to it) removed the hint. Net effect: 85 GB of ISO reads pinned in the
page cache + no readahead widening → mux throughput collapse from
18 MB/s historical to 2.7-8 MB/s on 0.21.x. Restored in 0.21.6.
## `prefetch`
Async-prefetch `len` bytes at `offset` into the page cache. The kernel
`readahead(2)` syscall queues the I/O and returns immediately — it
does NOT wait for completion. Called right after each consumed read
so the next batch's I/O overlaps with the caller's processing of the
current batch (decrypt + demux + mux).
Without this hint, with a synchronous demux consumer running at
~50 MB/s and a single-spindle disk capable of ~150 MB/s, the disk sits
idle ~70% of each iteration because kernel readahead alone (capped at
`/sys/block/<dev>/queue/read_ahead_kb`, default 128 KB) can only
pre-stage a tiny slice of the next batch. An explicit `readahead()` of
the same size as the current batch tells the kernel to queue the full
next-batch read now.
-29
View File
@@ -1,29 +0,0 @@
# file_sector_source::macos
macOS has no direct `POSIX_FADV_SEQUENTIAL` equivalent; the idiomatic hint is
`fcntl(F_RDADVISE, &radvisory)` describing the byte range you intend to read
soon. The open-time sequential hint points it at the whole file (clamped to
`RDADVISE_MAX_BYTES` so a multi-TB ISO doesn't ask the kernel to prefetch
everything at once).
## RDADVISE_MAX_BYTES
Cap on the byte length passed to `F_RDADVISE`. Asking for a multi-GB
readahead window is counterproductive — the OS doesn't have that much cache
to throw at one fd. 64 MiB is generous for our use case (sweep, mux) so the
kernel's prefetch keeps pace with our app-level pipeline depth.
## drop_window
macOS has no direct `POSIX_FADV_DONTNEED` equivalent for a byte range.
`fcntl(F_NOCACHE)` would disable caching globally on the fd (too coarse — we
want the unread region to still benefit). Best approximation: no-op.
macOS's unified buffer cache is generally less prone to the pin-everything
pathology that triggers the regression on Linux NFS clients.
## prefetch
Async-prefetch the byte range `[offset, offset+len)`. macOS uses the same
`fcntl(F_RDADVISE, &radvisory)` primitive as the open-time sequential hint,
just targeted at a moving window instead of the whole file. The kernel
queues I/O for the requested range and returns immediately.
-77
View File
@@ -1,77 +0,0 @@
# `FileSectorSource` design rationale
`FileSectorSource` reads 2048-byte sectors from an ISO file on disk via
direct `seek + read_exact` (`pread`-equivalent) calls, letting the kernel's
own readahead policy manage prefetch.
## Why no app-level buffer
Pre-0.21.3 this source held a 32 MiB (later 4 MiB) read-ahead buffer to
amortise per-sector NFS round-trips. Empirically that buffer hurt: 32 MiB
refills bursted the NFS TCP connection hard enough to starve the concurrent
writer, and even a 4 MiB window gave the kernel less freedom to pipeline
reads with writes. Direct pread per call lets Linux's readahead widen as it
detects the sequential pattern, and naturally interleaves with writeback.
## DONTNEED on the consumed window
Without page-cache eviction an 85 GB streaming ISO read pins the entire
file in memory, starves the concurrent writer, and collapses mux throughput
(observed: 2.7 MB/s mux on 0.21.5 vs. 70 MB/s isolated NFS reads). Every
`READ_DROP_CHUNK_BYTES_DEFAULT` of consumed bytes we call
`posix_fadvise(DONTNEED)` over that window, mirroring the write-side
`crate::io::writeback::WritebackPipeline` policy.
The drop window is accounted by a monotonic forward byte counter, which
matches the sequential streaming pattern the mux highway drives. Under
random or backward access the dropped range no longer lines up with the
bytes actually read — but `DONTNEED` is purely an advisory cache hint with
no correctness impact, so this degrades to a slightly imprecise hint rather
than a bug.
`READ_DROP_CHUNK_BYTES_DEFAULT` (32 MiB) is the empirically tuned value on
a 7200rpm HDD via SATA: smaller windows (8 / 16 MiB) shorten the
kernel-readahead overlap and slow the producer; larger windows (64 / 128
MiB) let the page cache pin enough of the ISO to pressure concurrent
writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
`READ_DROP_CHUNK_MIB_MAX` (64 GiB) bounds `FREEMKV_READ_DROP_CHUNK_MIB`
before the MiB→byte multiply, mirroring `WRITEBACK_CHUNK_MIB_MAX`, whose
identical multiply is bounded for exactly this reason: without the bound,
a value above 2^44 overflows — a panic on the first ISO open in an
overflow-checked build, and in release a wrap to a near-zero window that
fires `drop_window` on every read. Out-of-range values fall back to the
default.
## Platform open hint
On `open()` each platform issues its "sequential access expected" hint so
OS-level readahead widens. The hint and the DONTNEED call live in per-OS
sibling modules (`linux::hint_sequential` et al.) — no inline `#[cfg]` in
this file.
## Read-ahead prefetch
After every consumed read we issue an OS-level prefetch hint for the next
equivalent-sized window (`platform::prefetch`). The kernel queues that I/O
asynchronously and returns immediately, so the next batch's read overlaps
with the caller's processing of the current batch (decrypt + demux + mux).
Without this the disk sits idle ~70% of each iteration because kernel
SEQUENTIAL readahead alone (capped at `read_ahead_kb`, default 128 KB) is
far smaller than our 16 MiB app-level batch.
## Test notes
- `read_sectors_with_an_undersized_buffer_errors_rather_than_panicking`:
this is a public `SectorSource` impl, so buffer length is caller input,
and the guard used to be a `debug_assert!` — compiled out in release,
where the `out[..bytes]` slice then panicked with 'range end index out
of range'. `Drive::read_fua` already carries this exact guard with a
comment recording the same panic being fixed there, and
`PrefetchedSectorSource` has `direct_read_too_small_buffer_errors` for
the same case; this impl had neither.
- `dontneed_eviction_does_not_affect_data`: reads past the default 32 MiB
drop chunk (16384 sectors) so the eviction block fires at least once,
asserting every sector still reads correctly. Avoids mutating
`FREEMKV_READ_DROP_CHUNK_MIB` to sidestep a parallel-test env race with
`drop_chunk_size_env_override`.
-16
View File
@@ -1,16 +0,0 @@
# FLAC elementary-stream decodability gate (`src/mux/codec/flac.rs`)
FLAC frames carry no length field, so a raw stream is delimited only by
sync-scanning + CRC validation. In freemkv, though, FLAC never arrives raw:
it comes from mp4/mkv, where each packet is exactly one container-delimited
FLAC frame (a complete, pre-delimited frame per packet). So this parser
is a per-packet gate, not a framer: every FLAC frame ends with a 16-bit CRC
(poly 0x8005, init 0, non-reflected) computed so the residue over the whole
frame — footer CRC included — is zero (per the FLAC format specification,
RFC 9639, frame footer). A nonzero residue is definitive corruption → drop
the frame (a silence gap, never a shift — each packet keeps its own PTS),
logged via the shared tally.
A packet that does not begin with the FLAC frame sync is not a delimited
frame we can validate, so it is passed through unchanged (never
false-dropped).
-36
View File
@@ -1,36 +0,0 @@
# fMP4 muxer (`src/mux/fmp4/mod.rs`) — stub status and design notes
## Status
**STUB**. The muxer can emit the init segment (`ftyp` + a minimal HEVC
`moov` skeleton with one video track) via
[`Fmp4Mux::write_init_segment`], so the shape and call site are
validated, but media fragments (`moof`/`mdat`) are NOT emitted.
[`Fmp4Mux::write_video`] therefore returns
[`Error::Fmp4Unimplemented`](crate::error::Error::Fmp4Unimplemented)
rather than silently accepting and discarding frames. It buffers
nothing, so it cannot accumulate memory.
## Not yet implemented
- `moof` box: `mfhd` (sequence_number) + `traf` (`tfhd` + `tfdt`
+ `trun` with sample sizes, durations, flags, composition offsets).
- `mdat` box: concatenated sample data.
- Fragment cadence: one fragment per GOP or every N seconds,
whichever comes first.
- HEVC `hvcC` box inside `moov.trak.mdia.minf.stbl.stsd` so the
init segment is self-describing (`stsd` currently has zero entries).
- Sample-flags computation (sync vs. delta, depends_on, etc.).
- Edit lists / fragment_duration for accurate seeking.
Reference: ISO/IEC 14496-12 §8 (Movie Fragments).
## `wrap_box` size-cast rationale
All callers build tiny init-segment boxes (kilobytes at most), so the
`u32` size never overflows; the saturating cast plus the debug assert
documents and guards that invariant rather than silently emitting a
truncated, structurally corrupt size field. `body` is always
internally constructed here, never untrusted input — a future caller
feeding a multi-gigabyte body trips the debug assert instead of
writing a malformed box.
-70
View File
@@ -1,70 +0,0 @@
# Fox — `dcx.xml` label parser
Older Fox authoring (e.g. a real Fox release, `/BDMV/JAR/05001/dcx.xml`) ships a
human-readable XML manifest alongside the BD-J jar. Its root is `<dcx>`, and
under `<disc>` it lists every playlist the disc plays. The main-feature
playlists carry nested per-stream `<audio>`/`<subtitle>` elements naming
language, editorial purpose and forced/SDH state outright — everything a
label parser wants, in attributes, with no bytecode to walk.
NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this file
is one authoring house's internal metadata that happens to press onto the
disc. Every field meaning below was read off a real disc; treat an
unfamiliar value as unknown rather than guessing.
## Confirmed schema
A real Fox release, `/BDMV/JAR/05001/dcx.xml`:
```xml
<dcx>
<disc>
<playlist id="00001" lang="eng" name="topmenu"/>
...
<playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="eng" type="rnib"/>
<audio id="03" lang="spa" dial="lat" type="feature"/>
...
<subtitle id="01" lang="eng" type="feature" form="sdh"/>
<subtitle id="02" lang="spa" dial="lat" type="embed"/>
...
<subtitle id="11" lang="eng" type="text"/>
<properties> ...chapter marks... </properties>
</playlist>
<playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628"> ... </playlist>
</disc>
</dcx>
```
## Field meanings (read off the disc, not documented)
* `<audio type>`: `feature` = a normal program track; `rnib` = a
descriptive/narration track ("Royal National Institute of Blind People"
described-video), mapped to `LabelPurpose::Descriptive`. A `type`
containing "comment" maps to `LabelPurpose::Commentary`.
* `<subtitle form>`: `sdh` marks a subtitles-for-the-deaf-and-hard-of-
hearing track (`LabelQualifier::Sdh`).
* `<subtitle type>`: `embed` marks a dedicated forced/embedded-subtitle
track (`LabelQualifier::Forced`); `feature`/`text` are full tracks.
* `id` on a nested stream is its 1-based STN slot WITHIN ITS TYPE
(audio ids 01..N, subtitle ids 01..N independently), matching the vendor
`stream_number` convention the ordinal binder in `super::apply_labels`
reads.
The `<audio>`/`<subtitle>` elements are NESTED inside one feature playlist,
so extraction is scoped to a single `<playlist name="feature">` element —
never a document-wide `<audio>` scan, which would merge the regional
`00800` (eng) and `00801` (jpn) tables into one and collide their slots.
## `select_feature_playlist` scoping rationale
Among all `<playlist name="feature">` elements — Fox presses one per
regional variant (`00800` eng, `00801` jpn) — the one carrying the most
`<audio>`/`<subtitle>` streams wins, ties to the first.
Returning ONE element is the whole point: each feature playlist is its own
STN table with its own 1-based slot ids, so merging two would put two
different streams on slot `01`. The richest table is the fullest label set
and anchors to its matching title by language sequence in
`super::apply_labels`.
-13
View File
@@ -1,13 +0,0 @@
# Linux `f_type` classification
`classify_f_type` in `src/platform/fs_type/linux.rs` is the single source of
truth for the magic-number comparisons used by both the path-based
(`detect_impl`) and fd-based (`detect_fd_impl`) entry points.
## Why the `i64` cast
`statfs::f_type` is signed `__fsword_t` on glibc and unsigned `c_ulong` on
musl; a portable comparison needs a common type, so the value is cast to
`i64`. On glibc x86_64 both are already `i64` — clippy flags the cast as
unnecessary on that target only, but the cast is required for musl, hence
the `#[allow(clippy::unnecessary_cast)]`.
-12
View File
@@ -1,12 +0,0 @@
# fs_type module
The buffering architecture (Phase 2) selects different output sinks
for local vs network filesystems: NFS gets the adaptive
`WritebackFile` machinery on Linux; local disks get `LocalFileSink`
and rely on the kernel's default writeback policy. This module
provides the construction-site primitive that picks which one.
Per the per-OS file-split convention, the actual `statfs` call lives
in the matching platform file (`linux.rs`, `macos.rs`, `windows.rs`,
`other.rs`); this `mod.rs` exposes only the cross-platform enum and
the `detect` entry point.
-20
View File
@@ -1,20 +0,0 @@
# `io::fsync` — platform-aware crash-durability primitives
Two flush operations need OS-specific handling to make a write survive a
crash / power loss:
- **`dir`** — fsync a directory so a prior `rename(2)` into it is durable.
After a crash a renamed file's dirent can otherwise be lost even though
the rename returned, because it is still page-cache-only. This is a POSIX
concept: on Windows std cannot even open a directory as a `File` (it does
not set `FILE_FLAG_BACKUP_SEMANTICS`), and NTFS/ReFS commit the rename's
dirent without an explicit directory flush — so it is a no-op there
rather than a failed open that logs on every marker write.
- **`file_durable`** — fsync a file's contents + metadata. Opens the file
**read+write**: on Windows `File::sync_all` maps to `FlushFileBuffers`,
which requires a handle with write access and returns
`ERROR_ACCESS_DENIED` (os error 5) on a read-only handle. (A read-only
`File::open` + `sync_all` is legal on POSIX, which is why that bug only
bit Windows.) The open mode is platform-uniform, so this lives here with
no dispatch.
-37
View File
@@ -1,37 +0,0 @@
# `fvi://` sink notes
Rationale and design notes for `src/mux/fvi_sink.rs` that don't belong in the
public rustdoc contract.
## Relationship to `VideoMap`
The sink is a thin consumer of the reusable, pure-data
[`VideoMap`](crate::mux::videomap) model
([`MapHeader`](crate::mux::videomap::MapHeader) /
[`PictureRecord`](crate::mux::videomap::PictureRecord)): it builds the header
from the title, then writes one record per video
[`PesFrame`](crate::pes::PesFrame) straight to disk. Nothing here re-parses
the elementary stream, and the whole index is never buffered.
Serialization is inlined in the sink itself.
## Extensibility
A different output format would be a DIFFERENT sink reusing the same
`VideoMap` model (e.g. a future `fvi2://`), not a pluggable encoder —
extensibility is by adding a sink, like every other sink in this crate.
## `write_fvi_record`: coding-derived members (§7.1)
Coding-derived members (§7.1) are produced via the codec-agnostic
`PictureInfo` accessors, never the raw bitstream: `field_order` is emitted
only when measured (OMITTED on codec-type-only HEVC/H.264/VC-1),
`progressive` only when signalled.
## `write_fvi_record`: `dts` / future `recovered` member
`dts` is MAY per the format spec — the highway carries no DTS on a frame, so
it is always omitted.
TODO(provenance→recovery join): a `recovered` MAY member (sweep/patch mapfile
overlap with this AU's `src`) belongs here; unreachable at `PesFrame`.
-189
View File
@@ -1,189 +0,0 @@
# H.264 codec parser notes
## Slice-type mapping
`h264_slice_coding_type` maps an H.264 `slice_type` (Rec. ITU-T H.264 §7.4.3,
Table 7-6) to a coding type. Values 5..=9 repeat 0..=4 (the "all slices of
this type" forms), so `slice_type % 5`: 0 = P, 1 = B, 2 = I, 3 = SP
(predicted → P), 4 = SI (intra → I). Returns `None` for values outside 0..=9
(malformed header).
## `unescape_ebsp_prefix`
Copies the leading bytes of an EBSP with emulation-prevention bytes removed
(ISO/IEC 14496-10 §7.3.1: a 0x03 following 0x00 0x00 is not part of the
RBSP). Only a short prefix is produced, because the sole caller decodes two
ue(v) fields from the very start of a slice header. `first_mb_in_slice` and
`slice_type` are each at most 32 bits, so 16 octets is ample and keeps this
off the per-frame allocation path for anything larger.
## `unescape_ebsp`
Copies `ebsp` with emulation-prevention bytes removed, stopping after at most
`max_octets` output bytes (pass `ebsp.len()` for "no cap").
The zero run-length counter is cumulative across the whole scan and is reset
only when an escape byte is actually dropped — matching the reference
decoding process in ITU-T H.264 §7.3.1, which discards a 0x03 following ANY
run of two-or-more 0x00 bytes, not just an exact `00 00 03` sliding window. A
window-based scanner that advances by 3 on a match and by 1 otherwise
disagrees with this on a run of 3+ real zero bytes ahead of an 0x03:
`00 00 00 03` decodes here as `00 00 00` (the run is escaped, matching every
other H.264 decoder), where a window scan starting fresh after each
non-match would keep the 0x03 as real payload. Only one implementation of
this rule may exist in this module — see `parse_sps_high_profile_ext`, which
used to keep its own window-scan copy that disagreed with this one on
exactly that input.
## `handle_param_set`
Handles an SPS/PPS NAL (mirrors the HEVC fix). The strip/emit decision is
made against the currently-ACTIVE set `cur`, NOT the codecPrivate copy
`first`: a streaming MKV decoder applies avcC once at init and thereafter
updates a parameter set only from an in-band NAL, so a switch BACK to the
first-seen body (== codecPrivate) is still a change the decoder must be told
about. Stripping on `== first` silently dropped that revert.
- First of its type → seeds codecPrivate; stripped (decoder gets it from avcC).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Unconditional (not only when the active set differs from codecPrivate): a
streaming decoder applies the avcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IDRs
even though its body is unchanged; if the decoder then drops it (a reset
event), nothing re-sends it and every subsequent slice fails (param-set id
out of range) until the next genuine change. Re-asserting at EVERY keyframe
— what compliant muxers do at every IDR — makes streaming decode
self-healing. Re-sending an identical param set is benign; cost is a few
bytes per keyframe. This strictly supersets the change-only re-assert, so
the param-set-revert fix is unaffected.
## `parse_sps_high_profile_ext`
Parses `(chroma_format_idc, bit_depth_luma_minus8, bit_depth_chroma_minus8)`
from a High-Profile SPS NAL (profile_idc ∈ `HIGH_PROFILES` — the 14
chroma/bit-depth extended profiles `codec_private` invokes this for).
SPS RBSP layout (ITU-T H.264 §7.3.2.1.1) up to the fields we need:
```
byte 0 NAL header (already known to be type 7)
byte 1 profile_idc
byte 2 constraint_set_flags / reserved
byte 3 level_idc
ue(v) seq_parameter_set_id
— High-profile branch —
ue(v) chroma_format_idc
if chroma_format_idc == 3: u(1) separate_colour_plane_flag
ue(v) bit_depth_luma_minus8
ue(v) bit_depth_chroma_minus8
```
RBSP emulation-prevention bytes (0x00 0x00 0x03 → 0x00 0x00) are removed
before bit-parsing so the bit reader sees clean RBSP data.
Returns `None` if the SPS is too short or malformed (Exp-Golomb code
overflows 32 bits, leading-zero count > 31, etc.). The caller silently omits
the extension in that case.
## Test notes
### `open_gop_intra_access_unit_is_a_keyframe`
Open-GOP resync fix: a non-IDR intra-coded access unit MUST be flagged a
keyframe so the B1 resync gate (`mux/resync.rs`) can disarm on a BD
open-GOP tail instead of dropping every frame to EOF and truncating the
video. BD H.264 titles reach random access at a non-IDR I-frame (the
recovery point); HEVC gets this free from the CRA NAL type, H.264 does not,
so the measured intra coding type is the signal. Red before the intra
promotion in `parse` — a non-IDR I-slice used to yield `keyframe == false`.
### `inter_coded_access_unit_is_not_a_keyframe`
Guards the promotion's precision: a non-intra (P) access unit must NOT be
promoted to a keyframe, or the resync gate would resume on a frame that
carries inter-references to dropped data and the MKV cue index would gain
bogus random-access points.
### `slice_header_parse_removes_emulation_prevention_bytes`
The slice-header parse must remove emulation-prevention bytes first
(ISO/IEC 14496-10 §7.3.1/§7.4.1): the payload after the NAL header is EBSP,
and a 0x03 following 0x00 0x00 is not part of the RBSP.
Reachable when `first_mb_in_slice` is 65535 or more — a ue(v) that needs
sixteen leading zero bits, so the payload opens 0x00 0x00 and an encoder
must escape it. A UHD frame is ~32,400 macroblocks so conforming Blu-ray
never gets there, but 8K does, and the disc is untrusted input. Reading the
raw NAL decoded `slice_type` against the inserted byte and produced the
wrong picture type.
### `unescape_ebsp_drops_escape_after_a_run_of_three_zeros`
Regression: `parse_sps_high_profile_ext` used to re-derive the
emulation-prevention rule with its own window scanner (match `00 00 03` at
position i, advance by 3; else advance by 1) instead of calling the shared
`unescape_ebsp`. On a run of 3+ real 0x00 bytes ahead of an 0x03 —
non-conformant, but this is untrusted disc input, not a spec-clean encoder —
the two disagreed: `unescape_ebsp`'s cumulative zero counter (the rule the
H.264 reference decoding process in §7.3.1 specifies) drops the 0x03 as an
escape, while a fresh 3-byte window starting right after the non-matching
first byte kept it as real payload. Pin the shared function's behaviour
here so a second hand-rolled copy doesn't quietly reappear.
### `keyframe_param_reassert_emits_exact_bytes`
The keyframe SPS/PPS re-assert must produce EXACTLY these bytes: the active
SPS, then the active PPS, then the access unit's own NALs, each as a 4-byte
big-endian length prefix followed by the NAL body (ISO/IEC 14496-15
length-prefixed form, lengthSizeMinusOne = 3).
This pins the full byte string, not just its length, so the in-place
`splice` that replaced the build-a-new-buffer-and-copy approach is proven
byte-for-byte equivalent — including the ORDER (parameter sets ahead of the
slices, which is what makes the keyframe self-contained). The literals were
captured from the pre-splice implementation's output.
### `keyframe_param_reassert_does_not_reallocate_the_frame`
MEASURED: the keyframe parameter-set re-assert must be spliced into the
front of the already-assembled access unit IN PLACE, not built as a fresh
full-size buffer.
It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-hundred-KB allocation — memcpy'd the
whole frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a UHD title is
~200,000 frames of 150-400 KB with a keyframe every 1-2 s, i.e. thousands of
avoidable multi-hundred-KB copies per title, each large enough to go through
mmap. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates; this
counts the reallocations that happen, which must be zero. Mirrors the HEVC
parser's test of the same name.
### `avcc_profile_244_appends_extension_bytes`
ISO 14496-15 §5.3.3.1.2 regression: profile_idc=244 (High 4:4:4 Predictive)
ALSO mandates the chroma/bit-depth extension. It was missing from
HIGH_PROFILES, so a 244 stream took the Baseline/Main path and emitted an
avcC with NO extension bytes — non-conforming, and strict parsers then
assume 8-bit 4:2:0. The extension must be appended.
### `sps_reader_read_ue_thirty_one_leading_zeros_is_still_valid`
`SpsReader::read_ue`'s truncation guard is `leading_zeros > 31`: 31 leading
zero bits is the longest legal code and must decode, not abort. Mirrors the
equivalent guard in the shared `BitReader` in `startcode.rs`, kept here
because `SpsReader` is a separate, unshared implementation used only for SPS
parsing.
-16
View File
@@ -1,16 +0,0 @@
# halt
## Why `Ordering::Relaxed`
`Ordering::Relaxed` is sufficient on both load and store because this flag
is purely advisory — no other memory operations piggyback on it for
happens-before ordering. Callers that need to publish data across threads
do so via channels or other synchronization, not via this bit.
## Why 250ms for `POLL_INTERVAL`
250 ms is the sweet spot between responsiveness (operator presses Stop,
sees it take effect within ~quarter-second) and waste (atomic load + clock
read is cheap but not free at thousands of hertz). It's centralised here
so the half-dozen halt-polling loops across `io` can't drift apart
silently.
-35
View File
@@ -1,35 +0,0 @@
# Seeded robustness harness (`src/harness.rs`)
Every parser reached from here takes bytes that came off a disc, and this
crate's primary boundary is that the disc is untrusted: a malformed, damaged
or hostile image must never crash the library. These tests assert exactly
that one property — the parser returns `Ok` or `Err`, and never panics.
## Why this exists rather than `cargo-fuzz`
`cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
for libFuzzer's coverage feedback) and this project pins stable. So the
generator lives here instead. It gives up coverage-guided mutation — the real
loss — and keeps everything else: millions of cases, structure-aware input,
and a crash corpus. It also gains determinism, which a fuzzer does not have:
the same seed replays the same cases on any machine.
## Why no `proptest` or `arbitrary`
This crate has exactly one dev-dependency. That is a deliberate posture, and
a randomness crate is not worth ten transitive dependencies when the parsers
take plain `&[u8]` and a good enough generator is forty lines.
## Budget
`FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
enough that the per-commit gate stays under a second). The overnight run sets
it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
fixed so a failure in CI reproduces locally verbatim.
## On failure
The panic message carries the seed, generator and case index. Re-run with
`FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
into `tests/corpus/` as a permanent regression fixture — discovery happens
here, defence happens there.
-465
View File
@@ -1,465 +0,0 @@
# HD-DVD title scanning (`src/disc/hddvd.rs`)
HD-DVD is a **tree-level peer** of DVD and Blu-ray (not a stream variant like
FMTS): its content lives in `HVDVD_TS/` as `.evo` clips — Enhanced VOB, an
MPEG **program** stream — each with a small `.map` timemap sidecar, navigated
by `.xpl`/`.ifo` playlists in `HVDVD_TS/` and `ADV_OBJ/`. Because it is a
different tree with a different playlist format, it gets its OWN scanner
(this file), a peer to `Disc::scan_bluray_titles` — the two-format design
rule: a genuinely different format is a new enumerator, not an extension
bolted into the BD path.
## Title composition (authoritative path)
Title composition is authoritative, from the Advanced-Content playlist. HD-DVD
ships a real player playlist at `ADV_OBJ/VPLST000.XPL` (DVD-Forum
`HDDVDVideo/Playlist` XML). The scanner parses it (with a real XML parser,
`roxmltree`) into one `DiscTitle` per `<Title>`: its `<PrimaryAudioVideoClip>`
clips in playback order (each an EVO, referenced via its `.MAP` sidecar), the
`titleDuration`, the `displayName`, and the `<ChapterList>`. A layer-break
split (`FEATURE_1` + `FEATURE_2`, or `feature`/`feature_Divide`) is composed
into ONE title with the two parts as clips, carrying each clip's title-time
in/out points (45 kHz ticks) so a seamless join can be spliced onto one
timeline. Container is `ContentFormat::MpegPs`, so the existing PS mux path
handles it. Per-clip streams are enumerated by demuxing the clip head and
building one `Stream` per distinct elementary stream (video + DD+ audio),
codec sniffed from the ES bytes.
## Fallback path
When no playlist is present (or it fails to parse), the scanner falls back to
the older clip-name heuristic: parse the `HVA*.VTI` clip table, join the
`feature*`-named clips into one title, and emit every other clip on its own.
## Known gaps
Not parsed yet: subtitles (8-bit RLC on `0xBD` sub `0x20..=0x3F`) and per-track
audio languages (the XPL carries `<Audio description=...>` but they are not yet
wired onto the streams). Extents and size are real (the ripper images the
clips).
## VTI parsing
`parse_vti_clip_order` parses the `ADVANCED-VTS` VTI clip-name table. The
table is a run of `VTI_CLIP_ENTRY_STRIDE`-spaced records, each carrying a
NUL-terminated `<name>.EVO`. Rather than trust the (imprecise) header pointer,
it collects every NUL-terminated `*.EVO` name and keeps the largest group
sharing one residue modulo the stride — the clip table — in offset order.
Returns empty for a non-VTI blob or one with no recognizable table.
## Codec sniffing
`sniff_video_codec` reads MPEG / Annex-B start codes: `00 00 01 B3` → MPEG-2
(sequence_header), `00 00 01 0F` → VC-1 (BD/HD-DVD sequence-header BDU),
`00 00 01 [x7]` H.264 SPS NAL (type 7, forbidden_zero_bit clear) → H.264.
Returns `None` when no recognizable start code is present. The scan prefers
the unambiguous MPEG-2 / VC-1 sequence headers; H.264 is inferred from an SPS
NAL so a stray slice/picture code can't be mistaken for a different codec.
`sniff_audio_codec` recognizes only Dolby Digital Plus (E-AC-3) today, via its
`0x0B77` syncword — what real retail HD-DVD titles carry on sub-ids
`0xC0..=0xC7`. Returns `None` for an unrecognized sample so the caller drops
the stream rather than mislabeling it.
## probe_evo_streams
Demuxes the head of an `.evo` clip (through the disc's `SectorSource`) and
builds one `Stream` per distinct elementary stream found: the video track
(mapped to the canonical `DVD_VIDEO_PID` for plain 0xE0-0xEF video, or the
extended-stream-id PID for HD-DVD VC-1) and every DD+ audio sub-stream
(mapped via `dvd_audio_pid`). Codec is sniffed from the demuxed ES bytes.
Mirrors the stream construction in `Disc::scan_dvd_titles`; resolution /
language / channels use sane HD-DVD defaults (the muxer reads the true pixel
dimensions from the H.264 SPS, and E-AC-3 channel counts are not decoded
here). Returns an empty vec when the clip cannot be read or carries no
recognizable stream (e.g. an AACS-encrypted clip probed as ciphertext).
Cancellation is checked before every chunk read and is the ONE condition that
returns `Err` rather than an empty vec: a probe cut short by a Stop has not
established that the clip carries no streams, and reporting it as if it had
would enumerate a stream-less title as a scanned fact. A read that fails for
any other reason keeps the existing best-effort behaviour.
## collect_es
VC-1 video rides extended-stream-id `0xFD` with `stream_id_extension` `0x55`.
HD audio (MLP/TrueHD) can also use `0xFD` with other extensions — routing
those to their own audio tracks is deferred (see the HD-DVD program-chain
follow-up); until then only the VC-1 extension is treated as video, so an
audio `0xFD` sub-stream can never mis-stamp the video PID.
## XPL amplification bounds
The XPL parser guards against a crafted `ADV_OBJ/VPLST000.XPL` amplifying a
small file into an enormous amount of work or memory. Four separate axes are
bounded, each independently, because they are separate attacks:
- **`MAX_XPL_DEPTH` (32)** bounds element NESTING. A real `VPLST000.XPL` is
about six levels deep at its deepest
(`Playlist`/`TitleSet`/`Title`/`PrimaryAudioVideoClip`/`Video`, or
`.../ChapterList/Chapter`); 32 leaves a 5x margin for authoring tools that
wrap extra `ApplicationSegment`/`ObjectMappingList` layers, while staying
trivially stack-safe.
- **`MAX_XPL_TITLES` (512)** bounds how many `<Title>` elements one playlist
may declare — BREADTH, separate from the depth guard: a flat playlist
passes the depth check trivially and can still declare a title per 51
bytes, so a 64 MiB `VPLST000.XPL` (exactly `udf::MAX_FILE_BYTES`) reaches
~1.3 million of them. This matters here and not on the Blu-ray side because
of where the count comes from: `bluray.rs` learns its playlists from the
MPLS files actually present in the UDF directory, so the medium bounds it —
an XPL DECLARES its titles, so nothing does. The cost is drive time, not
memory: `compose_xpl_titles` runs `probe_evo_streams` once per title, and
each probe reads real sectors, so an uncapped count turns a `Disc::scan()`
the operator expects to take seconds into tens of terabytes of optical
reads on a disc that is only pretending to be large. The bound that matters
is `cap x EVO_PROBE_SECTORS`, since `compose_xpl_titles` memoizes on the
title's RESOLVED extent list, and distinct titles built from different
combinations of the same clips have different lists, so each one misses the
memo — at 4096 that product was 64 GiB of optical reads, the same worst
case, and the same contradiction with "seconds", as the sibling
`MAX_HDDVD_CLIPS` path. 512 keeps it at 8 GiB and remains far above any
real disc (retail HD-DVDs carry tens of titles).
- **`MAX_HDDVD_CLIPS` (512)** caps `.evo` clips taken from the `HVDVD_TS/`
directory listing. `MAX_XPL_TITLES` and `xpl_depth_within_limit` bound the
`ADV_OBJ/*.XPL` path, but a crafted disc simply omits `/ADV_OBJ`; control
then reaches the clip-name fallback in `Disc::scan_hddvd_titles`, which
emits one title per directory entry and probes each — the same unbounded-
probe cost, reached without the playlist. The directory is the only bound
the medium supplies, and `udf::MAX_DIR_BYTES` (1 MiB) still leaves room for
~24,000 FIDs. Memoizing the probe (`EvoProbeCache`) collapses entries that
resolve to the SAME extents, closing the many-names-one-File-Entry case —
but not this one: a File Entry is a single sector, so an attacker can give
every FID its own, each declaring its own extent, for ~48 MiB of image;
every probe then misses the memo and costs a full `EVO_PROBE_SECTORS`
(16 MiB) read, hundreds of GiB on a scan the operator expects to take
seconds. The cap's job is to bound `cap x EVO_PROBE_SECTORS`; it was 4096
(matching `MAX_XPL_TITLES`, which bounds a different, no-per-item-read-cost
path) — 4096 x 16 MiB is 64 GiB, more than dual-layer HD-DVD media
physically holds (reachable anyway, since distinct extent lists may overlap
on the medium). 512 keeps the product at 8 GiB and still leaves a wide
margin over any real disc (retail HD-DVDs carry TENS of `.evo` clips, ~10x
the real-world maximum, the sizing convention `MAX_XPL_CHAPTERS_PER_TITLE`
also uses against the 99-chapter authoring ceiling). Pinned by
`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`, which
asserts the PRODUCT rather than either constant, so raising one without
re-examining the other fails.
- **`MAX_XPL_CLIPS_PER_TITLE` (256)** bounds how many
`<PrimaryAudioVideoClip>` elements a SINGLE `<Title>` may contribute — the
fourth amplification axis, the one the other three guards leave open. A
64 MiB XPL holds ~1.7 million `<PrimaryAudioVideoClip src="A.EVO"/>`
elements at ~38 bytes each, and the collector is `descendants()`, not
`children()`, so a clip nested inside N ancestor `<Title>` elements is
collected by EVERY one of them — the depth cap still permits 32 such
ancestors (measured: a bare `<Title>` root nests 32 deep and still parses,
because the guard counts the self-closing clip element as non-nesting).
The product is ~1.7M x 32 ≈ 56 million heap-allocated `XplClip`s, each with
its own `String`, and then as many `Clip`s again in `compose_xpl_titles`.
Unlike the title cap, the cost here is MEMORY rather than drive time:
`compose_xpl_titles` de-duplicates by `.evo` name before probing
(`seen_evos`), so repeats cost no extra sector reads, but the `Vec`s are
built and held regardless. 256 is far above any real disc — a feature is
one clip, or two across a layer break; even seamless-branching/multi-angle
is tens — and keeps the aggregate bounded at `MAX_XPL_TITLES` x 256 clips
rather than the tens of millions above.
- **`MAX_XPL_CHAPTERS_PER_TITLE` (1024)** bounds `<Chapter>` elements per
`<Title>` — the same `descendants()` amplification, on the second unbounded
`collect()` in the same loop, bounded separately so capping clips doesn't
simply move the attack next door. A chapter is a bare `f64` rather than a
`String`-carrying struct so it's cheaper, but a
`<Chapter titleTimeBegin="…"/>` element is also smaller, so the element
count an XPL can reach is comparable. 1024 is far above any real disc: the
DVD/HD-DVD authoring convention tops out at 99 chapters per title, a 10x
margin.
## scan_hddvd_titles
Scans HD-DVD titles from the `HVDVD_TS/` `.evo` clips. The main feature is
authored as one or more `.evo` clips (a layer-break split —
`FEATURE_1`/`FEATURE_2` or `feature`/`feature_Divide`). The `HVA*.VTI`
navigation file names every clip in authored order; this parses it to
concatenate the feature clips into ONE title (so the largest-title pick gets
the whole movie, not just part 1), emitting every other clip as its own
title. Falls back to one title per clip when the VTI is absent or
unparseable, so a disc with no readable navigation still enumerates.
`chapters`/duration are left empty pending deeper VTI parsing.
Cancellation: `halt` is polled once per clip and again before every probe
chunk, and a read that fails with `Error::Halted` — how a live drive reports
a Stop, since `Drive::checked_exec` fails every command once its flag is set
— is propagated rather than swallowed. `Halted` is the only error this
returns; every other read failure keeps its best-effort behaviour. It has to
be an error and not a short title list: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it.
## xpl_depth_within_limit
Rejects a disc-supplied XML blob whose element nesting exceeds
`MAX_XPL_DEPTH`, BEFORE it reaches the parser. `roxmltree` is
recursive-descent and its depth-10 limit applies only to ENTITY expansion, so
element nesting is unbounded: a few hundred KB of well-formed XML (far under
the read cap) overflows the thread stack. That is a process ABORT, not an
`Err` and not an unwind, so neither the `Document::parse` fallback nor
`catch_unwind` can contain it; the only fix is to not hand the document over
at all. A byte-size cap is deliberately NOT added: file size is already
bounded by the UDF read path, and a size cap does not close this class (a
180 KB document already aborts) — depth is the bound that matters.
This is a conservative scan, not a validator: it tracks `<name …>` /
`</name>` while skipping comments, CDATA, processing instructions and
declarations, and treats `<… />` as non-nesting. A miscount can only cost a
pathological document its fast path, and the failure mode is the existing
one — fall back to the clip-name heuristic.
## Test rationale
Detailed rationale for tests in `src/disc/hddvd.rs`'s `#[cfg(test)]` module,
kept here so the in-code doc can stay within the internal-comment cap.
**`scan_hddvd_feature_composition_saturates_absurd_disc_declared_sizes`** —
The SECOND size accumulator. `compose_xpl_titles` sums an XPL's clip sizes;
this one sums the composed FEATURE title's parts, and it reads the same
disc-declared `u64` Information Length from the same `clip_extents` map. A
split feature whose two parts each declare a size near `u64::MAX` overflows
it exactly as the XPL path did. Nothing reconciles the declared size against
the extents that back it — here each part declares `u64::MAX` while
occupying ten sectors — because the allocation descriptor's length field is
only 32 bits wide, so the disc can always claim more than it holds.
Mutation: restore `size_bytes += *size` in the feature-composition path and
this goes red (debug: attempt to add with overflow).
**`scan_hddvd_does_not_compose_a_feature_over_an_unrecorded_part`** — A split
feature (FEATURE_1 + FEATURE_2) where ONE part carries an unrecorded extent
must not be composed into a FEATURE title at all. Composing around the
unusable part is the dangerous outcome: the title would play through as if
it were the whole movie while silently missing that part's runtime, and its
size still counts it. The other clips remain available as their own titles
— refusing the composition is not the same as refusing the disc.
**`scan_hddvd_logs_an_unrecorded_feature_part_with_its_own_code`** — The
refusal above must also be ACCOUNTED, with the unrecorded extent's own code.
This warn site carried a hardcoded `code = 6017` literal — in the very file
whose sibling arm was changed to stop doing exactly that — and nothing
tested it, so neither the literal nor its absence broke anything. Refusing a
feature and saying nothing leaves the operator with a disc that scanned
"fine" and is quietly missing its main title. Mutations: deleting the
`tracing::warn!` (no event to find); restoring the literal `6017` still
passes here BY VALUE, which is the point — what is pinned is
`E_UDF_UNRECORDED_EXTENT`'s value reaching the log, so a renumbering that
the literal could not follow goes red; logging a neighbouring code (e.g.
`E_UDF_NO_USABLE_EXTENT`) goes red immediately.
**`scan_hddvd_does_not_compose_a_feature_over_a_part_with_no_usable_extent`**
— The `Ok`-but-empty twin of the test above, and the one that actually bites
on an ordinary disc: a feature part whose `file_extents` call SUCCEEDS and
yields nothing usable — here a zero-byte `FEATURE_2.EVO`, no crafted ICB
required — must refuse the composition too, and must say so in the log.
Before this, the empty-`Ok` clip entered neither `clip_extents` nor
`unusable`, so the `any(|n| unusable.contains(..))` guard missed it and the
`filter(|n| clip_extents.contains_key(..))` beside it quietly DELETED the
part: the scan composed a `FEATURE` title out of part one alone, still
advertising itself as the feature, at rc=0, with no diagnostic anywhere.
Half a movie presented as a whole one — strictly worse than the `Err` case
the guard was built for, and reachable without a hostile disc. Mutations
this catches, all of which the pre-fix code exhibited: dropping the
`extents.is_empty()` branch entirely (a `FEATURE` title appears with one
extent); inserting into `clip_extents` anyway (same); marking the clip
unusable but NOT logging (the log assertion fails — absence of a log is
itself the defect); logging the neighbouring `E_UDF_UNRECORDED_EXTENT`
instead of `E_UDF_NO_USABLE_EXTENT` (the code assertion fails, because a
zero-length file is not an authoring hole and triaging the two together
sends whoever reads it at the wrong population).
**`parse_xpl_titles_caps_a_playlist_declaring_absurdly_many_titles`** — A
disc-supplied playlist nested far deeper than any real one must be REFUSED
BEFORE the XML parser sees it. `roxmltree` is recursive-descent and its
depth-10 limit covers only ENTITY expansion, so unbounded element nesting
blows the thread stack — an ABORT, which no `catch_unwind` and no `Err`
fallback can contain. ~50k levels is a few hundred KB of well-formed XML,
far under the read cap. The correct outcome is the same as any other
unusable playlist: an empty `Vec`, so the caller falls back to the clip-name
heuristic. Separately, this test also exercises BREADTH, not depth: a flat
playlist passes the nesting guard trivially and can still declare a title
per ~51 bytes, so a 64 MiB XPL (exactly `udf::MAX_FILE_BYTES`) reaches ~1.3
million of them. The cost is drive time, not memory: `compose_xpl_titles`
runs `probe_evo_streams` once per title and each probe reads real sectors,
so an uncapped count turns a scan the operator expects to take seconds into
tens of terabytes of optical reads. `bluray.rs` is not exposed to this
because its playlist count comes from the MPLS files present in the UDF
directory; an XPL DECLARES its titles, so nothing bounds them. Mutation:
delete the `.take(MAX_XPL_TITLES)` and this goes red at the declared count.
**`nested_title_xpl`** — Builds a playlist that nests `nesting` `<Title>`
elements inside one another and puts `clips` self-closing
`<PrimaryAudioVideoClip>` elements at the innermost level, plus `chapters`
`<Chapter>` elements. `parse_xpl_titles` collects a title's clips with
`descendants()`, not `children()`, so EVERY one of the `nesting` ancestors
collects the SAME innermost clip list — the amplification this exercises.
**`parse_xpl_titles_caps_clips_per_title_against_descendant_amplification`**
— CLIPS PER TITLE is the axis the title cap, the depth cap and the
directory cap all leave open. `MAX_XPL_TITLES` bounds how many titles a
playlist may declare and `MAX_XPL_DEPTH` bounds how deeply it may nest, but
NEITHER bounds how many `<PrimaryAudioVideoClip>` elements a single title
collects. Worse, the collector is `descendants()`, so one clip nested inside
N ancestor `<Title>`s is collected N times over — the clip count MULTIPLIES
by the nesting the depth guard still permits. A 64 MiB XPL
(`udf::MAX_FILE_BYTES`) holds ~1.7M clip elements at ~38 bytes each; with
the 32 `<Title>` ancestors the depth cap still permits, that is ~56 million
heap-allocated `XplClip`s, and then as many `Clip`s again in
`compose_xpl_titles`. This test uses small numbers that exercise the same
multiplication without a 64 MiB fixture: pre-fix it collected 756 clips per
title, 18,900 across the 25 nested titles, from a document declaring only
756 clip elements — an exact 25x, the nesting count. Mutation: delete the
`clips.len() >= MAX_XPL_CLIPS_PER_TITLE` break and this goes red at the
unamplified per-title count.
**`parse_xpl_titles_caps_chapters_per_title`** — CHAPTERS PER TITLE is the
same `descendants()` amplification on the second unbounded `collect()` in
the same loop. Bounded separately so closing the clip axis does not simply
move the attack next door. Mutation: delete the
`.take(MAX_XPL_CHAPTERS_PER_TITLE)` and this goes red at the declared count.
**`parse_xpl_titles_keeps_every_clip_of_a_realistic_title`** — The control: a
realistic multi-clip title still resolves EVERY one of its clips and
chapters. `SYNTH_XPL`'s main movie is a layer-break split — two clips on one
timeline — plus two chapters; losing either to the cap would cost a genuine
disc half its feature. Mutation: set `MAX_XPL_CLIPS_PER_TITLE` to 1 and this
goes red.
**`parse_xpl_titles_accepts_real_world_nesting_depth`** — The depth guard
must not reject real discs. A genuine `VPLST000.XPL` is a handful of levels
deep; this is the same document the parsing test uses, plus self-closing
tags, a comment and a processing instruction that the pre-parse scanner has
to account for without inflating its depth count.
**`scan_hddvd_composes_titles_from_xpl_playlist`** — The authoritative path:
when a VPLST000.XPL is present, titles come from the playlist — the
layer-break split (FEATURE_1 + FEATURE_2) is composed into ONE title with
the real duration, name, chapters, and per-clip title-time offsets — not the
clip-name heuristic.
**`compose_xpl_titles_saturates_absurd_disc_declared_clip_sizes`** — A
title's `size_bytes` is a running sum of DISC-DECLARED clip sizes: the `u64`
UDF File Entry Information Length, which nothing cross-checks against the
extent list that file actually occupies. Two clips can therefore each
declare a size near `u64::MAX`. Summed with a plain `+=`, that overflows: a
panic in a debug build — a crash on untrusted disc content, which this
library must never do — and a wrap to a small garbage total in release,
which misreports the title's size while looking entirely normal. Mutation:
restore `size_bytes += *size` in `compose_xpl_titles` and this goes red
(debug: attempt to add with overflow).
**`compose_xpl_titles_dedups_repeated_clip_references_by_evo`** — A crafted
Advanced-Content playlist can name the SAME `.evo` in
`<PrimaryAudioVideoClip>` many times over (the XML has no fixed element
count, unlike MPLS's binary PlayItem count). Each repeat must NOT push
another copy of that clip's extent list onto the title — mirroring
bluray.rs's `first_ref = seen_clips.insert(...)` gate, which pushes a clip's
extents only the first time its id is seen. Here the analogous key is the
`.evo` filename (this file has no clip_id; `.evo` is what `clip_extents` is
keyed by). Without the gate, extents grows by the referenced clip's full
extent list on EVERY repetition — unbounded by the number of on-disc files,
bounded only by playlist size.
**`scan_hddvd_titles_drops_a_split_feature_whose_part_cannot_be_read`** — A
clip whose extents cannot be resolved must drop the SPLIT FEATURE. Catches
reverting the `Err(e) => { warn; unusable.insert(..) }` arm in
`Disc::scan_hddvd_titles` back to the bare `Err(_) => {}` it replaced. RED
BEFORE GREEN, and TWO earlier attempts at this test did NOT go red. The
first asserted "no title names the broken clip" — that passes either way,
because a clip that resolves to no extents is never inserted into
`clip_extents` and so yields no per-clip title regardless. The second fixed
that but shipped no `.vti`: `order` is built solely from
`parse_vti_clip_order` of the navigation file, so with no `.vti` it is
EMPTY, `feature` is empty, and no composed title is ever built — the
assertion held vacuously with the fix reverted. Hence the synthetic
`HVA00001.VTI` fixture: it is what makes the composer run at all. The
defect lives one level up from the per-clip titles, in the composed
feature: `unusable` is what tells the composer that a part is MISSING
rather than merely absent, and the old bare `Err(_) => {}` populated it for
nothing but an unrecorded extent. So a scratched sector under FEATURE_2's
ICB (`Error::DiscRead`) left FEATURE_1 composing a title named "FEATURE" by
itself — half a movie offered as the whole one.
**`scan_hddvd_titles_excludes_a_clip_with_zero_sectors`** — A clip whose
file has a zero-byte size (a degenerate/empty allocation: its ICB's
allocation descriptor has `data_len == 0`, the UDF AD-list terminator, so
`file_extents` yields no extent at all) must not produce a title. NOTE: this
exercises the *upstream* `data_len == 0` terminator path in
`crate::udf::UdfFs::file_extents`, not the `sectors > 0 && lba > 0` guard in
`scan_hddvd_titles` itself — with this fixture (`file_extents` never returns
a `(lba, 0)` tuple, and `PART_START` unconditionally makes every resolved
`lba` positive) that guard is unreachable in a divergent way; kept here as a
regression check on the zero-byte-file behavior in its own right.
**`scan_hddvd_titles_probes_once_for_entries_resolving_to_identical_extents`**
— THE THIRD AMPLIFICATION AXIS on HD-DVD title composition, past both
existing guards. `xpl_depth_within_limit` bounds XML NESTING;
`MAX_XPL_TITLES` + the `.evo` de-dup bound what an XPL can DECLARE. Both
live on the playlist path — and an attacker simply omits `/ADV_OBJ`, so
`read_adv_obj_xpl` returns `None` and neither guard is ever consulted.
Control then reaches the clip-name FALLBACK, which emits one title per
`.evo` DIRECTORY ENTRY and probes each. Nothing de-duplicates a FID's ICB
LBA, so every name in a 1 MiB directory (`udf::MAX_DIR_BYTES`, ~24,000 FIDs)
can point at ONE File Entry. Each resolves to the SAME extents and each
costs a full `EVO_PROBE_SECTORS` (16 MiB) probe: ~375 GiB of optical reads
from a directory that describes a single file. Memoizing the probe on the
resolved extent list collapses that to ONE probe, because all of those
names resolve to one extent list. Mutation: drop the memo lookup in
`EvoProbeCache::streams` (always probe) and this goes red at `ENTRIES`
probes.
**`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`** —
The clip cap and the per-probe read budget are ONE bound, not two, and only
their product is meaningful: `MAX_HDDVD_CLIPS` probes each cost up to
`EVO_PROBE_SECTORS`, so the pair fixes the worst-case drive time of a
`Disc::scan()`. `EvoProbeCache` does not help against a crafted disc here:
it keys on the RESOLVED extent list, and distinct-but-overlapping extent
lists all reading one physical region miss the memo every time while every
read succeeds. So the product is genuinely reachable, and it is the number
that has to stay within what a scan can absorb. 8 GiB is the ceiling
asserted here. It is far above any genuine disc — retail HD-DVDs carry tens
of `.evo` clips, i.e. well under 1 GiB of probing — and far below the 64 GiB
the pair used to permit, which exceeded the capacity of the dual-layer media
this format tops out at and took tens of minutes of optical reads. This is
a relationship, not a magic number: raising EITHER constant without
re-examining the other trips it.
**`scan_hddvd_titles_still_probes_each_distinct_clip`** — CONTROL:
memoization must not silently collapse DISTINCT clips. A legitimate disc
with several different `.evo` files still probes each one, and each title
keeps the streams of ITS OWN clip. Mutation: make `EvoProbeCache`'s key a
constant (ignore the extents) and this goes red — the junk clip inherits
the first clip's H.264 streams.
**`scan_hddvd_titles_caps_a_directory_declaring_absurdly_many_clips`** —
Memoization alone is NOT the whole fix. Collapsing identical extent lists
bounds the one-File-Entry attack, but distinct extent lists are only as
bounded as the File Entries an attacker cares to lay down — and a File
Entry is ONE sector. A 1 MiB directory's ~24,000 FIDs can each point at
their own 1-sector File Entry (a ~48 MiB image) declaring its own huge
extent, so every probe misses the memo and the amplification is back.
`MAX_HDDVD_CLIPS` is the bound that closes that. Mutation: drop the
`clips.len() < MAX_HDDVD_CLIPS` gate in the directory scan and this goes
red at `ENTRIES`.
## EvoProbeCache
`EvoProbeCache` memoizes `probe_evo_streams`, keyed on the RESOLVED extent
list. Probing is the expensive half of title composition: each pass reads up
to `EVO_PROBE_SECTORS` (16 MiB) off the medium. Both composition paths can
reach the same physical clip many times over — an XPL naming one `.evo` from
many `<Title>`s, or a directory whose FIDs all point at one File Entry — and
a probe is a pure function of the extents it reads, so the second and later
passes over an identical extent list are re-reads of bytes already seen. The
key is the extent list itself, NOT the clip name or title: two names for one
File Entry share a key (one probe), while two genuinely different clips have
different `start_lba`s and so keep their own probes and streams. Cache size
is bounded by the caller's title/clip cap (`MAX_XPL_TITLES` /
`MAX_HDDVD_CLIPS`).
-99
View File
@@ -1,99 +0,0 @@
# HEVC Annex B muxer (`src/mux/hevc/mod.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## Module overview
Writes a raw `.hevc` / `.h265` Annex B byte stream:
`00 00 00 01 | NAL_unit | 00 00 00 01 | NAL_unit | …` with no container
framing. On the first frame the muxer emits the codec_private's VPS, SPS, PPS
(parsed from a `HEVCDecoderConfigurationRecord` in `length-prefixed-in-hvcC`
form), then converts each PES frame's length-prefixed NAL units to Annex B
and writes them.
## `write_frame`
Input may be either:
- Length-prefixed: `[u32-BE len][NAL bytes]` repeated. This is the form
emitted by libfreemkv's HEVC parser (the MKV-native layout). Converted to
Annex B.
- Already Annex B: a buffer beginning with a `00 00 00 01` or `00 00 01`
start code. Passed through unchanged.
## `hvcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §8.3.3.1.2):
- 22-byte fixed header
- byte 22 = `numOfArrays`
- each array: `array_completeness:1 | reserved:1 | NAL_unit_type:6`,
`numNalus:u16-BE`, then `numNalus` × `(nalUnitLength:u16-BE + NAL bytes)`.
We don't filter on NAL type — VPS (32), SPS (33), PPS (34), and any SEI
arrays included in hvcC all get the same Annex B treatment.
## `length_prefixed_to_annex_b`
Already-Annex-B input (a buffer beginning with a `00 00 00 01` or `00 00 01`
start code) is detected up front and passed through unchanged — some
upstream paths (raw HEVC ES from disc) hand Annex B straight through the
PES layer, and a genuine start code would otherwise be misread as a u32-BE
length prefix.
Truncation policy (single source of truth across all muxers): if a length
prefix runs past the end of the buffer (e.g. a NAL truncated by a bad disc
sector), the truncated trailing NAL is dropped and only the valid Annex-B
prefix accumulated so far is emitted. We never emit a half-NAL nor leak raw
length-prefixed bytes into the Annex-B stream.
## `nal_length_size`
Number of octets each NAL length prefix occupies in the elementary data of a
track described by `record` — the `lengthSizeMinusOne + 1` field of the
decoder configuration record (ISO/IEC 14496-15).
- avcC (`AVCDecoderConfigurationRecord`, §5.3.3.1.2): byte 4 is
`bit(6) reserved | unsigned int(2) lengthSizeMinusOne`.
- hvcC (`HEVCDecoderConfigurationRecord`, §8.3.3.1.2): byte 21 is
`constantFrameRate(2) | numTemporalLayers(3) | temporalIdNested(1) |
lengthSizeMinusOne(2)`.
The spec permits only 1, 2 or 4 octets (`lengthSizeMinusOne` of 0, 1 or 3);
a declared 3 is non-conformant but is decoded rather than rejected, since
reading N octets is the same operation for every N. A record too short to
carry the field, or a codec with no such record, falls back to
`DEFAULT_NAL_LENGTH_SIZE` — the width every freemkv parser emits.
This exists because assuming 4 is silent corruption for a legal source:
reading a 2-octet-prefixed frame as u32-BE yields an absurd first length,
the conversion loop bails with nothing parsed, and the raw length-prefixed
bytes are passed through as though they were already Annex B — a stream
with no start codes at all, and no error anywhere.
## `append_length_prefixed_as_annex_b`
Same conversion as `length_prefixed_to_annex_b` but writes directly into a
caller-owned buffer, avoiding an intermediate allocation on hot paths (e.g.
per-frame video muxing). If `data` doesn't parse as length-prefixed (no
NALs extracted), it's appended unchanged on the assumption it's already
Annex B.
## `append_length_prefixed_as_annex_b_sized`
For a source whose NAL length prefixes are `length_size` octets wide rather
than the 4 this crate's own parsers emit. Derive `length_size` from the
track's configuration record with `nal_length_size` — ISO/IEC 14496-15 lets
a legal avcC/hvcC declare 1 or 2 octet prefixes, and reading those as
u32-BE mangles the frame.
`length_size` outside `1..=4` is clamped to `DEFAULT_NAL_LENGTH_SIZE`; the
field it comes from is 2 bits wide, so that is unreachable from real input.
## `avcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §5.3.3.1.2):
- 5-byte fixed header
- byte 5 = `[reserved:3 | numOfSequenceParameterSets:5]`
- `numOfSPS` × `(sequenceParameterSetLength:u16-BE + SPS bytes)`
- 1 byte = `numOfPictureParameterSets`
- `numOfPPS` × `(pictureParameterSetLength:u16-BE + PPS bytes)`
-283
View File
@@ -1,283 +0,0 @@
# HEVC parser (`src/mux/codec/hevc.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## `hevc_num_extra_slice_header_bits`
`num_extra_slice_header_bits` from a HEVC PPS NAL (H.265 §7.3.2.3): after the
2-byte NAL header, skip `pps_pic_parameter_set_id` + `pps_seq_parameter_set_id`
(both `ue(v)`) and `dependent_slice_segments_enabled_flag` +
`output_flag_present_flag` (`u(1)` each), then read `u(3)`. `None` if the PPS
is too short to parse — the caller then declines to guess a slice type.
## `hevc_first_slice_coding_type`
Measures the coding type from the FIRST coded slice of an access unit
(H.265 §7.3.6.1 `slice_segment_header`). Reads only the leading fields of the
first slice segment: `first_slice_segment_in_pic_flag` u(1), the IRAP
`no_output_of_prior_pics_flag` u(1), `slice_pic_parameter_set_id` ue(v), the
`num_extra_slice_header_bits` reserved bits, then `slice_type` ue(v). Returns
`None` for a non-first slice or on truncation — never a guess. `num_extra`
MUST come from the active PPS so the bit offset to `slice_type` is exact.
## `HevcParser::hdr10`
Combines the accumulated mastering-display and content-light SEI into a
complete `Hdr10Metadata`, or `None` until BOTH HDR10 SEI messages have been
seen. Requiring both means an SDR / partially-signalled stream never emits a
half-populated (confidently-wrong) HDR10 record.
## `HevcParser::scan_sei`
Scans an SEI NAL (`[2-byte NAL header][RBSP]`) for the two HDR10 payload
types and captures each the FIRST time it appears (per-stream constants).
RBSP structure (Rec. ITU-T H.265 D.2 `sei_rbsp` / `sei_message`): a sequence
of messages, each `payloadType` then `payloadSize` encoded as a run of 0xFF
bytes plus a final <0xFF byte (the "ff-extension" coding), followed by
`payloadSize` payload bytes. Emulation-prevention (00 00 03) is stripped
before reading — unlike a slice header, an SEI payload can be deep enough
that an emulation byte falls inside the fields we read. Unknown payload
types are skipped by their size so a later HDR10 message in the same NAL is
still reached.
## `HevcParser::mark_clip_boundary`
MPLS `connection_condition` 0x05 and 0x06 are the non-seamless values (per
the BD-ROM spec: 0x01 = first item / seamless, 0x05/0x06 = non-seamless).
The first CRA at/after this call is rewritten CRA_NUT (21) → BLA_W_LP (16)
so a linear decoder sets NoRaslOutput and discards the now-dangling RASL
leading pictures with no "could not find ref" error.
It is a no-op for the rewrite unless a CRA actually follows: an
IDR/IDR_W_RADL boundary needs no fix (it carries no cross-splice
references), and the flag is cleared by the first IRAP-class NAL it reaches.
SAFETY: never call this for connection_condition 0x01 (seamless/first item)
or within a single-clip title — doing so could convert a legitimate
mid-content CRA to BLA. The default (never called) path leaves output
byte-identical.
## `handle_param_set`
Handles a VPS/SPS/PPS NAL: decides whether to strip it (the decoder already
has the value) or emit it in-band, and tracks the currently-active body.
The decision MUST be made against the currently-active set (`cur`), NOT the
codecPrivate copy (`first`). The two player behaviours for hvcC-in-MKV
diverge exactly here:
- A *seek-capable / Annex-B* player (one that converts hvcC to Annex-B by
inserting the parameter sets) re-applies the hvcC sets at every keyframe.
`reassert_active` handles it.
- A *streaming* decode (a decoder consuming the MKV directly — what most
integrity checkers do) applies hvcC ONCE at init and thereafter updates a
parameter set ONLY from an in-band NAL.
So when a title redefines a set mid-stream (id 0 body A → B) and later
switches BACK to A (== codecPrivate), the change to A must STILL be emitted
in-band: the streaming decoder is sitting on B and will never revert
otherwise, decoding the whole A-segment against B → CABAC/cu_qp_delta
desync. Stripping on `== first` (the old behaviour) dropped exactly that
revert and corrupted every "switch back to the first body" segment.
Rules:
- First of its type → seeds codecPrivate; stripped (the decoder gets it from
hvcC at init).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change, in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active VPS/SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Why unconditional (not only when the active set differs from codecPrivate):
a streaming decoder applies the hvcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IRAPs
even though its body is unchanged; if the decoder then drops it (a CRA reset
or SPS event), nothing re-sends it and every subsequent slice fails with
"PPS id out of range" until the next genuine change (observed as a ~24 min
corrupt band on one dual-layer UHD title). Re-asserting the active set at
EVERY keyframe — what compliant Matroska muxers do at every IRAP — makes
streaming decode self-healing. Re-sending an identical param set is benign
(decoders expect it at IRAPs); cost is a few hundred bytes per keyframe.
This strictly supersets the earlier change-only re-assert, so the
param-set-revert fix is unaffected.
## length-prefix self-check (issue #52)
`length_prefix_tiles` walks the assembled `frame_data` ([u32-BE len][body]
records) and confirms they EXACTLY tile it: every declared length fits, none is
zero, and the last body ends precisely at the buffer end with no trailing
bytes. `frame_data` is length-prefixed by construction (`push_length_prefixed`
/ `handle_param_set` write a 4-byte BE prefix — `LENGTH_PREFIX_SIZE`, which MUST
equal the hvcC `lengthSizeMinusOne + 1`), so a `false` means a framing desync a
downstream demuxer would report as "Invalid NAL unit size (N>M)".
`HevcParser::parse` runs this as defense in depth: if the self-check fails it
logs `src` at WARN and DROPS the access unit (`return Vec::new()`) rather than
emit a mis-framed frame. Because no real input can desync a
built-by-construction buffer, the drop branch is exercised end-to-end via a
test-only seam (`FORCE_FRAMING_DESYNC`) that appends a stray byte just before
the guard; see `parse_drops_desynced_access_unit`. Disconnecting the guard from
`parse` fails that test.
## `parse_mastering_display`
Parses a Mastering Display Colour Volume SEI payload (Rec. ITU-T H.265
D.2.28 / semantics D.3.28). Layout — 24 bytes total, all big-endian:
`display_primaries_x[c]` u(16), `display_primaries_y[c]` u(16) for c=0,1,2
(SEI primary order is c=0 Green, c=1 Blue, c=2 Red); white_point_x u(16),
white_point_y u(16); max_display_mastering_luminance u(32);
min_display_mastering_luminance u(32). Returns `None` if the payload is
shorter than 24 bytes (malformed → ignored, never partially populated).
## Test notes
**`hevc_hdr10_sei_keeps_the_first_value_and_ignores_later_repeats`** — the
`scan_sei` match arms are guarded with `self.sei_mastering.is_none()` /
`self.sei_content_light.is_none()`, so the FIRST mastering-display and
content-light SEI a title carries wins and later repeats (every AU of a real
HDR10 stream repeats both) are ignored. Each AU below carries only ONE of
the two messages, so the whole-scan early return
(`if self.sei_mastering.is_some() && self.sei_content_light.is_some()`)
never fires early and the per-arm guards are actually exercised.
**`sei_nal` (test helper)** — wraps one or more SEI messages in a prefix-SEI
NAL (type 39) preceded by an Annex-B start code. The assembled message bytes
are emulation-prevented (as a conforming encoder would) so they never form a
false start code; the 0x80 RBSP trailing-bits byte is appended.
**`scan_sei_stops_copying_once_both_hdr10_messages_are_captured`** — MEASURED,
not reasoned: an HDR10 stream carries a prefix SEI per access unit, and
`scan_sei` used to allocate + byte-copy the whole SEI RBSP through
`strip_emulation_prevention` on EVERY one, including after both HDR10
messages were already captured. On a ~200,000-frame UHD title that is
~200,000 wasted allocations/copies. Counted at the single
`strip_emulation_prevention` call site.
**`nonzero_num_extra_slice_header_bits_shifts_the_slice_type_offset`** —
`num_extra_slice_header_bits` (H.265 §7.3.2.3) is a PPS field, and the
`slice_reserved_flag[i]` bits it counts sit BETWEEN
`slice_pic_parameter_set_id` and `slice_type` in the slice segment header
(§7.3.6.1). Every earlier fixture used a PPS with the field == 0, so the
skip was never exercised: a parser that ignored the field entirely agreed
with all of them, and would then read `slice_type` from the wrong bit offset
on any real stream that sets it — mislabelling every picture's coding type.
**`reasserts_active_pps_at_bare_keyframe`** — regression (UHD banded
corruption): a stream redefines PPS id 0 mid-title, then a later keyframe
arrives WITHOUT repeating it (the source relies on the decoder retaining the
redefinition — valid for a raw bitstream). An hvcC player re-applies the
FIRST (codecPrivate) PPS at every keyframe, so the active redefinition must
be re-asserted in-band at that bare keyframe or the whole segment decodes
against the wrong parameter set.
**`keyframe_param_reassert_does_not_reallocate_the_frame`** — MEASURED: the
keyframe parameter-set re-assert must be spliced into the front of the
already-assembled access unit IN PLACE, not built as a fresh full-size
buffer. It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-MB allocation — memcpy'd the whole
frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a 2 h UHD title at
24 fps with a 1 s GOP is ~7,200 keyframes, ~14-28 GB of avoidable memcpy per
title. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates;
this test counts the reallocations that happen, which must be zero.
**`emits_switch_back_to_codecprivate_pps`** — regression (a UHD title, the
real bug): id 0 is body A (→ hvcC), then redefined to B, then the title
switches BACK to A. A streaming decoder (hvcC at init, in-band updates only)
is sitting on B; the switch back to A must be emitted IN-BAND even though A
== codecPrivate, or the whole A-segment decodes against B (cu_qp_delta
desync). Stripping on `== hvcC` dropped this revert.
**`cra_at_auto_detected_pts_backstep_rewritten_to_bla`** — regression for a
UHD Dolby Vision profile 7 dual-layer title: a multi-clip title is read as
one concatenated stream and the mpls connection_condition is never plumbed
to the parser, so the splice CRA opening the next clip kept its dangling
RASL leading pictures and a linear decoder flooded "Could not find ref with
POC N". The parser must AUTO-DETECT the boundary from the backward PES-PTS
reset between clips (each .m2ts has its own PTS base) and rewrite that
splice CRA → BLA_W_LP with no explicit `mark_clip_boundary` call.
**`cra_after_33bit_pts_wrap_not_rewritten`** — regression for the 33-bit PTS
wraparound false-trigger (rc.5.2 audit #1): a SINGLE clip whose raw 90 kHz
PES PTS crosses the 2^33 counter wrap (~26.5 h) must NOT be mistaken for a
non-seamless clip join. Before the fix the raw 2^33→0 backward step armed
`pending_clip_boundary` and the next in-clip CRA was wrongly rewritten
CRA→BLA_W_LP (dropping valid RASL pictures — visible corruption). After
unwrapping onto a monotonic timeline the wrap is absorbed and the CRA stays
CRA.
**`cra_splice_detected_at_large_pts_magnitude_not_masked_by_wrap_logic`** —
the wrap-vs-backstep test above keeps `high_pts` near the very top of the
33-bit range, close enough to the `PTS_WRAP_PERIOD / 2` threshold that
`high - unwrapped` and a hand-flipped `high + unwrapped` land on the SAME
side of the threshold at every step — it doesn't actually distinguish the
two. This test uses PTS magnitudes around 3e9 (order 2^32, well below the
wrap threshold but reached in well under an hour), where the subtraction and
the addition diverge: ordinary forward progression keeps `high - unwrapped`
small, but `high + unwrapped` would already exceed `PTS_WRAP_PERIOD / 2` on
the very next frame, which would pollute `pts_wrap_offset` by a full period
and mask the genuine two-clip splice that follows.
**`non_cra_nals_never_rewritten_at_boundary`** — non-CRA NALs are never
rewritten even when a boundary IS marked. IDR (19), RASL (8/9), VPS/SPS/PPS,
and a trailing slice all pass through unmodified; the IDR clears the pending
boundary so no later CRA is wrongly converted.
**`no_boundary_marker_is_byte_identical`** — a frame stream with NO boundary
marker is BYTE-IDENTICAL to a parser that has no splice-rewrite field at all
(the UHD-safety guarantee): asserts byte-equality of every emitted frame
across a multi-AU stream containing CRAs, IDRs, RASLs, VPS/SPS/PPS, and
trailing slices — none of which is ever marked.
**`seamless_boundary_no_rewrite`** — a SEAMLESS boundary (connection_condition
0x05/0x06) is expressed by NOT calling `mark_clip_boundary`, so a CRA across
a seamless join is left unchanged. This encodes the contract: only
non-seamless joins call `mark_clip_boundary`; seamless ones never do, so no
rewrite occurs.
**`redefined_pps_emitted_inline`** — a parameter set REDEFINED mid-stream
(same id, different body) must be emitted INLINE so the decoder
re-activates it. Some discs redefine PPS id 0 partway through the title;
the old parser kept only the first PPS, so the second segment decoded
against the wrong PPS (CABAC desync).
**`make_sps_full` (test helper)** — builds a stored SPS NAL with sub-layers
and a conformance window, so the parser must skip sub-layer PTL and the 4
conformance-window ue(v) fields before reaching the bit depths.
`max_sub_layers_minus1` controls the sub-layer loop.
**`hvcc_array_length_round_trips_above_256_bytes`** — the hvcC array length
is a 16-bit big-endian field written as two separate `push`es: `(len >> 8)
as u8` then `len as u8`. Every VPS/SPS/PPS the rest of the suite feeds is
well under 256 bytes, so the high byte is always 0 and a `>>` -> `<<`
mutation (which also always truncates to 0 for those inputs) is
unobservable there. A real HEVC SPS with an extended VUI/HRD block can
exceed 256 bytes, so this test uses 300+ byte VPS/SPS/PPS and decodes the
16-bit length fields back to confirm they round-trip to the exact NAL
length, not merely a byte that happens to be 0.
**`hevc_ps_reorder_is_installed_and_flush_drains_its_real_frames`** — HEVC
counterpart of `h264_ps_reorder_reconstructs_distinct_display_pts`. A
DVD/HD-DVD program stream stamps a PTS only on each GOP anchor, so the
parser must reconstruct display-order timestamps for the rest. Two things
are pinned here that nothing else constrains: `with_ps_reorder(true)` must
actually INSTALL the reorderer (a builder that returned a
default-constructed parser would silently leave the transport-stream path
in place on every DVD title), and `flush()` must drain the reorderer's real
buffered frames at EOF — not nothing, and not a manufactured empty frame at
PTS 0 (a zero-length Block whose timestamp jumps back behind every cluster
already written, RFC 9559 §5.1.3.2). Slice bodies follow H.265 §7.3.6.1:
`first_slice_segment_in_pic_flag`, the IRAP-only
`no_output_of_prior_pics_flag`, `ue(pps_id)`, `ue(slice_type)` (Table 7-7:
0 = B, 1 = P, 2 = I). PPS body 0xC0 sets `num_extra_slice_header_bits = 0`.
-26
View File
@@ -1,26 +0,0 @@
# hex
The single hex → bytes parser for the whole workspace.
## Why one parser
Key material arrives as hex from three third-party sources — the keydb, an
online key service, and the mapfile's `# freemkv-vid:` comment — and each
used to parse it slightly differently (one stripped `0x`/`0X`, one stripped
nothing, one stripped `0x` only). A key written with a prefix one parser
didn't expect was silently dropped → "can't decrypt" with no error. This is
the one parser they all call, so the prefix/case/validation rules live in
exactly one place.
## Byte-oriented, not char-oriented
Operates on BYTES, not `&str` char indices: the inputs are untrusted, so a
multi-byte UTF-8 scalar must reject as malformed, never panic on a
mid-codepoint slice.
## `byte()` internals
Combines two ASCII hex-digit bytes into one byte. `as char` is intentional:
for a non-ASCII byte it produces a Latin-1 scalar that `to_digit(16)` then
rejects — so non-hex (incl. `+`/`-` sign chars) and multi-byte input fail
cleanly rather than slipping through `from_str_radix`'s sign handling.
-75
View File
@@ -1,75 +0,0 @@
# identity.rs — drive identity test notes
## `OversizedCountTransport` (test double)
Returns the requested data length but reports a `bytes_transferred` larger
than the caller's buffer — models a drive that lies about its transfer
count. The old slicing code panicked on this; the clamps in `from_drive`
must keep it from indexing out of range.
## `inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive`
`ascii_field` with a buffer shorter than `start` returns empty string
rather than panicking.
- Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer (e.g.
a device that reports fewer than 8 bytes) must not panic.
- Mutation: removing the `data.len() > start` guard makes it panic on
short inputs.
A drive that answers INQUIRY with GOOD status but a short or empty data
phase must fail the probe, not present as a blank drive.
The buffer is pre-zeroed, so decoding it unconditionally yielded empty
vendor/product/revision strings and a byte 0 of 0x00. Every platform
enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
and 0x00 is DIRECT ACCESS — so the drive silently vanished from the device
list rather than reporting that its identity probe failed. A USB-SATA
bridge mid-wedge does exactly this.
The two GET CONFIGURATION calls in the same function already clamped on
`bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
lines above them, discarded it.
## `ascii_field_boundary_len_equals_start_is_empty`
`ascii_field`'s guard is `data.len() > start` (strictly greater), not
`>=`: a buffer whose length is exactly `start` has NO byte at that offset,
so it must still yield empty, not attempt to slice.
Mutation: `>` -> `>=` would try to slice `data[start..]` when
`data.len() == start`, which panics (empty range at the very end is fine,
but the guard's job is the `< start` case below it — pinning the exact
boundary catches an off-by-one either direction).
## `display_formats_trimmed_fields_space_separated`
`Display` renders the four trimmed identity fields space-separated — the
human-readable counterpart of `match_key`'s pipe-separated form. Not
exercised anywhere else in this test module.
Mutation: replacing the `fmt` body with `Ok(Default::default())` writes
nothing at all, so formatting any `DriveId` yields "".
## `FixedGcCountTransport` (test double) / `from_drive_gc_failure_yields_empty_firmware_date`
GET CONFIGURATION failure (transport error) must not abort the identity
probe — firmware_date is empty, raw_gc_010c is empty. Mutation:
propagating the GET_CONFIGURATION error with `?` aborts `from_drive`.
`FixedGcCountTransport` is a transport whose GET CONFIGURATION responses
report an exact, caller-chosen `bytes_transferred` for each of the two GC
features (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
boundary guards can be pinned precisely. INQUIRY always succeeds.
## `from_drive_firmware_date_boundary_exactly_12_is_empty`
`end > 12` in the firmware-date branch (`from_drive`) is a strict
inequality: `bytes_transferred == 12` reports the field absent (offset 12
is the first byte of the 12-char date; a count of exactly 12 covers bytes
0..12, none of which is the date), so `firmware_date` must be empty, not
the mutant's off-by-one read.
Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an empty
but non-panicking slice — silently reporting "present" data that is
actually all outside the transferred count.
-168
View File
@@ -1,168 +0,0 @@
# IFO parser notes
Overflow rationale relocated from `src/ifo.rs` comments (see `// See docs/ifo.md`
pointers in that file for the anchor point of each section below).
## `DvdTitle::feature_start_cell` (Bug-4: scene-selection/logo at head of feature)
The main feature's PGC can open with leading cells that are NOT part of the
movie — a scene-index segment or an interleaved-angle sub-block. This walks
the leading run and returns the index of the first cell that is a plain
feature cell (category `0x00`-class).
Conservative by construction — it can NEVER truncate a normal feature:
- It only ever skips a *prefix*; the scan stops at the first plain-feature
cell and keeps everything from there on.
- A normal single-angle feature has category `0x00` on cell 0, so the scan
stops immediately at index 0 and drops nothing.
- It never drops on duration or any heuristic — only on the spec category
bits — and it never drops the FIRST cell of an angle block (the angle we
keep).
- As a final guard it never returns past the last cell, and never drops when
that would leave zero cells.
For a real disc where every feature cell category is `0x00` and chapter 1
sits at 00:00:00, this returns 0 — a no-op — which is the correct result: the
disc's scene-index lives in a separate menu/title PGC, not in leading cells
of the feature PGC, so there is nothing to drop here.
## `bcd_to_frames` / `bcd_to_secs` (dvd_time_t is a timecode, not wall-clock time)
Format: `[hours_bcd, minutes_bcd, seconds_bcd, rate_and_frames]`
- Byte 0: hours in BCD (e.g. 0x01 = 1 hour, 0x12 = 12 hours)
- Byte 1: minutes in BCD
- Byte 2: seconds in BCD
- Byte 3: bits 7-6 = frame rate flag (01=25fps, 11=29.97fps),
bits 5-0 = frame count in BCD
The seconds field advances once every [`DvdRate::nominal_fps`] frames. On
NTSC discs that is every 30 frames, but 30 frames of 30000/1001 fps video
last 1001/1000 s — so reading H:M:S as literal seconds under-reports real
time by exactly 0.1% (3.6 s per hour). Callers that need real seconds must go
through the frame count, which is what `bcd_to_secs` does.
## `MAX_TT_SRPT_TITLES` / `parse_tt_srpt`
DVD-Video caps a disc at 99 titles (VMGI TT_SRPT `TT_Ns`, and the 99-title /
99-title-set structure the format is built around), but the on-disc count is
an untrusted `u16`: a ~800 KB crafted IFO can declare 65535 entries, each
re-parsing a PGC into a full `DvdTitle` (~540 MB of `DvdInfo`). Headroom: this
IS the format maximum, so it clips no conformant disc — a real DVD cannot
address a 100th title through TT_SRPT.
`parse_tt_srpt` has two bounds on untrusted input: the declared entry count
is clamped to `MAX_TT_SRPT_TITLES`, and entries naming a `(vts_number,
vts_title_num)` pair already seen are dropped. De-duplication is a
correctness fix as well as a bound: two TT_SRPT entries pointing at the same
VTS title are the same title, and `parse_pgcit` would otherwise re-parse
that one PGC into a separate `DvdTitle` per entry.
## `parse_audio_attr` visibility
`pub(crate)` for the CROSS-MODULE tests only. The sole production caller is
`parse_vts_attributes` in this file; `src/mux/mkv.rs`'s `#[cfg(test)]` block
calls it directly so its language-mapping tests run the real parser over real
on-disc IFO bytes end to end, instead of a hand-built `DvdAudioAttr` that
could agree with the muxer while both disagree with the disc. Narrowing this
would mean either a `#[cfg(test)]`/`#[cfg(not(test))]` pair of signatures
that can drift apart, or moving those tests away from the code they exist to
pin — both worse than the widened crate-internal visibility, which reaches no
public API.
## `assign_audio_sub_stream_ids`
On DVD-Video the sub-id's low nibble is the audio-stream *number* (0-7),
shared across all codecs — the single stream index the PGC `audio_control`
table / navigation registers select — and the high nibble is the codec base.
So the sub-id is `codec_base | position`, where `position` is the stream's
index in the IFO audio-attribute table (NOT a per-codec running count):
- AC-3 → `0x80 | i`
- DTS → `0x88 | i`
- LPCM → `0xA0 | i`
- MP1/MP2 and anything else → `None` (regular MPEG-audio PES, not a
private-stream-1 sub-id).
A per-codec ordinal was wrong: it only coincides with the wire id when a
codec's first stream is also the disc's audio stream #0. Any codec that is
not the first audio stream (e.g. a DTS track after an AC-3 track) then got
a sub-id one-too-low, so the demux routing key (`0xBD00 | sub_id`) never
matched and the track muxed silent. The positional index is the real wire
number, so distinct positions still give distinct sub-ids (no collision).
Position saturates at 7 so a malformed over-count never produces an
out-of-range sub-id.
## `parse_raw_dvd_lang_bytes`
The salvage is deliberately not narrowed to a-z: whatever survives is only
ever a lookup key for `dvd_lang_to_iso639_2`, which degrades anything it does
not recognize to `und`, so a stray `X` or `5` costs nothing and cannot reach
an output stream as a language code.
## `dvd_lang_to_iso639_2`
Uses `labels::vocab::iso639_1_to_iso639_2`, which spans the WHOLE of ISO
639-1 (plus the withdrawn spellings `iw`/`in`/`ji` that DVD-Video's
frozen-1988 language list still puts on disc). The narrower `vocab::menu_lang`
table is deliberately NOT used here: it exists for Blu-ray menu-graphic
filename tokens and knows only 25 languages, so a Region-2 disc's Romanian,
Bulgarian, Croatian, Serbian, Slovak, Slovenian, Hebrew, Estonian, Latvian,
Lithuanian and Icelandic tracks would all fold onto `und` together. DVD
streams carry an empty `label`, so the language is the only thing
distinguishing one subtitle track from the next — a valid code that is
identical for six tracks is worse for the user than the invalid one it
replaced. Both tables normalize to ISO 639-2/T, so they agree wherever they
overlap.
An empty or unrecognized code degrades to `"und"` (ISO 639-2 / Matroska's own
"undetermined" value) — a valid element value — rather than passing through
an invalid 2-letter code or an empty string. Never guesses.
## `video_attr_absolute_bytes_pin_real_layout` test
ABSOLUTE-BYTE pin (audit §3 #2): the existing video-attr tests build the byte
via `v_atr_byte(...)`, which uses the SAME shift constants the parser reads
with — a co-edit of constant + helper would silently re-introduce the
PAL-as-NTSC bug and every test would still pass. This test feeds
`parse_video_attr` HARDCODED bytes captured from real DVD-Video layouts
(DVD-Video video attributes: mpeg_version[7-6] video_format[5-4]
display_aspect[3-2] permitted_df[1-0]) — no `v_atr_byte`. If the parser's bit
positions drift, these fail.
## `dts_after_ac3_uses_positional_substream_id` test
Regression (The Punisher 2004): audio[0]=AC-3 5.1, audio[1]=DTS 5.0. The DTS
track sits at audio position 1, so its wire sub-id is 0x89 (0x88 | 1), NOT
the per-codec 0x88. With the old per-codec ordinal it got 0x88 → demux
routing key 0xBD88 had no match → every DTS packet (which carries 0x89) was
dropped → the track muxed present-but-silent while the AC-3 (at position 0,
where ordinal and position coincide) played fine. Positional numbering fixes
it end-to-end.
## `pgc_chapter_times_ntsc_no_pulldown_drift` test
Regression test for issue freemkv#25 (NTSC chapter drift). Builds a PGC from
the real cell table of the disc in the bug report and checks the chapter
marks against the reference tool's values. Before the fix, chapter 14 landed
4.019 s early; the drift was proportional to elapsed time (0.1%), so a short
synthetic fixture would not have caught it.
## Test additions note
The block of tests starting at `cell_category_low_flags_are_bit_isolated`
covers cell-category bit isolation, palette/program-map arithmetic, and the
malformed-language salvage paths.
## `build_pgc` test fixture
Builds a standalone PGC starting at offset 0. Layout: 0xEA-byte header, then
the program map (one byte per program, the 1-based first cell number), then
the 24-byte cell playback table. `cells` are `(category, BCD time,
first_sector, last_sector)`.
## `build_pgcit` test fixture
Builds a VTS_PGCIT at offset 0: VTS_PGC_Ns(2) + reserved(2) +
VTS_PGCIT_EA(4), then one 8-byte VTS_PGCI_SRP per PGC (VTS_PGC_CAT(4) +
VTS_PGCI_SA(4), the PGC's byte offset from the VTS_PGCIT start). `pgcs` are
appended after the SRP table.
-35
View File
@@ -1,35 +0,0 @@
# `src/io/image_writer.rs`
## Why this is not `freemkv_engine::copy`
The engine's `copy` is the RECOVERY path — mapfile sidecar, `--multipass`
sweep/patch, damage-jump, ECC-aware batching, auto-resume. Every one of those
exists because an optical drive returns read errors on marginal media. A
file-backed or synthesized source has no marginal media: a read either
succeeds or the underlying file is broken, and retrying it is pointless.
Routing a non-drive source through the recovery path is not merely wasteful,
it is wrong. Its mapfile identity check compares AACS unit keys and the VID,
both of which are empty for an already-decrypted source, so identity passes
for ANY such source: a second run with a different input to the same output
path would resume over the previous image and produce wrong content at exit
zero. Keeping the two paths separate makes that unrepresentable.
So: drive sources get `freemkv_engine::copy`. Everything else gets this.
## `write_image` rationale
The output is a faithful image of whatever the source presents — decrypted
if the caller wrapped the source in a
[`DecryptingSectorSource`](../src/sector/decrypting.rs), ciphertext if it did
not. `write_image` performs no decryption itself and makes no decryption
decision; that belongs to the caller, which knows whether the run is
`--raw`.
On cancellation the partial file is left in place — the caller decides
whether a partial image is worth keeping, and deleting a multi-gigabyte file
the user may want to inspect is not this function's call to make.
A short read from the source is treated as an error rather than zero-filled:
silently padding a truncated source would produce an image that looks
complete and is not.
-10
View File
@@ -1,10 +0,0 @@
# `/BDMV/index.bdmv` layout
Binary layout of the BD navigation index (First Play, Top Menu, and the
title table), as parsed by `src/bdnav/index.rs`.
`"INDX"` + version(4) + `indexes_start`(u32 @8) + … ; at `indexes_start`:
`index_len`(u32), First-Play object(12), Top-Menu object(12),
`num_titles`(u16), then `num_titles` title objects(12 each). Every object's
`object_type` is the top two bits of its first byte; for an HDMV object the
`id_ref` into `MovieObject.bdmv` is a big-endian u16 at object offset 6.
-7
View File
@@ -1,7 +0,0 @@
# FMTS index selection
Where the resolved index comes from is a separate concern from disposition
classification (`resolve_disc_index` in `src/aacs/index_select.rs`): today it
is read off the index keys the key source handed us; when Processing Keys are
available it will come from the VK derivation instead. Either way the
disposition logic in `unit_disposition` is identical.
-100
View File
@@ -1,100 +0,0 @@
# `aacs::inf` — notes
## `UnitKeyFile`'s hand-written `Debug`
Redacting `Debug`, per the policy `aacs::types` documents: this struct holds
the disc's ENCRYPTED CPS unit keys — exactly the material a keydb entry stores
— plus the disc hash they are looked up by. A derived `Debug` printed every key
byte verbatim, so any `{:?}` (a downstream crate, an `assert_eq!` failure
message, a future `tracing::debug!` in this module) leaked them. Only
non-secret shape is printed. Guarded by `unit_key_file_debug_is_redacted`.
The disc hash is the public keydb lookup key, printed as hex the same way
`DiscEntry` prints its own — never as raw bytes.
## `parse_vtkf`
Layout — AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, a fixed
2480-byte file, verified byte-exact against real discs (Freedom `VTKF090`,
Dukes of Hazzard `VTKF000`):
```text
[0x00..0x0C] magic "DVD_HD_V_TKF"
[0x0C..0x10] BE32 HD_VTKF_SIZE (2480)
[0x10..0x1C] associated playlist name ("VPLST%%%.XPL")
[0x1C..0x80] reserved
[0x80..] 64 entries × 36 bytes:
BIFO (1) | reserved (3) | ENCRYPTED title key (16) | binding MAC (16)
BIFO bit 7 (AV_FLG) set = this slot holds a title key
(pre-recorded discs fill the binding MAC with 0xFF)
[0x9A0..2480] 16-byte TKF MAC (CMAC keyed by Kvu — NOT a key)
```
The slot index (1-based) is the CPS unit number, so an absent slot is
SKIPPED (not a terminator) — collapsing gaps would renumber later keys and
hand the wrong title key to CPS unit N+1. The title→CPS mapping is
playlist-driven (`VPLST%%%.XPL`) and owned by the HD DVD enumerator, so
`title_cps_unit` is left empty here.
The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
(real multi-CPS-unit HD-DVD titles) yielded garbage keys for CPS unit ≥2.
## `synth_vtkf` test helper
Build a synthetic `VTKF%%%.AACS` matching the real on-disc layout (AACS
HD DVD Book Table 3-8, verified against Freedom `VTKF090` and Dukes
`VTKF000`): magic, BE32 size, playlist name, reserved to 0x80, then 64
entry slots of 36 bytes (the first `keys.len()` present with `AV_FLG`
set, the rest empty), a reserved gap, and the 16-byte trailing TKF MAC.
## `unit_key_file_debug_is_redacted` test
`UnitKeyFile` holds the disc's ENCRYPTED CPS unit keys. A derived `Debug`
printed every byte; the hand-written impl must not. Sentinel key byte
0xD5 = decimal 213 (a derived `Debug` renders `[u8; 16]` in decimal), the
same probe `aacs::types::redaction_tests` uses. Mutation guard: putting
`#[derive(Debug)]` back fails this.
## `read_mkb_from_drive_returns_the_concatenated_pack_payload` test
`read_mkb_from_drive` is the in-drive MKB source: every AACS derivation
downstream (`mkb_find_mk_dv`, the subset-difference walk, the whole
Media Key ladder) consumes exactly what it returns. An empty return is
not a benign "no MKB" — it is a total read failure reported as success,
and every derivation then fails with a key-not-found code that points
the operator at their keydb rather than at the drive.
This pins the CONTENT: the concatenated payload of all packs, in pack
order, byte for byte.
## `read_mkb_from_drive_issues_the_exact_mmc_cdb_for_each_pack` test
The CDB is what the drive actually acts on, and every byte of it is
load-bearing: a wrong format code returns a different disc structure
entirely, and a wrong allocation length truncates the pack. The existing
test above pins the opcode, the format code and the pack number; this
pins the WHOLE 12-byte CDB, so no field can drift unnoticed.
Expected layout (MMC-6 READ DISC STRUCTURE, AACS MKB format):
`[0]` opcode, `[1]` media type 0x01, `[2..6]` address = pack number
(BE32), `[6]` layer 0, `[7]` format 0x83, `[8..10]` allocation length
BE16 = 32772 = `0x80 0x04`, `[10..12]` reserved/control.
## `read_mkb_from_drive_accepts_a_full_size_pack` test
A pack payload filling the FULL 32768-byte window must come back whole.
The `len > 0 && len <= 32768` bound is what stands between a maximal
pack and a silently dropped one, and the small payloads used elsewhere
in this module never reach it.
## `read_mkb_from_drive_ignores_a_pack_declaring_more_than_the_buffer_holds` test
A drive that DECLARES more payload than it returned must not be
believed. The BE16 length in the response header is drive-supplied data:
a firmware bug, a short transfer, or a hostile device can put a value in
it that runs past the 32772-byte buffer. Copying `len` bytes on that word
alone panics the rip thread mid-scan.
Both the first-pack read and the per-pack loop carry the same bound, so
both are exercised here: the over-declared pack contributes nothing and
the honest pack still comes through.
-34
View File
@@ -1,34 +0,0 @@
# `src/io/mod.rs` — background
## `WritebackFile`
Drop-in wrapper around `std::fs::File` for any call site that performs
large sequential writes (sweep, patch, mux, etc.). It implements `Write`
and `Seek` so existing code paths can swap `File` for `WritebackFile`
with no body changes. Internally it drives a `WritebackPipeline` that,
on Linux, drains dirty pages continuously at 32 MB granularity to avoid
the kernel's accumulate-then-burst flush behaviour. macOS and Windows
use a no-op pipeline — their default cache policies have not been shown
to exhibit the same pathology for this access pattern.
## `FileSectorSource`
The read-side dual of `WritebackFile` — it implements
[`crate::sector::SectorSource`] for an ISO file using direct
`pread`-equivalent calls so the kernel's own readahead policy runs
(which interleaves naturally with the concurrent writeback). It pairs
that with periodic `posix_fadvise(DONTNEED)` drops on the consumed
window so an 85 GB streaming ISO read doesn't fill the page cache and
starve the concurrent MKV write.
## `Pipeline` + `Sink`
The generic producer/consumer primitive used by sweep, patch, and mux
to overlap reads with writes via a bounded channel + dedicated consumer
thread.
## `byte_prefetcher`
The read-ahead producer feeding the mux pipeline for `io::Read`-backed
sources: a worker thread fills a recycled pool of buffers and ships
them through a channel, exposing `BytePrefetcher` / `PrefetchShell`.
-25
View File
@@ -1,25 +0,0 @@
# src/labels/jar.rs — relocated comment prose
## `MAX_CLASS_BYTES` rationale
A real BD-J `.class` is far under this ceiling (64 MiB); a lying header
simply gets truncated and the class fails to parse, which is skipped like
any other bad entry. The buffer is grown incrementally and the read is
capped here rather than pre-sized from the declared size.
## `for_each_jar` "top-level" vendor examples
"Top-level" means entries directly under `/BDMV/JAR/`, not nested under a
subdir. Pixelogic, Criterion, Paramount, etc. put their data files inside
`/BDMV/JAR/<x>/`; dbp and Deluxe put their jar directly at
`/BDMV/JAR/<name>.jar`.
## `read_is_bounded_by_cap` test rationale
The read is bounded by `MAX_CLASS_BYTES`: a stored entry whose real payload
exceeds the cap yields only the first `MAX_CLASS_BYTES` bytes to the
parser, never the full (unbounded) entry. Verified here on a small cap via
the entry-count path: the truncated bytes still parse a valid class
prefix, but no read beyond the cap occurs. The test asserts the entry is
still surfaced exactly once (the cap does not drop legitimate entries) and
that the call returns.

Some files were not shown because too many files have changed in this diff Show More