Compare commits

...
121 Commits
Author SHA1 Message Date
Matthew Jackson 76e71dc20a v1.7.0-rc5: bump version (freemkv-unlock git-pinned for the tag) 2026-09-03 15:16:22 +00:00
Matthew Jackson 3c8fce3031 disc/encrypt: don't fall through to the AACS cert unlocker on a firmware-claimed drive
A drive claimed by a firmware unlocker (freemkv/LD/Renesas) at init() is that
unlocker's sole VID source. When it stashed no VID, do_handshake_cert now returns
(None, None) instead of running the AACS cert route (run_bus's AacsUnlocker) whose
REPORT KEY AGID cycle poisons the drive's AGID selector so every later bare 0xAD
VID read aborts until the medium is reloaded. A key source can still supply the key.

Adds a red-before-green test proving the guard issues zero SCSI on that path.
2026-09-03 07:41:43 -07:00
Matthew Jackson 5578e42961 Cargo: restore dev path-dep for freemkv-unlock
The dev branch tip must path-dep ../freemkv-unlock so CI's sibling checkout
(and local dev) resolves the branch tip; release.sh swaps it to a git tag
only in the tagged commit. dev had been left in the released git-tag
(v1.6.14) form after the last release, so the new 1.7.0 unlock API couldn't
resolve. Restore the path form the comment already describes.
2026-09-02 18:02:26 -07:00
Matthew Jackson a7db00c0a1 unlock_bridge: drive unlockers through the unified single-unlock contract
Adopt freemkv-unlock's new Unlocker::unlock(). The bridge assembles the
firmware set (freemkv / LD / Renesas) for drive-prep and the disc set
(AACS cert + DVD) for bus removal, injecting host certs into the AACS
unlocker at construction — the loop no longer knows any unlocker's guts.

- run_features / run_bus now loop unlock() and return Ok(Option<Unlocked>);
  drive init and do_handshake_cert adapt to the Option.
- unlocker_matrix reports the LD name and credits Renesas as a drive-level
  (raw-read) route, so AACS shows "no" when any firmware route took the bus.
- Cert route sets HandshakeResult.drive_unlocked=false: the bus is removed
  via read_data_key (AKE), not at the drive; the gate credits it that way.
2026-09-02 17:53:33 -07:00
Matthew Jackson 09957e066b ci: identity-guard — block unrelated-corp / non-canonical committer emails 2026-09-02 14:46:16 -07:00
Matthew Jackson a88b7486ac release: 1.7.0
Fixed
- mux/mp4: honour 64-bit largesize (size==1) and size==0 (run-to-end) in the
  capped child-box scan; a largesize sibling previously ended the walk early and
  later boxes vanished from a MOV/MP4.

Version 1.6.15 (unreleased) -> 1.7.0; changelog folded under 1.7.0.

Earlier 1.7.0 work remains as its own commits below this one: the
GHSA-j8ww-f5fg-9pmh sector-recycle soundness fix, CI/lint hardening (#15/#17),
docs, and Rob Ekl's scsi raw_command / fd-handoff change (#16).
2026-09-02 12:40:55 -07:00
Rob Ekl 6957c7251c scsi: route raw_command through checked_cdb_len; add the fd hand-off release edge (#16)
raw_command clamped CDBs with cdb.len().min(16) instead of routing through
the shared checked_cdb_len guard, so an over-length or empty CDB became a
different command rather than a rejected one; it now returns the crate Result
so a rejected CDB is distinguishable from an I/O failure.

The recovery/Drop fd hand-off lacked a release edge: the teardown swap was not
a release op and the recovery CAS was not an acquire load, so a recovery thread
publishing its fd after Drop drained the slot could miss dead==true and leak the
descriptor. Both sides are now AcqRel, with the execute() drain left at Acquire
(sound via the release-sequence rule). A loom model (cfg(loom) dev-dep, its own
CI job) provides the regression coverage cargo test cannot.

Module docs for fd_handoff and timeline now resolve in their own scope, and item
docs stay within comment-guard's caps.

Fixes #16.
2026-09-02 12:40:55 -07:00
Matthew Jackson 6e14fc0c0c libfreemkv: 1.7.0 groundwork — sector-recycle soundness + CI hardening
Security
- Remove the unsafe Vec::set_len over recycled sector buffers — UB reachable
  from safe code (GHSA-j8ww-f5fg-9pmh, low). Uses Vec::resize; pooling unchanged.

CI / tooling
- clippy on stable with the MSRV single-sourced from Cargo.toml; cargo-deny
  dependency-audit gate with first-party git sources allowed (#15); audience-based
  comment-guard (#17).

Docs
- README (keydb ownership, BD-J parser count), drive-access transport/reset
  notes, mux timeline module-doc links.
2026-09-02 12:40:55 -07:00
Matthew Jackson 962aa5d852 v1.6.14: bump version (freemkv-unlock git-pinned for the tag) 2026-08-31 23:04:25 +00:00
Matthew Jackson 719b63e376 chore: reconcile v1.6.13 release commit from main into dev 2026-08-31 15:48:53 -07:00
Matthew Jackson 3cf7922574 docs(changelog): 1.6.14 entry 2026-08-31 15:32:40 -07:00
Matthew Jackson 0c953e9340 chore(msrv): floor to 1.88, pin CI on the floor, allow uninlined_format_args
Declared rust-version and the ci.yml/qa.yml toolchain pins both sit at 1.88 (the
edition-2024 floor once zip 8 and let-chains are accounted for), so CI builds on
exactly the MSRV: an accidental bump to a newer toolchain fails CI by design and
moving the floor takes a deliberate pin edit. The uninlined_format_args allow
reconciles 1.88's clippy with the allow-by-default it became in 1.97+.
2026-08-31 11:06:35 -07:00
Matthew Jackson 2e90a7585d drive: move --share capture orchestration to freemkv-engine
libfreemkv keeps only generic SCSI primitives; the app-specific drive-info
capture (INQUIRY / GET_CONFIG / vendor buffers) now lives in its consumer.
2026-08-31 10:08:36 -07:00
Matthew Jackson 3e11225389 audit: resolve remaining v1.6.10..HEAD findings (parallel fan-out)
Correctness/robustness:
- scsi/macos_shim: guard g_handle / g_da_bsd read-modify-write with a
  process-global mutex (scoped OUT of the 5 s async-claim wait) — closes the
  DiskArbitration global-state TOCTOU on concurrent/re-entrant open+close.
- disc/profile: the public DiscProfile now surfaces `encrypted` and a numeric
  `key_error` (Error::code of aacs/css failure) so a disc with failed key
  resolution no longer serializes identically to a rippable one.

Performance:
- disc/dvd: read VIDEO_TS.IFO once and reuse it for both parse_vmg and
  resolve_main_title (parse_vmg_with / resolve_main_title gain an optional
  vmg_bytes) — eliminates a duplicate VMGI read on every DVD scan.

Tests:
- bdnav: MAX_OBJECTS count-cap rejection, MAX_STEPS non-convergence guard, and
  an end-to-end index.bdmv + MovieObject.bdmv resolver test (all red-green).
- drive/capture: cover the Renesas "SAT"-signature capture gate (match + miss).

Craft/docs:
- css/keyless: factor the duplicated LFSR0 output-tap into one helper.
- drive/capture: name the Renesas vendor CDB constants.
- labels/deluxe: use the named ANEWARRAY/NEWARRAY opcodes, not bare hex.
- dvdnav/vmcmd: tighten pub -> pub(crate). dvdnav/mod: mark the parked stub.
- disc/bluray: document that whole-clip extents are intentional (trimming is
  PTS-level in SeamPlan). disc/mod + error: doc/comment drift corrected.
2026-08-31 08:36:08 -07:00
Matthew Jackson 82511d84b6 audit: resolve macOS shim lifetime bug + close coverage/DRY gaps
- scsi/macos_shim: the DA claim used a stack DAClaimResult passed to an async
  callback and never released its semaphore — on the 5 s timeout that is both a
  use-after-free (the callback outlives da_hold) and a semaphore leak per drive
  open. Heap-allocate it, reference-count it (rc=2), and let whichever side
  finishes last release the semaphore and free. (Compiles; needs on-hardware
  DA validation.)
- bdnav/index: add a regression test that a title count over MAX_TITLES is
  rejected before any record read (untrusted index.bdmv count).
- dvdnav/nav: share ifo::MAX_TT_SRPT_TITLES instead of a second private copy so
  the two 99-title caps can never diverge.
2026-08-31 08:11:26 -07:00
Matthew Jackson 20afbfdccb audit: fixes from the v1.6.10..HEAD codeaudit sweep
Correctness:
- dvd: join the nav main-feature target on the REAL vts_title_num (carried on
  DvdTitle) instead of the title's position in the vec, which desyncs when a
  sibling PGC is dropped as unparseable or TT_SRPT is non-monotonic.
- diag: the "demoted: no-video" dump now gates on has_probable_video (the gate
  main_feature_order actually uses), not the stricter has_video.

Robustness:
- labels/deluxe: clinit_enum_field_names now caps retained bytes
  (MAX_CLINIT_LDC_BYTES), like its sibling, so a malicious BD-J .class with a
  huge CONSTANT_Utf8 field name can't grow the output unbounded.

Mux (FlagInterlaced correction cleanup):
- factor the PGS/FlagInterlaced deferred byte patches through one patch_bytes
  helper; name the InterlacedRewrite record; add ebml::void_element (validated)
  for the demotion Void; bound the muxed_* test scans to the pre-Cluster region.

Tests:
- add the FlagInterlaced majority tie-break test (tie stays progressive).
- de-vacuum the profile subtitle-count and meta_sink channel-count assertions.

Docs:
- clpi module doc described an EP-map/sector-range parser that was never
  implemented; corrected to what the code actually reads.
2026-08-31 07:51:05 -07:00
Matthew Jackson 6d78a36d22 lib: trim feature-gate comment blocks to satisfy comment-guard 2026-08-30 22:41:40 -07:00
Matthew Jackson a9d41f53f8 mux: correct FlagInterlaced from the whole-stream scan majority
FlagInterlaced was written from only the FIRST coded picture, but MPEG-2
progressive_frame is a per-picture flag: a progressive leader on an
interlaced feature (or an interlaced first picture on progressive film)
mislabeled the whole track, so players either deinterlaced progressive
frames or left interlaced frames combed.

Tally every picture's measured scan and correct the reserved FlagInterlaced
byte to the dominant scan at finish() -- the same reserve-then-patch idiom as
PgsForcedFixup and the AC-3/duration fixups. On a demotion to progressive the
now-stale FieldOrder element is Void'd. Also pins the interlaced direction at
the muxed-byte level and adds promote/demote regression tests.
2026-08-30 22:38:51 -07:00
Matthew Jackson c7c53989f7 Feature-gate the crate: scsi (transport-only) vs rip (full)
Split the crate into two additive features so a transport-only consumer
(freemkv-flash) can depend on the SCSI passthrough layer without pulling the
ripping dependency tree (serde, zip, rayon, aes, freemkv-unlock, …):

- scsi = the SCSI passthrough transport (src/scsi/ + error): ScsiTransport,
  CDB builders, per-OS backends + the macOS cc shim, sense parsing. Pulls only
  tracing + the always-on libc target dep.
- rip  = the full ripping tree (drive/disc/UDF/MPLS/CLPI/AACS/CSS/mux/keydb +
  the freemkv-unlock seam) and all its heavy deps. rip implies scsi.
- default = ["rip", "scsi"], so the full build stays byte-for-byte identical for
  the existing consumers; transport-only users set default-features = false,
  features = ["scsi"].

All ripping modules are gated #[cfg(feature = "rip")]; error + scsi stay always
compiled. Two read-path timeout consts are rip-gated so a transport-only build
does not warn on unused.
2026-08-30 22:10:05 -07:00
dependabot[bot] 7ec2dbfbcb build(deps): bump codecov/codecov-action from 5 to 7
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5 to 7.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v5...v7)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-30 21:53:05 -07:00
avon662 16e5b5ee38 Write FlagInterlaced from the measured picture, not the declared resolution
MkvTrack::video sets `interlaced` from the source scan's DECLARED
resolution. An NTSC DVD's IFO declares 480i whether or not its pictures
are coded progressive -- film and animation routinely are -- so
progressive content ships as FlagInterlaced=1, and every player that
honours the flag runs a deinterlacer over progressive frames, softening
every frame of an otherwise bit-exact remux.

The measurement was already there and already trusted for the sibling
property. `apply_coding_to_track` receives the first coded picture's
PictureInfo and derives FieldOrder::Progressive from it
(progressive_sequence / progressive_frame, ISO/IEC 13818-2 6.3.5 /
6.3.10) -- it used that verdict only to clear FieldOrder and then
discarded it. It now also corrects the scan type, exactly as a measured
TFF/BFF already overrides the declaration.

Measured on a real NTSC DVD: ffmpeg's `idet` reports 100% progressive
(multi-frame detection: 0 TFF, 0 BFF, 0 repeated fields) on titles that
shipped FlagInterlaced=1. progressive_frame is set on every picture
coding extension in the stream while progressive_sequence stays 0, which
is the ordinary shape for a film or animation DVD.

Video payload is unaffected -- byte-identical before and after; only the
track header changes.
2026-08-30 21:47:19 -07:00
Matthew Jackson d40540ce9e v1.6.13: bump version (freemkv-unlock git-pinned for the tag) 2026-08-28 15:48:58 +00:00
Matthew Jackson ba14d66c18 Release 1.6.13: capture additional drive buffers during info --share 2026-08-27 21:22:11 -07:00
Matthew Jackson f16abe1cf4 restore freemkv-unlock path dep for local dev (post-v1.6.12) 2026-08-27 22:57:33 +00:00
Matthew Jackson 477c0da6dd v1.6.12: bump version (freemkv-unlock git-pinned for the tag) 2026-08-27 22:57:29 +00:00
Matthew Jackson 41c5b8b785 changelog: 1.6.12 (unreleased) 2026-08-27 13:11:27 -07:00
Matthew Jackson 9f43ae94d3 Comment and Documentation Cleanup
- Cap every inline `//` comment block at 3 lines; distil the rationale
  rather than delete it. Enforced by a new comment-guard CI check.
- Fix stale and orphaned rustdoc: broken intra-doc links (renamed or
  removed items), doc blocks that had merged onto the wrong item (leaving
  the intended item undocumented), and doc-contradicts-code drift.
- Add a doc-links CI check (cargo doc, deny broken intra-doc links) so
  both classes of comment rot are caught going forward.
2026-08-27 12:07:53 -07:00
Matthew Jackson 9ed6f86255 Fix macOS optical E1000: hold a DiskArbitration claim for the session
The macOS SCSI transport took exclusive SCSI access plus a one-shot
`diskutil unmountDisk`, but never claimed the disc from DiskArbitration.
diskarbitrationd was therefore free to remount the volume mid-rip (a
Spotlight re-probe is the usual trigger), yanking the device out from
under an in-progress read; the failure surfaced to the user as E1000.

Hold a DADiskClaim plus a mount-approval dissenter scoped to our disc for
the transport lifetime, so the OS cannot remount it until shim_close().
Link the DiskArbitration framework in build.rs.
2026-08-27 11:17:01 -07:00
Matthew Jackson 70ae7b6c10 restore freemkv-unlock path dep for local dev (post-v1.6.11) 2026-08-25 23:17:11 -07:00
Matthew Jackson 3f03c6db06 v1.6.11: bump version (freemkv-unlock git-pinned for the tag) 2026-08-25 23:17:07 -07:00
Matthew Jackson 562bb8bee5 docs: neutralize tool-comparison and technique wording (opsec) 2026-08-25 22:57:01 -07:00
Matthew Jackson 7c13ed32b5 fix: align drive-prep unlocker token with unlock rename; neutralize product name 2026-08-25 22:54:38 -07:00
Matthew Jackson 574aa2c506 docs: neutralize retail title references (opsec) 2026-08-25 22:45:57 -07:00
Matthew Jackson 3912762b52 docs: neutralize comment provenance and external references (opsec) 2026-08-25 22:31:19 -07:00
Matthew Jackson 028768e71b docs: 1.6.11 changelog 2026-08-25 22:08:48 -07:00
Matthew Jackson bf4e5ac395 docs: standardize README badges (License, CI, codecov) 2026-08-25 20:26:28 -07:00
Matthew Jackson 11948d513b nav: taint control-transfer decisions only, and track SPRM-laundered GPRMs
Refine the First-Play SPRM-taint abstain to fix two defects:

- Over-abstention: a tainted compare that guards a SetGPRM store (not a
  control transfer) no longer forces abstention. The predicate only
  decides whether the store runs; a following unconditional dispatch is
  still resolvable. Taint now gates only JumpTT/Goto (the control
  transfers the resolver follows); a store under a tainted guard taints
  its destination GPRM (unknown post-state) instead of abstaining.

- Under-abstention: an SPRM laundered through a GPRM (SetGPRM g = SPRM,
  then a later GPRM-only compare on g) previously evaded taint and let
  the resolver commit to the cold-machine arm. Add per-GPRM taint bits:
  a store propagates its source's taint (SPRM or tainted GPRM) and a
  concrete/immediate store clears it; eval marks a compare tainted when
  it reads a tainted GPRM.

Adds red-before-green tests for both plus taint-clear guards.
2026-08-25 20:26:03 -07:00
Matthew Jackson 7d5c0a5c41 Fix round-3 audit findings: VM operand decode, MPLS field offset, log escaping, nav SPRM abstain, tests
1. dvdnav/vmcmd: the register operand of the if_v1/if_v3 compare layouts is the
   LOW byte of the reg-or-immediate field (b[5] / b[7]), not the high byte
   (b[4] / b[6]). Fix both layouts and their comment; add register-mode KATs
   with distinct high/low bytes.

2. mpls: connection_condition is the low nibble of PlayItem byte[10]
   (byte[9] is fully reserved), corroborated by the byte-11 stc_id and
   byte-12 in_time reads. Relocate every fixture's encoded value and update the
   regression test to assert the reserved byte does not leak.

3. labels/deluxe: escape the two disc-derived class-name tracing fields via the
   Debug path, matching the other log sites.

4. disc/dvd: add an integration test that drives scan_dvd_titles with a resolved
   First-Play nav target so the 1-based promotion join is verified end-to-end.

5. bdnav/vm: add unit tests for the SET/SWAP immediate gate — full register
   exchange, and that an immediate-flagged operand is never a write target.

6. dvdnav/nav: honor the documented contract — a First-Play compare that reads a
   system parameter is undecidable at scan time, so eval now taints on SPRM reads
   and the executor abstains rather than committing to the cold-power-on arm.
   GPRM reads never taint.

7. labels/deluxe: narrow four now-internal helpers from pub(crate) to private.
2026-08-25 20:26:03 -07:00
Matthew Jackson 1ede962aec ci(coverage): authenticate Codecov upload with org CODECOV_TOKEN 2026-08-25 19:32:30 -07:00
Matthew Jackson 014ea870a4 ci(coverage): add --ignore-run-fail so instrumentation flake never blocks upload 2026-08-25 19:04:41 -07:00
Matthew Jackson bdd2876d78 ci: add non-required Codecov coverage job, config, and badge
Add an additive coverage job to CI that generates lcov via cargo llvm-cov
and uploads to Codecov (tokenless). Mirrors the test job's pinned 1.94
toolchain and dual freemkv-unlock checkout for the path dep. Informational
only: no gate depends on it. Adds codecov.yml and a README badge.
2026-08-25 18:54:23 -07:00
Matthew Jackson 44de7a8d12 harness: fuzz the Deluxe bytecode decoder directly, not through a dead zip open
Audit round-2 (fix-adversary): the round-1 deluxe sweep fed the zip local-file-
header magic, but ZipArchive::new validates the end-of-central-directory at the
buffer's end — which the byte generators (front magic + noise) can never
synthesize — so the archive open always failed and decode_binding was never
reached: false coverage for the very bytecode-decoder panic path it guarded.
Drive decode_binding_class directly on a parsed .class with an empty master-enum
table instead, so the symbolic stack machine is actually exercised.
2026-08-25 18:45:28 -07:00
Matthew Jackson 552076a943 disc/labels/harness: audit round-1 batch — DoS bound, fuzz coverage, API surface
Continuing the /codeaudit HEAD..v1.6.10 round:

- disc/mod.rs rank_titles: bound the O(n^2 * clips) composite scan to a
  sub-second work budget (both title count and per-title clip count are
  untrusted disc metadata); above it, skip composite classification and degrade
  to the has-video + canonical order. Reorder the per-pair checks so the O(len)
  is_subset runs only after the cheap video/size gates. (regression test)
- bdnav/vm.rs SWAP: honour the imm_op1/imm_op2 store gates so a SWAP to an
  immediate operand does not corrupt a register.
- harness.rs: fuzz the previously-uncovered new parsers — the Deluxe bytecode
  decoder, the Fox dcx.xml scanner, and the combined bdnav index+mobj+VM resolve
  path (labels::{deluxe,fox} and bdnav::vm made pub(crate) for in-crate fuzzing).
- dvdnav: pub(crate) the module + submodules (was pub) to match bdnav and stop
  leaking the DVD IFO/VM internals as public API; drop an unused re-export.
- labels: escape disc-derived strings in diagnostics (Debug not Display) so a
  hostile disc cannot inject terminal control sequences into logs.
2026-08-25 18:33:37 -07:00
Matthew Jackson 3ecbeb7eba disc/bdnav: audit round-1 — VM title/PSR faithfulness + main_title never-panics
From the /codeaudit HEAD..v1.6.10 round (each verified against the libbluray
HDMV VM reference, and design-refuted SOUND on two models):

- bdnav/vm.rs title_obj: resolve First Play via the spec sentinel 0xFFFF (was
  num_titles+1). A JumpTitle 0xFFFF now reaches First Play, and an out-of-range
  JumpTitle(N+1) correctly abstains instead of mis-jumping there.
- bdnav/vm.rs wr: refuse stores to PSR-tagged registers (matches _store_reg),
  so an illegal SET/SWAP to a PSR is a no-op and later compares agree with a
  real player instead of diverging.
- disc/profile.rs main_title(): return Option<&TitleProfile> instead of indexing
  titles[0], which panicked on a disc that scanned to zero titles.

Regression tests added, each shown red-before-green.
2026-08-25 18:16:17 -07:00
Matthew Jackson b405154fd0 ci: run the consumers job at the highest dependent MSRV (1.98)
The MSRV split raised the dependents above libfreemkv's own floor — the
bins (freemkv, autorip, bdemu) are 1.98 and freemkv-keysources is 1.97 —
so the consumers job could no longer build them at libfreemkv's 1.94
toolchain (`rustc 1.94.0 is not supported by ... freemkv-keysources
requires rustc 1.97`), leaving dev red since the MSRV change. Pin that one
job to 1.98, which satisfies every dependent's floor. libfreemkv's own
lib/lint/test jobs stay at its 1.94 MSRV.
2026-08-25 13:49:35 -07:00
Matthew Jackson 8a630c8839 disc: payload-plausibility floor on nav/authoring feature promotion
A nav- or authoring-resolved playlist could promote a video-bearing but
negligible-payload branch above the substantial feature it sits beside:
Spotlight (2015) UHD navigates to a Dolby-Vision seamless-branch playlist
(00001, 0.4 GB across 102 tiny clips) instead of the 91.5 GB single-clip
feature (00002). A player plays the branch; a rip wants the feature.

Require a nav/authoring-promoted title to also carry at least 10% of the
largest video-bearing title's bytes; otherwise selection falls through to
the composite/size ranking, which recovers the real feature. Adds the
Spotlight-shaped regression test.

Verified against the 88-disc hoard: title selection is identical to the
prior release on every disc except the one it fixes (Spider-Man 3 UHD,
00245 wrapper decoy -> 00001 feature).

Also clears three 1.97 clippy lints in the HDMV VM and drops two
reference-implementation filename citations from the bdnav module docs.
2026-08-25 13:44:04 -07:00
Matthew Jackson 6c00090308 labels/fox: wire the extracted feature-playlist id into the selection hint 2026-08-25 13:01:25 -07:00
Matthew Jackson 7f9f5a9d3e Merge branch 'label-fox' into integrate-1.6.11 2026-08-25 13:00:06 -07:00
Matthew Jackson b3c23c19ee Merge branch 'label-deluxe' into integrate-1.6.11 2026-08-25 12:59:54 -07:00
Matthew Jackson f3923cf587 Merge branch 'dvd-nav' into integrate-1.6.11 2026-08-25 12:59:48 -07:00
Matthew Jackson 002912abb7 dvdnav: resolve DVD main feature by following First-Play navigation
Implement the DVD navigation feature resolver (#40): read the disc's own
First-Play program out of VMGI (VIDEO_TS.IFO) and execute it on a bounded
register machine to identify which title a player would land on, then prefer
that title in DVD main-feature selection.

- src/dvdnav/nav.rs: new module.
  * Parses the VMGI First-Play PGC command table (FP_PGC pointer at VMGI+0x84,
    PGC command-table pointer at PGC+0xE4, pre-command list) and TT_SRPT
    (title -> VTS/VTS_TTN, pointer at VMGI+0xC4).
  * Executes the pre-command list with the existing vmcmd decoder over 16 GPRMs
    + SPRMs from a cold machine: SetGPRM (mov/add/sub/mul/div/mod/and/or/xor),
    compare predicates, Goto, Break/Exit, and JumpTT (title dispatch). Menu /
    sub-domain jumps and intra-domain links are treated as undecidable.
  * resolve_from_vmg(&[u8]) is pure and total; resolve_main_title(reader, udf)
    reads the IFO and returns the resolved (title, vtsn, vts_ttn) or None.
  * Every offset/count is bounds-checked; a step budget bounds non-convergence.

- src/disc/dvd.rs: scan_dvd_titles now resolves the nav main title and returns
  its playlist_id alongside the titles; test call sites updated for the tuple.
- src/disc/mod.rs: the DVD nav pick feeds sort_titles_by_main_feature as
  nav_feature, the same seam the BD path uses, guarded to a video-bearing
  title. None -> today's leading-cell heuristic stands (strict improvement).
- src/harness.rs: add resolve_from_vmg to the never-panic sweep.

Tests: 11 nav unit tests (unconditional/conditional/set-then-dispatch convergence,
menu/empty/out-of-range/self-loop/truncation/overflow abstention) built to the
IFO spec using the real First-Play command bytes from the existing decoder KATs
(unconditional JumpTT 1; menu entry). Full suite green on 1.94.
2026-08-25 12:55:34 -07:00
Matthew Jackson b5fd193d54 disc: add normalized DiscProfile output surface
Add a thin, format-agnostic typed VIEW over the scanned Disc model so
downstream reads `profile.titles[i].subtitles[j].forced` (and analogous
audio/video flags) for any disc with no per-format conditionals.

New src/disc/profile.rs:
- DiscProfile / TitleProfile / VideoTrack / AudioTrack / SubtitleTrack
  (serde Serialize + Deserialize), re-exported from the disc module.
- DiscProfile::from_disc / From<&Disc>, Disc::profile(), and accessors
  (DiscProfile::main_title, TitleProfile::video/audio/subtitles).
- Splits each title's Stream enum into three typed vectors; flattens
  normalized fields (language default "und", forced/sdh from the subtitle
  qualifier, commentary/descriptive from the audio purpose, label -> name).
- Hoists derived flags so downstream never recomputes: main_title (the
  pre-sorted main feature at index 0), is_main, and the default
  audio/video track (first non-secondary, keep only the first), subtitle
  default always false.

Make the profile types the single serialization source of truth: the
json:// sink now serializes TitleProfile instead of hand-building JSON,
so json:// and any downstream share one schema. Stream arrays are split
by kind and qualifier/purpose decomposed into booleans; json://-only
extras (clips, full chapter_marks list) are spliced back on. meta_sink
tests reconciled to the new schema.

Tests: profile normalization across BdTs + MpegPs titles, typed-vector
split, populated fields, hoisted main/is_main/default flags,
qualifier/purpose mapping, and a serde round-trip.
2026-08-25 12:43:24 -07:00
Matthew Jackson 49011d577f labels/deluxe: decode real per-stream labels, emit High (Universal + others)
The Deluxe parser detected com/bydeluxe discs but emitted nothing, so
mpls_universal won at Low. Grounding the decoder on real Universal discs
(Fast Five / F9 / Bourne, studio="uni") surfaced three defects that
kept the full Phase A-D pipeline from ever resolving a stream on any
real disc, all fixed here:

1. Master enums were keyed by the ZIP ENTRY name ("pd.class") but the
   binding class references enum constants by JVM INTERNAL name
   (getstatic pd.<field>), so Phase C's getstatic count never matched
   the identified enums and found zero binding classes. Key master enums
   by this_class_name instead.
2. MasterEnumTable resolved getstatic operands by enum VALUE string
   ("English") but the operand is the obfuscated static-FIELD name
   ("a"), so every language/purpose ordinal came back unresolved.
   Capture the ordered putstatic field names (clinit_enum_field_names)
   and key the resolver on them.
3. The Language-enum fingerprint demanded 70+-4 values; Universal's
   enum has 65, so Phase A rejected it outright. Give each fingerprint
   its own count window (Language widened to span 65 and 70); the
   ordered 4+-string prefix stays the real discriminator.

Also: model anewarray/newarray so array construction keeps the symbolic
stack aligned, and skip constructors with an array parameter so the
Universal title-wrapper object (oq.<init>(...,[Lnp;,[Lwb;,...)) is not
mistaken for a subtitle binding. Recover SDH/RNIB from the language
display name (Universal encodes it as a distinct Language value, not a
Purpose). Add config.xml <TitleConfig studio="..."> detection. Raise
confidence to High when the binding table decodes >=1 stream; the
mpls_universal floor still back-fills un-named STN slots.

Fixtures: verbatim pd/lp/tl .class files from FastFive.iso decode to
English/French/Spanish + English Commentary (Dolby Digital) audio and
per-title eng/spa/fra subtitles. Validated end to end: labels-analyze on
FastFive/F9/Bourne now reports parser=deluxe High with real per-stream
languages, names, purposes and codecs instead of mpls_universal Low.
2026-08-25 12:25:38 -07:00
Matthew Jackson f8e39abe54 labels: harden class_reader/jar as first-class .class format readers
Treat Java bytecode as a documented format we READ (never execute), a
peer of mpls/clpi. The class reader's bounds checking was already sound
(typed Reader with checked u8/u16/u32/u64 + slice, saturating switch
arithmetic, iterator halt-on-truncation); this adds the untrusted-input
entry points the never-panic harness was missing and the malformed-input
regression tests the format warranted.

- harness: add class_reader (magic CAFEBABE) and jar (zip PK header)
  sweep targets. The class_reader target also drives the accessors a
  label parser exercises (this_class_name, member Code walk, cp_index,
  member_ref, load_constant_display) so their indexing is fuzzed, not
  just the structural parse. The jar target opens the sweep bytes as a
  zip and walks every .class through for_each_class exactly as deluxe/dbp
  do.
- class_reader tests: oversized constant_pool_count and interfaces_count
  reject with UnexpectedEof (u16 counts keep with_capacity bounded);
  out-of-range cp indices return None from every accessor; a Code
  attribute whose declared code_length runs past its body is rejected.
2026-08-25 12:25:19 -07:00
Matthew Jackson 65d719ee5d disc: wire bdnav as the top nav-feature tier in main-feature selection
scan_with runs the bdnav VM for BDMV discs and feeds the video-gated result
as the new top MAIN_FEATURE_ORDER key (nav-feature > authoring-feature >
standalone > has-video). A nav result is only honoured for a video-bearing
title, so a mis-resolve can never reopen the decoy hole; when the VM abstains
(BD-J, non-convergence) the structural/heuristic order decides exactly as
before. Adds nav selection tests and updates the diag decision reason.
2026-08-25 12:19:16 -07:00
Matthew Jackson c3bdbdcfc9 bdnav: faithful HDMV navigation VM (index.bdmv + MovieObject.bdmv)
Add src/bdnav: a libbluray-faithful, bounded, never-panic HDMV navigation VM
that runs First-Play with correct power-on PSR/GPR state to resolve the
playlist the disc itself plays as the feature. One generic VM handles the
obfuscated Sony dispatcher and every other HDMV disc with no vendor
special-casing; it abstains (None) on the BD-J boundary, malformed data, or
non-convergence. index/mobj parsers added to the never-panic harness.
2026-08-25 12:19:16 -07:00
Matthew Jackson 3f4e69f7b8 labels: add Fox dcx.xml vendor parser
Older Fox discs ship a loose plain-XML manifest at
/BDMV/JAR/<id>/dcx.xml (root <dcx>) that names every feature stream's
language, editorial purpose and forced/SDH state in attributes — fully
parseable and previously unowned (2 Fox discs matched no parser, 11 were
mis-owned by the deluxe stub).

New src/labels/fox.rs:
  - detect(): loose dcx.xml under /BDMV/JAR/*/ (cheap dir walk), falling
    back to a com/foxbd/ jar central-directory prefix so newer Fox discs
    are attributed to Fox even though their bytecode form isn't decoded
    yet (Phase 2, sketched in-file).
  - parse(): scopes extraction to one <playlist name="feature"> element
    (the richest, to avoid merging regional 00800/00801 STN tables),
    reusing labels/xml.rs. audio type=rnib -> Descriptive, comment* ->
    Commentary; subtitle form=sdh -> Sdh, type=embed -> Forced; nested
    id="NN" -> 1-based per-type STN slot. High confidence.
  - Feature playlist id recorded via tracing::debug for now; dev's
    ParseResult has no feature_playlist field (that FeaturePlaylistHint
    slot lives on selection-hardening). An in-code note documents wiring
    it at reconciliation, matching paramount::parse.

Registered ahead of mpls_universal, alongside the High-confidence vendor
parsers; registry-order lock test updated.

Tests embed the real Life of Pi dcx.xml (/BDMV/JAR/05001), asserting the
00800 feature selection + id, all 11 audio labels (incl. the eng rnib
descriptive track) and 11 subtitle labels (sdh/forced/none), the
no-cross-playlist-merge scope rule, plus negative-detect cases.
2026-08-25 12:09:37 -07:00
Matthew Jackson 89b03610e7 disc: demote play-all/wrapper composites in main-feature selection
The has-video gate alone fixed Spider-Man 3 UHD only by accident: the
00245 decoy = [bumper][00001 feature clip][outro], and streams come from
PlayItem 0 (the bumper) only, so it reported v=0. That is fragile — a
video-bearing composite [feature][outro] still beat the feature on size
(audit R2), and a real feature with a non-video first PlayItem would be
wrongly disqualified (R6).

Add structural clip-set composite detection: a title whose clip set
properly contains another substantial video title's whole clip set is a
play-all/wrapper composite and is demoted below the standalone title it
wraps — independent of stream counts, physical size, and disc capacity
(which is 0/unknown when READ CAPACITY fails, disabling the oversize
gate). Replace the has-video disqualification with the more-permissive
has_probable_video (real video, or size+duration+clips consistent with
video) so a bumper-lead feature is never disqualified while streamless
menus still are. Corroborate the authoring hint by duration (a hint at a
short branch/trailer can't override the feature) and require real video
streams for it. Precedence: authoring -> standalone-over-composite ->
has-video -> canonical (with a reserved top slot for the future BD-nav
key). Unify the session.rs key-sampling/verdict pickers and
image_crack_extents on has_probable_video.

Tests: video-bearing composite (T2), oversize composite with unknown
capacity (T3), short-branch hint not honoured (T4), bumper-lead feature
not disqualified (T6), all-streamless deterministic + single-source pick
(T7), equal-clip twins not composite, video-only title. T2/T3/T6 fail
under the prior has-video-only logic.
2026-08-25 11:54:04 -07:00
Matthew Jackson a7cbb3afcd disc: harden main-feature selection against streamless decoys
Main-feature selection ranked purely on physical size -> duration ->
audio (canonical_title_order), with no requirement that the chosen title
carry a decodable video stream. A playlist-obfuscation decoy can inflate
a streamless playlist past the real feature on size AND duration (Spider-
Man 3 UHD: 00245.mpls, 78.4GB/8350s/v=0 beat 00001.mpls, 76.5GB/8038s/
v=1), so the mux ran a title with zero streams -> zero frames -> E6008.

- Add Disc::main_feature_order layering two gates atop canonical_title_order:
  authoring-feature (a title the disc's menu authoring designates AND that
  has video) wins outright; then has-video demotes any streamless title
  below every title that has video. scan_with re-sorts with it after labels
  run so titles[0] is the selected feature. New MAIN_FEATURE_ORDER_KEYS +
  DiscTitle::has_video().
- Surface the authoring feature-playlist id: labels::FeaturePlaylistHint,
  captured from Paramount playlists.xml <playlist name="Feature" id=..>,
  threaded through ParseResult/extract; labels::apply now returns it.
- Unify pickers so key-sampling and the structural verdict (session.rs) and
  image_crack_extents prefer the has-video feature, not a streamless decoy.
- diag: decision row logs video= count + main_feature_order reason and a
  tag=decision.demoted line per no-video title (closes the field-diagnosis
  gap this incident exposed).

Tests: main_feature_order_demotes_streamless_decoy (pre-gate vs post-gate
contrast), _honours_authoring_hint, authoring_hint_never_promotes_a_
streamless_title, feature_playlist_hint_matches_on_id_or_filename; diag
reason test updated. QA green on 1.94 (fmt/clippy -D warnings/test).
2026-08-25 11:23:44 -07:00
Matthew Jackson f75766b4d5 chore: lower MSRV to 1.94, extend disc provenance tests, tidy comments
Lower the crate MSRV from 1.97 to 1.94 -- the lowest toolchain on which
`cargo build`, `cargo clippy --all-targets -D warnings`, and
`cargo test --tests` all pass clean. Dependencies floor the build at 1.90
(freemkv-unlock 1.90, zip 1.88), but clippy is only warning-clean from
1.94 (older clippy has a `duplicated_attributes` false positive on
`#![cfg(test)]` and flags a now-documented `let_and_return`). Update
`rust-version` and every `dtolnay/rust-toolchain` CI pin to 1.94.0, and
add a CHANGELOG Unreleased entry.

Extend the byte-provenance coverage added in f7d38ca (currently one
TS-path test) with two PS-path tests driving the real `Mpeg2Parser`:
- `ps_stream_stamps_source_provenance`: the PS path forwards `ps.source`
  onto `PesPacket`, so demuxed frames carry `Some(SourcePos)`.
- `ps_stream_carries_feed_base_across_reads`: the cumulative `fed_bytes`
  base carries ACROSS read buffers -- a frame decoded from the second
  sector is stamped `source.byte >= 2048`, not reset per read.

Also record the f7d38ca provenance fix in the CHANGELOG, inline a
`let_and_return` test binding, and trim a redundant sentence from
`PsDemuxer::feed_at`'s doc comment.
2026-08-25 10:35:47 -07:00
dependabot[bot] 1c313fe882 Bump actions/checkout from 5 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-24 08:52:57 -07:00
Pascal Harprecht f7d38caf0b mux/disc: pass feed base offset to demuxer to stamp byte provenance
On direct disc reads (`disc://...`), `DiscStream` called `demuxer.feed(&self.read_buf[..bytes])`
instead of `feed_at(buf_base, ...)`. Without a running feed base offset, demuxed `PesPacket`s
had `source = None`, stripping byte provenance from downstream `PesFrame`s.

Under a `SeamPlan`, when `src_byte` is `None`, `SeamPlan::place` falls back to a timestamp
heuristic across clips. On multi-clip playlists where a subsequent clip starts at the same
base timestamp as the initial clip (`in_time = 600s`), B-frame PTS reorder dips (where
`raw_ns < last` and `|raw_ns - next_in| <= 250ms`) falsely triggered a `stepped_back` clip
transition on the first reordered frame. Because the subsequent clip's span was short (e.g. 1s),
all subsequent video frames across the rest of the title were rejected and dropped as outside
clip marks. Audio tracks (which have monotonic PTS without reordering) were unaffected.

Track `fed_bytes` in `DiscStream` and route through `feed_at` for both TS and PS demuxers,
forwarding `ps.source` to `PesPacket` on the PS path.

Test: `mux::disc::tests::stream_surface_tests::disc_stream_stamps_source_provenance`
2026-08-24 08:52:19 -07:00
Matthew Jackson e98e98c8fd TrueHD: resync forward to next major sync on a source discontinuity
MLP/TrueHD carries predictor + restart state across access units, so after a source transport-stream discontinuity (continuity-counter gap) the parser was splicing post-gap audio mid-stream, decoding against stale state and producing a decoder-choking seam (a burst of "restart header sync incorrect" / "Invalid blocksize" errors that could flag the whole track corrupt).

Honor the source's own gap signal: drop forward to the next CRC-valid major-sync AU (a clean re-init point) before resuming, guarded so it never arms before a validated baseline exists (no stream-head whole-track drop). Mirrors the video keyframe resync and the ResyncGate design. Corrects the "audio has no inter-frame refs" comments in truehd/dts/resync. Adds a regression test.
2026-08-23 16:30:29 -07:00
Matthew Jackson 7c085db133 restore freemkv-unlock path dep for local dev (post-v1.6.9) 2026-08-22 08:34:31 -07:00
Matthew Jackson 587e26353c v1.6.9: bump version (freemkv-unlock git-pinned for the tag) 2026-08-22 08:34:26 -07:00
Matthew Jackson 3eb3e649b1 changelog: 1.6.9 unified-release entry 2026-08-22 08:24:55 -07:00
Matthew Jackson b01505983c restore freemkv-unlock path dep for local dev (post-v1.6.8) 2026-08-21 12:54:53 -07:00
Matthew Jackson 87bc07dad8 v1.6.8: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 12:54:48 -07:00
Matthew Jackson 743c5830b6 docs: 1.6.8 changelog (unified release) 2026-08-21 12:47:34 -07:00
Matthew Jackson 80c8e4ca24 ci: require a CHANGELOG entry for the crate version on main
Adds a 'changelog' job that fails when Cargo.toml's version has no matching
'## [X.Y.Z]' heading in CHANGELOG.md, hard on main and PRs into main, warning
on dev/qa — so the unified release can't ship a crate with a blank changelog.
2026-08-21 11:23:45 -07:00
Matthew Jackson f49e79c198 restore freemkv-unlock path dep for local dev (post-v1.6.7) 2026-08-21 09:31:40 -07:00
Matthew Jackson 231cbf3d76 v1.6.7: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 09:31:31 -07:00
Matthew Jackson 21c7b12e2b docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson 799039df21 restore freemkv-unlock path dep for local dev (post-v1.6.6) 2026-08-20 20:03:48 -07:00
Matthew Jackson 497b7726dc v1.6.6: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 20:03:44 -07:00
Matthew Jackson 94b549a9d8 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
Matthew Jackson fc2d35a19c v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew Jackson 3c51190c32 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson dfe0390445 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 8678a2bf94 drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson d953b46023 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 443ac6bcb1 Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 794393f24e CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit b02c402 ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-b02c402
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson c6e2614c4c Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson b02c402dae CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 128e3bb120 Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson dc100f1c80 Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 286b5d4527 ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson d9f3f82c65 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson e30209091e restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
Matthew Jackson ac46e2382f v1.6.4: bump version (freemkv-unlock git-pinned for the tag) 2026-08-15 09:13:26 -07:00
Matthew Jackson 862c1460c3 Trim single-clip titles to their playlist marks (1.6.4) 2026-08-15 08:14:31 -07:00
Matthew Jackson fea64d07e2 Stop an uncrackable VTS borrowing another VTS's title key
`resolve_vts_key` called the `Option`-returning `css::crack_key`, which
collapses "no scrambled sector was seen" with "scrambled sectors were
seen and no key came out", and then fell back to the disc-wide key. A
multi-VTS CSS DVD whose second title set resists the Stevenson scan was
descrambled under the FIRST set's key: corrupt PES behind an intact
header, written out with `complete = true` at exit 0.

`CrackOutcome` exists precisely to keep those two apart, and its doc says
callers must surface the second as a hard error. Every sibling path in
the crate already does — `Disc::decrypt_keys_for_title` and the mux path
both map `ScrambledUncracked` to `CssKeyMissing`. This was the one path
that did not. The ordering comment 15 lines above describes this exact
outcome as the bug it exists to prevent; ordering makes the crack far
more likely to succeed, but it cannot make a failed crack safe.

Also, three things nothing could catch:

- The demux output filename took the stream's language raw while the
  base beside it was sanitised. A language code is three raw STN bytes
  through `from_utf8_lossy`, and `00 00 00` is the ordinary "undefined"
  encoding on real discs — a NUL in a path fails `File::create` with
  InvalidInput, taking the whole export down before one track file
  opened. `sanitize` now maps control characters too; it did not.

- `css::crack_key_scan`'s short-read handling was dead code under test:
  every source in the module returned the full request, so reverting
  `advance` to the requested count, or dropping the `.max(1)`, left the
  suite green. The `.max(1)` is load-bearing — without it a source
  returning `Ok(0)` never moves the cursor and never increments the
  budget, so the scan spins forever. That mutation now HANGS the test
  rather than failing it, which is the honest demonstration.

- `MAX_SUBDIRS`'s const-assert carried `#[cfg(not(test))]` inside a
  `#[cfg(test)] mod tests`, so it compiled in no configuration and could
  never fire — the dead gate the test above it was written to replace.
  Moved to module scope, and verified it now rejects a wrong constant at
  compile time.
2026-08-11 17:47:56 -07:00
Matthew Jackson 6e8da15c34 Bound the third hostile CSV, and stop two docs overclaiming
`forced_sub` was the last unbounded attacker-controlled list in
paramount.rs. `MAX_COM_INDICES` capped the two `*_com1_idx` attributes;
this one had nothing capping it at all, and unlike them it has no value
to filter — a cell is a classification of the position it sits at, so
its bound has to be positional. Extracted as `forced_subs` for the same
reason `com_indices` was extracted: through `labels_from_feature` the
bound is unobservable, because the subtitle loop cannot reach those
cells either, so the assertion could not fail.

The `MAX_COM_INDICES` doc claimed the entry-allocation argument for the
whole constant. It is the VALUE filter that caps the set (values below
the bound, so at most that many distinct entries, however long the
attribute); the `take` caps the WORK. Both are real and they are not
the same bound; the doc now says which is which.

`jar_inventory_dedup_is_not_quadratic` called itself proof by deadline.
It is a hang guard — a return to the linear scan runs for minutes and
would wedge CI rather than fail it — and no assertion in it can tell a
BTreeSet from any other sub-quadratic dedup. Renamed and documented for
what it does. Its margin was measured before keeping it: 0.14s debug /
0.07s release against 10s, ~70x, unlike the 6x that made paramount.rs's
wall-clock test flake.
2026-08-11 16:59:33 -07:00
Matthew Jackson 8898351cb1 Round 2: stop two of round 1's tests claiming more than they prove
Both are mine, and both pass with the bound they "cover" deleted.

`a_hostile_commentary_index_list_is_bounded_not_merely_fast` asserted
only label purposes. The out-of-range filler it feeds is unobservable at
the label level and a HashSet collapses the repeats, so removing
MAX_COM_INDICES entirely leaves it green. What it DOES catch is a bound
set too low — verified at 2, where the real indices stop resolving. Named
and documented for that, and it no longer implies it guards enforcement.

`an_index_that_cannot_address_any_cell_is_not_retained` asserted through
the labels, where retention is by definition unobservable: the loop never
queries a cell that high. It now reads the set through `com_indices`,
where the claim is checkable.

Enforcement was and remains proven by
`distinct_unaddressable_indices_are_refused_not_stored`, which was red at
50,000 retained entries and green at zero. Two tests, two properties;
neither pretends to the other's job now.
2026-08-11 11:54:28 -07:00
Matthew Jackson 7a03be5496 Bound the commentary-index parse, and stop proving it with a clock
The gate caught `commentary_index_lookup_is_not_quadratic` failing, then
passing on a re-run. Measured both ways: 1.62s alone, OVER 10s against
its own 10s deadline while the suite's other 3,347 tests ran
concurrently. A 6x margin against a shared CPU is not a margin, and a
test that gets re-run until it passes is not a test.

It was also measuring the wrong thing. Replacing the linear scan with a
HashSet bounded the LOOKUP; the set was still built from every entry the
disc declared. `playlists.xml` is attacker-controlled and has no length
of its own, so a hostile disc could still force an unbounded allocation
before any lookup happened — the parse, not the query, was the exposure.
The code's own comment said "unbounded parsed input" and only fixed half
of it.

`MAX_COM_INDICES` bounds both halves, at the one value that cannot
change behaviour: an index at or beyond `u16::MAX` can never match a
cell, because the labelling loops break at `u16::try_from(i + 1)`. Real
authoring is nowhere near it — the BD STN table admits 32 streams.

The parse moved into a `com_indices` helper so the bound is OBSERVABLE.
Through `labels_from_feature` it is not: a HashSet collapses repeated
values, and an out-of-range index changes no label, so the obvious test
passes with or without the cap — an assertion that cannot fail, which is
what the first draft of this fix shipped. The test now hands in 50,000
DISTINCT unaddressable indices and asserts they are refused. Proven red
with the bound removed (50,000 kept), green with it.

Applies to both `sub_com1_idx` and `aud_com1_idx`.

NOT done here: `labels/mod.rs:1421`'s `jar_inventory_dedup_is_not_quadratic`
is the same wall-clock shape and has the same flakiness. Named so it is
not lost.
2026-08-11 11:30:53 -07:00
Matthew Jackson de94e12864 Cover the dir:// input door, and correct the CSS scramble-gate docs
Three round-1 audit findings on v1.6.0..HEAD.

Tests: `mux::resolve::input("dir://…")` — the door the CLI rips a folder
through — was never driven with a real folder. The only dir:// input test
uses a missing path, which fails in DirImage::open long before
`image_input`, so dropping the `is_folder` argument (and with it
`session::apply_folder_encryption_verdict`) passed the whole suite. Two
tests now close it, stated as a DIFFERENTIAL between the two doors,
because that is the invariant the shared function exists to hold: a
clear folder that kept its AACS/ directory must scan the same and select
the same extents through `scan_dir` and through `dir://`, and a
scrambled one must be refused through both with E9063. Proven red
against a mutated guard (both fail with E7022, the tree-shape verdict
the probe overrides) and green with it restored.

Docs: css/mod.rs documented `is_scrambled`, a function this line renamed
to `has_scramble_flag_bits`. Eleven stale sites, two of them broken
rustdoc links. Two were not stale names but false statements — they said
the descramble loop keeps the looser raw-flag test, when
`descramble_sector` and `descramble_region` both gate on
`is_scrambled_pack`; that parenthetical is rewritten to say what the
code does and why (the measured VIDEO_TS.IFO case: 38 titles became 10,
silently, at exit 0). The one mention that must stay is the sentence
explaining why the name was rejected.

Constants: io::image_writer and dirimage::encode each re-declared the
2048-byte sector that consts::SECTOR_BYTES already exports; both now
alias it. Note `BATCH_SECTORS = 2048` in image_writer is a different
quantity (sectors per batch, not bytes) and is deliberately left alone.
Bare 2048 literals elsewhere in the crate are out of scope here.
2026-08-11 10:23:37 -07:00
Matthew Jackson 21ec1d3f80 restore freemkv-unlock path dep for local dev (post-v1.6.3) 2026-08-10 09:04:40 -07:00
Matthew Jackson 6658a80819 v1.6.3: bump version (freemkv-unlock git-pinned for the tag) 2026-08-10 09:04:35 -07:00
Matthew Jackson 3db11a192c Carry the same licence and community files as the other repos
These are eight public repos that ship one product on one version, and
they had drifted: freemkv-engine carried no LICENSE at all (GitHub
reported its licence as none), two crates had no code of conduct, four
had no contributing guide, and none had a security policy — so there was
no private route to report a vulnerability in a disc-decryption tool.

SECURITY.md names GitHub Security Advisories on each repo rather than an
address, so there is nothing to keep in sync and no inbox to go stale.
2026-08-09 22:08:22 -07:00
Matthew Jackson a425a71cd5 Keep the cross-platform jobs for the release candidate
dev is where work lands and is meant to be pushed to often, so what runs
there should be the cheap answer to "did I break it": lint, tests and the
Linux build. The macOS and Windows jobs now run on qa and main instead of
on every push to dev.

Nothing is deleted and no platform goes unchecked before a release. qa.yml
already covers macOS and Windows independently, and the jobs that live only
here -- the Intel macOS build, the Windows release build -- still run, on
the branches where a cross-platform break is worth blocking on.

They are SKIPPED on dev via `if`, not left unscheduled. A queued job would
be worse than a slow one: release.sh's CI gate refuses while any run for
the commit is still in progress, so a job that never gets a runner blocks
releases silently, with no error anywhere. That is the failure the
real-media note in qa.yml describes, and it is why this is an `if` on the
job rather than a narrower set of trigger branches.
2026-08-09 20:24:07 -07:00
Matthew Jackson f79745d815 Release 1.6.3
Version bump and changelog for the 1.6.3 sync. All eight crates ship the
same version, so the crates with no functional change this cycle say so
rather than carrying an empty section.
2026-08-09 20:01:33 -07:00
Matthew Jackson 5d8f77e049 Drop six unused crates, and align the rest with the workspace
Two problems, both invisible until the whole graph is looked at together.

DEAD: num-bigint, sha2, num-traits, num-integer, cmac and cbc are
declared here and referenced nowhere -- not in src, tests or benches.
They were being compiled, audited and offered version bumps forever for
no reason. Removing beats bumping.

cbc nearly survived the sweep: a substring search for "cbc" matches 44
occurrences of ycbcr_to_rgb in the DVD subtitle decoder, so it looked
used. Only a word-boundary search exposed it.

SKEW: this crate was the outlier on every shared dependency -- aes 0.8,
rand 0.8, base64 0.22.1 and zip 2 against 0.9 / 0.10 / 0.23 / 8
elsewhere. Cargo cannot unify across a major version, so it compiled
BOTH: 32 duplicated crates in the freemkv binary's graph, including two
complete AES implementations (aes 0.8 + 0.9, cipher 0.4 + 0.5), two
digest stacks and two getrandom. Two crypto stacks in one product is
worth removing on its own.

The aes bump is an API rename -- BlockCipher-prefixed traits, Array for
GenericArray -- and the obvious translation uses Array::from_slice,
which the new version deprecates and clippy's -D warnings would reject.
These use the From<[T; N]> conversion the crate points at instead.

3441 tests pass in debug and release. The AACS crypto here is covered by
known-answer tests, so a byte-order or sizing mistake in that rename
could not have passed.
2026-08-09 18:37:24 -07:00
dependabot[bot] 79a16a9279 Bump softprops/action-gh-release from 2 to 3 (#4)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 17:14:30 -07:00
dependabot[bot] 219a77b581 Bump actions/checkout from 5 to 7 (#5)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:58:27 -07:00
Matthew Jackson be378cbc34 Have Dependabot watch the dependencies, and land its PRs on dev
Version updates were never configured here -- only security alerts, which
report but never open a pull request. So the absence of Dependabot PRs was
not "nothing to update", it was "nothing was checking".

Updates target `dev`, never `main`. main is a release pointer that
release.sh moves to each tag, so a bot commit on it would put work there
that no tag contains -- the exact state that aborted the 1.6.2 cascade at
its last step, and pointing a scheduled bot at main would recreate it
weekly.

Minor and patch bumps are grouped into one PR rather than one per crate:
eight repos times a handful of dependencies is a volume nobody reads, and
an unread PR queue is indistinguishable from no updates at all.

The freemkv crates are excluded. They depend on each other by git tag,
re-pinned by release.sh inside the release commit, and Dependabot cannot
see that cascade -- a PR bumping one could pin a version whose tag does
not exist yet.
2026-08-09 16:48:31 -07:00
Matthew Jackson 63200a1d1f Retire the README version bot
It fired on `release: published` and committed to main, the branch the
release points at its tag, so it could only ever leave main ahead of the
tag. release.sh now makes the same edit inside the release commit.

In this repo it was a no-op regardless: the pattern it rewrote does not
appear in this README, so it has been running and changing nothing.
2026-08-09 07:20:44 -07:00
Matthew Jackson 526dd3552b restore freemkv-unlock path dep for local dev (post-v1.6.2) 2026-08-08 22:18:33 -07:00
Matthew Jackson 8d17904459 v1.6.2: bump version (freemkv-unlock git-pinned for the tag) 2026-08-08 22:18:28 -07:00
Matthew Jackson 1bdb8add2f Stop trusting a byte offset when the feed is not what was measured
A clip's span is measured over the title's full extents when the disc is
scanned, and a frame is placed by the offset it was read from. Those only
agree while the mux reads every byte the scan counted.

On a disc carrying a forensic segment it does not: the read plan omits the
units belonging to another device group, so fewer bytes are fed than the
spans describe, and the shortfall grows through the title. Every frame past
the first segment then looks earlier than it is — placed in a clip it did
not come from, or dropped at a join for failing marks it was never inside.
The spans still tile one another perfectly, so the check that asks whether
they can be trusted cannot see any of it.

The plan is now compared against the extents it was built from. When they
differ the spans are dropped and placement falls back to timestamps, which
is what that path is for and what the surrounding comment already promised
would happen when an offset stops meaning anything. Ordinary discs are
untouched: with no forensic segment the plan IS the extents.
2026-08-08 19:42:53 -07:00
Matthew Jackson c9562c07a3 Make four guards testable, and one guard see both spellings
The provenance guard knew one way to lose a frame's source offset —
writing it as absent. Omitting the field entirely does the same thing,
because the frame type fills it in by default, and the guard read straight
past that. A parser rewritten into the second spelling would have kept its
green light while its track silently went back to being placed by guesswork.
It now reports which spelling it found and where.

The title-count clamp was asserted over a fixture too small to hold more
titles than the cap allows, so the walk stopped when the buffer ran out and
the clamp was never what bounded it — the assertion held with the clamp
deleted. The fixture now carries more entries than the cap.

The subdirectory limit was checked by restating the constant's own
definition; the guard itself had never run, and deleting it changed
nothing. The limit is lowered under test so a real folder can exceed it,
and the test now walks one and requires the refusal.

One test also carried two unrelated grounding notes while the test they
described had none, so reading the note above a test told you about a
different one.
2026-08-08 19:06:37 -07:00
Matthew Jackson 7f9dcf1870 Keep both halves of a clip file that two play items share
A playlist may point at the same .m2ts from several play items — a
seamless split, a looped segment, multiple angles. The file has one set of
bytes, so it has one span, and the byte a frame was read from therefore
identifies the FILE, not which play item's range it falls in. Placement
took the first of them and judged every frame against its marks, so
everything past that range's end was treated as material the playlist
excludes and dropped: half the clip missing from the rip, with the
timeline still charged for its full duration.

The offset narrows a frame to the run; only its timestamp can finish the
job, and each play item carries its own marks. A frame is now matched
against the marks of the entry it actually falls in, falling back to the
first when it falls in none, which leaves genuinely-excluded material
dropped as before.

The test for this construction asserted only that such a playlist is
trusted, and never placed a frame from the second range — so the loss it
described in prose was invisible to it. It now places one from every range
and requires them all to survive.

The deferred-mux replay of buffered frames wrote them without their source
offset, quietly sending the head of every such title down the timestamp
heuristic the rest of this work exists to retire. It passes the offset
through now, and the frame writer that omits it is compiled out of the
library entirely: no production path can discard provenance any more.

Also: the AC-3 buffer-reuse test could not see the regression it named.
Feeding equal-sized packets, a fresh allocation per call yields the same
capacity as a reused one. It now feeds a large packet then a small one,
where only a reused buffer keeps the larger capacity.
2026-08-08 17:18:35 -07:00
Matthew Jackson 3c5c681399 Audit round: an ADTS frame that cannot hold its own CRC, and three drifts
An ADTS header that declares a CRC follows must be at least nine bytes —
seven of header plus the two the CRC occupies — because the declared frame
length counts them. The structural gate compared against a flat seven and
never read the bit that says whether a CRC is there at all, so a frame
whose own header describes something impossible was accepted and handed to
the muxer as decodable.

The pipeline's spawn doc named Sweep, and the thread Sweep would have
created, as callers to look for. Neither has been in this crate since the
recovery passes moved out. The same paragraph already records fixing this
once, for a different departed caller — it simply drifted again a sentence
later, so it now says to name callers that live here or name none.

Whether a disc is structurally AACS-encrypted was spelled out by hand in
both the fast identify and the full scan. They agreed today; nothing made
them agree tomorrow, and disagreeing would mean the same disc reported
encrypted by one path and clear by the other. There is one definition now,
and the comment that pointed at it by line number points at its name.

The AC-3 parser built a fresh buffer on every packet — of the order of a
hundred thousand times per title — to work around a borrow it cannot
avoid. The copy stays; the allocation does not. The buffer is now lent out
and handed back, and a test pins that, because reverting it would be
invisible in behaviour.

Left alone deliberately: send/send_with_halt and finish/finish_with_halt
look like one action under two names, and are not. After the consumer
fails, one must still accept items and the other must refuse them; that
difference is what stops a producer reading an entire disc for a write
that died on its first frame. Collapsing them was tried here and the
existing test caught it. Both now say so where the choice is made.
2026-08-08 16:55:55 -07:00
Matthew Jackson 43d9fb583a Add an error for an image shorter than its recovery data
A recovery resumed against an image that has been truncated since the
previous pass — a full disk, an interrupted transfer, a remount — cannot
repair it: the pass only revisits the ranges the recovery data calls bad,
so everything past the cut stays a hole while the counts still describe a
whole disc. There was no way to say that. Reusing the recovery-data error
would have been wrong, because the recovery data is intact; the image is
not.

Registered in the uniqueness list, the code table and the range check, so
a future variant cannot silently reuse 6015 or map to the wrong code.
2026-08-08 13:35:44 -07:00
Matthew Jackson 60ed45c047 changelog: 1.6.2 2026-08-08 08:53:13 -07:00
Matthew Jackson 07cc9fa5f9 ci: run the dependents' test suites, not just a type-check
The `consumers` job checked out all five dependents against this commit and
ran `cargo check --all-targets`. That proves they still COMPILE, which
catches a changed signature and nothing else. The failures worth catching
here keep every signature intact and change behaviour: the library still
builds, the dependent still builds, and the dependent's tests are what go
red. Those never ran.

Run their suites instead. Same checkouts, same patched paths — only the
verb changes.
2026-08-08 08:46:17 -07:00
Matthew Jackson 1e6bb8babd Give a frame that outruns its epoch's video a provisional offset
The demuxer interleaves, so at a boundary the streams do not reset on the
same frame: audio for the next segment can reach the muxer before the video
frame that opens the epoch it belongs to. Only the primary video track may
open one, so those frames rode the just-ended offset and landed a whole
segment in the past. Downstream the strictly-monotonic block nudge then
crushed the entire run onto one instant a tick apart, which is audible.

Such a frame is recognised on its OWN raw PTS, which within an epoch only
advances, so a large backward step is unambiguous — a different signal from
the shared frontier, which is what the old false-positive ratchet keyed on.
It then rides a provisional offset private to its track, computed the same
way the video path computes a real one, and drops it the moment the video
retires an epoch, so the run rejoins with no seam.

The offset is deliberately private: it never writes offset_ns, never
advances the frontier and never retires an epoch, so it cannot move the
video timeline. Letting a passive track open a REAL epoch was tried first
and inflated a 476.776 s title to 656.216 s, because every track meets a
boundary at its own pace and the video path rebased again on top.

Measured on a real DVD title with 8 cell boundaries: audio frames stamped
inside one cadence 120 -> 0, outsized gaps 7 -> 0, subtitle 1 -> 0, and the
video span byte-identical at 476.484. On the HD-DVD title the remaining
boundary gap on its first audio track falls from 0.999 s to 0.275 s.
Acceptance: 76 pass, 0 fail, 0 skip.
2026-08-08 00:48:10 -07:00
Matthew Jackson 64f894c9f3 Place a straggler by its own epoch's end, not the current frontier
A non-video frame can reach the muxer after the video that opened the next
epoch has already been processed. It carries an old-epoch PTS, so adding
the new offset flings it forward by a whole clip. The remap that exists to
catch this compared the frame against the CURRENT frontier and only
accepted it within one backstep, which is the wrong yardstick: by the time
a straggler is seen the frontier has moved on into a new epoch, and how far
below it the frame lands says nothing about where it belongs.

Retiring an epoch now records the frontier it closed at alongside its
offset, and a straggler is judged against the end of the epoch its PTS came
from. Keeping the whole history rather than one previous offset also fixes
the case a single `prev_offset_ns` cannot express at all: a frame from two
or more epochs back.

Measured on a real HD-DVD title whose second audio track put its final
packet at 12834.587 s in a 6434.100 s file. That frame sits 23 s below the
current frontier — outside the old window, so it was refused — but 0.15 s
below the end of the clip it actually came from. It now lands at 6417.216,
one 32 ms cadence step after its neighbour, and the track spans 6416.160
with no outsized gaps. The other two streams are unchanged.

A title that never rebases retires no epoch, so the lookup finds nothing
and the mapping is bit-identical to before.
2026-08-08 00:02:57 -07:00
Matthew Jackson 3820348d2f test: cover the 1.6.1 provenance work where it was assumed, not asserted
Four parsers (dvdsub, flac, lpcm, mpegaudio) stamp a source offset on
every frame but had no test that read one back, so a regression to
`source: None` would have been caught only by the brace-balanced audit
in codec/mod.rs — a lint, not a behavioural check. Each now asserts the
emitted frame carries the offset of the packet that supplied its first
byte.

The Blu-ray feed spans had no direct test at all. Add one that walks a
multi-item playlist and requires the spans to tile the feed with no gap
or overlap; it catches a one-sector-per-clip drift, which is exactly the
error class that would misattribute frames near a seam.

`no_provenance_still_places_by_marks` asserted only that placement
returned something, which passes for a frame placed in the wrong clip.
Its probe timestamp lands in an overlap between two clips in the real
mark table, so pinning one clip would assert a coin-flip; instead
require the offset to be one that a clip actually containing that
timestamp would produce.
2026-08-07 22:16:02 -07:00
Matthew Jackson 102ae82933 Test the DVD title table's sector-offset arithmetic
The TT_SRPT pointer at 0xC4 is a SECTOR offset from the start of
VIDEO_TS.IFO. Reading it as a byte offset lands 1/2048th of the way in and
does not error — it silently returns a different title list, which is how
a disc enumerating 38 titles and an image enumerating 10 looked like a
scan difference rather than a failure.

The table already had tests for its entry layout, dedup and the 99-title
clamp. What it did not have was anything pinning the offset unit itself,
or the grouping of titles by title set, or a truncated table keeping the
rows it can read rather than failing a whole disc.
2026-08-07 22:01:37 -07:00
Matthew Jackson 9d05f54ddb restore freemkv-unlock path dep for local dev (post-v1.6.1) 2026-08-07 19:08:46 -07:00
342 changed files with 42840 additions and 27583 deletions
+63
View File
@@ -0,0 +1,63 @@
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+244 -35
View File
@@ -11,10 +11,56 @@ on:
pull_request:
jobs:
lint:
# ── Single source of truth for the MSRV ─────────────────────────────────────
#
# `rust-version` in Cargo.toml is the ONE place the MSRV is declared. This job
# reads it; every job that must build/test AT the MSRV consumes
# `needs.resolve-msrv.outputs.v` instead of a hardcoded pin, so the floor
# cannot drift out of sync across workflow files. The `test` job building
# green at this value IS the verification that the declared MSRV is real.
# clippy and the `consumers` job deliberately do NOT use it — they run on
# `stable` (lint debt is caught against current Rust; dependents float highest).
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# CI owns the changelog: main may never carry a released version without a
# CHANGELOG.md entry for it. The release orchestrator dates an existing
# `## [X.Y.Z]` heading — it does NOT invent one — so a version bump that
# forgot its changelog entry must fail here rather than ship a blank release.
# Hard-fails on main (and PRs targeting main); warns elsewhere so an
# in-progress dev/qa branch whose entry isn't written yet is not blocked.
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
set -euo pipefail
version=$(grep -m1 '^version = ' Cargo.toml | sed -E 's/.*"([^"]+)".*/\1/')
if grep -qE "^## \[${version}\]" CHANGELOG.md; then
echo "CHANGELOG.md has an entry for ${version}."
exit 0
fi
msg="CHANGELOG.md has no '## [${version}]' entry"
if [ "${GITHUB_REF_NAME:-}" = "main" ] || [ "${GITHUB_BASE_REF:-}" = "main" ]; then
echo "::error::${msg} — main requires a changelog entry for the released version."
exit 1
fi
echo "::warning::${msg} (allowed on ${GITHUB_REF_NAME:-unknown}; required before it can reach main)."
lint:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -26,33 +72,45 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
# fmt runs on the pinned MSRV toolchain; clippy runs on stable. Third
# parties compile our source on their own toolchain and never run our
# clippy, so the lint toolchain need not match the MSRV — only build/test
# must (see the test/cross jobs). Linting on stable makes new-lint debt a
# BLOCKING gate the pinned version would otherwise hide. The MSRV pin is
# installed second so it stays the default toolchain (fmt uses it
# implicitly); clippy is invoked explicitly with `+stable`.
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy, rustfmt
components: clippy
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
working-directory: libfreemkv
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
# Cargo.lock is now committed (issue #15) and the `deny` job audits the
# resolved graph, but --locked is still omitted: CI's sibling path-patch
# layout resolves the freemkv crates as path packages, which --locked
# can't satisfy — the same reason the binary crates omit it in ci.yml.
# --all-targets so TEST code is linted too. Without it this crate — the
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo clippy --all-targets -- -D warnings
- run: cargo +stable clippy --all-targets -- -D warnings
working-directory: libfreemkv
test:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -64,22 +122,67 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --tests
working-directory: libfreemkv
check-macos:
runs-on: macos-latest
# The memory orderings in `scsi::fd_handoff` — the fd hand-off between a
# recovery thread and `SgIoTransport::drop` — cannot be checked by `cargo
# test`: they compile to the same instructions on x86_64, so the WRONG ones
# pass every run. loom enumerates the executions the C++20 model permits and
# reports the leaked descriptor directly, which makes it the only regression
# coverage that part of the module has. If this job stops running, that
# coverage disappears silently behind a green build.
#
# Its own job rather than a step in `test` because `RUSTFLAGS` is part of the
# rust-cache key: folding it into `test` costs that job a full cache miss on
# every run. It takes the same sibling checkout as the jobs above even though
# `--no-default-features` drops freemkv-unlock from the BUILD — `--features`
# does not exempt an optional path dependency from manifest RESOLUTION, so
# without the sibling this job would fail the moment the branch tip carries
# the path-dep form of that dependency rather than a git tag.
loom:
needs: resolve-msrv
runs-on: ubuntu-latest
env:
RUSTFLAGS: --cfg loom
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --lib --no-default-features --features scsi fd_handoff
working-directory: libfreemkv
# `lint` runs clippy without `--cfg loom`, so this is the only place the
# model code is linted at all.
- run: cargo clippy --lib --no-default-features --features scsi --profile test -- -D warnings
working-directory: libfreemkv
deny:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -91,12 +194,54 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@stable
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check advisories bans licenses sources
manifest-path: libfreemkv/Cargo.toml
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -104,9 +249,23 @@ jobs:
working-directory: libfreemkv
check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
@@ -118,12 +277,14 @@ jobs:
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -149,21 +310,21 @@ jobs:
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { path: libfreemkv }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
@@ -178,7 +339,11 @@ jobs:
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
# The dependents span a higher MSRV than libfreemkv itself — the binaries
# (freemkv/autorip/bdemu) always track the highest supported Rust. Build
# them on `stable` so this job satisfies the highest dependent floor; the
# library's own lower MSRV (verified by the `test` job) cannot compile them.
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
@@ -187,13 +352,57 @@ jobs:
freemkv
autorip
bdemu
- run: cargo check --all-targets
# `cargo check` alone only proves the dependents still COMPILE against
# this commit. It cannot see a behavioural change — the library keeps its
# signatures and a dependent's tests start failing. That is the shape of
# every defect worth catching here, so run their suites too.
- run: cargo test --tests
working-directory: freemkv-keysources
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: freemkv-engine
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: freemkv
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: autorip
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: bdemu
# ── Coverage (non-required, additive) ───────────────────────────────────────
#
# Reports line coverage to Codecov. Deliberately NOT wired into any release
# gate: it is informational only (see codecov.yml) and no other job `needs:`
# it. Mirrors the `test` job's layout exactly — same MSRV toolchain (resolved
# from Cargo.toml), same dual checkout so `../freemkv-unlock` resolves, cache
# workspace and working-directory. cargo-llvm-cov comes as a prebuilt binary
# from taiki-e/install-action, so it is toolchain-independent.
#
# No `--locked`: like the fmt/clippy/test jobs above, the sibling path-patch
# layout resolves the freemkv crates as path packages, which --locked can't satisfy.
coverage:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: libfreemkv
- uses: codecov/codecov-action@v7
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: libfreemkv/lcov.info
slug: freemkv/libfreemkv
fail_ci_if_error: false
+17
View File
@@ -0,0 +1,17 @@
name: comment-guard
# Caps explanatory prose by COMMENT INTENT (not visibility):
# //! / /*! module docs — unbounded (module-scoped, can't hide per-item text)
# /// / /** */ item docs — 8 prose lines (fenced ``` examples excluded)
# // / /* */ asides — 3 lines (single-blank-separated runs count as one)
# Overflow belongs in the module //! or docs/<topic>.md with a one-line // pointer.
on: [push, pull_request]
jobs:
comment-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Run comment-guard
run: python3 ci/comment-guard.py
+32
View File
@@ -0,0 +1,32 @@
name: doc-links
# Fails on broken intra-doc links in `///`/`//!` doc comments (e.g. a `[Type]`
# link to a renamed/removed item). Catches doc drift that `cargo build` misses.
#
# libfreemkv path-deps ../freemkv-unlock on the branch tip, so — like ci.yml —
# both repos are checked out into sibling subdirectories for the path dep to
# resolve.
on: [push, pull_request]
jobs:
doc-links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- name: cargo doc (deny broken intra-doc links)
working-directory: libfreemkv
env:
RUSTDOCFLAGS: "-D rustdoc::broken_intra_doc_links"
run: cargo doc --no-deps --document-private-items --lib --bins
+41
View File
@@ -0,0 +1,41 @@
name: identity-guard
# Prevention: no commit may carry an unrelated-business or non-canonical author/
# committer email. dev@example.com (a dead corp address) once slipped onto the
# release cascade + a batch of local commits; this fails the build if it recurs.
on:
push:
branches: [dev, qa, main]
pull_request:
jobs:
identity:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Reject banned author/committer emails
env:
BEFORE: ${{ github.event.before }}
AFTER: ${{ github.sha }}
EVENT: ${{ github.event_name }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
PR_HEAD: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
# Unrelated business domains + a personal address normalized to the
# GitHub noreply. Real external contributors are unaffected.
BANNED='@example-a\.invalid|@example-b\.invalid|owner@example\.invalid'
if [ "$EVENT" = "pull_request" ]; then
range="$PR_BASE..$PR_HEAD"
elif [ -n "${BEFORE:-}" ] && [ "$BEFORE" != "0000000000000000000000000000000000000000" ] && git cat-file -e "$BEFORE" 2>/dev/null; then
range="$BEFORE..$AFTER"
else
range="$AFTER~1..$AFTER"
fi
bad=$(git log --format='%an <%ae>%n%cn <%ce>' "$range" 2>/dev/null | sort -u | grep -iE "$BANNED" || true)
if [ -n "$bad" ]; then
echo "::error::commit(s) carry a banned author/committer identity:"; echo "$bad" | sed 's/^/ /'
echo "::error::Commit as your GitHub noreply (1085847+MattJackson@users.noreply.github.com). Fix: git config user.email <noreply>; git commit --amend --reset-author."
exit 1
fi
echo "identity-guard: clean"
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Compute commit range
+31 -10
View File
@@ -24,6 +24,19 @@ on:
workflow_dispatch:
jobs:
# Single source of truth for the MSRV — see ci.yml. Jobs that build/test AT
# the MSRV read `needs.resolve-msrv.outputs.v` from Cargo.toml's rust-version;
# clippy (cross-lint) runs on stable instead.
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
@@ -43,7 +56,7 @@ jobs:
permissions:
contents: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
@@ -65,21 +78,24 @@ jobs:
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
needs: resolve-msrv
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
@@ -92,15 +108,17 @@ jobs:
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
# clippy runs on stable (matches ci.yml's lint job); build/test stay on
# the MSRV pin. See ci.yml for why the lint toolchain need not be the MSRV.
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
@@ -115,17 +133,20 @@ jobs:
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
needs: resolve-msrv
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
+14 -8
View File
@@ -17,7 +17,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -29,7 +29,7 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# gated by the local precommit (same MSRV, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
@@ -37,11 +37,17 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.97.0
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
# Cargo.lock is now committed (issue #15), but --locked is still omitted:
# the sibling path-patch layout resolves the freemkv crates as path
# packages, which --locked can't satisfy.
- run: cargo test
# NOTE: there is no crates.io publish job. libfreemkv is git-tag-only
@@ -56,8 +62,8 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
generate_release_notes: true
-36
View File
@@ -1,36 +0,0 @@
name: Update README version
on:
release:
types: [published]
permissions:
contents: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main
token: ${{ secrets.ORG_DISPATCH_TOKEN }}
- name: Update version in README
run: |
VERSION="${{ github.event.release.tag_name }}"
FILE="README.md"
# Update cargo dependency version (e.g. "0.2" -> "0.3")
MAJOR_MINOR="${VERSION#v}"
MAJOR_MINOR="${MAJOR_MINOR%.*}"
sed -i "s|libfreemkv = \"[0-9]*\.[0-9]*\"|libfreemkv = \"${MAJOR_MINOR}\"|" "$FILE"
- name: Commit and push
run: |
VERSION="${{ github.event.release.tag_name }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet || git commit -m "Update to libfreemkv ${VERSION}"
git push
-1
View File
@@ -1,5 +1,4 @@
/target
Cargo.lock
*.swp
*.swo
.DS_Store
+317
View File
@@ -1,5 +1,322 @@
# Changelog
## [1.7.0] — 2026-09-02
### Fixed
- `mux/mp4`: the capped child-box scan treated any `size < 8` as a stop, which dropped a 64-bit `largesize` child box (`size == 1`, real length in the following 8 bytes) and every sibling after it; a `size == 0` (run-to-end) box was mishandled too. Both are now honoured, so an MP4/MOV whose boxes use 64-bit sizes is walked fully instead of truncated.
- Linux: `SgIoTransport::raw_command` clamped an over-length CDB with `cdb.len().min(16)` instead of routing it through the shared `checked_cdb_len` guard that `execute()` uses. Under SPC-4 a CDB's length is fixed by its opcode, so truncating one does not shorten the command — it sends a descriptor the drive will read as something else. An empty CDB was likewise unguarded, and went to the sg driver as a zero-length command descriptor. Unreachable — `raw_command` is private and its one caller passes a fixed 6-byte ALLOW MEDIUM REMOVAL — but it was the one transport path that did the thing the guard exists to prevent.
- Linux: the fd hand-off between a background recovery thread and `SgIoTransport::drop` had no release edge, so a recovery thread that published its fd after teardown had drained the slot was not guaranteed to observe `dead == true` and could leave the descriptor unclosed. `Drop`'s claiming `swap` was `Acquire` (a relaxed store, heading no release sequence) and the recovery thread's `compare_exchange` was `Release` (a relaxed load, acquiring nothing) — neither side alone was enough, and upgrading only one still leaks. Both are now `AcqRel`. Worst case was a leaked file descriptor on a transport already being torn down.
### Changed
- The Linux recovery-thread fd hand-off moved to `scsi::fd_handoff`, where it is compiled and unit-tested on every platform (`linux.rs` is built on one host only) and model-checked under `loom`. No public API change.
### Security
- Fixed a soundness bug (GHSA-j8ww-f5fg-9pmh, low severity): `PrefetchedSectorSource::into_channels` handed out a `Sender<Vec<u8>>` recycle channel whose buffers the producer re-exposed with `unsafe { Vec::set_len }` guarded only by capacity, so a downstream caller recycling a `Vec::with_capacity(n)` (len 0) could drive `set_len` over uninitialized memory — undefined behaviour reachable from safe code. The producer now uses `Vec::resize`, removing the `unsafe`; buffer pooling (the real cross-thread alloc/free win) is unchanged. No in-tree caller triggered it.
## [1.6.14] — 2026-08-31
### Changed
- Version aligned to 1.6.14 for the unified release.
## [1.6.13] — 2026-08-28
### Added
- Capture a few additional drive buffers during `info --share` collection on supported drives.
## [1.6.12] — 2026-08-27
### Fixed
- macOS: hold a DiskArbitration claim (plus a mount-approval dissenter) for the SCSI transport's lifetime so `diskarbitrationd` can't remount the disc mid-rip — the cause of the E1000 "disc stolen mid-rip" failure on macOS.
### Changed
- Comment and documentation cleanup.
## [1.6.11] — 2026-08-26
### Added
- Navigation-driven main-feature selection: selection now follows the disc's
own on-disc menu/navigation logic — a Blu-ray HDMV navigation VM and a DVD
First-Play resolver — to reach the main feature, instead of guessing by
title size alone.
- Clip-set composite/decoy detection, so a title assembled from a composite
or decoy clip set is recognized as such during selection.
- A feature-payload floor, giving main-feature selection a minimum-size
sanity check.
- Normalized `DiscProfile` output.
- Vendor navigation-label parsing.
- Substantially expanded unit-test coverage across the navigation and
clip-selection paths.
### Fixed
- Main-feature selection no longer picks an obfuscated decoy playlist over
the real feature.
- Two rounds of code-audit hardening: navigation VM operand/field-offset
conformance, navigation now abstains rather than guessing on an
undecidable system-parameter branch, panic-safety fixes, log-escaping
fixes, and expanded fuzz/test coverage.
- Direct disc reads (`disc://…`): `DiscStream` now threads a cumulative feed
base offset into the demuxer (`feed_at`) for both the TS and PS paths and
forwards it onto each `PesPacket`, so demuxed frames carry byte-exact source
provenance (`SourcePos`). Without it, `SeamPlan::place` fell back to a PTS
heuristic that a B-frame PTS reorder dip could misread as a `stepped_back`
clip transition, dropping all subsequent video frames in multi-clip playlists.
### Changed
- Lowered the minimum supported Rust version (MSRV) from 1.97 to 1.94. This is
the lowest toolchain on which `cargo build`, `cargo clippy --all-targets
-D warnings`, and `cargo test --tests` all pass clean; CI toolchain pins were
moved to match. (The crate's dependencies floor `cargo build` at 1.90, but
clippy is only warning-clean from 1.94.)
## [1.6.10] — 2026-08-23
### Fixed
- TrueHD/MLP audio: on a source transport-stream discontinuity (lost or gapped
packets, signalled by a continuity-counter break), the parser now drops forward
to the next major-sync access unit before resuming, instead of splicing the
post-gap audio mid-stream. MLP carries predictor + restart state across access
units, so resuming mid-stream decoded against stale state and produced a
decoder-choking seam — streaming decoders emitted a burst of "restart header
sync incorrect" / "Invalid blocksize" errors and could flag the whole track
corrupt. This mirrors the video path (resync to the next keyframe after a gap)
and how the independent-frame audio codecs (AC-3, DTS, FLAC, AAC) already
recover; the fix is guarded so it never arms before a validated major-sync
baseline exists (no whole-track drop at stream head). The gapped instant itself
is in the source and cannot be recovered, but the muxed track is now
decoder-clean.
## [1.6.9] — 2026-08-22
### Changed
- Version aligned to 1.6.9 for the unified release. No functional changes to
this crate; the release was driven by autorip (automatic per-episode TV
ripping — each episode named `S{NN}E{MM}`, with TMDB runtime-aligned episode
numbering across multi-disc seasons — a Manual Rename option, and a unified
per-disc staging state file — see the autorip 1.6.9 notes).
## [1.6.8] — 2026-08-21
### Changed
- Version aligned to 1.6.8 for the unified release. No functional changes to
this crate; the release was driven by autorip (webhooks now fire per pipeline
stage — Rip / Mux / Move — with the Rip hook firing the moment the drive is
free again, plus a Ripper-tab activity-banner fix so it also shows during
moves — see the autorip 1.6.8 notes).
## [1.6.7] — 2026-08-21
### Changed
- Version aligned to 1.6.7 for the unified release. No functional changes to
this crate; the release was driven by autorip (per-webhook event selection,
a progress bar per moved artifact, and move-queue / webhook-error fixes —
see the autorip 1.6.7 notes).
## [1.6.6] — 2026-08-20
### Changed
- Version aligned to 1.6.6 for the unified release. No functional changes
to this crate; the release was driven by autorip (webhooks may now target
private/LAN addresses — see the autorip 1.6.6 notes).
## [1.6.5] — 2026-08-20
### Fixed
- **On some Blu-ray/UHD titles the picture stopped short of the declared
end.** When a title is stitched across a clip seam, freemkv drops video
after the join until it sees a fresh keyframe it can restart decoding
from. It only recognised one kind of keyframe (an IDR frame), but a
Blu-ray's final segment can open on a different kind of self-contained
frame instead — so on those discs the restart point was never
recognised and every remaining frame to the end of the film was
dropped, losing up to a group-of-pictures-plus tail of picture. That
kind of frame is now recognised as a valid restart point, matching how
the HEVC path already worked, so the tail is kept. HEVC and DVD titles
were never affected.
- **A split HD-DVD feature could be exported as half the film while still
claiming the full running time.** A feature stored across parts (for
example `FEATURE_1.EVO` + `FEATURE_2.EVO`) is composed back into one
title. If a part resolved with no usable data — a zero-length file, or
an extent map that yielded nothing — that part was neither used nor
flagged, so the feature was quietly composed from the surviving parts
alone, still advertising the whole runtime, and exited success with no
log. The missing part is now marked unusable and logged under its own
code (E6019), so half a movie can no longer be presented as a whole one.
- **A Blu-ray title with an unreadable clip could be exported short while
still reporting the full running time.** A scratched or malformed
clip-info (`.clpi`) sector, or a clip whose extents could not be
resolved (a bad sector, a broken allocation chain, an embedded-data
file), was silently skipped — but the title's duration had already been
counted from the playlist, so it shipped short of the runtime it claimed
at exit success, with nothing logged. Previously only one narrow failure
kind counted; now every unresolvable clip drops the title and warns with
the read's own error code. A truly absent optional file (such as the 3D
`.ssif` on a 2D disc) is still treated as benign.
- **Pressing Stop during a disc scan could leave the disc looking like it
holds fewer titles than it does — or none at all.** Once cancelled,
every remaining drive command fails, but the Blu-ray and DVD title
enumerators treated those failures as ordinary skips: the Blu-ray scan
returned a truncated title list and the DVD scan returned zero titles,
both at exit success, so a cancelled scan was indistinguishable from a
disc that simply held that many titles. A cancel is now propagated as an
error out of every enumerator and at every read site, so it can never be
cached, displayed, or ripped from as if it were the real disc. HD-DVD
already behaved this way and is the model the others now follow.
- **A decrypted HD-DVD could be refused with a DVD copy-protection error
(E7023).** HD-DVD and DVD share the same MPEG program-stream container,
and the scramble detector mistook an HD-DVD navigation packet for a
CSS-scrambled sector — so a good, already-decrypted HD-DVD was reported
as carrying an unrecoverable CSS key and hard-failed, sending anyone
triaging it hunting for a missing DVD key on a disc that never had one.
The detector now excludes the structural packet types that CSS never
scrambles, identified from a field that is readable even on ciphertext,
so a decrypted HD-DVD scans clean. The DVD CSS crack is unchanged and a
genuinely scrambled, uncrackable DVD still hard-fails as before, so
ciphertext can never be muxed as plaintext.
- **A file with an allocated-but-never-written region could splice
undefined sectors into the rip.** The UDF reader treated an
allocated-but-unrecorded extent as ordinary content and read whatever
happened to be on those sectors into the output, and it decoded
embedded-data files (which store their content inline, not as a sector
map) as if their bytes were an extent list, pointing the reader at
unrelated sectors. Such extents are now refused when they actually
occupy space, and embedded files are handled as their own case; a
legitimately zero-length file still reads as empty rather than dropping
its title.
- **Audio and subtitle tracks in many languages were all labelled
"undefined."** The language mapping recognised only a handful of codes
and collapsed fifteen others to `und`. Every ISO 639-1 language code is
now mapped, so those tracks carry their real language. The DVD subtitle
colour palette, which was written in the wrong order, is also corrected.
- **If the disc read-ahead thread died mid-rip, a whole title could be
fabricated and reported complete.** A prefetch producer that terminated
returned an end-of-data signal that the reader legitimately read as a
short read and zero-filled, so a failed read could be papered over with
zeros and the pass still reported as successful. It now reports a
distinct source-terminated error instead. Dead-bus drive faults are
likewise classified rather than flattened, so the wedged-drive recovery
path can see them, and the drive now responds to Stop during spin-up and
spin-down instead of staying deaf for up to ~30 seconds.
- **A holed extraction could climb to a clean 100% on the live progress
channel.** The progress feed hardcoded its unreadable-byte count to zero
and counted every zero-filled hole as good data, so a progress-only
consumer saw a damaged extraction finish spotless — even though the
authoritative result was already truthful. The real good/unreadable
split is now threaded through the live channel. Separately, an
unreadable HD-DVD authored clip order is now logged with its error code
instead of being silently discarded before falling back to the
per-clip heuristic.
### Security
- **Bounded several unbounded amplification axes in the HD-DVD and Blu-ray
scanners.** A crafted disc could drive the playlist nesting depth, the
title count, and the clips and chapters per title without limit, and
could force a repeated clip-name fallback probe — each of which alone
left the worst-case scan unbounded. All now carry positional caps (512
titles and clips, roughly ten times any retail disc) and the fallback
probe is memoized. No effect on a well-formed disc.
### Changed
- **How freemkv picks the per-platform drive code was consolidated, with
no change to reading a disc.** The drive layer chose its
operating-system-specific module separately inside each entry point; it
now selects that module once, so a new entry point cannot silently
forget a platform. Purely internal.
## [1.6.4] — 2026-08-15
### Fixed
- **On a few Blu-ray/UHD titles the sound ran on for half a minute after the
picture had ended.** A disc stores each part of a film as a clip, and the
playlist marks exactly where that clip's content begins and ends. Where a
title is a single clip, freemkv trimmed the picture to those marks but not the
sound — and some discs leave extra audio in the file past the end mark (a
quiet fade authored after the last frame of picture). That trailing audio was
copied through, so the file claimed one running time while carrying up to ~36
seconds more sound than picture. Measured on `The Bourne Supremacy`: the
picture ends at 1:48:26 as declared, but every sound track ran to 1:49:02.
Single-clip titles are now trimmed to their playlist marks the same way
multi-clip titles already were, so sound and picture end together at the
declared duration. A title that had no extra material past its marks is
byte-for-byte unchanged. Multi-clip titles were never affected.
- **A multi-title-set CSS DVD could descramble one title set under another
set's key.** When a DVD's second title set resisted the keyless title-key
recovery, the decrypt fell back to the disc-wide key instead of failing —
writing a corrupt title behind an intact header and reporting success at exit
0. A failed recovery is now a hard error, the same as every other path in the
crate already does; ordering makes the recovery more likely to succeed but
cannot make a failed one safe.
- **A disc whose stream language field was all-zero could abort the whole track
export.** An all-zero language code (the ordinary "undefined" value on real
discs) put a NUL byte into a demux output filename and failed file creation
before a single track opened. Control bytes in that field are now sanitised
the same way the rest of the name already was.
### Security
- **Bounded the last unbounded attacker-controlled list in the DVD label
parser.** A crafted IFO could grow the forced-subtitle index list without
limit; it now carries the same positional cap as the neighbouring command
lists. No effect on a well-formed disc.
## [1.6.3] — 2026-08-10
### Changed
- **Six crates that were never used have been removed, and nothing about reading
a disc has changed.** They were declared as dependencies but referenced
nowhere, so they were compiled into every build for nothing. The rest were
aligned with the versions the other freemkv crates use.
- **Two complete AES implementations were being built into the product.** Two
different releases of the cipher crates had been pulled in by different parts
of the tree and both were compiled. They are now one.
## [1.6.2] — 2026-08-08
### Fixed
- **A stray moment of sound at the end of an HD-DVD title, and a click at every
chapter break on a DVD.** Both came from the same thing. A disc's sound and
picture do not arrive in lockstep, and where a title is stitched from
segments, a few frames of sound can reach the muxer just before or just after
the picture that marks the join. Those frames were timed against the wrong
segment. On one HD-DVD title a single trailing sound frame was placed at
3h33m in a 1h47m film. On a DVD, roughly half a second of sound was squeezed
into an instant at each of eight chapter breaks — audible as a click, eight
times in an eight-minute title. Sound is now timed against the segment it
belongs to. Measured on both discs: the stray frame lands one frame after its
neighbour, and the DVD's collapsed runs and jumps are gone, with picture
timing unchanged to the millisecond. Blu-ray was never affected.
## [1.6.1] — 2026-08-07
### Added
Generated
+884
View File
@@ -0,0 +1,884 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "aes"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8eb277bec05f56a0e0591f155a484cbd0f4f07ff2905051a48c72f004f7ed58"
dependencies = [
"cipher",
"cpubits",
"cpufeatures 0.3.1",
]
[[package]]
name = "aho-corasick"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
dependencies = [
"memchr",
]
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "base64"
version = "0.23.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "bitflags"
version = "2.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da"
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "block-buffer"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
dependencies = [
"hybrid-array",
]
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "cc"
version = "1.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ad534f4357a5264cce5019c989cf66a4f0dc4e0d1b1d15f8aacec0ff7360273"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "chacha20"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06"
dependencies = [
"cfg-if",
"cpufeatures 0.3.1",
"rand_core",
]
[[package]]
name = "cipher"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c"
dependencies = [
"crypto-common 0.2.2",
"inout",
]
[[package]]
name = "const-oid"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c"
[[package]]
name = "cpubits"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "cpufeatures"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566"
dependencies = [
"libc",
]
[[package]]
name = "crc32fast"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8498c871161e1742aaa9d52551b2d6ebdd4c3d45a3be423e3728f33b955be550"
dependencies = [
"cfg-if",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb"
dependencies = [
"crossbeam-epoch",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-epoch"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "crypto-common"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
dependencies = [
"hybrid-array",
]
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer 0.10.4",
"crypto-common 0.1.7",
]
[[package]]
name = "digest"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
dependencies = [
"block-buffer 0.12.1",
"const-oid",
"crypto-common 0.2.2",
]
[[package]]
name = "either"
version = "1.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "errno"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "fastrand"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223"
[[package]]
name = "find-msvc-tools"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d45db016d36b838f563236e9193d0ee6ce38f3f68b6c94e914b4929c96bbb890"
[[package]]
name = "flate2"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
dependencies = [
"zlib-rs",
]
[[package]]
name = "freemkv-unlock"
version = "1.7.0"
dependencies = [
"aes",
"base64",
"num-bigint",
"num-traits",
"rand",
"serde",
"serde_json",
"sha1",
"sha2",
"tracing",
]
[[package]]
name = "generator"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3b854b0e584ead1a33f18b2fcad7cf7be18b3875c78816b753639aa501513ae"
dependencies = [
"cc",
"cfg-if",
"libc",
"log",
"rustversion",
"windows-link",
"windows-result",
]
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]]
name = "getrandom"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"rand_core",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "hybrid-array"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "707114b52a152fa7bdb290cd7cd5912d9467273b6d74e21b8d81aca1f8533f6b"
dependencies = [
"typenum",
]
[[package]]
name = "indexmap"
version = "2.14.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07aa2048142242915a31d35844fb311e0e53fcca590c3a0a40dcf1b841fa09eb"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]]
name = "inout"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4250ce6452e92010fdf7268ccc5d14faa80bb12fc741938534c58f16804e03c7"
dependencies = [
"hybrid-array",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "lazy_static"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]]
name = "libc"
version = "0.2.189"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libfreemkv"
version = "1.7.0"
dependencies = [
"aes",
"base64",
"crossbeam-channel",
"freemkv-unlock",
"libc",
"loom",
"memchr",
"rand",
"rayon",
"roxmltree",
"serde",
"serde_json",
"sha1",
"tempfile",
"tracing",
"zip",
]
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "log"
version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "loom"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "419e0dc8046cb947daa77eb95ae174acfbddb7673b4151f56d1eed8e93fbfaca"
dependencies = [
"cfg-if",
"generator",
"scoped-tls",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "matchers"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d1525a2a28c7f4fa0fc98bb91ae755d1e2d1505079e05539e35bc876b5d65ae9"
dependencies = [
"regex-automata",
]
[[package]]
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "nu-ansi-term"
version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys",
]
[[package]]
name = "num-bigint"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93e7820bc0a80a0238e650327316f929ba18d5be054b647490a3a6a339f3e7c0"
dependencies = [
"num-integer",
"num-traits",
]
[[package]]
name = "num-integer"
version = "0.1.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b"
dependencies = [
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "proc-macro2"
version = "1.0.107"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "rand"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80"
dependencies = [
"chacha20",
"getrandom",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
[[package]]
name = "rayon"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
dependencies = [
"either",
"rayon-core",
]
[[package]]
name = "rayon-core"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
dependencies = [
"crossbeam-deque",
"crossbeam-utils",
]
[[package]]
name = "regex-automata"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "roxmltree"
version = "0.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c20b6793b5c2fa6553b250154b78d6d0db37e72700ae35fad9387a46f487c97"
[[package]]
name = "rustix"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
dependencies = [
"bitflags",
"errno",
"libc",
"linux-raw-sys",
"windows-sys",
]
[[package]]
name = "rustversion"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
[[package]]
name = "scoped-tls"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
[[package]]
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.4",
]
[[package]]
name = "serde_json"
version = "1.0.151"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "sha1"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"digest 0.10.7",
]
[[package]]
name = "sha2"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
dependencies = [
"cfg-if",
"cpufeatures 0.3.1",
"digest 0.11.3",
]
[[package]]
name = "sharded-slab"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f40ca3c46823713e0d4209592e8d6e826aa57e928f09752619fc696c499637f6"
dependencies = [
"lazy_static",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "simd-adler32"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
[[package]]
name = "smallvec"
version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9be42f50aa861c555654aa3a37f52f4b1074bacf4e48fe0ef7fa584e80f1f0f"
[[package]]
name = "syn"
version = "2.0.119"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6275cddf4610d1775e6d1fe9469b2e77d0f39fd98fb7450901b821e0c53649f"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "tempfile"
version = "3.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom",
"once_cell",
"rustix",
"windows-sys",
]
[[package]]
name = "thread_local"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ad99c4c6d32803332c548b1af0540b357b3f5fc0be8f6c6bfe8b2e6ae784070"
dependencies = [
"cfg-if",
]
[[package]]
name = "tracing"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"pin-project-lite",
"tracing-attributes",
"tracing-core",
]
[[package]]
name = "tracing-attributes"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "tracing-core"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
"valuable",
]
[[package]]
name = "tracing-log"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee855f1f400bd0e5c02d150ae5de3840039a3f54b025156404e34c23c03f47c3"
dependencies = [
"log",
"once_cell",
"tracing-core",
]
[[package]]
name = "tracing-subscriber"
version = "0.3.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319"
dependencies = [
"matchers",
"nu-ansi-term",
"once_cell",
"regex-automata",
"sharded-slab",
"smallvec",
"thread_local",
"tracing",
"tracing-core",
"tracing-log",
]
[[package]]
name = "typed-path"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
[[package]]
name = "typenum"
version = "1.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "valuable"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "zip"
version = "8.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b"
dependencies = [
"crc32fast",
"flate2",
"indexmap",
"memchr",
"typed-path",
"zopfli",
]
[[package]]
name = "zlib-rs"
version = "0.6.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34b31d188d9d685a4f9c7b46d6e36631b07058d2cfe190267adce54dc230bf12"
[[package]]
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[patch.unused]]
name = "freemkv-engine"
version = "1.7.0"
[[patch.unused]]
name = "freemkv-i18n"
version = "1.7.0"
[[patch.unused]]
name = "freemkv-keysources"
version = "1.7.0"
+78 -22
View File
@@ -1,8 +1,8 @@
[package]
name = "libfreemkv"
version = "1.6.1"
version = "1.7.0-rc5"
edition = "2024"
rust-version = "1.97"
rust-version = "1.88"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
@@ -18,51 +18,79 @@ publish = false
lto = "thin"
codegen-units = 1
# ── Feature layout ───────────────────────────────────────────────────────────
# `scsi` — the SCSI passthrough transport layer ONLY (`src/scsi/` + `error`):
# ScsiTransport/execute, ScsiResult/ScsiSense/parse_sense, DriveInfo,
# list_drives, drive_has_disc, CDB builders, the per-OS backends and
# the raw-`cc` macOS shim (built by build.rs). Pulls only `tracing`
# (transport instrumentation) plus the always-on `libc` target dep —
# none of the ripping dependency tree.
# `rip` — the full ripping tree (drive lifecycle, disc/UDF/MPLS/CLPI parsing,
# AACS/CSS decrypt, PES/mux pipeline, keydb, freemkv-unlock seam). All
# heavy deps (serde, zip, rayon, roxmltree, rand, crossbeam, memchr,
# sha1, aes, base64, freemkv-unlock, serde_json) live here.
# `default = ["rip", "scsi"]` preserves today's full build byte-for-byte;
# consumers wanting transport only use `default-features = false, features =
# ["scsi"]`. `rip` implies `scsi` because the ripping tree reads through it.
[features]
default = ["rip", "scsi"]
scsi = ["dep:tracing"]
rip = [
"scsi",
"dep:serde",
"dep:serde_json",
"dep:sha1",
"dep:aes",
"dep:freemkv-unlock",
"dep:rand",
"dep:zip",
"dep:base64",
"dep:roxmltree",
"dep:crossbeam-channel",
"dep:rayon",
"dep:memchr",
]
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10"
sha2 = "0.10"
aes = "0.8"
cbc = "0.1"
serde = { version = "1", features = ["derive"], optional = true }
serde_json = { version = "1", optional = true }
sha1 = { version = "0.10", optional = true }
aes = { version = "0.9", optional = true }
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.1" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.7.0-rc5", optional = true }
rand = { version = "0.10", optional = true }
zip = { version = "8", default-features = false, features = ["deflate"], optional = true }
base64 = { version = "0.23", optional = true }
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = "0.20"
roxmltree = { version = "0.20", optional = true }
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Permitted
# and friends). Part of the `scsi` feature — the transport layer traces here.
# Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
tracing = { version = "0.1", optional = true }
# Bounded MPSC channel with kernel-wakeup send_timeout. Used by `io::pipeline`
# so the halt-aware send/finish loops can BLOCK on consumer drain instead of
# polling — the 50 ms poll cadence of the previous mpsc-based impl capped mux
# throughput at ~1 MB/s (0.21.7).
crossbeam-channel = "0.5"
crossbeam-channel = { version = "0.5", optional = true }
# Persistent work-stealing thread pool for parallel AACS unit
# decryption. Per-call std::thread::scope spawned fresh OS threads
# and that overhead dominated for typical batch sizes (60 units).
# rayon's global pool initialises once on first use.
rayon = "1"
rayon = { version = "1", optional = true }
# SIMD-accelerated bytestring search. Drives the HEVC/H.264 start-code
# scan in `mux::codec::h264::find_start_code` — naive byte-by-byte
# walk is ~500 MB/s single-thread on x86_64; memchr's vectorised
# `memmem::find` for the 3-byte `00 00 01` needle hits ~5 GB/s on
# AVX2-capable hosts.
memchr = "2"
memchr = { version = "2", optional = true }
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
@@ -73,7 +101,35 @@ libc = "0.2"
[dev-dependencies]
tempfile = "3"
# Permutation model checker for the `scsi::fd_handoff` atomics — the fd
# hand-off between a Linux recovery thread and `SgIoTransport::drop`. A normal
# test cannot see a missing release edge there: the orderings compile to the
# same instructions on x86_64, so the wrong ones pass every time. loom
# enumerates the executions the C++20 model permits and reports the leaked
# descriptor directly.
#
# Behind `cfg(loom)`, which nothing sets unless you ask for it, so this is out
# of the graph for every ordinary build and for every CI job but the dedicated
# `loom` one. Run it with:
#
# RUSTFLAGS="--cfg loom" cargo test --lib --no-default-features \
# --features scsi fd_handoff
#
# `cargo test --lib` specifically: as a DEV-dependency loom is only in scope for
# test targets, which is why `fd_handoff` gates its import on `all(loom, test)`
# and falls back to std elsewhere. Without that, `--cfg loom` would break plain
# `cargo build`/`clippy` with an unresolved-crate error.
[target.'cfg(loom)'.dev-dependencies]
loom = "0.7"
[[bench]]
name = "sgio_read"
harness = false
[lints.rust]
# `loom` is set by RUSTFLAGS for the model checker above, never by a feature.
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(loom)'] }
[lints.clippy]
uninlined_format_args = "allow"
+8 -2
View File
@@ -1,7 +1,7 @@
# libfreemkv — local dev helper.
# Mirrors the workspace-wide CI checks but scoped to this single crate.
.PHONY: test build check ci clean
.PHONY: test build check loom ci clean
test:
cargo test --tests
@@ -13,7 +13,13 @@ check:
cargo fmt --check
cargo clippy --all-targets -- -D warnings
ci: check build test
# Model-check the scsi::fd_handoff atomics. `cargo test` cannot fail on those
# orderings — they compile to the same instructions on x86_64 — so this is the
# only check on them. Mirrors the `loom` CI job.
loom:
RUSTFLAGS="--cfg loom" cargo test --lib --no-default-features --features scsi fd_handoff
ci: check build test loom
clean:
cargo clean
+8 -8
View File
@@ -1,10 +1,12 @@
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![CI](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml/badge.svg)](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/freemkv/libfreemkv/branch/dev/graph/badge.svg)](https://codecov.io/gh/freemkv/libfreemkv)
# libfreemkv
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Drive-level unlocking is handled internally; consumers work with disc access and decryption only.
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, and content reading in one crate. Drive-level unlocking is handled internally; consumers work with disc access and decryption only.
DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require a `keydb.cfg` (default `~/.config/freemkv/keydb.cfg`) supplying disc-specific volume unique keys; no AACS key material is compiled in.
DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require disc-specific volume unique keys, supplied by the consumer (e.g. via [freemkv-keysources](https://github.com/freemkv/freemkv-keysources), which owns `keydb.cfg` lookup/download); libfreemkv itself never reads `keydb.cfg` or downloads keys, and no AACS key material is compiled in.
**12+ MB/s** sustained read speeds on BD. Drive prep (`init()`) handles unlocking internally via the `freemkv-unlock` crate — clients never see it; when no drive unlock applies, the library rips via the host-certificate AACS handshake.
@@ -70,9 +72,8 @@ cannot call into it; front-ends get recovery from the engine directly. See
- **Drive access** — open, identify, internal unlock + prep, speed control, eject
- **12+ MB/s reads** — auto-detects kernel transfer limits, sustained full speed
- **Disc scanning** — UDF 2.50 filesystem, MPLS playlists, CLPI clip info
- **Stream labels** — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
- **Stream labels** — 7 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, DBP, Deluxe, Fox)
- **AACS decryption** — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
- **KEYDB updates** — download, verify, save from any HTTP URL (zero deps, raw TCP)
- **Content reading** — adaptive batch reads with automatic decryption
- **Stream I/O** — unified stream pipeline for reading and writing any format
@@ -94,7 +95,7 @@ Streams implement a single unified `pes::Stream` trait (re-exported as `PesStrea
DVDs (CSS) decrypt out of the box, with no external key file needed.
Blu-rays and UHD (AACS) require a `keydb.cfg` at `~/.config/freemkv/keydb.cfg` (or passed via `ScanOptions`). No AACS key material is compiled into the binary.
Blu-rays and UHD (AACS) require Unit Keys passed in via `ScanOptions`/`KeySpec`. libfreemkv does not read `keydb.cfg` or fetch keys itself — that's the consumer's job (see [freemkv-keysources](https://github.com/freemkv/freemkv-keysources)). No AACS key material is compiled into the binary.
## Architecture
@@ -109,10 +110,9 @@ Disc — scan titles, streams, AACS/CSS state
├── MPLS parser — playlists → titles + clips + streams
├── CLPI parser — clip info → EP map → sector extents
├── IFO parser — DVD title sets, PGC chains, cell addresses
├── Labels — 5 BD-J format parsers (detect + parse)
├── Labels — 7 BD-J format parsers (detect + parse)
├── AACS — key resolution + content decryption
├── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
└── KEYDB — download + verify + save
└── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
Streams — unified PES pipeline
├── PesStream — pes::Stream: one trait, read()/write() PES frames
+22
View File
@@ -0,0 +1,22 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/libfreemkv/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.
+1
View File
@@ -5,6 +5,7 @@ fn main() {
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
println!("cargo:rustc-link-lib=framework=DiskArbitration");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
+252
View File
@@ -0,0 +1,252 @@
#!/usr/bin/env python3
"""comment-guard: cap explanatory prose by COMMENT INTENT, not visibility.
Three tiers, keyed on the syntax the author reached for:
* `//!` / `/*!` — a module (or crate) design doc. UNBOUNDED. There is one per
file, at the top, documenting the module as a whole, so it can't be used to
smuggle per-item rationale the way item docs can. Long correctness / design
essays (e.g. a memory-ordering proof) legitimately live here.
* `///` / `/** */` — item documentation. Up to DOC_CAP *prose* lines. Fenced
``` code blocks do NOT count, so real examples aren't punished. This holds
whether or not the item is `pub`: a `pub(crate)` fn deserves a real doc
paragraph just like a `pub` one. Capping `///` here (rather than exempting
it, as the old guard did — issue #17) is what stops rationale being hidden
behind a doc sigil, while still leaving room for a genuine paragraph.
* `//` / `/* */` — a quick internal aside. Up to INLINE_CAP lines. This is the
shape that sprawls into walls, so it's kept tight: stanzas separated by a
single blank line count CUMULATIVELY, so one essay can't be split into runs.
Overflow belongs in the module `//!`, or in docs/<topic>.md with a one-line `//`
pointer — both show up in review.
Scans src/ and tests/ under the given root (default cwd). `--selftest` runs
built-in cases. Exit non-zero if any block exceeds its cap.
"""
import re
import sys
from pathlib import Path
INLINE_CAP = 3
DOC_CAP = 8
_RAW_OPEN = re.compile(r'r(#*)"') # raw string opener: r"…", r#"…"#, br##"…"## …
def _strip_line_comment(s):
"""Return (kind, body) for a whole-line comment, else (None, None).
kind is 'doc' for `///`/`//!`, 'plain' for other `//`. A trailing comment on
a code line is not a whole-line comment and is ignored (single-line, cheap).
"""
if s.startswith("//!"):
return "doc", s[3:]
if s.startswith("///") and not s.startswith("////"):
return "doc", s[3:]
if s.startswith("//"):
return "plain", s[2:]
return None, None
def _fence_toggle(body):
"""True if this doc body line opens/closes a ``` fenced code block."""
return body.strip().startswith("```")
def _classify(lines):
"""Yield per-line dicts: comment kind ('doc'/'plain'/None), doc-ness of
block comments, and code info. Block comments are tracked across lines.
"""
info = []
in_block = False
block_is_doc = False
raw_hashes = None # None = not in a raw string; int N = open, needs '"' + N*'#'
for raw in lines:
s = raw.lstrip()
rec = {"raw": raw, "s": s, "comment": None, "is_doc": False, "code": False}
# Inside a raw string literal (e.g. an embedded HTML/CSS/JS asset): the
# bytes are string data, not Rust comments, so the guard is blind to them
# until the string closes. Prevents `/* */` or `//` lines in an embedded
# page from being counted as comment blocks.
if raw_hashes is not None:
if ('"' + "#" * raw_hashes) in raw:
raw_hashes = None
info.append(rec)
continue
if in_block:
rec["comment"] = "doc" if block_is_doc else "plain"
rec["is_doc"] = block_is_doc
if "*/" in raw:
in_block = False
info.append(rec)
continue
# A block comment only starts a comment-line if it's the first token.
if s.startswith("/*"):
rec["is_doc"] = s.startswith("/**") or s.startswith("/*!")
rec["comment"] = "doc" if rec["is_doc"] else "plain"
if "*/" not in s[2:]:
in_block = True
block_is_doc = rec["is_doc"]
info.append(rec)
continue
kind, _ = _strip_line_comment(s)
if kind is not None:
rec["comment"] = kind
rec["is_doc"] = kind == "doc"
info.append(rec)
continue
# Code (or blank). Blank = empty after strip.
rec["code"] = s != ""
# A code line may OPEN a raw string that spans following lines. Detected
# only on code (comments were handled above), so `// see r"x"` is safe.
m = _RAW_OPEN.search(raw)
if m and ('"' + "#" * len(m.group(1))) not in raw[m.end():]:
raw_hashes = len(m.group(1))
info.append(rec)
return info
def violations(path):
lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
info = _classify(lines)
n = len(info)
out = []
i = 0
while i < n:
rec = info[i]
if rec["comment"] is None:
i += 1
continue
start = i
if rec["is_doc"]:
# Doc run: contiguous doc lines. `//!` / `/*!` module docs are
# UNBOUNDED; any other doc run caps PROSE (fenced ``` excluded) at
# DOC_CAP, regardless of the documented item's visibility.
j = i
prose = 0
in_fence = False
module_doc = False
while j < n and info[j]["comment"] == "doc":
s = info[j]["s"]
if s.startswith("//!") or s.startswith("/*!"):
module_doc = True
body = (
s[3:] if (s.startswith("///") or s.startswith("//!")) else s
)
if _fence_toggle(body):
in_fence = not in_fence
elif not in_fence and body.strip() != "":
prose += 1
j += 1
if not module_doc and prose > DOC_CAP:
out.append((start + 1, prose, DOC_CAP, "doc prose"))
i = j
continue
# Internal (plain) run, with single-blank bridging (cumulative).
j = i
count = 0
while j < n:
r = info[j]
if r["comment"] == "plain":
count += 1
j += 1
continue
if r["comment"] == "doc":
break
if r["comment"] is None and not r["code"]:
# blank: bridge only a SINGLE blank, else stop
if j + 1 < n and info[j + 1]["comment"] == "plain":
j += 1
continue
break
break # code
if count > INLINE_CAP:
out.append((start + 1, count, INLINE_CAP, "inline comment"))
i = j
return out
def main(argv):
if "--selftest" in argv:
return _selftest()
root = Path(argv[1]) if len(argv) > 1 else Path.cwd()
files = sorted(p for d in ("src", "tests") for p in (root / d).rglob("*.rs"))
total = 0
for f in files:
for start, length, cap, kind in violations(f):
rel = f.relative_to(root)
print(f"{rel}:{start}: {kind} block is {length} lines (max {cap})")
total += 1
if total:
print(f"\ncomment-guard: {total} comment block(s) exceed their cap.")
print(
f"Inline // asides cap at {INLINE_CAP}; /// item-doc prose caps at "
f"{DOC_CAP} (fenced examples excluded); //! module docs are unbounded. "
"Trim, move the rationale into the module //!, or docs/<topic>.md "
"with a // pointer."
)
return 1
return 0
def _selftest():
import tempfile
cases = [
# (rust source, expected number of violations)
# short /// doc -> ok
("/// short doc\npub fn a() {}\n", 0),
# /// on a PRIVATE item now gets the doc cap (8), so 4 lines is fine
("/// l1\n/// l2\n/// l3\n/// l4\nfn priv_a() {}\n", 0),
# ...but /// is NOT unlimited: 9 prose lines on a private item fails
("".join(f"/// l{k}\n" for k in range(9)) + "fn priv_b() {}\n", 1),
# 8 prose /// on a pub item -> ok (boundary)
("".join(f"/// l{k}\n" for k in range(8)) + "pub fn c() {}\n", 0),
# 9 prose /// on a pub item -> violation
("".join(f"/// l{k}\n" for k in range(9)) + "pub fn d() {}\n", 1),
# /// with a long fenced example -> example excluded, ok
(
"/// summary\n/// # Examples\n/// ```\n"
+ "".join("/// code\n" for _ in range(12))
+ "/// ```\npub fn e() {}\n",
0,
),
# pub(crate) gets the SAME doc cap as pub (no pub/private split anymore)
("".join(f"/// l{k}\n" for k in range(6)) + "pub(crate) fn f() {}\n", 0),
# bare inline 4 lines -> violation
("// a\n// b\n// c\n// d\nlet x = 1;\n", 1),
# blank-split dodge: 3 + blank + 3 inline -> cumulative -> violation
("// a\n// b\n// c\n\n// d\n// e\n// f\nfn z() {}\n", 1),
# plain /* */ block, 5 lines -> inline cap -> violation
("/* one\n two\n three\n four\n five */\nfn q() {}\n", 1),
# comment-like lines INSIDE a raw string are string data, not comments
(
'const H: &str = r##"<style>\n'
+ "".join("/* css note */\n" for _ in range(8))
+ '</style>"##;\nfn h() {}\n',
0,
),
# //! module doc may run long -> EXEMPT (no cap)
("".join(f"//! l{k}\n" for k in range(40)) + "\npub fn m() {}\n", 0),
# /*! block module doc may run long -> EXEMPT
("/*! l0\n" + "".join(f" l{k}\n" for k in range(30)) + "*/\nfn n() {}\n", 0),
]
ok = True
with tempfile.TemporaryDirectory() as d:
for idx, (src, want) in enumerate(cases):
p = Path(d) / f"case{idx}.rs"
p.write_text(src)
got = len(violations(p))
flag = "ok" if got == want else "FAIL"
if got != want:
ok = False
print(f" selftest {idx}: want {want} got {got} [{flag}]")
print("selftest PASSED" if ok else "selftest FAILED")
return 0 if ok else 1
if __name__ == "__main__":
sys.exit(main(sys.argv))
+18
View File
@@ -0,0 +1,18 @@
# Coverage is informational only: it never fails CI or blocks a merge/release.
# The `coverage` job in .github/workflows/ci.yml is non-required and no gate
# depends on it.
coverage:
status:
project:
default:
informational: true
patch:
default:
informational: true
comment: false
ignore:
- "tests"
- "examples"
- "benches"
+42
View File
@@ -0,0 +1,42 @@
# cargo-deny — gates the RESOLVED dependency graph (issue #15).
# Runs in CI against the committed Cargo.lock: RUSTSEC advisories, a permissive
# licence allow-list, and source/ban hygiene. Update the lock (cargo update)
# to clear an advisory rather than adding an `ignore` entry without cause.
[advisories]
yanked = "deny"
# ignore = ["RUSTSEC-0000-0000"] # only with a written justification
[licenses]
# The whole tree is permissive. r-efi's LGPL arm is satisfied by Apache-2.0
# (its expression is "Apache-2.0 OR LGPL-2.1-or-later"), so no copyleft is
# allowed here. Add a licence only when a real dependency introduces it.
allow = [
"MIT",
"Apache-2.0",
"Apache-2.0 WITH LLVM-exception",
"0BSD",
"Unicode-3.0",
"Unlicense",
"Zlib",
"BSD-2-Clause",
"BSD-3-Clause",
"ISC",
"CDLA-Permissive-2.0",
]
confidence-threshold = 0.9
[bans]
multiple-versions = "warn" # transitive dupes (digest 0.10/0.11 etc.) are noise, not a gate
wildcards = "deny"
# The freemkv siblings resolve as path deps under CI's path-patch layout, which
# reads as a wildcard version req. That is internal, not a supply-chain wildcard.
allow-wildcard-paths = true
[sources]
unknown-registry = "deny"
unknown-git = "deny"
# The freemkv siblings are first-party git-tag deps; allow git sources from
# our own org only (CI resolves them as git, not the local path override).
[sources.allow-org]
github = ["freemkv"]
+190
View File
@@ -0,0 +1,190 @@
# AACS Content Decryption — Implementation Notes
Rationale and background trimmed out of `src/aacs/content.rs` doc comments to
satisfy `ci/comment-guard.py`'s per-audience prose caps. See
[AACS Encryption](aacs.md) for the public-facing key resolution story and
[AACS Crypto Primitives](aacs-crypto.md) for the low-level AES primitives.
## `is_unit_aligned`
AACS aligned units (6144 B / 3 sectors) are anchored at the start of each
clip's encrypted region, so a read must begin a whole number of units past
that base for `decrypt_sectors` (which anchors units at buffer offset 0) to
align the CBC correctly. This is the single source of truth for the
decrypt-on-read gate, the inline and highway mux read paths, and the
key-validation sample reader — all key off this, never absolute `lba % 3`. A
disc whose clip `start_lba` is not itself 3-aligned would otherwise mis-gate
(reject readable units, then report "Decryption failed") on exactly the
titles whose clips land off a 3-boundary.
`saturating_sub` makes the `lba < unit_base` case well-defined — it clamps
the offset to 0, a unit boundary — rather than the latent `wrapping_sub` trap
where an underflow wraps to ~2^32 and, because `2^32 ≡ 1 (mod 3)`,
mis-reports the alignment (e.g. `lba == unit_base - 1` would falsely read as
aligned).
## `aacs_unit_encrypted`
The mux is the principal consumer of this signal for `BdTs` discs (via
[`is_clean`], see below). It is readable WITHOUT a key. CRITICAL: only
meaningful when `unit` is read at the correct clip-FILE-anchored boundary — a
disc-absolute / mis-aligned read makes the flag byte arbitrary mid-stream
data, which is why per-unit checks run clip-anchored, not in the whole-disc
sweep.
## `aacs_unit_needs_decrypt`
Buffer-iterating sites that may run twice over the same `buf` (the post-fetch
re-decrypt, sample collection, failure diagnosis) need an IDEMPOTENT "does
this still need work?" test, so `aacs_unit_encrypted` (the authoritative flag,
which decryption never rewrites) is composed with a "structure restored?"
check that flips once decrypted: TS syncs come back for `BdTs`; valid
`00 00 01 BA` packs come back for `MpegPs`.
## HD-DVD scramble flag (`PS_SCRAMBLE_OFF` / `PS_SCRAMBLE_MASK`)
BD/UHD/FMTS flag encryption uses the Copy Permission Indicator in the top 2
bits of byte 0 (the M2TS `TP_extra_header`). HD-DVD `.evo` is Program Stream —
byte 0 is a `00 00 01 BA` pack_start_code, NOT a CPI — so AACS reuses the
MPEG-2 `PES_scrambling_control` field instead: pack_header (14 bytes) + PES
start-code/`stream_id` (4) + `PES_packet_length` (2) puts the PES flags byte at
offset 20, with `PES_scrambling_control` in bits 5-4 (`& 0x30`). Non-zero =
encrypted. Derived from BackupHDDVD (`Header[20] & 0x30`) cross-checked
against MPEG-2 systems (ISO/IEC 13818-1).
UNVERIFIED against a real ENCRYPTED HD-DVD disc — none available to confirm
byte-exactly. Two open questions a real disc must settle:
1. A pack carrying an MPEG `system_header` (`00 00 01 BB`) before the PES
packet shifts this offset past 20.
2. Whether offset 20 is even readable pre-decrypt. BD keeps only the first 16
bytes clear (the seed) and AES-CBC-encrypts 16..6144 — under that model
offset 20 is ciphertext. BackupHDDVD reading `Header[20]` pre-decrypt
implies HD-DVD instead encrypts PES *payloads* with *clear* pack/PES
headers (per-PES model). If so, `decrypt_unit`'s 16-byte-seed model also
would not fit HD-DVD and needs its own path. TS is unaffected either way.
## `KEY_PROOF_PACKETS`
Four `0x47` syncs are 32 bits of MPEG-TS structure, but the per-UNIT
false-pass risk is NOT 2^-32: `is_clean_ts` accepts ANY four of the ~31
encrypted packets in a 6144-byte aligned unit, so for a wrong key (uniform
AES noise, `0x47` at 1/256 per packet) it is ≈ C(31,4)·256^-4 ≈ 7e-6, i.e.
~1e-5. 1-in-4-billion is the probability for four SPECIFIC packets and
overstates the margin by ~4000x; at `KEY_PROOF_PACKETS = 3` the per-unit rate
is ≈ C(31,3)·256^-3 ≈ 2.6e-4, so do NOT lower it on the strength of slack
that is not there. It is an ABSOLUTE proof floor, NOT a proportion — a unit
the key opened but whose content is bad-encoded (many non-conforming
packets) is proven by ANY four good packets, not rejected for the bad ones.
## `is_clean_ts`
The mux is its principal consumer for `BdTs` discs, reaching it through
`is_clean`: `mux::resolve`'s multi-CPS `pick` closure selects a unit key by
it, `probe_index_phase` reports each FMTS index's interleave parity by it,
and `decrypt::decrypt_sectors_mapped` uses it as the forensic-range verify
net. (The doc used to say "the mux never calls this", which invited a
maintainer to tighten or loosen the proof rule believing only whole-disc read
verification was affected — while it in fact changes which unit key a
multi-CPS disc muxes with and which phase an FMTS index is muxed at.)
Rule — evidence is ABSOLUTE, scaled to the packets that exist. Over the
ENCRYPTED packets (skip packet 0: its `0x47` sits in the clear 16-byte seed,
so it reads `0x47` for ANY key and is never evidence), let `E` = non-padding
content packets and `synced` = those carrying `0x47`. The key opened the unit
iff `E == 0` (nothing encrypted to prove) OR `synced >= min(E, KEY_PROOF_PACKETS)`.
* WRONG key → ~0 synced → fails (reaching 4 by chance ≈ 1e-5/unit, and every
unit of a clip would have to fluke — astronomically safe).
* RIGHT key, bad-encoded content → any 4 good packets pass; the bad ones are
the muxer's problem. (This is the false-negative the old 75% PROPORTION
caused — a mostly-bad unit the key opened was wrongly rejected.)
* `min(E, 4)` handles the end-of-clip fragment TAIL: a unit with only E=1
real packet (then source-zero padding) needs just that one to sync, so a
sparse-but-valid tail is never false-rejected. Padding (all-zero payload)
is excluded throughout.
## `is_clean_ps`
The Program-Stream arm of `is_clean` (HD-DVD `.evo`): every 2048-byte pack
begins with the pack_start_code `00 00 01 BA`; a 6144-byte AACS unit spans
three packs. Pack 0's start sits in the clear 16-byte seed (present
regardless of the key — mirrors `is_clean_ts` skipping packet 0); packs 1
and 2 (offsets 2048 and 4096) are in the encrypted region, so a wrong key
garbles them and this returns false (64 bits of discrimination). Validated
against real decrypted HD-DVD `.evo` (two real retail titles): pack starts
are exactly 2048-aligned, three per aligned unit, at offsets 0 / 2048 / 4096.
UNVERIFIED-HDDVD-DECRYPT: the pack structure here is confirmed on decrypted
rips, but that a real ENCRYPTED `.evo` decrypts to it via the same
6144-byte aligned unit (16-byte seed + AES-CBC over 16..6144) as BD is the
unverified assumption — we have no encrypted HD DVD. If HD-DVD decryption
yields garbage with a known-good key, the unit granularity is the suspect.
## `decrypt_unit` / `encrypt_unit`
Block Key = AES-128E(Kcu, seed) ⊕ seed (`[BD]` §3.10.1 Fig 3-8: encrypt the
clear 16-byte seed under the CPS Unit Key, XOR the seed back in — the
trailing ⊕ seed is load-bearing); then AES-128-CBC decrypt/encrypt bytes
16..6144 under the AACS IV. `encrypt_unit` is the exact inverse of
`decrypt_unit`, for authoring an encrypted disc image (and for building
genuinely-encrypted read-path fixtures); it does NOT set the encrypted flag,
because where that flag lives is container-specific, and keeping it out
keeps the crypto container-agnostic.
Source-zero padding packets (all 192 bytes zero on disc) are restored to
zero by `decrypt_unit`: their decrypted bytes are AES-noise from decrypting
zeros, but the source WAS zero, so writing the true source back is faithful
and gives the demux a tidy gap. Content packets are left EXACTLY as
decrypted — the decrypt path never rewrites content, so an authored-bad
packet passes through verbatim. `encrypt_unit` deliberately does NOT mirror
this: an all-zero plaintext packet encrypts to ciphertext that is not
all-zero, so it is not mistaken for padding on the way back and the round
trip is still exact. Authoring that wants true source-zero padding leaves
those packets unencrypted instead.
`encrypt_unit` returns `false` — encrypting nothing — when `unit` is shorter
than `ALIGNED_UNIT_LEN`. That case MUST be checked: the caller has already
set the container's encrypted flag by then (the contract requires
flag-before-crypto, since the header is the key seed), so ignoring the
result leaves a unit marked encrypted while still carrying plaintext, which
is the worst possible outcome for an authoring tool.
## `decrypt_bus` key-schedule regression (test: `decrypt_bus_expands_the_read_data_key_once_per_unit`)
MEASURED, not reasoned: `decrypt_bus` used to call `aes_cbc_decrypt` once per
2048-byte sector, and each call built its own AES-128 key schedule, so a
6144-byte aligned unit performed THREE key expansions under the same
loop-invariant `read_data_key`. On a 90 GB UHD read on a stock
(non-firmware-unlocked) drive — ~14.6 million aligned units — that is ~29
million redundant expansions on the per-unit decrypt hot path, for a key
that is constant for the whole disc. The counter is incremented inside
`crypto::new_cipher`, the single construction site.
## `IV0_PUBLISHED` (test fixture)
`[C]` §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation. Both
IV tests used to compute their expected value from `crypto::AACS_IV` itself,
so the constant was asserted against itself and NOTHING in the suite pinned
its bytes: swapping `AACS_IV` for `[0u8; 16]` left both tests passing (one
builds its ciphertext with the same value and the other cancels the change
in a triple XOR) while every real AACS disc decrypted to noise — block 0 of
every 6144-byte aligned unit and of every bus-encrypted sector XORed with the
wrong IV. `IV0_PUBLISHED` is the independent witness the tests assert
against instead.
## Encrypted-flag readers (`aacs_unit_seed_encrypted`, `aacs_unit_encrypted`)
`aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the guard
that stops a truncated encrypted fragment from being emitted as clear
content. It reads ONLY the two Copy Permission Indicator bits (`[BD]`
§3.10.2, byte 0 bits 6-7); the remaining six bits are `TP_extra_header`
arrival-timestamp bits and carry no encryption meaning. Both failure
directions are damaging and silent: a reader that answers "encrypted" for a
clear fragment discards good content, and one that answers "clear" for an
encrypted fragment writes ciphertext into the output as if it were video.
`aacs_unit_encrypted` is the AUTHORITATIVE gate and requires a WHOLE
6144-byte aligned unit: on anything shorter the flag byte is not guaranteed
to be the unit's, so it must answer `false` and leave the partial-unit case
to `aacs_unit_seed_encrypted`. A reversed length guard would both classify
fragments off arbitrary mid-stream bytes and, on an empty slice, index out
of bounds.
+89
View File
@@ -0,0 +1,89 @@
# AACS Crypto Primitives — Implementation Notes
Rationale and background for the internal (non-public) primitives in
`src/aacs/crypto.rs`. See [AACS Encryption](aacs.md) for the public-facing
key resolution and decryption story.
## `aes_cbc_encrypt`
Forward direction of `aes_cbc_decrypt`, and its exact inverse (`[C]` §2.1.2,
`AES-128CBCE`). `data.len()` must be a multiple of 16; the `debug_assert!`
documents/enforces that contract.
Constructs the cipher ONCE for the whole slice. Driving this from the
single-block `aes_ecb_encrypt` instead rebuilds the AES key schedule per
16-byte block, which for a 6144-byte aligned unit is 383 redundant key
expansions.
## `aes_cbc_decrypt`
AES-128-CBC decrypt in-place with the fixed AACS IV (`[C]` §2.1.2,
`AES-128CBCD`). `data.len()` must be a multiple of 16; any trailing partial
block is silently ignored. All callers pass aligned regions (6128 and 2032
bytes), and the `debug_assert!` documents/enforces that contract.
This doc was once orphaned onto `aes_cbc_encrypt` above it, when that
function was inserted directly after it with no separating blank line:
rustdoc rendered the crate's only forward-direction AACS primitive as
"decrypt" and cited the spec's DECRYPT clause for it, while this function
had no doc at all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in
`content.rs` pins the directions behaviourally, so a maintainer "fixing" the
contradiction by swapping the two bodies fails the suite instead of shipping
a second decryptor behind an already-set encrypted flag.
## `cbc_decrypt_blocks`
Split out of `aes_cbc_decrypt` so a caller that decrypts several regions
under one loop-invariant key expands the schedule once. `decrypt_bus`
(`super::content::decrypt_bus`) is that caller: bus encryption (`[C]` §4.2 /
the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY 2048-byte
sector, so a 6144-byte aligned unit is three regions under one
`read_data_key` — three key schedules where one suffices, on the per-unit
decrypt hot path of a whole 90 GB read.
## `aes_g`
AES-G(x1, x2) = AES-128D(x1, x2) XOR x2 (`[C]` §2.1.3, note: uses
AES-128**D**). The Media Key Variant chain uses AES-G to derive both the
variant number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
(`Kvu = AES-G(Km, VID)`). See `super::derive::derive_vuk` for the classical
VUK form — the math is identical, this exposes it as a neutral primitive
for the variant chain.
## `aesg3`
AACS-G3 (`[C]` §3.2.2, Triple AES Generator): `left = D(k,s0)⊕s0` inc 0,
`pk = D(k,s0+1)⊕(s0+1)` inc 1, `right = D(k,s0+2)⊕(s0+2)` inc 2. Shared
with `super::variant` (its variant chain runs the same SD tree); a single
definition keeps the two walks byte-identical.
## Test: `S0`
The AACS-G3 seed `s0`, transcribed independently from `[C]` §3.2.2 rather
than read from `AESG3_SEED` — a test that sourced the seed from the
production constant would assert that constant against itself and would
still pass if it were edited.
## Test: `aesg3_inverts_to_the_spec_seed_under_aes_encrypt`
`aesg3` is the node function of the AACS subset-difference tree: every
Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every descent
step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that returned a
fixed block would make every device key in the crate derive the SAME
Processing Key, and a `^` that became `|` or `&` would derive a
wrong-but-plausible one — in both cases the MKB walk simply stops finding
Media Keys, with no error to say why.
Pinned through the spec relation rather than a re-implementation: `[C]`
§3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for `s = s0 + inc` (added
into the last seed byte), so applying the FORWARD primitive
`aes_ecb_encrypt` — a different function from the one under test — to
`aesg3(k, inc) XOR s` must reproduce `s` exactly.
## Test: `aesg3_yields_three_distinct_subkeys_for_the_three_increments`
The Triple Generator's three outputs (`[C]` §3.2.2: left = inc 0, the
Processing Key = inc 1, right = inc 2) are the two child node keys and the
Processing Key of ONE tree node. They must be three different keys — if
`inc` were ignored, a descent would revisit its own parent and the walk
would derive the same key at every level of the tree.
+68
View File
@@ -0,0 +1,68 @@
# `src/aacs/derive.rs` — key derivation notes
Rationale and background relocated here from doc comments in `derive.rs` to
keep those comments within the repo's comment-length limits.
## `derive_media_key_from_pk` — why the direct PK × cvalue scan
A Processing Key is **terminal**: it is the key at its Subset-Difference
node, one `AES-G` from the Media Key. So the fast path tries each PK
*directly* against the MKB cvalue tables (no tree descent). On a large
AACS 2.x UHD MKB (~181k cvalues) this is ~15x faster than treating a PK as
a device-node label and walking the tree.
MKB record types relevant to this path:
| Type | Meaning |
|------|---------|
| `0x10` | Type and Version Record (has MKB version) |
| `0x81` | Verify Media Key Record, AACS 1.0 (has `mk_dv`) |
| `0x86` | Verify Media Key Record, AACS 2.0/2.1 (has `mk_dv`) |
| `0x04` | Subset-Difference Index (has UVS entries) |
| `0x05` | Media Key Data Record (cvalues, 1:1 with `0x04`) |
| `0x07` | Explicit Subset-Difference Record (NOT cvalues) |
## `recover_dk_position` — search strategy and cost
This finds a device key's tree node empirically: for each MKB
subset-difference record, it tries the device at the record's node AND at
every ancestor v-position (the device may sit one or more levels ABOVE the
record, descending via AES-G3 to reach it), deriving the candidate
Processing Key DIRECTLY (one `calc_pk_from_dk` per candidate, no full
re-walk) and checking it validates against that record's cvalue.
Cost is `O(slots × tree_depth)` — linear in the MKB's subset-difference
index, not the quartic cost of re-deriving per candidate. Slots are
independent so the scan parallelizes (~181k slots, ~26s serial on a UHD
MKB); `find_map_any` returns the first match and cancels the rest.
## `resolve_candidate` — composed primitives and `None` conditions
Composes the raw derivation primitives (`derive_media_key_from_pk`,
`derive_media_key_and_pk_from_dk`, `derive_vuk`, `derive_unit_keys`) and
parses `Unit_Key_RO.inf` at the version the disc's MKB declares, so a
multi-CPS disc yields all its unit keys from the one candidate.
Validate `unit_keys` against a real encrypted unit with `decrypt_unit` +
`is_clean_ts` to prove the candidate actually opens the disc — this function
is pure derivation, no sampling or validation.
Returns `None` only when derivation itself cannot proceed: a PK its MKB
rejects, a `Dk` the MKB can't process, a missing VID on a path that needs
one, or an unparseable/empty `Unit_Key_RO.inf`.
## Test fixtures — inverting the AACS relations
There are no published AACS test vectors, but none are needed: the AACS
relations (`[C]` §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
key can be constructed with `aes_ecb_encrypt` and the same `aesg3` the walk
uses as its node function. `plant_mkb` and its siblings in
`position_recovery_tests` do exactly this — no real key material, and the
assertions check the DERIVED Media Key, not any intermediate the code under
test also produces.
`plant_mkb`'s single-slot fixture picks `uv = 0x0400` (lowest set bit 10)
with `u_mask_shift = 12` so a gating device node exists: `resolve_dk_node`
flips one bit `b < 12`, and the walk's gate (`[C]` §3.2.4) needs that bit
inside `v_mask` (`0xFFFF_FFFF << 11`) and outside `u_mask`
(`0xFFFF_FFFF << 12`) — i.e. `b == 11`.
+78
View File
@@ -0,0 +1,78 @@
# `src/aacs/resolve.rs` — internal rationale
Overflow detail relocated here by the comment-guard pass; each section is
pointed to by a short `//` comment at the named site in `resolve.rs`.
## `resolve_keys_classical`
Resolve all AACS keys for a disc using the classical (single-stage Media Key
derivation) paths. Used by both V10 and V20.
Paths run in root-of-trust -> per-disc-leaf order. A match at any path
returns immediately:
1. MKB + device keys -> processing key -> media key -> VUK
2. MKB + processing keys -> media key -> VUK
3. KEYDB MK + matching VID -> derived VUK
4. KEYDB disc-hash -> VUK
5. KEYDB disc-hash -> pre-decrypted unit keys (no VUK)
## `unique_verifying_mk`
The MK-pool selection rule of path 2.5, split out of `resolve_keys_v1` so the
ambiguity guard has a reachable test.
`verifies` is the MKB check -- in production
`AES-D(mk, mk_dv)[..8] == MK_VERIFY_MAGIC`. Returns a Media Key only when
EXACTLY ONE DISTINCT candidate passes. Duplicates of the same key are one
candidate (a pool aggregated across providers routinely repeats a key), but
two DIFFERENT keys that both verify mean the pool cannot say which is this
disc's Km: picking either derives a wrong VUK, and a wrong VUK decrypts to
plausible-looking garbage rather than failing loudly. Bail and let the later
hash/VID paths answer instead.
The predicate is a parameter rather than the inlined AES check because a
genuine two-key multi-hit cannot be synthesised: it needs one ciphertext
that decrypts under two distinct AES-128 keys to plaintexts sharing a 64-bit
prefix -- a 2^64 search. Injecting the verifier is the only way the
ambiguity branch is reachable from a test at all.
## Test: `mk_pool_ambiguity_bails_rather_than_picking_a_media_key`
Path 2.5's ambiguity guard: when MORE THAN ONE DISTINCT pooled Media Key
verifies against the MKB, the resolver must return no key at all rather than
pick one. A wrong Km derives a wrong VUK, and a wrong VUK does not fail
loudly -- it decrypts the title to garbage that muxes and plays as a corrupt
rip.
The real MKB check cannot be forced into a multi-hit: two distinct AES-128
keys decrypting one `mk_dv` to plaintexts that share the 64-bit verify magic
is a 2^64 search, not a fixture. So the rule is tested through
`unique_verifying_mk`, whose verifier is a parameter -- the same function
`resolve_keys_v1` calls, with the same pool semantics.
## Test: `resolve_keys_v21_treats_the_all_zero_volume_id_as_no_vid`
`resolve_keys_v21` gates paths 1 and 3 on `has_vid`, and an all-zero Volume
ID is the crate's "the VID was never read" sentinel -- the SCSI handshake
leaves the buffer zeroed when it does not run or fails.
Both directions matter and both fail silently:
- treating the zero sentinel as a real VID runs path 3 and derives
`Kvu = AES-G(Km, 0...0)`, a perfectly well-formed but WRONG VUK. It
unwraps the title keys to garbage, and nothing downstream errors -- the
rip just decodes to noise.
- treating a real VID as absent skips paths 1 and 3 entirely, so a disc
that could have been resolved from its Media Key reports no key.
Asserted through the final VUK, not through the flag.
## Test: `classify_vid_present_with_material_is_no_material_not_vid`
A NON-zero VID present, but resolution still fails (the providers carry
material that doesn't resolve this disc). The VID is available, so the
failure is NOT "VID unavailable" -- it must classify NoMaterial regardless
of how much derivation material is present, because re-acquiring the VID is
not the fix. This is the `has_vid == true` short-circuit, which no existing
test covers (the gate test only uses the zero-VID sentinel).
+14
View File
@@ -0,0 +1,14 @@
# AACS resolution trace shape
`src/aacs/trace.rs` defines the structured trace of a resolution attempt (the
unlock phase, then the key-resolution phase).
- No user-facing English: every step's STATE is a typed enum variant.
Applications render these into localized text — the library never does.
This module only defines the shape and is wired through the
resolve/handshake return path far enough to compile.
- The `who` of each step is the source's `label()` / unlocker's `name()` — a
stable identifier string (a NAME, like a codec id, NOT user-facing prose),
carried verbatim so an app renderer never has to match an enum back to a
name it already has. Only the OUTCOME / path enums are structured states
the app maps to i18n English.
+97
View File
@@ -117,3 +117,100 @@ starting with `;` or `#` are comments) holding the host credentials and per-disc
entries the library uses to resolve a disc. autorip can auto-download and
refresh it from a configured URL. The library does not ship any AACS keys
compiled into the binary.
## `UnitKey::is_default_index` test rationale
`is_default_index` is the public predicate that separates ordinary (index-0)
content keys from FMTS forensic index keys (see `UnitKey` docs; AACS 2.1
`IndividualSegment.tbl` tagging). A body answering `true` for everything would
present a forensic index key as an ordinary content key -- the caller would
decrypt the bulk of the title with a key that only opens 1/32nd of it;
answering `false` for everything would hide every ordinary key.
`is_default_index_separates_the_two_constructors` is pinned against the two
named constructors, which are the contract: `UnitKey::new` builds the ordinary
key, `UnitKey::forensic` builds an index key for `1..=32`.
`is_default_index_agrees_with_the_forensic_index_resolver` checks the
predicate agrees with the one consumer of `index_number` in the crate:
`crate::aacs::index_select::resolve_disc_index` resolves the disc's forensic
index from exactly the keys that are NOT default. If the two disagree, a disc
resolves an index whose key the rest of the pipeline treats as ordinary (or
vice versa).
## `src/aacs/mod.rs` module notes
### KEYDB.cfg line format
```
| DK | DEVICE_KEY 0x... | DEVICE_NODE 0x... | KEY_UV 0x... | KEY_U_MASK_SHIFT 0x...
| PK | 0x...
| HC | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
| HC2 | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
0x<disc_hash> = <title> | D | <date> | M | 0x<media_key> | I | 0x<disc_id> | V | 0x<vuk> | U | <unit_keys>
```
### Spec provenance tags
The crypto in `src/aacs/` carries `[TAG] §x.y` citations back to the published
AACS specification (Final Rev 0.953), so each primitive links to the section it
implements:
- `[C]` -- AACS Introduction and Common Cryptographic Elements Book (primitives,
MKB/key-management).
- `[PR]` -- AACS Pre-recorded Video Book (Volume/Title Key layer).
- `[BD]` -- AACS Blu-ray Disc Pre-recorded Book (CPS Unit Key, Aligned Unit,
Block Key).
- `[RE]` -- reverse-engineered from real discs, cited only where the public
spec is silent (the `0x86` verify record and the Category-C MKB type values).
### AACS key-input path discovery (`AacsRole`, `role_paths`, `find_hddvd_aacs_dir`)
BD and UHD keep their key material under a fixed `/AACS/...` tree, so those
paths are constants. HD DVD keeps the equivalents in a reserved root directory
whose NAME is authoring-house-specific -- observed `ANY!` (Dukes of Hazzard)
and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK` mirror -- and
whose title-key file is NOT always `VTKF000.AACS` (Freedom ships
`VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED from the
parsed UDF tree (`find_hddvd_aacs_dir` + `role_paths`), never hardcoded.
Each key role (`AacsRole`) resolves to an ordered candidate list -- the BD/UHD
constants first, then whatever the HD DVD directory actually holds -- which
every reader walks with `read_first`, first-that-reads. No reader ever
branches on disc type: a BD/UHD disc has the `/AACS/` files so those win; an
HD DVD has none of them, so it falls through to the discovered entries.
Centralised so `resolve_vid_only`, `read_aacs_inputs`, `read_mkb_content`, and
`read_aacs_version` can never silently diverge the disc_hash / MKB / VID that
another reader feeds a key service.
`find_hddvd_aacs_dir` identifies the HD DVD AACS directory structurally, NOT by
a hardcoded name: the root child directory whose name ends in `!` (so the
`<name>!_BAK` backup mirror, which also ends in a non-`!` char, is not
mistaken for it) and which contains `MKBROM.AACS`. Observed real names:
`ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD disc has no such
directory -> `None`.
`role_paths` builds the ordered candidate list: fixed BD/UHD paths first, then
the discovered HD DVD files. For `AacsRole::UnitKey` every `VTKF*.AACS` in the
directory is appended in sorted name order -- a disc may carry more than one
variant (Freedom: `VTKF090` + `VTKF100`), not just `VTKF000`.
`read_first` walks an AACS role's candidate paths and returns the first that
reads; `read` performs the actual per-path read (full file or bounded prefix),
so callers share the same first-present walk regardless of read style. Returns
`Error::AacsNoKeys` if no candidate is present. Generic over the path element
(`&str` or owned `String`) so it accepts the `Vec<String>` that `role_paths`
builds from the discovered HD DVD directory.
### Test: `a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory`
The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required -- the
discovery is a conjunction, not a disjunction. The existing fixtures only ever
present a directory that satisfies both (`AAC!` with `MKBROM.AACS`) alongside
one that satisfies neither (`AAC!_BAK` -- which contains `MKBROM.AACS` but is
ALSO reached only after the real dir), so either half of the conjunction could
be dropped and the same directory would still be found. This test's fixture
has a directory that satisfies the name half and NOT the contents half: it
must not be picked. If it were, the HD DVD path would resolve `MKBROM.AACS`,
`CONTENT_CERT.AACS` and the title-key file under a directory that holds none
of them -- the disc reports "no AACS key files" and never rips.
+220
View File
@@ -0,0 +1,220 @@
# AC3 / E-AC3 parser notes (`src/mux/codec/ac3.rs`)
## Why a frame set is one sample, never split into separate tracks
A legacy AC-3 syncframe is a complete access unit on its own, but an E-AC-3
access unit (ETSI TS 102 366 / ATSC A/52 Annex E) is a whole FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream frame that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7) and their own dependents. A decoder needs the whole set
to reconstruct the programme (a 5.1 independent substream plus a dependent
substream carrying the extra channels of a 7.1 programme, and the AC-3-core +
E-AC-3-dependent arrangement used for backwards-compatible Dolby Digital
Plus), and every substream of the frame set covers the SAME time period.
WHY the whole frame set is ONE sample, rather than splitting an additional
independent substream (an associated / commentary service) into a track of
its own: Annex E orders the substreams of a frame set inside a single
elementary stream, all covering one time period, and a decoder handed a frame
set renders the programme it is asked for from it — that is exactly what the
container's E-AC-3 sample and its `dec3`/`num_ind_sub` description denote.
Splitting would mean rewriting the bitstream (re-numbering `substreamid` so
the extracted service becomes substream 0, and rebuilding its frame sets),
because a substream numbered 1..7 with no substream 0 is not a conforming
stream. That is a transcode, not a remux; this parser is lossless, so the
frame set stays whole and player-side programme selection decides what is
heard.
## `SAMPLE_RATES` / fscod2
Sample rates indexed by fscod (0=48kHz, 1=44.1kHz, 2=32kHz). fscod=3 is
reserved in AC-3; in E-AC-3 it signals "fscod2" (reduced rates: 24/22.05/16
kHz, selected by byte-4 bits `[5:4]`). `frame_sample_rate` decodes fscod2 in
the E-AC-3 case; the table's index-3 entry (48 kHz) is only the fallback when
the header is too short to read fscod2.
## `MAX_AC3_BUF` derivation
An AC-3/E-AC-3 syncframe is at most 8192 bytes (the `frame_size > 8192`
reject) and an access unit is a whole frame set: up to 8 independent
substreams, each with up to 8 dependent substreams (ETSI TS 102 366 Annex E)
— 72 syncframes worst case. A cap of one straddling frame set plus slack
therefore has to exceed 72 × 8192 = 576 KiB. If the buffer grows past the cap
without yielding a frame (pathological / never-syncing input) it is dropped
and resync begins, rather than accumulating one PES worth of data per call
for the whole title.
## `scan_access_units`
Per ETSI TS 102 366 (ATSC A/52) Annex E an access unit is one FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7 — associated/commentary services) with their own
dependents. The access unit therefore closes only when the next
`substreamid`-0 independent substream (or, with `at_eos`, the end of the
stream) is seen, and it carries THAT substream's PTS and duration: every
other substream of the frame set describes the SAME time period and adds no
duration of its own.
`base_pts_ns` times the access unit that begins at `data[0]` — i.e. the
running cadence carried over from the previous call. `anchor` re-anchors the
running PTS to this PES's own timestamp at the first access unit that STARTS
in the newly-appended bytes: a PES timestamp applies to the first access unit
beginning in that PES, never to one that began in an earlier PES and is only
being completed (or was held) here.
Returns the emitted access units, the offset in `data` from which bytes must
be carried over to the next call, and the PTS to stamp on the access unit
that begins that carry-over.
## `substream_role`
Byte 2 of an E-AC-3 syncframe is `strmtyp(2) | substreamid(3) | frmsiz[10:8]`
(ETSI TS 102 366 Annex E BSI). `strmtyp` 0 and 2 are independent substreams
(type 2 being an independent substream that is not the first of the bit
stream); `strmtyp` 1 is the dependent substream.
Annex E defines a FRAME SET as independent substream 0 — mandatory, always
first — with its dependent substreams, followed by the OPTIONAL additional
independent substreams 1..7, each with their own dependents; all of them
carry the same time period. So the access-unit boundary is an independent
substream with `substreamid` == 0, and NOT merely "an independent substream":
an additional independent substream (an associated or commentary service)
sits INSIDE the frame set already open. Keying the boundary on `strmtyp`
alone made every such substream close the access unit and advance the
running PTS a second time over the same ~32 ms, doubling the timeline (about
a second of A/V drift per second of audio).
`strmtyp` 3 is reserved: its BSI layout is not defined, so its `substreamid`
bits cannot be trusted and it is treated as starting a fresh access unit — an
unknown frame is never merged into an unrelated programme, and never
discarded as an orphan either.
Legacy AC-3 (`bsid < 11`) has no substream structure — byte 2 there is the
crc1 field, never `strmtyp` — so it always starts an access unit.
A frame set MAY carry a substream as several consecutive syncframes of fewer
than six blocks each (Annex E allows numblkscod < 3). Those extra syncframes
are `substreamid` 0 too, so each starts its own access unit here rather than
merging into one frame set. That is deliberate: each carries its OWN
numblkscod-derived duration (see `frame_duration_ns`), so the timeline total
stays exact, and the substreams are still delivered in bitstream order.
## `frame_crc_ok`
Per ETSI TS 102 366 (ATSC A/52) the frame carries a CRC-16/ANSI (poly 0x8005,
init 0, non-reflected) over the bytes after the 2-byte syncword — i.e.
`crc16_ansi(&buf[2..]) == 0` covers `frame_size - 2` bytes; the trailing crc
word makes a clean frame's residue zero. A nonzero residue is a
~1-in-65536-certain sign of payload corruption, so the frame is dropped
(silence gap) rather than shipped as a glitch.
## `acmod_channels` bit layout
This is the AUTHORITATIVE channel count for the track header: the DVD IFO
`audio_attr_t.channels` nibble is a well-known unreliable/stale field, so the
muxer prefers this over the IFO-claimed count (the bitstream acmod is
authoritative; the IFO audio nibble is not trusted). LFE adds one channel
(e.g. acmod=7 + lfeon → 6 = 5.1).
Bit layout from the syncword (A/52 §5.3.2 BSI):
```text
byte 5: bsid(5) | bsmod(3)
byte 6: acmod(3) | [cmixlev(2) if acmod has a centre and acmod!=1]
| [surmixlev(2) if acmod has surround]
| [dsurmod(2) if acmod==2] | lfeon(1) | ...
```
`acmod` therefore always occupies byte-6 bits 7-5; `lfeon` follows a variable
number of optional 2-bit fields, so the decoder tracks the bit cursor.
## `ACMOD_CHANNELS` table
Base channel count per AC-3 `acmod` (A/52 Table 5.8), BEFORE the LFE. Index is
the 3-bit acmod value; add 1 when `lfeon` is set.
```text
0 = 1+1 (Ch1, Ch2) -> 2 4 = 2/1 (L,R,S) -> 3
1 = 1/0 (C, mono) -> 1 5 = 3/1 (L,C,R,S) -> 4
2 = 2/0 (L, R) -> 2 6 = 2/2 (L,R,SL,SR) -> 4
3 = 3/0 (L,C,R) -> 3 7 = 3/2 (L,C,R,SL,SR) -> 5
```
## Test helpers and coverage notes
### `finalize_ac3_crc`
Relies on the CRC-16/ANSI residue property: appending `crc16([2..n-2])`
(big-endian) zeroes the register over `[2..n]`.
### `the_working_buffer_is_reused_across_packets_not_reallocated`
`parse` runs once per PES packet — of the order of 10^5 times on a feature's
audio track — and used to build a fresh `Vec` every call. The copy itself
cannot go without restructuring the borrow relationship, but the allocation
can: the buffer is taken out of `self`, filled, and put back with its
capacity intact. Asserting on capacity is deliberately white-box, because
that is exactly what regresses if someone reverts to `to_vec()` — the
behaviour would be identical and no other test would notice.
### `make_eac3_frame` byte layout
ETSI TS 102 366 Annex E: byte 2 is strmtyp(2) | substreamid(3) | frmsiz[10:8],
byte 3 is frmsiz[7:0], and the frame is `(frmsiz + 1) * 2` bytes. byte 4 =
fscod 0 (48 kHz), numblkscod 3 (6 blocks → 1536 samples → 32 ms), acmod 7 +
lfeon (5.1); byte 5 = bsid 16 so the E-AC-3 paths are taken.
### `a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet`
An access unit closes only at the next `substreamid`-0 independent substream,
so one that keeps gaining dependent substreams stays OPEN across PES
boundaries and its bytes stay in the carry-over. The carry-over must not be
re-scanned — and re-CRCed — from the access unit's first byte on every
packet: the buffer only stops growing at `MAX_AC3_BUF` (1 MiB), and a PES on
a DVD is about 2 KiB, so re-deriving the held access unit costs work
quadratic in the packets fed.
Measured directly, because a work bound has no frame-level shadow:
`frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
Mutation: pass `None` for `held` in `parse` (or drop the `if let Some(h) =
held` resume) — the count returns to the quadratic figure.
### `a_discontinuity_drops_the_held_access_unit_with_its_bytes`
`parse` clears `self.acc` on a discontinuity because the buffered bytes are a
truncated frame. The held access unit is described by OFFSETS into exactly
those bytes, so it has to go with them. Without `self.held = None`, the next
packet resumes a HeldAu whose `start`/`end` were computed against the
pre-gap buffer but are applied to the unrelated post-gap bytes — splicing
audio across the gap at best, and indexing past the end of the new, shorter
buffer at worst.
The two existing discontinuity tests use plain AC-3 (bsid < 11), which never
holds an access unit open, so neither of them reaches this reset.
### `a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes`
Mutation this catches: deleting (or inverting) the resume-path re-check `if
drop_reason.is_none() && self.tally.is_poisoned()` at the top of
`scan_access_units`. The verdict on a held access unit is frozen at the
moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
previous access unit is closed — so the very drop that crosses the poison
threshold lands after the next unit's verdict was already taken as `None`.
Without the re-check that unit is emitted as an ordinary frame after the
track has been judged too damaged to mux: corrupt audio passed through as
success, and worse, it is the ONE frame that escapes a whole-track fallback
whose entire point is that nothing after the verdict ships.
Neither existing held-AU test reaches this: both keep a pristine tally.
The fixture drives the exact interleaving above. `DropTally` poisons once
`verified_dropped * 2 > kept + dropped` past the 200-AU gate, so one PES
carries 200 CRC-failing `substreamid`-0 syncframes followed by a clean one.
Closing corrupt unit #200 (which happens only when the clean frame is
reached) latches the poison — after that clean frame's own verdict was
computed. Being an E-AC-3 unit that can still gain substreams, it is then
HELD across the boundary with `drop_reason: None`, which is precisely the
state the re-check exists for.
+37
View File
@@ -0,0 +1,37 @@
# ADTS decodability gate
## Module rationale (`src/mux/codec/adts.rs`)
Per the ADTS framing defined in ISO/IEC 13818-7 / ISO/IEC 14496-3, a header
is structurally invalid in exactly three ways this gate treats as hard
rejects: syncword != 0xFFF, a reserved `sampling_frequency_index` (the sample
rate table has 13 valid entries, so index >= 13 is reserved), and
`aac_frame_length < 7` (shorter than the fixed+variable header itself). The
optional 16-bit ADTS CRC is not verified here — it is simply skipped. So the
gate enforces those three rejects: a packet that begins with the ADTS sync
but is otherwise malformed is dropped; a packet with no ADTS sync is raw AAC
(e.g. from an MP4 container, which carries no ADTS header) or a continuation
and passes through unchanged — never false-dropped. Raw AAC has no per-frame
integrity data, so like LPCM it cannot be gated.
## Test: `a_crc_present_header_shorter_than_its_own_crc_is_invalid`
A header that CLAIMS a CRC (protection_absent = 0) but declares a frame
length too short to contain one.
`aac_frame_length` counts the header and the CRC, not just the payload,
so with a CRC present the smallest structurally possible frame is 9
bytes: the 7-byte fixed+variable header plus the 16-bit crc_check.
The gate compared against a flat 7 and never read protection_absent at
all, so a frame whose own header says it is impossible was classified
Valid and forwarded to the muxer.
## Test: `dropped_frames_are_counted_but_their_duration_is_not_invented`
A dropped ADTS frame is dropped BECAUSE its header failed validation, so
the very fields a duration would come from (sampling_frequency_index, and
the 1024-samples-per-AAC-frame constant applied to it) are the ones known
to be untrustworthy. This gate therefore reports the drop's duration as
zero rather than deriving a number from a header it has just rejected —
the honest answer, and the one the count alongside it must be read with.
A nonzero constant here would report silence that was never measured.
+217
View File
@@ -0,0 +1,217 @@
# `src/mux/au_assembly.rs` — access-unit assembly
## Why this exists
The contract a codec parser converts is `PES → access units (Frames)`. A
*transport* stream hands the parser one AU per PES for free (BD aligns one
access unit per PES; the TS demuxer reassembles to the
`payload_unit_start_indicator`). A *program* stream does not — the PS muxer
chops the elementary stream into fixed-size PES fragments with no AU
alignment, and only the first fragment of an AU carries a PTS. So a parser
that assumes one-AU-per-PES (h264/hevc/vc1, written against TS) mis-frames a
program stream, while `mpeg2` — the DVD/PS codec — must reassemble across PES.
`AuAssembler` is that reassembly, factored out so EVERY program-stream video
parser shares one implementation instead of hand-rolling the buffer. The
h264/hevc/vc1 parsers (`Mode::StartCode` / `Mode::Vc1`) and the MPEG-2 parser
(`Mode::Mpeg2`, via `AuAssembler::mpeg2`) all drive it. It buffers PES-fragment
bytes and emits one AU per codec AU boundary, carrying the AU-start
timing/source forward. Since the boundary is a codec start code, it lives with
the codec parser (which picks the marker); only the generic buffering +
timing-carry is shared here.
This is *inside* the parser, not a pipeline stage: the pipeline stays
`Demuxer → PES → Parser → Frames`, and the demuxer stays codec-agnostic. Every
stream a parser sees runs through one of these — self-framing codecs (MPEG-2,
audio) use `Mode::Passthrough` so the parser code path is uniform.
## `MAX_AU_BUFFER`
Safety cap on a single in-progress access unit. A real coded picture is far
below this; a stream that never yields a second AU boundary is force-flushed
at the cap rather than buffering without bound on hostile/corrupt input.
## `MAX_MARKS`
Cap on buffered timing/discontinuity marks. A real access unit spans a few
hundred PES fragments at most; this bounds the mark deques so a run of
zero-length (or start-code-free) timed fragments — which grow no buffer bytes
and so never trip the `MAX_AU_BUFFER` mark-prune — cannot accumulate marks
without bound on hostile/corrupt disc input.
## `AuAssembler::for_codec`
Video codecs whose parsers assume AU-complete PES (H.264 / HEVC / VC-1) get a
`Mode::StartCode` assembler; MPEG-2 (self-reassembles) and audio/subtitle
codecs (self-framing) get `Mode::Passthrough` so callers can run every stream
through this uniformly.
## `AuAssembler::mpeg2`
The MPEG-2 parser owns one of these directly (rather than hand-rolling the
buffer): the demux layer runs MPEG-2 through `Mode::Passthrough` and hands
each fragment to the parser, which feeds them here to be reframed on picture
boundaries.
## `AuAssembler::push`/`push_owned`
For a self-framing (`Passthrough`) stream the payload is MOVED straight into
the emitted unit with no copy — the common DVD/HD-DVD case (MPEG-2 video, all
audio). A buffering mode copies into `buf` exactly as `push`.
## `take_front`
Detaches `buf[..end]` as the emitted AU's own `Vec` and leaves `buf` holding
the tail.
The AU's bytes are HANDED OVER — `buf`'s allocation becomes the returned `Vec`
and a fresh buffer (pre-sized to the same capacity, so the next AU accumulates
without re-growing) takes its place holding only the short tail.
`buf[..end].to_vec()` + `drain(..end)` instead copied every AU out in full: on
a UHD HEVC title that is a whole-frame memcpy (hundreds of KB) per coded
picture, ~200k times, for bytes that are about to be discarded from `buf`
anyway.
The allocation COUNT is unchanged (one per AU either way — the frame `Vec`
before, the replacement buffer now), so the only difference is the copy that
no longer happens. Nothing depends on `buf` keeping its identity: the only
state tied to `buf[0]`'s position is `base`/`scan_pos`/`opener_pos`, which the
caller updates immediately after.
Falls back to a copy when the buffer's capacity is far larger than the AU (a
small AU after a multi-MB one): handing over would otherwise attach an
oversized idle allocation to a small frame for as long as the frame queues
downstream, trading a copy for resident memory.
That fallback must not become permanent. `buf`'s capacity used to be a one-way
high-water mark — the replacement buffer was created with `cap.max(tail_len)`,
and the copy path's `drain` also preserves `cap` — so once ONE large AU had
been assembled, every later smaller AU satisfied `cap > 2*end` and took the
copy path forever. On a UHD HEVC title the first IDR grows `buf` to ~4-8 MB,
after which each ~200-400 KB P/B AU paid a whole-AU allocation plus a
whole-AU memcpy plus a tail memmove for ~99% of the ~200,000 coded pictures —
tens of GB of exactly the memcpy this handover exists to remove. So the copy
path now also RELEASES the high-water capacity, which re-arms the handover
for the next AU: one copy after a size step down, not one per frame forever.
## `drop_marks_before`
This is the STREAM-START case: bytes ahead of the first AU boundary are the
tail of an access unit that began before we had sync, and there is no prior
AU for them to be discontinuous *from*. Carrying a mark forward here would arm
the resync gate at the head of every title and drop its first GOP.
## `discard_gap_before`
This is the BACKSTOP case: `MAX_AU_BUFFER` bytes accumulated with no AU start
code in them, so the run is unusable and gets thrown away. Unlike the
stream-start trim above, there IS a prior AU here, and whatever follows
definitively does not continue it — a decoder handed the next picture would
resolve its references against frames separated from it by megabytes of
discarded data.
So the discard is itself a discontinuity, whether or not the source signalled
one. It is recorded as a sticky flag rather than an offset mark because a
mark placed at the new base would be retired moments later by the pre-sync
trim that follows resync — the gap has to outlive the bytes that caused it.
It arms the resync gate, which drops to the next keyframe instead of emitting
a picture with dangling references.
Timing marks before `off` are still retired — they describe bytes that no
longer exist, and the AU that eventually emits takes its PTS from the
fragment that actually opened it.
## Tests
### `a_backstop_discard_marks_the_next_au_discontinuous`
The 8 MiB backstop throws away a start-code-free run as unusable. The AU that
eventually emits after that discard MUST be marked discontinuous, whether or
not the source ever signalled a discontinuity: megabytes of the stream are
simply gone, so the next picture cannot resolve its references against the
last one that was emitted.
`discontinuity` is what arms the resync gate downstream (`resync.rs`, driven
from `mux/disc.rs`), which drops to the next keyframe rather than emitting a
picture with dangling references. If the flag is retired with the discarded
bytes, the gate never arms and the broken picture goes out — a silent
corruption, which is the one class of loss this crate refuses to have.
### `a_stream_start_trim_does_not_mark_the_first_au_discontinuous`
The opposite case, and the reason the two call sites are separate. Bytes
ahead of the FIRST access-unit delimiter are the tail of an AU that began
before we had sync. There is no prior AU for them to be discontinuous from, so
retiring the marks there is right — and necessary: marking the first AU of
every title discontinuous would arm the resync gate at the head of each one
and drop its opening GOP.
### `a_source_signalled_discontinuity_reaches_the_au_it_opens`
A source-signalled discontinuity reaches the AU it opens. This is the
`disc_marks` path — the ORIGINAL mechanism, distinct from the sticky
`pending_gap` the backstop sets. Nothing else pins it: the two tests above
drive `pending_gap`, and a mark placed on a fragment that is later discarded
is retired by design.
Deliberately NOT combined with the backstop. A previous version of this test
signalled the discontinuity on the first over-cap push and asserted the flag
on the AU after the discard — but that first run still has the next AU's
delimiter at `buf[0]`, so it force-flushes as an over-long AU, and THAT AU
consumes the mark. The assertion was then satisfied entirely by
`pending_gap`, making the test a duplicate of the one above it under a name
promising something else. The two mechanisms cannot be isolated in one
fixture, so they get one test each.
### `drained_au_takes_over_the_buffer_allocation_without_copying`
MEASURED: a drained AU must be HANDED the accumulation buffer's allocation,
not copied out of it. The emitted `Vec`'s data pointer is the buffer's own
pointer — which is only true if no full-frame copy happened.
(`buf[..end].to_vec()` allocates fresh, so the pointers differ.) One
whole-AU memcpy per coded picture is ~200k memcpys of a few hundred KB each
on a UHD feature.
### `handover_survives_a_large_au_instead_of_copying_every_later_one`
MEASURED: `take_front`'s copy fallback must not become permanent.
`buf`'s capacity used to be a one-way high-water mark, and the copy path's
`drain` preserves it, so after ONE large AU every later smaller AU satisfied
`cap > 2*end` and copied forever. On a UHD HEVC title the first IDR grows
`buf` to multiple MB, after which ~99% of the ~200,000 coded pictures each
paid a whole-AU allocation + whole-AU memcpy + tail memmove — tens of GB of
exactly the copy the handover exists to remove. Counted at the copy path
itself: one copy is expected right after the size step down; a per-frame
copy is the bug.
### `discarded_pre_sync_marks_do_not_time_the_first_access_unit`
After the pre-sync bytes are discarded, the emitted AU must take the timing
of the fragment that ACTUALLY opened it. `drop_marks_before` is what retires
the discarded fragment's marks; a no-op there stamps the first real access
unit with the PTS and source of bytes that were thrown away — a whole-title
A/V sync offset, since every later frame is timed relative to it.
### AU-opener detection: the per-mode start-code rule
`au_opener_from` duplicates a rule each codec parser also encodes; the tests
below pin it to the normative byte values so a drift between copies shows up.
### `au_opener_from_locates_the_real_start_code_per_codec`
The opener offset must be the position of the real start code, never a fixed
0. A constant `Some(0)` makes every pre-sync run of junk bytes look like the
head of an access unit, so the first AU of every stream that does not begin
exactly on a start code is emitted with junk glued to its front.
### `for_codec_routes_each_video_codec_to_its_reassembly_mode`
`for_codec` is the dispatch that decides whether a stream is REASSEMBLED
across PES fragments or passed straight through. Getting it wrong is silent:
an H.264/HEVC/VC-1 stream routed to `Passthrough` on a program source emits
one "frame" per PES fragment — a few hundred bytes of a coded picture, framed
as a whole access unit — and the output plays as corruption, not as an error.
Each mode is identified BEHAVIOURALLY (feed a two-AU stream in two halves and
see whether it reassembles), so the case cannot pass by matching a constant.
+42
View File
@@ -0,0 +1,42 @@
# BDMV disc-library metadata (`bdmt.rs`)
Every commercial Blu-ray carries a disc-library metadata directory with one
XML file per shipped language. The schema is the Blu-ray "disc library
metadata" namespace (`urn:BDA:bdmv;disclibmeta`), conventionally prefixed
`di:`. Fields commonly present:
- `<di:title>` or `<di:name>` — the title string. Vendor practice varies
(Paramount discs tend to use `<di:name>`).
- `<di:description>` — optional synopsis (often absent on retail discs;
common on box sets and special editions).
- `<di:discNumber>` / `<di:numSets>` (or `<di:numberOfSets>`) — set position
for multi-disc releases.
This module is intentionally separate from the BD-J `StreamLabel` parsers
under `labels/*.rs`. The XML here is disc-level (title, description, set
position), not per-stream.
Real-world XML is irregular: missing description elements, multiple title
elements (first one wins), and occasional malformed content. Extraction is
best-effort — a malformed file is treated as "no metadata" (returns `None`
from the helper), and the caller can still get metadata from
sibling-language XML files.
## `MAX_BDMT_BYTES`
The size comes from attacker-controlled UDF metadata; real files are a few
KB, so 1 MiB is generous while preventing a crafted huge-size entry from
triggering an oversized allocation in `read_file`.
## `parse_bdmt_xml`
Title-element preference: `<di:name>` → `<di:title>` →
`<di:tableOfContents>/<di:titleName>` (first match wins, per the
authoring-tool conventions documented above).
## `looks_like_xml`
Rejects candidate description strings that are themselves XML fragments —
observed on a captured disc, where `<di:description>` contained
`<di:thumbnail href="…"/>` child elements and no actual prose. Surfacing
that raw to the JSON output is worse than dropping the field entirely.
+21
View File
@@ -0,0 +1,21 @@
# bdnav — HDMV navigation resolver
Notes relocated from `src/bdnav/mod.rs` doc comments (kept out of the source
to respect the comment-guard's prose caps).
## No per-disc special-casing
There is no per-disc special-casing: one correctly-booted VM resolves
densely-branched dispatchers as well as simple ones. The honest boundary is
BD-J (the feature is chosen by a Java Xlet, not by the HDMV VM); there — and
on any malformed data or non-convergence — the resolver ABSTAINS (`None`)
and selection falls back to the structural/heuristic order.
## `resolve_feature`
Resolves the playlist id the disc's First-Play navigation plays as the
feature, restricted to ids the caller marks as feature candidates
(`is_feature_candidate` — typically video-bearing, non-trivial titles, so a
short logo/pre-roll `PlayPlayList` is skipped). Returns `None` for BD-J
discs, missing/malformed nav data, or when navigation does not converge on
a candidate.
+239
View File
@@ -0,0 +1,239 @@
# src/disc/bluray.rs — extended notes
Overflow prose for comments that were trimmed to the internal 3-line cap in
`src/disc/bluray.rs`. Each section is pointed to by a `// See docs/bluray.md
— <topic>` comment at the corresponding site.
## CLIP_STREAM_EXTS
Stream-file extensions probed for a BD-family playlist clip, in priority
order. A clip is normally `.m2ts`; AACS 2.1 (FMTS) discs name the main
feature `.fmts` (an M2TS transport stream plus forensic variant segments)
and 3D discs use `.ssif`. `.m2ts` is tried first, so a normal clip is
unaffected — the fallback only runs when `.m2ts` is absent (exactly when
`file_extents` errors).
Scope: these are all variants that live in `BDMV/STREAM/` and are reached
through an MPLS playlist. HD-DVD's `.evo` does NOT belong here — HD-DVD is
a different tree (`HVDVD_TS/`) with `.XPL` playlists and needs its own
enumerator (a peer to `parse_playlist`), not another extension in this
list.
## scan_bluray_titles
Cancellation: `halt` is polled between playlists and once more after the
loop, and a read that fails with `Error::Halted` — how a live drive
reports a Stop, since `Drive::checked_exec` fails every command once its
flag is set and `Drive::read` preserves the variant — is propagated rather
than swallowed. `Halted` is the only error this returns from the
enumeration itself; an unreadable or unparseable playlist keeps its
best-effort skip.
It has to be an error and not a short title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it. Before this, a Stop
pressed mid-scan failed every remaining `.mpls` read in turn, each one
silently skipped, and the scan returned `Ok(truncated)` at rc=0.
## parse_playlist
Sums PlayItem durations; returns `Ok(None)` if the playlist is under 30
seconds (skips menu / clip-info stub playlists), fails to parse, or names a
clip that cannot be resolved. Physical sector extents are pulled from the
UDF allocation descriptors of each referenced `.m2ts` (deduplicated by
clip_id).
`Ok(None)` keeps its two benign meanings (unparseable MPLS, sub-30s
playlist) plus the deliberate "drop this title" outcomes below. `Err` means
the scan is over: today that is only `Error::Halted`, the operator's Stop,
which must not be reported as a disc that simply holds fewer titles.
## Test: parse_playlist_keeps_exactly_30_seconds
At exactly 30s the playlist is kept (`< 30.0` is strict). The fixture is a
fully wired BDMV (`make_bdmv_fs`), not the bare `make_min_fs` this used to
use. `make_min_fs` lays no STREAM/CLIPINF, so the play item's `.clpi` read
now fails and the title drops — which is correct behaviour for an
unresolvable clip but has nothing to do with the 30-second boundary this
test exists to pin. Wiring the clip keeps the test measuring exactly one
thing.
## make_bdmv_fs_ssif
Full BDMV with a real Blu-ray 3D layout: `.ssif` files under
`BDMV/STREAM/SSIF/<clip>.ssif` (note the SSIF subdirectory, unlike
`make_bdmv_fs_ext`) plus a matching `.clpi` in CLIPINF. Resolving the SSIF
is what latches `is_3d = true` in `parse_playlist`.
## Test: parse_playlist_fmts_clip_resolves_extent
AACS 2.1: the feature clip is `00001.fmts`, NOT `.m2ts`. The
`CLIP_STREAM_EXTS` fallback in `parse_playlist` must still resolve the
physical extent — before the fix the hard-coded `.m2ts` path errored,
yielding empty extents (a silent empty rip and 0 encrypted samples for key
resolution). Size still comes from the `.clpi`, which parses regardless.
## Test: parse_playlist_dedups_repeated_clip_extents_and_size
THE 0.31.0 DEDUP PATH. A playlist that references the SAME clip_id from
multiple PlayItems (seamless split / looped segment) must count the
physical extents and packet bytes exactly once — mux reads extents in
order, so a duplicate would mux the A/V twice and inflate `size_bytes`
(bluray.rs: `first_ref = seen_clips.insert(...)` gates both `total_size +=`
and the `extents.push`). Per-PlayItem Clip entries are still recorded for
both.
## Test: parse_playlist_missing_clpi_yields_no_title
A clip whose `.clpi` is missing must yield no title.
This test used to assert the opposite — a title with `size_bytes == 0` and
empty extents — and its docstring quoted the buggy control flow ("bluray.rs
only fetches extents inside the `if let Ok(clpi_data)`") as if it were the
specification. It was blessing the defect: the title's `duration_secs` is
summed from the PlayItems before the clip is resolved, so the returned
title advertised the movie's full runtime while carrying none of its
bytes, and the discarded error meant not one log line said so. That is the
flagship failure class of this crate — a failure that looks like success —
reached through an ordinary missing or scratched CLIPINF file.
The correct behaviour is the same as for a clip whose extents cannot be
resolved (see `parse_playlist_unreadable_clip_icb_yields_no_title`): drop
the title and log the read's own error code.
That last clause is now asserted, not merely asked for. The site's comment
insists on `e.code()` because a missing `.clpi` (E6003), a scratched one
(E6000) and a malformed one (E6002) are different populations, but nothing
checked it: putting a literal back compiled and passed. Mutation: `"E6017"`
(or any fixed code) in place of `"E{}", e.code()` fails here, and so does
dropping the warn entirely — a silent drop is the same invisible title
loss this test was written for, one step later.
## Test: parse_playlist_unrecorded_extent_yields_no_title
A clip stream whose ICB declares an UNRECORDED (ECMA-167 4/14.14.1.1
type-1) extent must not yield a title at all.
The extent is allocated to the file but was never written, so the file's
content there is zeros while the media holds whatever was left at those
sectors. Neither answer a `(lba, sector_count)` read plan can give is true
— reading it splices undefined sectors into the rip as content, dropping
it slides every later extent's byte space — so the title is refused rather
than mis-ripped. This fixture is the shape a crafted disc uses to get such
a range into a title's extent list.
(The `sectors > 0 && lba > 0` filter below the resolver stays as defence in
depth; a zero-length AD is only reachable as an unrecorded descriptor,
since a zero-length TYPE 0 one terminates the AD list.)
## Test: parse_playlist_unreadable_clip_icb_yields_no_title
An unrecorded extent was never the only way a clip fails to resolve.
RED BEFORE GREEN: this fixture gives the .m2ts an ICB whose descriptor tag
is neither 261 nor 266, so `file_extents` returns `DiscRead` — the same
variant a scratched sector under a real clip's ICB produces, which is the
ordinary way this happens on real media. Before the fix only
`UdfUnrecordedExtent` set the drop flag, so this fell through to the "file
absent" path and `parse_playlist` returned a title: full declared duration
from the play item, `total_size` already counted from the .clpi, and zero
extents — a movie advertising its runtime with the content missing, and
not one log line. The clip must drop the title exactly as an unrecorded
extent does.
## Test: parse_playlist_logs_a_non_absence_ssif_failure_the_m2ts_fallback_hid
A non-absence SSIF failure that the `.m2ts` fallback then papers over must
be logged, with its own code.
`unresolved` had exactly one reader — `if let (None, Some(code))` — so a
code recorded for `/BDMV/STREAM/SSIF/<clip>.ssif` was thrown away whenever
the fallback succeeded. The disc here IS a 3D disc: the SSIF is present and
carries both eyes, and only its ICB is unreadable (a scratched sector, the
ordinary way this happens). The title shipped base-view 2D at rc=0 with
`is_3d` false and not one log line, so the operator's rip is silently
missing the dependent view and the journal cannot tell this disc apart
from one that was only ever 2D.
The title is deliberately still returned — the base view resolved, so
refusing would trade a degraded rip for no rip. The defect being fixed is
the silence, not the fallback.
Mutations this catches: deleting the new `(Some(_), Some(code))` arm, or
restoring the `if let (None, Some(code))` shape, leaves no event to find;
logging a fixed code instead of `e.code()` fails the code assertion;
making the arm `return Ok(None)` fails the title assertion.
## Test: parse_playlist_id_falls_back_to_zero_when_suffix_is_not_mpls
A filename that is long enough (>= 5 bytes) but does NOT end in ".mpls"
must NOT have its last 5 bytes stripped — the whole string is handed to
the numeric parse instead, which fails and falls back to playlist_id 0
(bluray.rs `filename.len() >= 5 && filename[len-5..].eq_ignore_ascii_case(".mpls")`).
## Test: scan_bluray_titles_skips_non_mpls_extension_file
A non-directory PLAYLIST entry whose name does NOT end in ".mpls" must be
skipped even though its content parses as a perfectly good (long) MPLS
playlist — extension gating, not content sniffing, decides eligibility
(bluray.rs `!entry.is_dir && entry.name...ends_with(".mpls")`).
## HaltingReader
A `SectorSource` that fails every read in `halt_range` with
`Error::Halted` — exactly how a live drive behaves once the operator
presses Stop: `Drive::checked_exec` fails every SCSI command with `Halted`
from then on, and `Drive::read` deliberately preserves the variant. Reads
outside the range still succeed, so a test can aim the cancel at one
structure and leave the scan far enough along to have something to
truncate.
## Test: halted_playlist_read_is_not_reported_as_a_shorter_disc
A Stop on a live drive never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The Blu-ray enumerator must not swallow that into a
successful scan.
RED BEFORE GREEN: with the propagation reverted this returned
`Ok([00800])` — the `if let Ok(mpls_data)` skipped the cancelled read of
00801.mpls, the loop ended, and a half-enumerated disc came back at
success. One title from a two-title disc is indistinguishable from a disc
that genuinely holds one title, and the caller caches and rips from it.
The halt lands on the second playlist deliberately: it is the last
iteration, so nothing after it would poll a flag — only propagating the
read's own error catches it.
## Test: halted_clpi_read_is_not_accounted_as_an_unresolvable_clip
The same cancel landing on a `.clpi` read must not be classified as an
unresolvable clip either.
RED BEFORE GREEN: with the `Err(Error::Halted)` arm removed from the
CLIPINF match, the cancel fell into the generic "clip could not be
resolved" arm, which logs a disc-defect code and drops the title —
accounting an operator Stop as a scratched disc, and (in the scan loop)
dropping every remaining playlist in turn for a truncated `Ok`. With the
fix the cancel is propagated with its own variant intact.
## Test: halted_extent_resolve_is_not_a_title_missing_its_clip
And the same cancel landing on `file_extents` — the clip's ICB, not its
CLIPINF — must propagate too.
RED BEFORE GREEN: `note` used to exempt `Halted` from the unresolved
classification (correctly — a cancel is not an authoring hole) but had no
way to propagate it, so the resolver simply produced no extents and
`parse_playlist` returned a title with the clip's runtime counted, its
`size_bytes` counted from the .clpi, and zero bytes behind it. Measured
with the propagation reverted: `Ok(Some((768000, [])))` — the flagship
defect shape, wearing a cancel.
## Test: read_meta_title_ignores_non_xml_file_regardless_of_content
A non-.xml file must be ignored even if its content looks like a valid
meta XML (contains a `<di:name>`) — extension gating, not content
sniffing, decides eligibility (bluray.rs `!e.is_dir &&
e.name...ends_with(".xml")`).
+61
View File
@@ -0,0 +1,61 @@
# `src/io/bounded.rs` — bounded-syscall primitive
## Why this exists
[`crate::halt::Halt`] is cooperative: callers poll `is_cancelled()`. It
cannot reach inside a syscall the kernel currently owns the thread for —
`libc::sync_file_range`, `libc::fsync`, `File::write` on NFS, and so on.
`/api/stop` from autorip therefore can't unstick a thread sitting in
such a syscall.
`bounded_syscall` is the escape hatch: it runs `op` on a fresh worker
thread, then `recv_timeout`s on a rendezvous channel for the result.
The wait is broken into ~250 ms slices so the calling thread can poll
the supplied `Halt` in between. If the deadline elapses or the halt
fires, the worker is intentionally leaked — the syscall will unwind
whenever the kernel decides, or at process exit, but the caller is
free to fall back to a degraded code path (skip the sync, log loudly,
etc.).
## Trade-offs
- **Thread per call.** Cheap (`std::thread::spawn` is < 100 µs on
Linux/macOS), but not free. Use on coarse-grained finalisation
syscalls (`sync_all`, `sync_file_range(WAIT_AFTER)`), not on hot
inner-loop writes.
- **Leak on timeout.** A wedged syscall keeps a kernel slot and a
user-space thread around for the rest of the process's life. Bounded
by the number of independent rip/mux sessions, which is one per
disc. The alternative — trapping the caller forever — defeats the
entire purpose of `/api/stop`.
- **Halt granularity ~250 ms.** Halt observation is not instant; it's
the worst-case latency of the `recv_timeout` slice. Good enough for
human-driven stop requests; not suitable for hard real-time
deadlines.
## Single source of truth
Do NOT inline this pattern. Every blocking-syscall wrapper in the rip
+ mux pipeline calls this helper, so changes (e.g. swapping the
channel impl, adjusting the poll slice, adding metrics) land in one
place.
## Platform
Pure `std::thread` + `std::sync::mpsc`. No `cfg(target_os)` needed
here — the helper itself is platform-agnostic. Callers that wrap
Linux-only syscalls (`sync_file_range`) still need their own
`#[cfg(target_os = "linux")]` gates; this helper does not.
## `bounded_syscall` contract details
- `halt` is polled at `POLL_INTERVAL` granularity. Pass `None` for
callers that don't (yet) have a halt token plumbed through —
behaviour degrades to deadline-only, matching the 0.20.5
`wait_after_with_timeout` shape this helper generalises.
- `op` returns `R: Send + 'static`. The closure must own everything it
touches because it may outlive this call (timeout / halt cases).
- On `Halted` / `Timeout` the worker thread is intentionally leaked:
the syscall will unwind whenever the kernel decides, or when the
process exits. The calling thread is never trapped inside a kernel
call.
+111
View File
@@ -0,0 +1,111 @@
# `byte_prefetcher` — design notes and test rationale
`BytePrefetcher` (`src/io/byte_prefetcher.rs`) is the `std::io::Read`
analogue of `crate::sector::PrefetchedSectorSource`. It spawns a
producer thread that fills a bounded pool of `Vec<u8>` chunks from the
underlying reader and ships them through a channel; the consumer pulls
filled chunks, uses them, and sends the empty `Vec<u8>` back through a
recycle channel so the producer can re-fill in place. Result: zero
allocations and zero cross-thread frees in the steady-state hot loop.
This is the byte-stream half of the freemkv mux highway —
`BytePrefetcher` feeds `crate::mux::demux_thread::DemuxThread` for
`m2ts://` (the only in-tree caller today, via `crate::mux::resolve`),
and works for any stream whose source is an `io::Read` rather than a
`SectorSource`.
## `PrefetchShell`
Drop blocks the calling thread until the producer exits. To guarantee
a prompt exit, drop the forward receiver and the recycle sender first
so the producer observes channel disconnection (or cancel the `Halt`
passed to `BytePrefetcher::new`, which the producer polls at
`POLL_INTERVAL` granularity even while parked on a channel op).
## Test coverage rationale
- **`recycle_depth_is_forward_depth_plus_one`** — `RECYCLE_DEPTH` must
be one MORE than `FORWARD_DEPTH` per its own doc comment: the
producer needs at least one buffer to fill while the consumer holds
the other `FORWARD_DEPTH`-worth in flight. A `+` -> `*`/`-` mutation
on `FORWARD_DEPTH + 1` would under-size the recycle channel (e.g.
`FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one short), which starves the
producer of a spare buffer.
- **`into_channels_drop_releases_producer`** — the CRITICAL
regression: after `into_channels`, dropping the returned forward
receiver + recycle sender must let the producer observe
disconnection and exit, so dropping the `PrefetchShell` (which
joins the producer) returns promptly. With the old clone+forget the
leaked endpoints kept the producer blocked and this join hung
forever.
- **`delivers_all_bytes_in_order_across_chunks`** — CORE CONTRACT: the
prefetcher must deliver every source byte, in order, exactly once —
never silently truncate or duplicate. Source is 5000 bytes; chunk
size 1024 forces multiple chunks (4 full + 1 short of 904). The
reassembled stream must equal the source. Mutation: replacing
`buf.truncate(n)` with a no-op would over-report bytes on the final
short read and this fails.
- **`short_read_truncates_to_actual_length`** — a reader that returns
fewer bytes than requested per call must NOT leave stale tail bytes
in the delivered chunk. Cursor over 10 bytes with a 4096 chunk
yields a single 10-byte chunk; the consumer must see exactly 10
bytes, not 4096. Grounds `buf.truncate(n)`. Mutation: delete the
truncate and the chunk would carry 4086 zero bytes of padding,
failing the length assert.
- **`empty_source_yields_clean_eof_no_batches`** — EOF semantics: an
empty source (Cursor over `[]`) yields `read() == Ok(0)` on the
first call, which the producer treats as EOF and returns, dropping
tx. The consumer sees RecvError (zero batches), NOT an Err batch and
NOT a zero-length Ok batch. Grounds the `Ok(0) => return` arm.
Mutation: changing `Ok(0) => return` to `Ok(0) => continue` would
spin forever (test times out).
- **`read_error_is_propagated_as_err_batch`** — a reader that fails
mid-stream must surface the `io::Error` as an `Err` batch on the
forward channel, not swallow it. We deliver one good chunk then an
error. The consumer must see the good bytes followed by the error.
Mutation: changing `let _ = tx.send(Err(e)); return;` to a plain
`return` would drop the error silently and this fails.
- **`read_panic_surfaces_as_err_batch_not_clean_eof`** — PANIC
propagation: a reader that PANICS mid-stream must NOT be read as a
clean EOF at the demux boundary. The producer's `catch_unwind` sends
an explicit `Err` sentinel before the thread unwinds, so the
consumer sees the good bytes followed by an error batch — never a
silent truncation. Without the `catch_unwind` the panic would just
drop `tx`, the consumer would see RecvError (== clean EOF) and the
partial output would be finalized as if complete.
- **`recycled_buffer_carries_no_stale_tail`** — recycle-buffer reuse
must NOT leak stale bytes between chunks of different lengths. After
a full chunk, a short read reuses the same recycled buffer; the
regrow-then-truncate sequence must yield only fresh bytes. Verified
by reassembling the full stream. Source: 8 bytes of 0xAA + 3 bytes
of 0xBB, with chunk_bytes=8 → chunk0 = 8×0xAA, chunk1 = 3×0xBB.
- **`exact_multiple_length_no_trailing_empty_batch`** — exact-multiple
boundary: when the source length is an exact multiple of
chunk_bytes, the final non-empty chunk is followed by an `Ok(0)` EOF
read, NOT a spurious empty Ok batch. 12 bytes with chunk_bytes=4 →
three 4-byte chunks then clean EOF. Total bytes must equal 12 and no
zero-length batch may appear.
- **`drop_finite_prefetcher_joins_cleanly`** — dropping the
`BytePrefetcher` directly (without `into_channels`) must join the
producer cleanly when the source is finite. The producer reaches
EOF, drops tx, and exits; Drop's join returns. Grounds the
`BytePrefetcher` `Drop` impl. Mutation: removing the `Ok(0) =>
return` EOF exit would hang this join.
- **`drop_endless_prefetcher_joins_cleanly`** — regression: dropping a
`BytePrefetcher` directly (without `into_channels`) with an ENDLESS
source must not deadlock. Before the fix, `Drop` joined the producer
while `rx`/`recycle_tx` were still alive (sibling field drop order),
so the producer filled the depth-2 forward channel and then spun in
`send_timeout` forever (rx never drained, halt=None). The fix drops
`rx`+`recycle_tx` BEFORE the join so the producer sees
`SendTimeoutError::Disconnected` and exits.
+48
View File
@@ -0,0 +1,48 @@
# `labels::class_reader`
Hand-rolled JVM `.class` file reader, tailored to the subset needed for BD-J
label extraction (Deluxe / dbp / similar frameworks). Implements JVMS §4
(class file format) and §6 (bytecode) only as far as: constant pool,
methods, the `Code` attribute, and a non-allocating bytecode iterator.
No external deps beyond `std`. No `unsafe`. No panics on malformed input —
every parse fault is a typed `Error`. This is shared infrastructure for any
label parser that needs structured access to `.class` files inside a
`/BDMV/JAR/<x>.jar`.
## `ConstantPool::from_entries`
Test-only constructor — builds a constant pool directly from a vector of
entries. Real callers go through `ClassFile::parse`, which builds this from
class-file bytes. Used by parser unit tests (e.g. `labels::deluxe`) that
need to exercise bytecode walkers against synthetic class fixtures without
hand-rolling valid `.class` byte buffers.
Caller is responsible for: prepending a `CpInfo::Empty` at index 0 (the
spec-reserved slot), and inserting a `CpInfo::Empty` after each Long/Double
entry (the 2-slot quirk).
## `decode_modified_utf8`
Decodes JVM "modified UTF-8" (JVMS §4.4.7). Practically identical to
standard UTF-8 for the BMP-printable subset seen in label strings, but with
two notable deviations:
- U+0000 is encoded as the two-byte sequence `0xC0 0x80`, not as `0x00`.
- Supplementary characters (U+10000..) are encoded as a UTF-16 surrogate
pair, each surrogate emitted as 3-byte modified UTF-8.
For label data (mostly ASCII / Latin-1 / CJK in BMP), the simple
implementation here covers everything we'll encounter. The `0xC0 0x80` →
U+0000 case is tolerated explicitly; supplementary characters would need
surrogate-pair stitching, but no label-relevant string uses them.
## `Reader::slice` overflow test
`Reader::slice` takes an attacker-supplied length: a JVMS `u4`
`attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8 length
(§4.4.7). Adding it to `pos` without a wrap check panics on overflow in
debug and, in release, wraps to a small end offset that slips past the
bounds check and then panics inside the slice index. Both are panics
escaping a parser whose whole input is untrusted disc bytes; the contract
is an EOF error.
+18
View File
@@ -0,0 +1,18 @@
# CLPI vs MPLS cross-validation diagnostic
`src/labels/clpi_audit.rs` walks both per-clip CLPI program info and
per-playlist MPLS STN tables, normalizes their stream lists by `(PID,
language, coding_type)`, and classifies each PID into one of four
buckets:
1. **CLPI only** — a stream present in a `.clpi` ProgramInfo that no
playlist STN table references (orphan on disc).
2. **MPLS only** — a stream a playlist references that no `.clpi`
ProgramInfo lists (indicates a parser disagreement).
3. **Match** — both sources agree on coding_type and language.
4. **Divergent** — both sources see the PID but disagree on
coding_type or language.
`audit` returns a structured `ClpiVsMplsAudit` report. This is a
diagnostic surface only; it does not feed the label-selection
pipeline.
+36
View File
@@ -201,6 +201,42 @@ The full ripping pipeline chains three parsers:
The `Disc::scan()` method in `src/disc/mod.rs` orchestrates this: for each play item in each playlist, it loads the corresponding CLPI, calls `get_extents()` with the play item's in/out times, and collects the resulting sector ranges into the title's extent list.
## ProgramInfo Section (Per-Stream Metadata)
`parse_program_info` in `src/clpi.rs` reads per-stream (pid, coding_type,
language, codec sub-fields) from the ProgramInfo section, per the Blu-ray
Disc Read-Only Format Part 3 CLIPINF (CLPI) specification:
```text
ProgramInfo:
length: 4 bytes
reserved: 1 byte
num_programs: 1 byte
for each program:
spn_program_sequence_start: 4 bytes
program_map_pid: 2 bytes
num_streams: 1 byte
num_groups: 1 byte
for each stream:
pid: 2 bytes
stream_coding_info_length: 1 byte
stream_coding_info: (varies by coding_type)
coding_type: 1 byte
per-type bytes (see match arms in parse_program_info)
```
It returns `Vec::new()` on any structural mismatch — errors aren't
propagated because the EP map is the primary CLPI output, and a corrupt
program_info shouldn't break sector-range lookups.
### Test fixture notes
- `prog_info_start_zero_does_not_parse_header_as_program_info`: the
fixture is crafted so that parsing from offset 0 WOULD yield a stream
(num_programs at `[5]`, a second program header whose num_streams
byte at `[20]` is 1, then a well-formed stream record), so an empty
result can only come from the `prog_info_start > 0` gate in `parse`.
## References
- BD-ROM Part 3, Section 5.5: Clip Information file format
+79
View File
@@ -0,0 +1,79 @@
# `src/mux/codec/mod.rs` — extended notes
Relocated prose from doc comments in `mod.rs`, trimmed there to stay within
the comment-guard's prose caps. Each `//`/`///` pointer in the source names
the section here it corresponds to.
## `Frame::discontinuity`
Carried per-FRAME (not per-PES) because buffering parsers decouple the frame
from the PES that carried the gap signal: MPEG-2 emits whole GOPs, H.264/HEVC
lag one access unit behind the PES that produced them. A clean rip leaves
every frame `false`; the flag is only ever set on the degraded/conceal path
(P3/B1), when a frame's data begins after packets the demuxer never received
— an undecryptable unit concealed as NULL-TS upstream, or a continuity break
in a damaged source.
## Drop-on-undecodable policy across codecs ("clean muxes always")
- **Audio with independent access units** (DTS, AC-3/E-AC-3, …) gates each AU
through a per-codec corruption check and drops the ones that fail, keeping
A/V sync (a drop is a silence gap, never a shift) and logging every drop
via the shared `dropgate::DropTally`. DTS validates via its core-frame
header (ETSI TS 102 114); AC-3 uses its native frame CRC.
- **LPCM is excluded on purpose**: raw PCM carries no framing or integrity
data, so a corrupt sample is indistinguishable from a quiet one — there is
nothing to detect, so nothing can be honestly dropped.
- **Video is excluded on purpose**: H.264/HEVC/MPEG-2/VC-1 are inter-frame
predicted, so dropping one frame corrupts every frame that references it
until the next keyframe. Video instead resyncs at GOP/IDR boundaries (the
ResyncGate) and lets the decoder conceal — a fundamentally different model
than per-frame audio dropping.
- TrueHD/MLP, FLAC, MP2/MP3 and AAC-ADTS also gate undecodable frames via a
`DropTally` (poison/drop-forward for MLP's inter-AU restart state on a
major-sync boundary; CRC/sync-verdict drops for the passthrough codecs).
## `codec_private()` — why "absent" is the only truthful answer for some parsers
A codec parser's `codec_private()` feeds `DiscStream::codec_private`, which
the MKV muxer turns directly into the track's `CodecPrivate` element (RFC
9559 §5.1.4.1.24): `Some(bytes)` writes an element holding exactly those
bytes, `None` omits the element entirely. The two are NOT interchangeable —
a zero-length `CodecPrivate` asserts that the codec's initialisation data IS
empty, which is not true of any codec, and a one-byte one asserts a config no
decoder can parse.
The gating parsers (ADTS, MPEG audio, FLAC) and the passthrough parser
extract no configuration at all: they validate and forward frames whose
configuration is carried in band (ADTS headers, MPEG-1 audio frame headers,
FLAC frame headers) or supplied by the source container. Having derived
nothing, the only truthful answer they can give is "absent" — any `Some`
would be a value they invented. Nothing else in the test suite distinguished
the two, so each of these impls could have returned a fabricated `Some` and
produced a malformed track header unnoticed.
## `provenance_guard` test module
Every emitted frame must carry the source byte offset of the packet it came
from. That invariant held only for video for as long as it existed: `dts`,
`ac3`, `adts`, `truehd`, `pgs`, `dvdsub`, `flac`, `lpcm`, `mpegaudio` and the
passthrough parser all built frames with `source: None`, so a multi-clip
title could not place audio or subtitles by byte and fell back to inferring
the clip from timestamps — which is what made branched titles run minutes
long.
Nothing asserted it, so nothing caught it. Finding it took a brace-balanced
scan of the tree by hand, which also turned up five sites a regex had
missed. `provenance_guard` is that scan, as a test.
### `code_only` / `frame_literals` design
`frame_literals` walks `Frame { .. }` literals with balanced braces. A regex
cannot do this — `Frame` blocks contain nested braces (`coding: Some(..)`,
closures) and a non-greedy match stops at the first `}`, which is how five
sites survived the first pass.
`code_only` strips line comments before scanning. A comment that MENTIONS
`source: None` is prose, not a construction — this guard's own doc comment
tripped it on the first run, which is the same mistake as grepping a file
and matching its commentary.
+17
View File
@@ -0,0 +1,17 @@
# `src/mux/codec/coding.rs`
## Spec references
ITU-T H.273 (CICP code points, shared elsewhere), ISO/IEC 13818-2 §6.3.10
(MPEG-2 picture coding extension: `top_field_first`, `repeat_first_field`,
`progressive_frame`), RFC 9559 §5.1.4.1.28 (Matroska `FieldOrder` element
0x9D).
## HDR10 unit scaling
`Hdr10Metadata` values are stored in their raw SEI integer units (not yet
scaled to the Matroska float domain); the muxer applies the H.265 → Matroska
unit conversion at emit time so the scaling lives in exactly one place.
Spec: Rec. ITU-T H.265 D.2.28 (Mastering Display Colour Volume, payloadType
137) and D.2.35 (Content Light Level Info, payloadType 144).
+11
View File
@@ -0,0 +1,11 @@
# `consts::coding_type` — spec citations
`coding_type` is one registry used in two places that share the same value
space: the MPEG-TS PMT `stream_type` (ISO/IEC 13818-1 Table 2-34) and the
Blu-ray STN/CLPI `stream_coding_type` (BD-ROM Part 3).
The standardized video codes (`0x02`, `0x1B`, `0x24`) are ISO assignments
(ISO/IEC 13818-1 Table 2-34); `0xEA` (VC-1) is a BD-ROM convention in the ISO
user-private range. The `0x80..=0xA2` audio/graphics codes also sit in the
user-private range and follow the Blu-ray Disc Association / ATSC A/52
convention.
+14
View File
@@ -0,0 +1,14 @@
# CRC helpers (`src/mux/codec/crc.rs`)
## `crc16_mlp` reversed byte order
MLP's checksum is the "reversed" scheme. `crc16_mlp` emits its two bytes in
the OPPOSITE order to a standard little-endian CRC readout, so the caller
swaps them back and compares against the stored trailer word read
LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
Comparing big-endian instead is precisely the bug that function was fixed
for: it could never validate a real extended major sync, so whole TrueHD
tracks were dropped silently. This comment used to prescribe exactly that,
and to point at a `truehd::mlp_major_sync_ok` that does not exist.
Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
+87
View File
@@ -0,0 +1,87 @@
# `src/labels/criterion.rs` — internal notes
## `assign_stream_numbers`
Assign a 1-based stream number per `StreamInfo`, parallel to `infos`.
A stream mapped in `playbackconfig.xml` (`stream_map`) keeps its
mapped number. Streams with no mapping (absent or incomplete
`playbackconfig.xml`, or an unmatched `StreamInfo_ID`) are numbered
1-based per type — but the fallback counter SKIPS any number already
claimed via the map, so a synthesized number can never collide with a
map-assigned one. (Both numbering domains are 1-based per type, and
`apply_labels` matches on `(type, stream_number)`, so a collision
would mislabel tracks.)
Returns `None` when the 1-based stream-number space is exhausted —
every number in `1..=u16::MAX` for that type is either already
claimed by the map or already synthesized. That is unreachable on
real media: the BD STN_table carries at most 32 primary audio and 32
PG streams per playlist, so the 65535-wide space leaves >2000x
headroom. It IS reachable from a crafted `streamproperties.xml`
listing >65535 stream entries, and the only correct answers there are
"fail the parse" or "emit colliding numbers"; we fail.
The skip search is bounded by the numbering space itself: a `u16`
`saturating_add` here parked the counter at `u16::MAX` forever
whenever the map also claimed `u16::MAX`, turning an overflow guard
into a hang that `apply()`'s `catch_unwind` cannot interrupt. The
counters are therefore widened to `u32` so the skip loop strictly
increases toward a fixed ceiling (guaranteeing termination) and
exhaustion is reported rather than absorbed.
## Test: immunity pin, section-boundary half
Covers `an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it`.
`parse_stream_infos` emits one `StreamInfo` per `*StreamInfos` element
unconditionally — no filter, no `continue` — so an element whose
fields are missing or unrecognized still occupies its position, and
`assign_stream_numbers` still spends a number on it.
That is the property that keeps this parser out of the failure mode
where a skipped entry pulls every later label one stream forward. It
is load-bearing for the fallback path specifically: with no
`playbackconfig.xml` the numbers come purely from position in this
list, so dropping an element there would shift the rest.
Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
real audio streams renumber to 1 and 2.
Each stream here is one closed XML element, and every field is read
out of `&text[start..end]` — the range `xml::find_element` returned —
so one element can never absorb the next one's fields, however the
document is malformed around it. Contrast the flat-string walk in
pixelogic, where a section whose end marker is missing keeps
consuming entries as STN slots.
The missing-boundary case fails closed. An element with no close tag
of its own ends at the NEXT close tag, so it absorbs the element
behind it — the list comes back SHORTER. It cannot come back longer:
nothing outside a returned range is ever read as a stream, and
`find_element` yields `None` rather than a range running to EOF when
no close tag exists at all. A malformed document can cost this parser
a slot; it can never invent one.
Mutation: read fields from the document rather than the element's
range, or let a close-less element run to EOF → the trailing elements
re-enter the list as extra streams.
## Test: `exhausted_numbering_terminates_instead_of_looping`
A crafted `streamproperties.xml` can drive the fallback counter to
the top of the 1-based u16 stream-number space and then present one
more unmapped stream whose successor number is also claimed by the
map.
This must TERMINATE. The bound is the numbering space itself, so the
assertion is on the spec-derived exhaustion behaviour (`None`), not
on any tunable constant. Run on a worker thread with a deadline so a
non-terminating loop fails the test in 20 s instead of hanging CI.
## Test: `full_u16_numbering_space_is_usable_and_unique`
The whole 1-based u16 space must remain usable: 65535 unmapped audio
streams get 65535 distinct numbers with no panic and no wrap. The
literals here are the JVMS-independent, spec-derived size of a u16
1-based numbering domain, not a tunable cap.
+245
View File
@@ -0,0 +1,245 @@
# CSS keyless title-key recovery (`src/css/keyless.rs`)
## The cipher this attacks
The content descrambler ([`super::lfsr::descramble_sector`]) seeds its two
LFSRs **directly** from `key = title_key XOR sector_seed` (seed =
`sector[0x54..0x59]`): LFSR1 from key/seed bytes 0-1, LFSR0 (24-bit) from
bytes 2-4 with the pre-conditioning `r0 = r0*2 + 8 - (r0 & 7)`, and each body
byte recovered as `plain = TAB1[cipher] ^ (keystream & 0xff)`. There is no
title-key mangling on the content path, so the recovery is a single inversion
of the sector cipher.
## The attack
1. **Known plaintext → keystream.** Because descramble applies TAB1 to the
ciphertext, the per-byte keystream is `TAB1[cipher[i]] ^ plain[i]`.
2. **Brute the 16-bit LFSR1 seed.** For each of 2^16 seeds, run LFSR1
forward; for the first four steps deduce the LFSR0 output bytes from the
keystream (carry-tracked), reconstructing LFSR0's state. For the next six
steps clock LFSR0 normally and check it reproduces the keystream — a wrong
LFSR1 seed fails fast.
3. **Back-clock LFSR0.** Run four backward steps (each a 256-way search for
the byte shifted in) to reach the initial state, then undo the
`r0*2 + 8 - (r0 & 7)` pre-conditioning to recover key[2..5].
4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]`.
Known plaintext for step 1 comes from the longest periodic run in the
cleartext `sec[0x00..0x80]`, assumed to continue into the encrypted region at
0x80.
## `recover_title_key_from_plain`
Core of the recovery. `crypted` is the ciphertext starting at sector byte
0x80; `decrypted` is the matching known plaintext; `seed` is
`sector[0x54..0x59]`. On success returns the recovered 5-byte title key;
`None` if no LFSR seed reproduces the keystream. At least 10 bytes of
`crypted`/`decrypted` are required (the cipher is iterated 10 times: 4 to
reconstruct LFSR0, 6 to validate).
## `attack_crib`
Scans the clear header `sec[0x00..0x80]` (never scrambled) for the longest
run that repeats with a cycle length in 2..0x2F. If the run is long enough
(`plen > 3` and at least two full cycles), the plaintext at 0x80 is taken to
be that periodic run continuing forward. Returns `None` for an unscrambled
sector or one with no usable run — such a sector can be neither cracked nor
key-validated, only descrambled with an externally-cached key.
The header is untouched by `descramble_sector`, so the crib is identical
before and after descramble: the decrypt path uses it as a per-sector
"did the cached key descramble correctly?" oracle (the predicted plaintext
must reappear at 0x80), and the cracker uses it as its known plaintext.
## Tests
### `synth_sector`
Builds a synthetic scrambled sector for a given title key and seed, with
`plain` placed as the plaintext at byte 0x80, scrambled with EXACTLY the
cipher `descramble_sector` inverts. Returns
`(scrambled_sector, full_plaintext_body)`.
### `synth_periodic_sector`
Builds a synthetic scrambled sector whose CLEARTEXT (0x00..0x80) ends in a
periodic run that continues into the encrypted region — the case
`crack_title_key` is designed to crack.
### `recover_round_trips_known_keys`
MANDATORY round-trip (Task C.1): synthesize a scrambled sector for a known
(title_key, seed), then assert `recover_title_key` returns a key that
descrambles the body back to plaintext. CSS title-key recovery is
well-defined up to keys that scramble identically; we assert the full body
round-trips (the true correctness property), and additionally that the EXACT
key is returned for the common case.
### `descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with`
`descramble_matches` is the ONLY gate between the LFSR search and a key
handed back to the caller: both `recover_title_key` and the crib-driven
`crack_title_key_inner` return a candidate only if this says the key really
descrambles the sector to the known plaintext. A body that always answered
`true` would let the first spurious LFSR-seed match through as the title
key — the ripper would then descramble the whole title with a key that opens
nothing, producing garbage rather than a "no key" error.
Pinned both directions: the genuine key is accepted, and EVERY key one bit
away from it is rejected. The one-bit neighbours are the strongest form of
wrong key — a gate that only rejects wildly different keys would still pass
a near-miss out of the 2^16 seed search.
### `descramble_matches_does_not_disturb_the_caller_s_sector`
The gate is applied to a COPY: verifying a candidate must not modify the
caller's sector. `recover_title_key` runs the gate and then hands the sector
on to be descrambled for real — if verification descrambled in place, that
second descramble would run over already-transformed bytes (and, worse, a
rejected candidate would leave the sector corrupted).
### `recover_rejects_a_sector_that_ends_inside_the_encrypted_region`
A sector buffer that ENDS inside the encrypted region must be refused, not
sliced. `recover_title_key` slices `sector[0x80..0x8A]` unconditionally
after its length guard. The existing short-sector test uses
`SECTOR_BYTES - 1`, which is still long enough for that slice to succeed —
so the guard was never the thing producing the `None`, and dropping it (or
weakening the `||` to `&&`, which a full-length crib satisfies) changed
nothing observable. On a real short read this is an out-of-bounds panic on
the rip thread.
### `a_buffer_longer_than_one_sector_still_yields_its_key`
A buffer LONGER than one sector is still one sector: both entry points read
the first `SECTOR_BYTES` and must recover the key from it. Callers read DVD
data in multi-sector blocks, so an over-long slice is the normal case, not
an exotic one. A length guard that rejected it (`len > SECTOR_BYTES` instead
of `<`) would make every block-read caller silently unable to crack
anything.
### `recover_accepts_more_than_ten_bytes_of_known_plaintext`
`recover_title_key` accepts MORE than ten bytes of known plaintext, and uses
all of it: the extra bytes tighten the `descramble_matches` gate. The
ten-byte figure is a MINIMUM (the cipher is iterated ten times), not an
exact requirement — a guard reading it as an upper bound would reject every
caller that knows a longer crib.
### `a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused`
The scramble-flag gate on a sector whose BODY really is ciphertext. Both
entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
unscrambled sector has no title key to recover, and its bytes at 0x80 are
already plaintext. Every prior test of this gate used an all-zero or
all-`0x11` sector, where the recovery would have found nothing anyway — so
widening the mask test (`&` to `|`, which makes it true for EVERY flag byte)
produced the same `None` and went unseen.
Here the sector is genuinely scrambled and its key IS recoverable; only the
cleared flag stands in the way. If the gate stops working, both functions
start returning keys for sectors the disc says are in the clear.
`attack_crib` carries its own copy of the same gate and is the one that
actually stops the crack (`crack_title_key`'s is defensive duplication); it
also doubles as the decrypt path's cached-key oracle.
### `descramble_matches_forces_the_scramble_flag_on_its_own_copy`
The gate must verify a candidate against the sector's CIPHERTEXT regardless
of what the sector's own flag byte says. `descramble_matches` forces `0x10`
on its copy precisely because `super::lfsr::descramble_sector` is a no-op
when the scramble bits are clear — without that, verifying a
scrambled-but-unflagged sector compares raw ciphertext against the crib, and
every candidate key is rejected. Nothing exercised it: every fixture already
had the flag set, where forcing the bit is a no-op.
### `descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector`
The gate compares the WHOLE supplied plaintext, clamped to the encrypted
region. Two properties in one, because they are the two halves of
`plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
- it must compare beyond the first sixteen bytes, or a key that opens only
the head of the crib is accepted; and
- it must never compare past the end of the sector — a caller that knows
more plaintext than the 1920-byte encrypted region holds otherwise
indexes off the end of the buffer and panics.
### `sector_with_trailing_run`
Builds a sector whose clear header ends in a `period`-length repeating run
of exactly `run_len` bytes immediately before 0x80. The run is anchored to
ABSOLUTE sector offset (`sec[x] = pat[x % period]`), which is what makes
"the run continues past 0x80" a statement independent of the code under
test: the byte at `0x80 + i` of the underlying plaintext is
`pat[(0x80 + i) % period]`. Everything before the run is `0x00` (the pattern
bytes are all >= 0xD0, so the run cannot be extended backwards by accident),
and the encrypted region is filled with `0xFF` — so a crib that reads past
0x80 into "ciphertext" is immediately visible.
### `attack_crib_predicts_the_periodic_run_continuing_past_0x80`
KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly at
0x80, the crib is the run continued forward — the same ten bytes for every
run length, because the prediction depends only on the pattern and the
phase, never on how many cycles happened to be visible. The short lengths
are the load-bearing ones: at `run_len = 11` the crib window starts at 0x76
and is only 10 bytes from the end of the header, so any drift in
`plain_start`, in `cycles * best_p`, or in the `i % best_p` wrap reads the
0xFF "ciphertext" instead of the run.
### `attack_crib_refuses_a_run_shorter_than_two_cycles`
A run of exactly ONE cycle (plus the trivial tail the detector counts) is
not enough to predict forward: `attack_crib` requires at least two full
cycles. Weakening that guard would let a one-off byte sequence be declared
periodic and produce a confidently wrong crib — which the decrypt path uses
as its "is my cached key still right?" oracle.
### `attack_crib_refuses_a_buffer_shorter_than_a_sector`
`attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
own length guard is the only thing between a short buffer and an
out-of-bounds read. Nothing reached it: every caller-level test used a full
sector, and the entry points' guards fire first.
### `attack_crib_survives_a_header_that_is_periodic_to_offset_zero`
A header that is periodic ALL THE WAY to offset 0 must not walk the
backward scan off the front of the sector. The detector counts backwards
from 0x7f while `j < 0x80`. On a fully periodic header the run never
breaks, so `j` reaches 0x7f and the bound is the ONLY thing that stops it —
one step further and `0x7f - j` underflows a `usize` and panics. A constant
or fully-patterned 128-byte header is ordinary DVD data (padding, a run of
zeros), not a crafted input, and every existing fixture had a
filler/run boundary well before offset 0 that stopped the scan early.
### `attack_crib_is_independent_of_the_encrypted_region`
The crib is read from the CLEAR header only. A run that reaches 0x80 must
predict from the header bytes, never from the encrypted region — the
previously-fixed bug this function's doc comment records. Pinned by
rewriting the encrypted region and requiring the crib not to move.
### `recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input`
`recover_title_key_from_plain` unconditionally builds a 10-byte keystream
buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard is
the only thing standing between a short slice and an index-out-of-bounds
PANIC. Nothing reached that guard before: `recover_title_key` rejects
`plain.len() < 10` at its own door and always hands on exactly ten
ciphertext bytes, and `crack_title_key_inner` always passes a fixed
`[u8; 10]` crib. The guard is a live contract for any future caller and was
executed by no test at either boundary.
### `recover_title_key_from_plain_xors_the_sector_seed_back_out`
The seed XOR-back (`recover_title_key_from_plain`'s last step) is what turns
the recovered LFSR key into the TITLE key: `key ^= sector_seed`. Pinned as a
known answer across seeds that differ only in one byte — the same
ciphertext/plaintext pair therefore must yield title keys differing in
exactly that byte. Without this, a body that ORed the seed in (or dropped
the step) still round-trips on any fixture whose seed is zero, and on the
non-zero ones the failure looks like "no key found" rather than a wrong
step.
+311
View File
@@ -0,0 +1,311 @@
# `src/css/mod.rs` — extended notes
Overflow detail relocated from doc comments in `src/css/mod.rs` (kept short
there per the comment-guard caps). Function/item names below match the code.
## `CSS_LOCKED_BAIL`
The bus-auth read gate is global (all-or-nothing), so a run this long of
`05/6F/03` (CSS-locked) reads means the gate is shut and nothing here is
crackable — bail instead of grinding the full 50_000-sector budget. That
grind is what made rc5 appear to hang on a wedged USB bridge.
## `crack_key`
Recovers the CSS title key with no keys, by scanning scrambled sectors and
running the known-plaintext attack (see the `keyless` module). The crib comes
from the periodic-run detector: a scrambled sector's cleartext region
(bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
constant fill); the attack assumes that run continues across the 0x80
boundary into the encrypted region, giving the known plaintext the 2^16 LFSR
recovery needs. Scans up to 50000 sectors across the extents and returns the
first sector that yields a key — no player keys, no disc-key crack. Works on
a live drive (after bus-auth unlocks reads) and on disc images alike.
This convenience form runs to completion (no cancellation) and returns just
the key; callers needing an operator-Stop / watchdog cancel, or the
three-way `CrackOutcome` (to distinguish "unencrypted" from
"encrypted-but-uncracked"), use `crack_key_outcome`, which takes a `halt`
token.
## `CrackOutcome`
Distinguishes the three cases the bare `Option<CssState>` conflated (and
which caused a silent-failure bug: scrambled-but-uncracked content was
treated as "unencrypted" and muxed as plaintext garbage at exit 0):
- `Cracked` — a scrambled sector yielded a title key.
- `Unencrypted` — NO scrambled sector was seen across the scanned extents
(`is_scrambled_pack` never true): the content is genuinely plaintext, so
proceeding without a key is correct.
- `ScrambledUncracked` — scrambled sectors WERE seen but no key could be
recovered (the recovery found no crackable crib, or the scrambled region
was unreadable). The content is encrypted; muxing it as plaintext would
emit garbage, so callers MUST surface a hard error instead of falling
through to "unencrypted" — the per-title `crate::error::Error::CssKeyMissing`
when it is ONE title's own re-crack that failed (skippable: a sibling VTS
may still crack), or the disc-level `crate::error::Error::CssNoDiscKey`
when it is the disc-wide scan (`Disc::css_error`, every title fails
identically).
## `resolve_dvd_title_key`
The SINGLE place every DVD read path obtains a title key, so the file-backed
mux highway (`crate::build_iso_pipeline`) and the live-drive single-pass
`crate::DiscStream` descramble a DVD identically ("reading is reading"). CSS
keys are per-VTS and crackable from the scrambled data itself, so a
`None`/MPEG-PS title cracks its own key here, in playback order over
`extents`. Everything else is left untouched:
- AACS keys (HD-DVD `.evo` is also MPEG-PS but arrives as `Aacs`) — no CSS.
- a title that already carries a key — nothing to resolve.
- a genuinely clear DVD (no scrambled sector) — stays `None`, a mux no-op.
A scrambled-but-uncrackable title is a hard `crate::error::Error::CssKeyMissing`,
never a silent scrambled-passthrough mux. That code is the PER-TITLE one
(`error::is_skippable_title_stub`), which is correct here: this function
cracks ONE title's own extents, and another VTS on the same disc may still
yield its key, so an all-titles rip skips this title and finishes the rest.
The whole-disc failure is `crate::error::Error::CssNoDiscKey`, raised by
`Disc::ensure_decryptable_keys` from the scan's `css_error`.
## `crack_key_scan` (private)
Tracks a `saw_scrambled` flag so a scrambled-but-uncracked disc is
distinguished from a genuinely-unencrypted one (the `crack_key` `Option`
wrapper collapses both to `None` via `CrackOutcome::into_state`).
## `descramble_sector`
A no-op unless the sector is a scrambled MPEG-2 PS PACK. The pack start code
is checked, not just the byte 0x14 flag bits, for the same reason
`descramble_region` checks it: 0x14 only means "scrambling control" inside a
pack, and in an IFO it is whatever that format stores there. A real
`VIDEO_TS.IFO` sector holds 0x15 there while starting `00 26 00 00`;
descrambling it destroyed 1912 of its 2048 bytes, and because that sector
carries TT_SRPT the disc enumerated 38 titles while an image decrypted from
it enumerated 10, silently, at exit 0.
This function has no callers inside the crate, but the module-level example
prescribes it — so the crate's own documented guidance led straight into
that defect. Making the guard part of the function, rather than something
each caller must remember, is what keeps the safe path the easy one.
## `descramble_region`
Descrambles a whole CSS buffer in place, re-cracking the title key on a VOB
region boundary. `title_key` is a CACHE of the last crack, not a fixed disc
key: it changes per VTS/VOB region, so it is validated on every scrambled
sector and re-cracked on a miss (the standard on-demand per-region rekey).
This CSS key acquisition is intrinsic to the cipher — CSS has no external key
source, the ONLY way to a title key is cracking the data — so it lives with
the CSS primitives and runs inside `decrypt::decrypt_sectors` (a public,
self-contained CSS decrypt), NOT at the post-decrypt recovery seam that AACS
key-fetch and FMTS segment-skip use (those consume external inputs).
The clear header (`<0x80`) is never scrambled, so its periodic crib predicts
the plaintext at `0x80`. Descramble with the cached key; if the crib fails to
reappear the key region changed (or the primed key was wrong) — restore the
ciphertext, re-crack from this very sector, and descramble again. A crib-less
sector (no periodic run) can be neither validated nor cracked, so it rides
the cached key — correct, because it lives in the same region as the nearby
crib sector that set the cache.
This section used to document an `Error::DecryptFailed` for the case where a
sector's crib rejects the cached key and the re-crack from that sector also
fails. That behaviour was tried and REVERTED: crib mismatch plus crack
failure is the signature of a crib FALSE POSITIVE, not of a stale key, and
failing there made real discs unrippable. The code now descrambles with the
cached key and returns `Ok`.
## `has_scramble_flag_bits`
Deliberately NOT called `is_scrambled`. Byte 0x14 only means "scrambling
control" inside an MPEG-2 Program Stream pack; in an IFO, UDF or ISO 9660
sector it is whatever that format stores there. Treating the flag alone as
proof of scrambling is what destroyed 1912 bytes of a real disc's
`VIDEO_TS.IFO` — the sector carrying TT_SRPT — so the disc enumerated 38
titles and an image decrypted from it enumerated 10, silently, at exit 0.
Callers want `is_scrambled_pack`. It asks the same question and also
requires the pack start code, which every genuinely scrambled VOB sector
carries and no IFO sector does.
It stays public only because an integration test asserts the flag
extraction directly. It has no production callers, and its previous doc
comment claimed one (`decrypt::decrypt_sectors`) that did not exist — so the
name was an invitation and the documentation was an argument for accepting
it.
## `is_scrambled_pack`
The HARDENED test the crack scan uses to set its `saw_scrambled` evidence
flag (Fix 3).
`has_scramble_flag_bits` keys solely on bits 4-5 of byte 0x14. That byte is
only meaningful inside a real DVD sector — an MPEG-2 Program Stream pack,
which ALWAYS begins with the 32-bit pack-start code `00 00 01 BA` at offset
0x00. A tiny clear / nav-only stub (a 0.5 s menu loop, an FBI-warning title)
can carry arbitrary bytes that happen to set bits 4-5 of byte 0x14; trusting
byte 0x14 alone there would flip the scan's `saw_scrambled` gate and make a
genuinely-UNENCRYPTED title report `ScrambledUncracked` — a false E7023.
Requiring the pack-start signature FIRST means only a sector that is
structurally a DVD video pack can be counted as scramble evidence. This does
NOT weaken the genuine "encrypted but uncrackable" hard-fail: a real
scrambled feature is made of valid PS packs, so its scrambled sectors still
pass this check and still drive `ScrambledUncracked` when no key cracks.
The pack-start check alone is not enough, and the sector's PES `stream_id`
(offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
(`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
stream_id even on a scrambled sector. The MPEG-PS structural packets that
are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
because on those the byte at 0x14 is NOT a PES scrambling-control field but
raw payload/structure whose bits 4-5 land set by chance.
This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
set. With no `0x11` gate those nav packs flipped the crack scan's
`saw_scrambled` flag on a disc that carries no CSS at all; the crack then
found no key (there is none) and the scan returned `ScrambledUncracked`,
hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
`0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
can act on (the recovery only recovers a key from a scrambled ES pack), so a
decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
private_stream_1 (`0xBD`) packs, none of which are excluded, so its
scrambled sectors still set `saw_scrambled` and still drive
`ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
The DESCRAMBLE path gates on this same function — `descramble_sector` and
`descramble_region` both call it, not the raw flag test — because the raw
test does not merely mis-skip a sector there: it descrambles one that was
never scrambled and destroys it. The measured case is written up above under
`descramble_region`: a `VIDEO_TS.IFO` sector holding 0x15 at offset 0x14 lost
1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38 titles
became 10 — silently, at exit 0. One gate, both paths.
## Tests (`mod tests`)
Detail relocated from test doc comments:
- `a_crib_false_positive_keeps_the_cached_key_rather_than_failing`:
`attack_crib` is a heuristic — it finds a periodic run in the unscrambled
header and predicts the run continues past 0x80. When that prediction does
not hold, the crib reports a mismatch even though the cached key is
correct, and the re-crack then fails BECAUSE the crib was never valid. So
this combination is the signature of a crib false positive, not of a stale
key, and the cached key remains the best available evidence. This test
exists because a prior round read the same code as "descrambling with a
key we just proved stale" and made it `DecryptFailed` to match the AACS
path; real DVDs hit this constantly, and the change made a real disc
unrippable (no unit test caught it — the real-media acceptance gate did).
CSS is not AACS: an AACS unit key either opens a unit or does not, whereas
a CSS title key is recovered from data whose recoverability varies sector
by sector.
- `has_scramble_flag_bits_short_buffer_is_false_no_panic`: grounding —
`sector.len() >= 2048 && (sector[0x14] >> 4) & 0x03 != 0` short-circuits so
a short buffer never reads index 0x14. Mutation: swapping operand order
would panic indexing a 20-byte slice; this test catches it.
- `has_scramble_flag_bits_uses_bits_4_5_only`: grounding —
`(sector[0x14] >> 4) & 0x03`. Mutation: widening the mask to `& 0x0F` makes
0x40 report scrambled; the 0x40 assert catches it.
- `has_scramble_flag_bits_exact_sector_length_accepted`: grounding —
`sector.len() >= 2048`. Mutation: `>= 2048` → `> 2048` makes an exact
2048-byte scrambled sector report false; this catches it.
- `is_scrambled_pack_requires_pack_start_signature`: Fix 3 hardening —
`is_scrambled_pack` requires BOTH the pack-start code and the 0x14 bits, so
a clear/nav-only stub with stray 0x14 bits isn't scramble evidence (else a
genuinely unencrypted title reports the false E7023).
- `is_scrambled_pack_excludes_nav_and_structural_stream_ids`: detection fix —
`is_scrambled_pack` must exclude system_header/padding/private_stream_2 by
`stream_id` at 0x11, since CSS never scrambles those and byte 0x14 there is
raw payload, not a scrambling-control field. This is the exact
decrypted-HD-DVD defect (an `.evo` RDI pack is 0xBF with pack-start and
0x14 bits set); the video/private_stream_1 cases prove the exclusion
doesn't weaken real-DVD detection.
- `MockSource` fields `lock_all`, `crackable`, `short_read`, `stream_id`: model
a CSS-locked drive (`05/6F/03` on every read), a designated crackable LBA
serving a full synthetic scrambled sector, a short-read count (the
`recovery: true` short-read contract, `Some(0)` being the degenerate
spin-guard case), and the PES `stream_id` written at 0x11 (used to model an
HD-DVD `.evo` RDI nav pack via `PRIVATE_STREAM_2`).
- `crackable_sector`: builds a crackable scrambled sector whose cleartext
header (0x59..0x80) carries a periodic run continuing across the 0x80
boundary, mirroring the `synth_periodic_sector` fixture in the keyless
tests but built from the crate-internal `scramble_sector`.
- `a_source_that_returns_zero_sectors_still_obeys_the_scan_budget`: `tried`
increments only per inspected sector inside `for s in 0..usable`; an
`Ok(0)` never runs that loop, so `advance` is forced to 1 to keep the
cursor moving and the extent's full declared length is walked. Mutation:
deleting the `tried` charge in the `usable == 0` arm goes red at 60,000
reads for a misbehaving/adversarial source.
- `crack_key_caps_total_tries_at_50000`: grounding — `max_tries = 50_000`,
`tried += 1` before the read, loop guard `tried < max_tries`. Mutation:
widening the cap, or removing the increment, both go red.
- `crack_key_budget_is_shared_across_extents`: grounding — `tried` is
declared outside the extent loop with a shared `if tried >= max_tries`
break. Mutation: moving the declaration inside the loop would give each
extent its own budget (80,000 total instead of 50,000).
- `crack_key_scans_from_extent_start_lba`: grounding —
`reader.read_sectors(ext.start_lba + i, ...)`. Mutation: dropping
`ext.start_lba +` would start LBAs at 0 instead of the extent's start.
- `crack_key_continues_past_read_errors`: grounding — a read `Err` falls
through to the next sector rather than aborting. Mutation: propagating the
error with `?` would stop the scan after the first failure.
- `crack_key_empty_extent_reads_nothing` / `crack_key_no_extents_is_none`:
grounding — `while i < ext.sector_count` / `for ext in extents` are no-ops
on empty input; an off-by-one (`<=`) or a pre-loop read would break these.
- `crack_outcome_reaches_cracked_with_span`: audit gap "MockSource never
yields a crackable sector" — drives the full `crack_key_scan` over a
synthetic ISO to exercise the `Cracked` branch and `crack_span` recording
end-to-end, previously untested.
- `is_scrambled_uncracked_is_true_for_that_case_and_false_for_the_other_two`:
a prior round introduced the three-way split precisely because conflating
the cases made an uncrackable disc exit 0 with garbage output; earlier
tests only asserted the TRUE direction, so a predicate answering "yes"
unconditionally was indistinguishable from a correct one. All three
outcomes here come from real `crack_key_outcome` scans, not hand-built enum
values.
- `resolve_dvd_title_key_*` tests: `resolve_dvd_title_key` is the SINGLE
shared per-title CSS step both read paths (`build_iso_pipeline` multi-pass
and `DiscStream::new` single-pass) call, so these pin its full contract at
the shared boundary — crack success, hard-fail on scrambled-uncrackable
(never leaving `keys` as `None`, which would mux scrambled bytes as
plaintext), the decrypted-HD-DVD RDI-pack regression (end to end), `--raw`
passthrough, the AACS-keys-untouched gate, the clear-DVD no-op, and the
halt-cancellation-surfaces-as-Halted fix (not a truncated-scan verdict).
- `all_locked_synthetic_iso_yields_css_key_missing_signal`: audit §2/§5 #7 —
pins that an all-locked synthetic ISO across MULTIPLE extents produces
`ScrambledUncracked`, the exact signal `disc/mod.rs` converts into
`disc.css_error = Some(Error::CssKeyMissing)`.
- `recrack_succeeds_on_other_vts_extents`: audit gap "success path missing"
— the prior re-crack test only covered the locked→None path; this proves a
key cracked for one VTS is genuinely re-derived (not reused) for another.
+32
View File
@@ -0,0 +1,32 @@
# Warner CTRM label parser — relocated rationale
Referenced from `src/labels/ctrm.rs` unit tests via `// See docs/ctrm.md` pointers.
## Immunity pin: no analog of the `paramount` forced_sub false positive
Immunity pin against the defect measured in the `paramount` parser, where a
vendor `forced_sub` cell hung off a FULL dialogue track's own slot to say
"this track also contains forced signs", and reading that cell as "this
track is forced" flagged 30 MB dialogue tracks forced.
This format cannot express that. The forced signal is not a flag beside a
track's entry — it IS the entry's stream-kind token, drawn from a closed
vocabulary in which `subtitle_production` (the full dialogue track) and
`subtitle_narrative` (the forced-narrative track) are mutually exclusive
alternatives in the same position. A row is one or the other; there is no
cell a full track can carry to acquire the qualifier, so the paramount
failure mode has no encoding here.
Mutation: give `subtitle_production` a `Forced` qualifier, or add a forced
side-flag that both kinds may carry.
## Immunity pin: stream numbers come from the row, not a counter
`language_streams.txt` states each stream's number in field 3, so a row the
parser cannot use is simply dropped — it can never renumber the rows behind
it. This is the property that keeps this parser out of the STN-slot-shifting
failure mode that bites parsers which count positionally: there, a skipped
entry silently pulls every later label one stream forward.
Mutation: replace `parts[2]` with a running per-type counter → the three
unusable rows in the corresponding test collapse the survivors onto 1/2 and 1.
+39
View File
@@ -0,0 +1,39 @@
# `labels::dbp`
## Implementation
Iterates `CpInfo::Utf8` constant-pool entries rather than raw
byte-scanning each class file. Equivalent label coverage (the literal
`TextField,...` strings live in the CP as Utf8 entries) with no
false-positive risk from method bytecode or attribute names that
happen to contain `TextField,`. Language / purpose / qualifier
classification lives in `super::vocab` so all Java-parser families
share one source of truth.
The parser ignores any prefix before the first `TextField,` occurrence
in a constant string — whatever string-pool ordering placed ahead of
it is irrelevant.
## `MAX_LABEL_BYTES` rationale
The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
constant contributes up to 65535 bytes, and the `u16` stream-number
keyspace admits 65536 of them per type.
Headroom: real dbp menu labels are short display names — "English Dolby
Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
bytes. 256 leaves >5x headroom over that, and any string past it is menu
geometry or padding, never a language name — `vocab::lang` would not
resolve it anyway.
## `MAX_LABELS_PER_TYPE` rationale
The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
type are reachable; paired with `MAX_LABEL_BYTES` this bounds the whole
scan at 2 x 512 x 256 bytes.
Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
streams per playlist, and dbp emits one menu TextField per stream. 512
leaves 16x headroom over the spec maximum.
+500
View File
@@ -0,0 +1,500 @@
# decrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/decrypt.md — <topic>` pointer
left at the original site.
## Parallel AACS decrypt
Each AACS aligned unit (6144 bytes) is decrypted INDEPENDENTLY of every
other unit — per-unit key derivation from the unit_key plus the unit's
own first-16-byte header. There is no cross-unit dependency, so a
buffer of N units can be decrypted on N threads in parallel via a
persistent rayon thread pool.
Small buffers (< `PARALLEL_MIN_UNITS` units) fall through to the serial
path to avoid pool dispatch overhead beating the per-unit AES work.
## Thread-count configuration — three layers
Resolution order (highest wins):
1. The most recent `set_decrypt_threads` call with `n > 0`. Calling
this *replaces* the live thread pool — useful for a settings-page
slider in a long-running daemon.
2. `FREEMKV_THREADS` env var, if set and `> 0`. Single knob covering
decrypt today, intended to also drive any future input-side /
output-side worker pools.
3. Default: all available cores. Algorithm optimisation comes first —
we measure single-thread performance to find serial bottlenecks
before throwing parallelism at it — but once a pool is engaged we
use the whole box. Hard cap at `MAX_THREADS` (rayon stack memory).
## DECRYPT_POOL
Current rayon pool. `RwLock<Option<Arc<...>>>` so that
`set_decrypt_threads` can swap the pool out without leaking the old
one and without blocking ongoing decrypt work (in-flight calls hold an
`Arc` clone via `decrypt_pool` and finish on the old pool; new calls
pick up the new pool).
## decrypt_pool()
Get (or lazily build) the active rayon thread pool. Returns an `Arc`
so in-flight work survives a concurrent `set_decrypt_threads` swap.
Returns `None` if the pool cannot be built (e.g. the OS refuses the
worker threads under a pid/thread limit). The caller falls back to the
serial decrypt path — library code never panics here.
## unit_is_our_phase
Does this unit belong to the phase we hold the key for?
An FMTS forensic segment interleaves two variants at the unit level:
the disc carries both, and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving
it as ciphertext is correct, because the muxer drops untouched
ciphertext cleanly.
`Phase::All` means the whole range is ours (the non-forensic case).
This lived inline inside `apply_aacs_map`'s per-unit closure, where
nothing could reach it: a mutation run flipped the `-` to `+` and the
`/` to `*` in the index arithmetic and every test still passed.
Getting either wrong silently decrypts the wrong half of a forensic
segment.
## AacsKeyMap (proactive AACS key-selection map)
Which held unit key decrypts each LBA of a title's encrypted content,
decided ONCE before mux from the disc's CPS-unit (and, later, FMTS
segment) structure — never by trial-decrypt-and-check per unit at mux
time.
This is the pivot that ends the mux "key-server storm": the old path
decrypts a unit, checks whether the plaintext looks like clean MPEG-TS,
and — because authored-bad content never reaches that bar — concludes
"wrong key, fetch a fresh one" and re-asks the key service for units it
already holds the correct key for. There is NO per-unit byte pattern
that separates "correctly decrypted but authored-bad" from "still
encrypted", so that check is unanswerable. The map removes the
question: we resolve one key per CPS unit / segment up front (see
`resolve_mux_key_map`), record which LBA ranges each covers, and at mux
time simply "decrypt this LBA with key K" and trust it — bad TS is the
muxer's concern, exactly as for a physically-read clear disc.
Ranges are `[start_lba, end_lba)` → index into the `Aacs { unit_keys }`
pool, sorted and disjoint. The map is a POSITIVE list: an LBA in no
range is passed through untouched (no default key). How a single-CPS
disc is mapped depends on the caller: the whole-disc EXTRACT path uses
one blanket range `(0, u32::MAX, 0)` so every encrypted unit — parsed
title or orphan clip — resolves to key 0; the per-title MUX/sweep path
(`resolve_mux_key_map` → `content_map`) maps only the title's own
extents, so an orphan clip outside them is left as pass-through. Either
way, clear nav/filesystem sectors (encrypted-flag off) pass through.
## AacsKeyMap::read_plan
Build the FMTS **read plan**: the title's aligned units filtered down
to only the units this rip must actually read — every default / CPS
unit, plus, inside each forensic segment, ONLY our-phase (`Phase::Even`
/ `Phase::Odd`) units. The alternate-phase units are a different device
group's variant: a licensed player never reads them, and neither do we.
They are omitted from the plan entirely, so they are never fetched,
decrypted, or handed to the demux — the demux therefore sees one
gapless our-variant stream, with no ciphertext to trip a
concealed-gap resync (the old behaviour that dropped good frames
around every segment).
`extents` are the title's clip extents (unit-aligned in the interior; a
shorter tail is ordinary content and always kept). `unit_sectors` is
the AACS aligned-unit size in sectors (3). Contiguous kept units
coalesce into as few extents as possible so the producer still issues
large sequential reads across default content; only inside a ~480 KB
forensic segment do reads become unit-granular (every other unit). A
map with no forensic (Even/Odd) range returns `extents` unchanged — the
common disc is not touched.
The parity test is byte-identical to the decrypt hot loop
(`(unit_lba - range_start) / unit_sectors`), so a unit kept here is
exactly a unit `decrypt_sectors_mapped` would open, and vice-versa.
## apply_aacs_map
AACS scheme step: apply `map`'s per-unit keys to `buf`, in-place.
`base_lba` is the absolute LBA of `buf`'s first sector; each aligned
unit (3 sectors) is decrypted with the key the map assigns to its LBA.
There is NO key trial and NO `is_clean` verdict: the map already
decided the key from disc structure, so this applies it and moves on —
a unit that decrypts to authored-bad TS passes through for the muxer to
drop, never re-fetched.
A SCHEME, not a policy. It reports what it could not open by returning
`Err(DecryptFailed)`; the decision that an unopenable unit must never
be emitted belongs to `decrypt_span`, which is the one place that
decides it for every scheme. A map index outside the held pool is
likewise a fail-loud `Error::DecryptFailed`: the resolver's job is to
guarantee every selectable index is present, so a gap here is a
resolver bug, not silent loss.
## decrypt_sectors — CSS / clear detail
For CSS: descrambles per 2048-byte sector, self-cracking the title key
from the data (no external input). For `None`: a no-op. For AACS:
**always** returns `Err(DecryptFailed)` — AACS decrypts exclusively
through the resolved key map (`decrypt_sectors_mapped`), which keys
every content unit up front and fails at RESOLVE time when a key is
missing. Reaching this arm with AACS keys means a reader was built
without installing its map (a bug), so it fails loud rather than apply
a guessed key.
`unit_key_idx` is a legacy parameter kept so the CSS / `None` wrapper
signature stays stable; it is ignored (the CSS arm self-gates on its
per-sector scramble flag). Returns `Err` if decryption was expected but
impossible; never produces silently corrupted output. The `usize`
return is a legacy unverified-byte count that is always `0` for the CSS
/ `None` arms.
## decrypt_span — orchestrator rationale
THE decrypt orchestrator. Every path into this crate's decryption goes
through here.
How a disc decrypts is one process — resolve a key for this span,
apply it, and refuse if no key can be proven. Only the resolve-and-apply
step is scheme-specific. This function owns the loop and the refusal;
the schemes below supply only what genuinely differs between AACS, CSS
and clear media.
That split exists because its absence caused six separate defects in
one release. There used to be TWO top-level paths — this one for CSS
and clear, and a wholly separate `decrypt_sectors_mapped` for AACS
whose arm here was a bare `return Err` stub — so each scheme decided
its own answer to "there is no key for these bytes" and nothing held
them to the same one. CSS drifted to descrambling with a key it had
just proven stale; the mapped path drifted to passing an unkeyable
encrypted unit through as ciphertext. Both looked like success to the
caller.
Adding a scheme means adding an arm here, which means answering the
refusal question. That is the point.
## Tests — relocated rationale
### a_forensic_read_plan_drops_units_the_full_extents_include
`DecryptKeys::None` is a no-op even with a content map + scrambled
bytes. A forensic map's read plan is NOT the extents it was given —
and that is the precondition the mux's provenance guard keys on.
A clip's feed span is measured over the FULL extents at scan time,
while the mux reads this reduced plan, so the byte offsets stamped on
frames and the offsets recorded in the spans describe different
streams. The deficit accumulates, so every frame after the first
segment resolves to an earlier clip than it came from. The spans still
tile each other, so the tiling check cannot see it; the mux compares
the plan against the full extents instead and stops trusting
provenance when they differ.
### content_gate_css_keys_is_noop
CSS ignores the content gate (it lives in the AACS arm) and always
reports `0` — confirming the gate is a no-op for CSS and the read
stays scheme-agnostic (the litmus test: adding CSS verify touches only
the CSS arm, never the read).
### content_gate_css_actually_descrambles_the_buffer
`decrypt_sectors_in_content` is the entry point `DecryptingSectorSource`
dispatches to whenever a content map is installed
(`sector/decrypting.rs` line ~211), so it is on the live read path for
every mapped rip. It must actually DECRYPT. The two `_is_noop` tests
above only assert its `usize` return is `0` — which is what a body
replaced by `Ok(0)` also returns, so neither one constrains it at all.
Here a genuinely scrambled CSS sector goes in and the CONSTRUCTED
plaintext must come out. Anything that skips `css::descramble_region` —
including a body that just reports `Ok(0)` — leaves ciphertext in the
buffer and the caller muxes scrambled MPEG at exit 0.
Expected bytes come from the plaintext this test built BEFORE
scrambling (CSS scrambles only 0x80..2048; the header stays clear), not
from re-running any descramble routine.
### content_gate_aacs_keys_fail_loud_not_ok_zero
The AACS arm of the same entry point must fail LOUD. Under the
keymap-only model AACS decrypts exclusively through
`decrypt_sectors_mapped`; reaching this wrapper with AACS keys means a
reader was built without installing its key map, and continuing would
hand the caller ciphertext under an `Ok`. `DecryptFailed` is the
correct verdict per the function's own contract — it must not be
softened into a success with a zero count.
### css_region_change_recracks_the_title_key
CHARACTERIZATION (recovery refactor safety net): the CSS arm's
per-region re-crack (the `title_key` cache is stale for a new VOB
region → restore ciphertext, `crack_title_key` this sector,
re-descramble). Two crackable sectors scrambled under DIFFERENT keys
sit back-to-back; the cache is primed to the FIRST key. Sector 0 rides
the cache (crib matches); sector 1 must trip the crib mismatch and
re-crack to its own key. Both must land correct plaintext, and the
cache must end on region 1's key.
This behaviour currently lives inline in `decrypt_sectors` (the `Css`
arm). It is the delicate logic the recovery refactor will move to the
input-stream seam, so it must stay green byte-for-byte across that
move.
### aacs_scrambled_trailing_partial_is_rejected
Whole leading unit plus a SCRAMBLED trailing partial that is FLAGGED
encrypted in its clear seed (the malformed danger case): an encrypted
unit split across an extent boundary cannot be CBC-decrypted
standalone. The mapped decrypt must fail loud with `DecryptFailed`
rather than emit the ciphertext partial as clear. Exercises the real
shipping path (`decrypt_sectors_mapped`) and its trailing-partial
guard.
### aacs_clear_trailing_partial_passes_through
A CLEAR trailing partial (encrypted flag NOT set) is a legitimate
content tail and must pass through, never trip the guard above.
"Passes through" means byte-for-byte unchanged, not merely `Ok`.
Asserting only `is_ok()` let a mutant that corrupts the clear partial
while still returning `Ok` pass — which is the whole failure this test
names. Mutation: XOR any byte of the tail before returning -> the
snapshot comparison fails.
### none_keys_is_noop
`DecryptKeys::None` is a pure no-op: the buffer must be returned
byte-for-byte unchanged with Ok, regardless of content (even content
that looks scrambled).
Grounding: the `DecryptKeys::None => {}` match arm does nothing.
Mutation: replace the empty arm with a call that mutates buf -> the
unchanged assert fails.
### is_encrypted_matches_variant
`is_encrypted` reflects the variant: None -> false, Css/Aacs -> true.
Grounding: `!matches!(self, DecryptKeys::None)`.
Mutation: invert the `!` -> None reports true, this fails.
### css_descrambles_with_title_key
The CSS path descrambles each 2048-byte sector with the title key. A
scrambled sector run through decrypt_sectors must come back to its
plaintext body (keystream XOR is involutive), proving the title key is
actually applied.
Grounding: `DecryptKeys::Css { title_key } => for chunk in
buf.chunks_mut(2048) { descramble_sector(title_key, chunk) }`.
Mutation: change `chunks_mut(2048)` to `chunks_mut(2049)` or pass a
fixed wrong key -> the body no longer matches the plaintext.
### css_processes_every_sector_in_buffer
The CSS path processes EACH 2048-byte sector independently in a
multi-sector buffer. Two scrambled sectors (with different seeds) in
one buffer must both round-trip — pinning that the loop steps by 2048
and applies the key to every sector, not just the first.
Grounding: `for chunk in buf.chunks_mut(2048)`.
Mutation: change the loop to descramble only the first chunk (e.g.
`.next()`) -> the second sector stays scrambled, assert fails.
### css_rekeys_when_title_key_region_changes
CSS title keys are per-VTS/VOB region: a real disc holds DIFFERENT keys
for different regions and the only way to get each is to crack it. The
decrypt path must re-crack when the cached key stops descrambling (its
crib no longer reappears at 0x80) instead of blindly applying one key
across a region boundary — the bug that pixelated every freemkv DVD
rip.
Two sectors scrambled under DIFFERENT keys, cache primed to ONLY the
first (exactly what the one-shot scan crack leaves). Sector 0 validates
+ descrambles with the cached key; sector 1's cached-key descramble
fails the crib, so the path re-cracks sector 1's own key and recovers
its plaintext. Before the fix (blind single-key apply) sector 1 was
garbage.
Grounding: the CSS arm's `attack_crib` → `chunk[0x80..] != crib` →
`crack_title_key` → `*title_key = fresh` rekey.
Mutation: drop the rekey branch (apply the cached key always) →
sector 1's body no longer matches its plaintext; this fails.
### css_leaves_clear_sector_unchanged
The CSS path leaves UNSCRAMBLED sectors (flag clear) byte-for-byte
untouched — descramble_sector early-returns on a zero flag. A clear
sector mixed into the buffer must not be corrupted.
Grounding: descramble_sector returns immediately when
`(sector[0x14] >> 4) & 0x03 == 0`.
Mutation: remove that early return in lfsr.rs -> a clear sector would
be XORed with a keystream and change; this fails.
### css_empty_buffer_is_ok
CSS decrypt always returns Ok (it cannot fail — descrambling is XOR,
no key validity check), even for an empty buffer.
Grounding: the CSS arm has no `return Err` path; `chunks_mut` over an
empty slice is a no-op; the function ends `Ok(())`.
Mutation: make the CSS arm return Err -> this fails.
### aacs_mapped_out_of_range_key_idx_errors
A map that selects a key index OUTSIDE the held pool must fail loud
with DecryptFailed — never silently apply a wrong key or pass
ciphertext through. This validates `decrypt_sectors_mapped`'s up-front
`key_indices()` bounds check (the real shipping AACS decrypt path).
Mutation: drop the `unit_keys.get(idx).is_none()` guard → the
out-of-range index would not error; this fails.
### aacs_via_unmapped_decrypt_sectors_fails_loud
SAFETY NET: reaching the CSS/`None` wrapper (`decrypt_sectors`) with
AACS keys means a reader was built with no map — a bug. It must fail
loud, never apply a guessed key. (AACS decrypts exclusively via
`decrypt_sectors_mapped`.)
### read_plan_forensic_reads_only_our_phase_units
FMTS: a forensic Even segment drops exactly its alternate (odd) units
from the read plan — they are never fetched — while default content on
either side stays one coalesced sequential run. The kept units are
byte-identical to the ones the decrypt hot loop opens.
### read_plan_phase_parity_is_measured_from_an_unaligned_range_start
A forensic range does NOT start on an aligned-unit boundary. Its start
LBA comes from a source-packet number — `start_spn * 192` put through
`clip_byte_to_lba` (`mux/resolve.rs`) — and 192-byte packets have no
relationship to the 3-sector aligned unit, so `range_start % 3` is
whatever the disc says.
That makes `unit_ix = (lba - range_start) / us` load-bearing in both of
its operations, and the existing coverage used a range starting exactly
on the extent's first unit, where several wrong formulas agree with the
right one by arithmetic accident.
Getting the parity wrong is not a crash. It reads and decrypts the
ALTERNATE variant's half of a forensic segment: the units this disc's
key does not open decrypt to garbage, and the units it does open are
skipped. AACS 2.1 forensic marking is exactly the mechanism that makes
the two halves different, so a phase inversion is silent — it produces
a full-length rip carrying the wrong variant.
### read_plan_gates_the_last_whole_unit_of_an_extent_not_just_the_remnant
An extent whose last whole unit is an alternate-phase unit must still
drop it. The tail guard exists for a REMNANT shorter than a unit —
bytes with no following unit to desync — and an extent ending exactly
on a unit boundary has no remnant at all.
With the guard widened to `remaining <= us`, the final unit of every
extent bypasses the phase gate and is read unconditionally. On a
forensic segment that lands at an extent end, that is one
alternate-variant unit pulled into the rip and decrypted with a key
that does not open it.
### every_scheme_gives_the_same_verdict_when_no_key_can_be_proven
Every scheme that CANNOT prove a key answers the same way.
This is the property `decrypt_span` exists to hold. There used to be
two top-level decrypt paths — one for CSS and clear media, one for
AACS — and each decided its own answer, so they drifted apart in
opposite directions within a single release: CSS descrambled with a
key it had just proven stale, and the AACS path passed an unkeyable
encrypted unit through as ciphertext. Both reported success.
Asserting one verdict across the schemes is what makes a future
divergence a test failure rather than a silent corruption. A
per-scheme test cannot do that: each would still pass while the two
disagreed.
CSS is deliberately NOT in this list. Its title key is recovered from
the data, sector by sector, by a heuristic that false-positives — a
crib mismatch whose re-crack fails means the crib was wrong, not that
the key is stale, so the cached key is kept and used. Round 9 folded
CSS in here on the reasoning that "no key" should mean one thing
everywhere; that made real DVDs unrippable, and the real-media gate
caught it. Uniform policy is right for schemes that can PROVE a key
wrong. CSS cannot.
### an_encrypted_unit_outside_every_key_range_fails_instead_of_passing_through
An ENCRYPTED unit that falls outside every key-map range must fail,
not pass through as ciphertext.
"The map has no key here" and "there is nothing to decrypt here" are
different statements, and only the second makes passing the unit
through correct. On a multi-CPS disc an orphan clip — referenced by no
playlist, so in no title extent and therefore in no range — hits the
first and was treated as the second. `extract_tree` then counted those
bytes as GOOD, dropped the `.partial` suffix, and reported `complete:
true`, exit 0: a scrambled file on disk with a clean bill of health.
A CLEAR unit outside every range is the ordinary case (filesystem and
nav on a whole-disc read) and must still pass through untouched — so
this asserts both directions.
### mapped_key_selection_is_positional_so_pool_order_matters
The mapped descramble indexes the committed key pool POSITIONALLY
(`unit_keys[key_idx].1`), so the ORDER of the `Vec<UnitKey>` a
`KeySource` returns is load-bearing — it is NOT "cosmetic, the decrypt
path strips it and tries every key", as
`keysource::resolve_and_apply_traced`'s doc used to claim. Trial-decrypt
was deliberately deleted; nothing here searches the pool. Reordering
the same two keys therefore sends each range to the WRONG key: the
range that decrypted clean now fails the correct-phase `is_clean` net
loudly (or, off a forensic phase, would decrypt a whole span under a
neighbour's key). Pins the corrected doc.
### decrypt_threads_within_valid_pool_range
The default (auto) decrypt thread count is always a usable pool size:
at least 1 (a 0-thread rayon pool is invalid) and never above
MAX_THREADS (rayon stack-memory cap). This test reads the resolved
value without mutating the process-global override, so it is safe to
run in parallel with other tests.
Grounding: `cores.clamp(1, MAX_THREADS)` in the default branch;
`env.min(MAX_THREADS)` in the env branch.
Mutation: change `.clamp(1, MAX_THREADS)` to `.clamp(0, MAX_THREADS)`
on a 0-core probe (unlikely) — more robustly, change the cap to
`MAX_THREADS * 2` -> on a many-core CI box the upper-bound assert can
fail. The lower-bound (>=1) guard is the load-bearing invariant.
### phase_gate_selects_only_our_parity_of_a_forensic_segment
The FMTS phase gate picks which half of an interleaved forensic
segment we decrypt. Both the `-` and the `/` in its index arithmetic
survived a mutation run, and getting either wrong silently decrypts the
alternate variant into garbage while reporting success.
### phase_gate_does_not_panic_on_a_malformed_map
A malformed key map must not take down a long-running service. A unit
below its own range start, or a zero unit size, are both map bugs —
they must return a defined answer rather than panicking on debug
overflow or dividing by zero.
Every case here asserts the DEFINED answer, not merely the absence of a
panic. The zero-unit-size case used to be written
`assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
accepts both answers and so pinned nothing at all: the guards could
invert and it would still pass. The answer is knowable —
`saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
is even — so pin it.
</content>
+34
View File
@@ -0,0 +1,34 @@
# `DecryptingSectorSource`
## Why this decorator is the single source of truth for decrypt-on-read
Every decrypt-on-read caller (e.g. `DiscStream`) wraps its source in this
decorator rather than calling the cipher helpers directly. The actual cipher
code lives in `crate::aacs` and `crate::css`; the decorator just calls the
existing `crate::decrypt::decrypt_sectors` helper that drives both of them
in-place after each read. Centralizing the call site means callers can wire
the decorator unconditionally and keep their pipeline shape uniform
regardless of encryption state — for `DecryptKeys::None` discs it is simply
a pass-through, so there is no branch in caller code for "is this disc
encrypted?".
## `KeyFetch`: why two explicit operations
`KeyFetch` exposes `unit_keys` and `fmts_indexes` as two separate operations
rather than one generic callback, so the "one base key vs. a whole forensic
set" contract lives in the type instead of a caller guessing at the return
length:
* `unit_keys` resolves the base Unit Key(s) for a CPS unit from real
encrypted samples drawn from it. The non-forensic path: one key per CPS
unit (the pool grows by whatever it returns). Used by the mux's base /
multi-CPS map resolution and by the sweep/patch recovery decorator.
* `fmts_indexes` resolves the disc's AACS 2.1 forensic index keys from an
index-1 single-phase anchor batch. The source hands back the COMPLETE set
(ordered index 1..N); the caller sizes the forensic map to `len()` and
never assumes a fixed N (32 is all we've seen, but the contract is
"whatever the source returns, >= 1, is all of them").
`KeyFetch` is a stateless, shared pair of `Arc<Fn>`, so one instance is built
once and cloned cheaply (two `Arc` bumps) into every read path; it is
`Send + Sync` so it can ride the mux highway's producer thread.
+227
View File
@@ -0,0 +1,227 @@
# Deluxe BD-J framework parser (`src/labels/deluxe.rs`)
Detected on discs whose `/BDMV/JAR/<x>.jar` contains a `com/bydeluxe/`
directory entry.
## What this parser reads
Deluxe-authored discs store stream labels as ordinal references into
enum classes whose names are obfuscated per-disc, so a name-based
match won't work. The label data is instead recovered by matching on
the **shape of each enum's `<clinit>`**, which is framework-stable:
| Enum | Signature |
|---|---|
| Language | 70 `ldc` operations in `<clinit>`, sequence starts `English, French, Spanish, Dutch, ...` |
| Purpose | 8 ldcs starting `Normal, Commentary, PiP, Trivia, ...` |
| VideoFormat | 7 ldcs starting `HD, HDR10 Plus, HD Dolby, ...` |
| Region | 22 ldcs starting `USA_D1, LIC1, LIC2, LIC3, ...` |
| Studio | 6 ldcs in `<clinit>` |
Matching on the shape rather than the class name keeps the parser
working across obfuscation variants. Codec strings come from the
standard BD-J `org/bluray/ti/CodingType` enum referenced directly by
the binding constructors (see `StackVal::CodingType`), not from a
Deluxe-internal enum.
## Implementation phases
- **Phase A** — master enum identification (`identify_master_enums`).
Walks every `.class`'s `<clinit>` ldc sequence and matches against
the framework-stable fingerprints. Output: `Vec<(label, MasterEnum)>`
with full ordinal → string-value tables.
- **Phase C** — binding-class identification (`find_binding_classes`).
The per-stream table is built by some class via repeated
`getstatic` references to the master enums identified in A.
That class has the highest such `getstatic` count in the jar.
Heuristic shape; precise threshold may need tuning.
- **Phase D** — binding-class bytecode decoder (`decode_binding`).
Walks the binding class's `<clinit>` with a tiny symbolic stack
machine. For each `new X / dup / ... / invokespecial X.<init>`
sequence, collects the int values and enum-reference operands
between the `dup` and the constructor call, then emits a
`DecodedStream`. The signal-to-StreamLabel mapping (which arg is
stream index? which is language? audio vs subtitle?) uses a
heuristic — see `interpret_streams` for the mapping rules.
## Confidence
`parse` returns `Some(ParseResult::high(labels))` when Phases A
through D produce at least one stream: the master enums matched their
framework-stable fingerprints (a strong ordered-prefix signature), the
binding class decoded, and at least one per-stream binding resolved to a
real (language, purpose, codec) tuple — the schema was fully recovered,
not guessed. `None` when the disc isn't Deluxe-authored or when decoding
produces zero streams (a recognized-but-broken state that the analyzer
still surfaces via `parsers_detected`).
## Size caps (Phase A candidate pool)
`MAX_CLINIT_LDC_STRINGS` (cap on `ldc` operands retained per class by
`clinit_ldc_strings`): unlike every other count cap in this crate, the
paired byte cap here cannot be the disc-file size — a `.class` entry
gated only by a `com/bydeluxe/` path prefix deflates from ~100 KB up to
the 64 MiB `MAX_CLASS_BYTES` read ceiling, so ~33M two-byte `ldc`
instructions — one retained `String` each — are reachable from a small
crafted disc. The bound has to be on the decompressed work, so it is
applied here. Headroom: the largest framework-stable enum is `Language`
at 70 values (`FINGERPRINTS`), and no fingerprint matches a count more
than `LDC_COUNT_TOLERANCE` away from its expected size, so anything past
~74 can never identify a master enum. 4096 leaves ~55x headroom over the
largest real enum for framework drift.
`MAX_CLINIT_LDC_BYTES` (companion byte cap): the count cap alone still
admits 4096 x 64 KiB of `Utf8` (a JVMS `CONSTANT_Utf8_info` length is a
u16), i.e. ~268 MB per class from repeated `ldc` of one huge constant.
Headroom: master-enum values are short display names ("English",
"HDR10 Plus", "USA_D1") well under 32 bytes, so a real Language enum
retains ~1 KB. 256 KiB admits 4096 values averaging 64 bytes each.
`MAX_CANDIDATE_TOTAL_BYTES` (aggregate companion, bounds retention across
ALL classes at once): `identify_master_enums` holds every class's
retained strings in one map simultaneously, so the per-class cap alone
still admits `classes x 256 KiB`: a 64 MiB jar of minimal `.class`
entries reaches tens of GiB. This is the same bound one level up.
Headroom: the five `FINGERPRINTS` enums together hold ~113 short values
(~1.2 KB). Every other class in a real BD-J jar contributes only
whatever string constants its own `<clinit>` loads — resource paths,
config keys — so a large authored jar lands in the low hundreds of KB.
16 MiB leaves ~40x headroom over a deliberately generous 400 KB estimate
for real media.
`MAX_CANDIDATE_CLASSES` (entry-count companion to
`MAX_CANDIDATE_TOTAL_BYTES`): the byte budget alone still admits ~16M
map entries when every class retains a single one-byte `ldc`, and the
per-entry `HashMap` + `String` overhead is not counted by that budget.
Headroom: a large retail BD-J title ships on the order of 1-3k classes,
and only those with a non-empty `<clinit>` ldc sequence become
candidates. 65536 leaves >20x headroom over the class count of any real
jar.
## Internal helpers
- `clinit_enum_field_names`: collects the static-field names an enum
class's `<clinit>` stores its own instances into, in declaration order
— the ordinal -> field-name mapping that mirrors `clinit_ldc_strings`'s
ordinal -> value mapping. An obfuscated Deluxe enum constant compiles
to `new E; dup; ldc "Value"; invokespecial E.<init>(…); putstatic
E.<field>`, so the `putstatic` whose owning class AND field descriptor
are both this class's own type names the field for that ordinal. The
binding class then references that constant as `getstatic E.<field>`;
without this mapping `MasterEnumTable` cannot turn the obfuscated field
name back into an ordinal. Bounded by `MAX_CLINIT_LDC_STRINGS` like the
value walk.
- `clinit_ldc_strings`: walks `<clinit>` and collects every `ldc`/`ldc_w`
operand that resolves to a `String` or `Utf8` constant, in declaration
order; `None` if the class has no `<clinit>`. Collection stops at
`MAX_CLINIT_LDC_STRINGS` operands or `MAX_CLINIT_LDC_BYTES` of retained
text, whichever comes first: the walk is driven by the decompressed
class, so it isn't bounded by the disc-file size cap the way the rest
of this module's counts are. Truncation cannot lose a real match — a
truncated sequence is far longer than any `FINGERPRINTS` entry's
`expected_count + LDC_COUNT_TOLERANCE`, so it would have been rejected
on count anyway.
- `find_binding_classes`: identifies all binding-class candidates by
getstatic-count to the master enums, using `MIN_GETSTATIC = 4` — the
binding class on a typical disc has 50+ such getstatic references (one
per slot x arity); the low floor lets a small disc with few streams
still qualify, while still filtering out classes that just reference
the language enum once for a config string. Some Deluxe discs split
the per-stream table across two binding classes (one for audio, one
for subtitle — the audio class commonly has the most getstatic refs,
the subtitle class somewhat fewer, both sharing the master Language +
Purpose enums), so this returns top-K candidates ordered by descending
getstatic count, filtered to a minimum concentration of references.
`MAX_CONSTRUCTIONS` (Phase D, cap on `Construction`s retained from a
binding `<clinit>` walk): one entry is appended per matched `new X / dup
/ invokespecial X.<init>` with no other bound — a `.class` gated only by
a `com/bydeluxe/` path prefix deflates to the 64 MiB `MAX_CLASS_BYTES`
ceiling, and the shortest matching sequence is a handful of bytes, so
millions of `Construction`s (each a `String` plus an arg `Vec`) are
reachable from a small crafted disc (~1 GiB). The same cap bounds the
per-class union in `decode_binding_class` (a crafted class may repeat
`<clinit>`, which JVMS §4.6 forbids but this reader tolerates) and the
cross-class union in `parse`, so the whole phase retains at most this
many. Headroom: the BD `STN_table` (BDAV stream-entry counts) admits at
most 32 primary audio and 32 PG streams per playlist, and a Deluxe
binding table covers the disc's playlists — low hundreds of entries on
the largest retail titles. 4096 leaves >20x headroom.
`Decoder::push` (symbolic-stack push during binding `<clinit>` walking):
honours the Code attribute's declared `max_stack`. JVMS 4.7.3 requires
that a method's operand stack never exceed `max_stack` at any point, so
a push past it can only come from bytecode that would fail JVM
verification. Dropping it costs nothing on real bytecode and bounds the
decoder: a `.class` gated only by a `com/bydeluxe/` path prefix deflates
from ~100 KB to the 64 MiB `MAX_CLASS_BYTES` ceiling, i.e. ~67M
single-byte `iconst_0` (~2 GiB of `StackVal`) on an unbounded `Vec`.
Headroom: `max_stack` is exactly what javac computed for the real
binding `<clinit>`, so no real construction can be clipped. The symbolic
stack counts `long`/`double` as one slot where the JVM counts two, so
its depth is never greater than the verified depth.
## interpret_streams: Constructions -> StreamLabels
Converts the per-construction tuples from Phase D into `StreamLabel`s.
Two binding-constructor shapes are handled:
- 5-arg: `BindingType.<init>(I, Lang;, Lpurpose;, I, LCodingType;)V`
- 4-arg: `BindingType.<init>(I, Lang;, Lpurpose;, LCodingType;)V`
Args are identified by **TYPE**, not position:
- First `EnumRef{kind: "Language"}` -> audio/subtitle language
- First `EnumRef{kind: "Purpose"}` -> Deluxe purpose ordinal
- First `CodingType(name)` -> codec field name (translated via
`coding_type_to_codec_hint`)
- First `Int(n)` -> stream index (preserved as ordering hint;
per-type sequential stream_number is what actually goes into the
StreamLabel, since BD spec stream-numbering is anchored on MPLS
data, not the binding code)
Stream type inference:
- Construction has a `CodingType` arg -> audio stream (subtitles on
Deluxe don't carry a CodingType; their codec is implicit PGS via
the BD spec).
- Construction has Language but no CodingType -> subtitle stream.
- Neither, and its binding type never yielded a stream -> not a
stream (skip). See `slot_kind` for why the binding type is
consulted rather than the language alone.
`slot_kinds`: which stream list each binding type enumerates, learned
from the constructions that DID resolve a language. A `<clinit>` walk
emits a `Construction` for every `new X; ...; invokespecial X.<init>`
it sees, so the list mixes real stream bindings with whatever else the
class initializer builds. `binding_type` is the constructed class name,
which is how the two are told apart: the stream bindings all share one
class (Deluxe splits audio and subtitle across two), and that class is
identifiable from the slots that resolved. A binding type that resolved
as both kinds is left out — with no consistent answer, guessing a list
to advance would be worse than not advancing.
`slot_kind`: the stream list an unresolved construction occupies a slot
in, or `None` when it is not a stream binding. A
`org.bluray.ti.CodingType` argument is decisive on its own: nothing but
an audio stream binding is handed one. Otherwise falls back to what the
binding type's resolved siblings showed (`slot_kinds`).
## Confirmed studio variants
- **Universal** (`studio="uni"`): Language enum `pd` (65 values,
`English, French, Spanish, Dutch, …`), Purpose enum `lp`
(`Normal, Commentary, PiP, Trivia, Descriptive, Score`), audio binding
`np.<init>(I, Lpd;, Llp;, Lorg/bluray/ti/CodingType;)`, subtitle binding
`wb.<init>(I, Lpd;, Llp;, Lmi;)`, all built in one binding class's
`<clinit>` alongside a title-wrapper object whose constructor takes the
per-stream arrays (filtered out by the array-parameter guard in the
decoder). SDH/RNIB is encoded as a distinct Language VALUE
("English SDH", "English RNIB"), recovered into the qualifier from the
name. Grounded on fixtures captured from a real Universal Blu-ray release.
- **Disney/Warner**: 70-value Language enum, same binding shape; the
original corpus this parser was written against.
+74
View File
@@ -0,0 +1,74 @@
# `demux://` sink — per-codec ES output
The `demux://` sink is a write-only `crate::pes::Stream`. Instead of muxing
the per-track `PesFrame` stream into a single container (as `MkvStream`
does), it routes each frame's payload to the file for `frame.track`,
post-processing where the internal codec `Frame` form differs from the
standalone on-disk ES form:
- **HEVC / H.264**: the codec parsers emit hvcC/avcC-style 4-byte
length-prefixed NALs (start codes stripped) and carry the parameter sets
out-of-band in `codec_private`. A standalone `.hevc`/`.h264` needs Annex-B
framing with the parameter sets prepended — see `AnnexBWriter`.
- **PGS**: the parser collapses a display/clear PCS pair into one
duration-bearing `Frame` with the raw segment payload but no `PG` magic /
timestamp header. A `.sup` needs the HDMV segment framing rebuilt — see
`PgsSupWriter`.
- **VobSub**: the `.sub` is the raw SPU stream; the `.idx` sidecar is
synthesized from per-SPU PTS + byte offsets — see `VobSubWriter`.
Every other codec writes `frame.data` verbatim (`PassthroughWriter`).
The sink does NOT touch the MKV mux path; it is purely additive.
## `PgsSupWriter`: rebuilding HDMV segment framing
The parser hands the demux sink the concatenated PGS segments of a display
set in `frame.data` (segment_type + segment_size + payload, repeated), with
no `PG` magic and no PTS/DTS. A `.sup` prefixes each segment with a 13-byte
header: `0x50 0x47` ("PG") | PTS u32 BE | DTS u32 BE | (segment_type|size
already present in the payload). When the parser folded a trailing clear
(`duration_ns` set), the writer re-emits it as an empty composition at
`pts + duration` so players time the subtitle out.
## `synthetic_clear_display_set`: synthesizing the clear PCS
Builds a synthetic "clear" display set: an empty PCS (0 composition objects)
followed by an END segment. The parser folds the original clear/end PCS
pair's wipe time into the display frame's `duration_ns` and drops the clear
bytes, so a faithful `.sup` re-emits one here at `display_pts + duration`;
without it every subtitle lingers to EOF. `width`/`height` are carried from
the display set's PCS so the clear PCS advertises the same video geometry —
they don't affect the wipe but keep the segment well-formed. Returned bytes
are concatenated `type(1)+size(2 BE)+payload` segments, the same shape
`emit_segments` consumes.
## Test: `epoch_driver_follows_ref_video_not_track_zero`
Regression for the hardcoded `frame.track == 0` epoch driver: on an M2TS/PMT
title the PMT can list an AUDIO ES before the VIDEO ES, so the video lands
at stream index 1. The sink already resolves the video reference
dynamically (`ref_video_track`); the epoch driver must use that SAME
reference, not the literal 0. In the test, track 0 is audio and track 1 is
video; a video-only clip boundary (track 1) must open a new epoch (bump
`offset_ns`). With the bug (only `frame.track == 0` drives epochs) the
video back-jump would be treated as a passive rider and `offset_ns` would
stay 0 — corrupting every track's rebased timeline.
## Test: `a_demux_export_the_seam_plan_emptied_fails`
Audit finding: this sink builds its timeline with
`TimelineContinuity::with_clips` but no test ever gave it a title with
usable PlayItem marks, so `frames_mapped` and `dropped_total()` were always
zero and BOTH gates in `finish` were unreachable. Deleting either left the
whole suite green while a `demux://` export of a seamless-branching title
wrote zero-byte track files beside a populated chapters document, at exit 0.
## `sanitize`: path-hostile filename characters
Control characters are replaced, NUL above all. Every string this touches is
disc bytes, and a language code is three raw STN bytes run through
`from_utf8_lossy` with no validation — `00 00 00` is the ordinary
"undefined" encoding on real Blu-rays. A NUL in a path aborts
`File::create` with `InvalidInput`, which took the whole demux export down
before a single track file was opened.
+65
View File
@@ -0,0 +1,65 @@
# `DemuxThread`
## Why a second worker thread
With `crate::sector::PrefetchedSectorSource` alone, read+decrypt already
runs on a producer thread; the *consumer* (main) thread still serialises
`ts_demuxer.feed` (M2TS parsing) with the codec parsers. Profiling showed
feed at ~37% and codec parse at ~44% of consumer wall time — i.e. feed is
heavy enough that pipelining it with parse pays for itself.
Splitting them: feed runs in `DemuxThread`; the consumer thread receives
`Vec<PesPacket>` batches and runs codec parse + frame emission only. Total
throughput becomes `1/max(feed, parse)` instead of `1/(feed + parse)`.
## Lifecycle
`DemuxThread::spawn_zero_copy` consumes the prefetch channels and the
demuxer state, returning a handle plus a `Receiver<DemuxBatch>`. Dropping
the handle closes the channel which signals the worker to exit; the join
in `Drop::drop` blocks until the worker observes channel closure and
returns (no timeout — a wedged downstream would block the drop until it
releases the channel).
## `spawn_zero_copy` rationale
Instead of taking a `SectorSource` and memcpy-ing through its
`read_sectors` API, this constructor consumes the prefetch channels
directly: filled buffers come in via `prefetch_rx`, the demux thread
feeds them, then returns them to `recycle_tx` for the producer to
re-fill. Eliminates the 16 MiB memcpy per batch that the SectorSource
adapter incurred (and, with the producer-side recycling pool, also
eliminates the per-batch heap alloc / cross-thread free that was costing
40%+ of demux-thread time before).
## Test: empty-batch disconnect fix (`worker_exits_promptly_on_consumer_drop_during_empty_batches`)
Regression: worker must detect consumer disconnect even when every demux
batch is empty (no matching PIDs / null packets).
Before the fix, `tx.send()` was never called for empty batches so the
worker never observed the consumer drop — it would spin through ALL
remaining extents before exiting, causing `DemuxThread::drop`'s `join()`
to block for minutes on a mostly-untracked disc region.
The watchdog: if the worker doesn't exit within 1s of the consumer drop
the test fails (rather than hanging forever as the bug would).
## Test: spawn-failure drop-order (`channels_disconnected_before_producer_join_on_spawn_failure`)
Regression: on thread-spawn failure the channels must be dropped BEFORE
the producer shell so the upstream producer observes disconnection and
exits, allowing `join()` to complete without hanging.
A true EAGAIN/pids-limit spawn failure cannot be reliably forced in a
unit test without root or ulimit co-operation, so the test exercises the
drop-order contract directly: a mock shell that panics if `join()` is
called while either channel end is still open.
The test constructs a `(prefetch_tx, prefetch_rx)` pair where the tx side
is held by a sentinel that stays alive as long as either channel end is
open, then asserts that the sentinel is gone by the time
`producer_shell`'s join logic would run. Because a real spawn failure
can't be forced, the test instead verifies the helper logic in isolation:
drop `prefetch_rx` and `recycle_tx` first, then observe the producer-side
sender is disconnected, which is the property the fix relies on.
+65
View File
@@ -0,0 +1,65 @@
# diag.rs — scan diagnostics rationale
## Why this module exists
A bug report log must be self-diagnosing: everything needed to explain *why*
freemkv made the choices it did at scan must be in the log, in a compact,
machine-parseable form. `diag.rs` emits one terse line per row (title, cell,
stream, decision) under the `tracing` target `freemkv::diag`, which the CLI
routes to `log.txt` when `--log-level 3` (debug) is set.
## Format conventions (stable, greppable)
- Every line is prefixed by a `tag=` so a log scraper can filter (`disc`,
`title`, `dvd.cell`, `dvd.vattr`, `dvd.aattr`, `bd.clip`, `bd.mark`, `aacs`,
`stream`, `decision`).
- Raw bytes are shown as `0xNN` next to their decode so a wrong decode is
obvious against the raw value.
- This module only READS already-parsed scan state — it never re-reads the
disc and never mutates anything.
## DVD per-cell table vs. Disc-level dump
The DVD per-cell table (with the raw cell-category byte) is emitted from the
IFO scan itself (`dump_dvd_cells`), because the per-cell `ifo::DvdCell` detail
is lowered away before the `Disc` is built. The `Disc`-level dump
(`dump_disc`) covers everything that survives lowering: titles, streams, the
picked main feature, and AACS state.
## `dump_dvd_substream_probe`
Emits the ACTUAL per-physical-sub-stream AC-3 channel counts read off the VOB
during the mux-time sub-stream probe (the Silence-of-the-Lambs wrong-stream
fix). This is the ground truth the IFO nibble is compared against: each row is
`sub_id=0x8x channels=N` for a physical `private_stream_1` AC-3 sub-stream
whose first frame was decoded. An empty probe (scrambled / unreadable / short
VOB) logs a single `probed=0` line so the absence is explicit in a bug log.
Self-sufficiency: with `tag=dvd.aattr` (the IFO's declared sub_id + claimed
channels) and these `tag=dvd.substream` rows (the physical reality), a bug log
alone shows whether the ordinal `0x80` actually carries the declared channel
layout — no disc needed to diagnose a wrong-substream rip.
## `dump_mkv_track`
Emits the MKV `TrackEntry` elements the muxer is about to WRITE for one track
— the Windows-fps-class metadata (FlagInterlaced, FieldOrder,
DefaultDuration, DefaultDecodedFieldDuration, Display dims) plus the
codecPrivate as hex. With this row a bug log alone is enough to verify why
Windows Explorer reports a given frame rate for an interlaced SD track: the
container values that drive its fps derivation are all present, no disc and
no MediaInfo needed.
`track_number` is the 1-based MKV track number; `track` is the built
`crate::mux::mkv::MkvTrack` whose fields map one-to-one onto the emitted
elements (see `MkvMuxer::new`).
## `main_feature_reason`
The `reason=` token on the main-feature decision row is DERIVED from
`Disc::CANONICAL_TITLE_ORDER_KEYS`, which lives beside the comparator that
actually implements them — never restated here. The previous hand-written
copy drifted (it advertised a `fewest-clips` key the comparator had replaced
with largest-physical-size), which made the self-diagnosing log explain the
pick with a rule the code does not apply. A diagnostic that disagrees with the
decision it documents is worse than no diagnostic.
+132
View File
@@ -0,0 +1,132 @@
# Rationale for `src/dirimage/tests.rs`
Longer-form notes for individual tests, referenced from `// See docs/dirimage-tests.md`
pointers in the source so the inline comments stay within the comment-guard caps.
## `Scratch::unique_path`
The path a scratch dir takes is kept separate from directory creation so its
uniqueness is testable without a syscall between draws. A monotonic counter,
NOT a timestamp, is what guarantees it: `SystemTime::now()` resolves to only
a MICROSECOND on macOS, so two of the seven parallel tests that share the
"bdmv" tag routinely read the same value within one microsecond, collide on
the same directory, and the first to finish `remove_dir_all`s it out from
under the others' reads — an intermittent `read_sectors` ENOENT. The counter
is unique regardless of clock granularity; pairing it with the pid keeps it
unique across test-binary processes.
## `scratch_paths_are_unique_even_at_clock_resolution`
Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
tag and run in parallel; if two land the same path, the first to drop
`remove_dir_all`s it out from under the other's reads, which surfaced as an
intermittent `read_sectors` failure. Regression: with the old
`SystemTime::now()` name this is RED, because macOS's clock advances only
per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
between them to nudge the clock, mirroring the real cross-thread collision —
hands back the same value many times over. The counter-based name is unique
regardless of clock granularity.
## `a_directory_spanning_many_blocks_reads_back_whole`
A directory with enough children that its FID list spans several 2048-byte
blocks, read back through the production parser.
This is the case the block-alignment question turns on: FIDs are packed
contiguously and a descriptor may straddle a block boundary, because
`read_directory` (`udf.rs:1312`) walks the directory extent as one flat byte
run and BREAKS at the first non-257 tag. Padding each block would truncate
the directory at the first pad. 200 entries also exceeds the 16-handle LRU
several times over, so it exercises handle eviction on the read path.
## `a_file_below_video_ts_is_placed_not_just_declared`
A file inside a SUBDIRECTORY of `VIDEO_TS` must still have its data placed.
Audit finding. The DVD branch places only the files directly inside
`VIDEO_TS`, because only those carry the IFO-relative constraints — and the
follow-up loop skipped that directory entirely, so anything one level deeper
got a File Entry declaring the file's real size with no extents behind it.
It appeared in the tree at full length and read back as nothing, at exit 0.
The identical folder under `BDMV/` was always placed correctly, which is
what made the gap easy to miss.
## `a_vob_offset_past_the_image_cap_is_refused`
A VOBS offset far past the content must be REFUSED, not honoured.
Audit finding. The planner honours a title set's declared VOBS offset
because honouring it is what makes a real backup readable, and nothing
bounded it: a regenerated `.BUP`, a rewritten IFO or a hand-assembled folder
naming a huge offset grew the image to wherever it pointed. A `u32` sector
count reaches roughly 8.8 TB, and writing that to an `iso://` destination
would fill a disk with zeros before anything noticed.
The companion (`an_oversized_vob_offset_leaves_a_gap_rather_than_failing`)
pins that a MODEST oversize is still honoured as a gap, so this cap refuses
only what it must.
## `playable_bdmv`
A BD folder that really enumerates a title, so the AACS content probe has
an extent to sample.
The `.m2ts` is built as real 192-byte BD source packets, because that is
what the probe judges: byte 0 carries the AACS Copy Permission Indicator in
its top two bits, and byte 4 is the MPEG-TS sync. `scrambled` sets the CPI
and withholds the sync — "flagged and not structurally clean", which is
exactly `aacs_unit_needs_decrypt`. An all-zero payload would prove nothing
either way: `is_clean_ts` skips zero payloads as padding.
## `both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory`
The two doors must AGREE. Same folder, same verdict, same extents.
Stated as a differential rather than a bare `is_ok()` so it cannot go
vacuous: if the fixture ever stops producing an AACS state, a one-sided
`Ok` assertion would still pass while guarding nothing, whereas "both doors
see the same title" is the invariant the shared function actually exists to
hold.
## `a_scrambled_folder_is_refused_through_the_dir_url_door_too`
The other verdict, through the same door: a folder whose content units are
really scrambled is refused with the TYPED code, not muxed into garbage.
This is the load-bearing half. `E_DIR_IMAGE_ENCRYPTED` is produced at
exactly one site in the crate (`session::apply_folder_encryption_verdict`),
reachable from here only through the `is_folder` argument — so this test
cannot pass if that argument is dropped, whatever else changes.
## `write_and_mount_externally`
Write a synthesized image to a real file and ask the OS to mount it.
This is the only check here that is not circular: `read_filesystem` shares
every assumption with the encoder, an operating system's UDF driver shares
none of them. Ignored by default because it shells out to `hdiutil` and
needs a host that can attach an image; run with
`cargo test -- --ignored write_and_mount_externally --nocapture`.
## `dvd_placement_invariant_on_a_real_folder`
Diagnostic (opt-in): verify the DVD placement invariant for every title set
in a REAL folder. `ifo.rs` derives a title's extents as
`file_start_lba(VTS_nn_0.IFO) + vtstt_vobs`, so that sum must land exactly on
`VTS_nn_1.VOB` or the mux reads the wrong sectors for that title.
Run with: `FMKV_DVD_FOLDER=/path/to/tree cargo test --lib
dvd_placement_invariant_on_a_real_folder -- --ignored --nocapture`
## `a_multi_clip_playlist_produces_feed_spans_that_tile_the_feed`
`Clip::feed_span` is the INPUT to the whole provenance feature — it is what
tells the muxer which clip a byte offset belongs to — and it is produced in
exactly one place, `disc/bluray.rs`. Every `SeamPlan` test synthesizes spans
by hand, so the consumer is thoroughly tested against fixtures written from
the same assumptions as the producer, and nothing checks the producer at all.
That is the shape of the original defect: the placement logic was tested and
correct, and the thing feeding it was wrong. `SeamPlan` only trusts spans
that TILE the feed contiguously from zero, so this asserts exactly that,
against the real scanner reading a real synthesized filesystem.
+185
View File
@@ -0,0 +1,185 @@
# `dirimage` — `dir://` as an image-level source
`dirimage` synthesizes a real, minimal, valid UDF 1.02 volume over a host
folder (a DVD `VIDEO_TS/` or Blu-ray `BDMV/` extracted by a MakeMKV-style
backup). Nothing is emulated:
* **Metadata sectors** (anchors, the volume descriptor sequences, the File
Set Descriptor, every File Entry, every directory's FID list) are encoded
into RAM by `encode` — a few MiB even for a large Blu-ray.
* **Data sectors** are not materialized at all. Each one maps to a byte
range of a real file, read on demand.
So `udf::read_filesystem` parses this image by exactly the same code path it
parses a real disc with, and every consumer above it is unchanged. The cost
is that a single-partition synthetic volume never exercises the UDF 2.50
Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
this module's tests do not cover that block and must not be read as if they
did.
## What this module deliberately does NOT do
* **3D / SSIF** — rejected up front (`Error::DirImageSsifUnsupported`). An
SSIF aliases the same sectors as its base and dependent `.m2ts`; the
planner allocates disjoint extents, so a 3D folder would produce silently
wrong output.
* **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
* **Encrypted folders** — a folder whose content is still AACS-scrambled is
rejected by the caller-side probe, not decrypted here.
## `layout.rs` — rationale detail
Full detail for the comments trimmed in `src/dirimage/layout.rs`, in file order.
**Module overview.** Two phases, kept apart because they fail for different
reasons: walk rejects what the image model cannot represent (3D SSIF, an
unrecognized tree, a case collision inside one directory); assign places
metadata first (File Set Descriptor, one File Entry per node, then the
directory FID lists), then file data. Data placement is the part with real
constraints. For a Blu-ray there are none — `.mpls`/`.clpi` address clips by
name, never by LBA — so files are packed sequentially. For a DVD the IFOs
record VOB positions as sector offsets from the IFO's own start, and `ifo.rs`
re-derives every title extent from them, so the placement must reproduce
those offsets exactly or the rip reads the wrong sectors (`place_video_ts`).
**`DATA_FLOOR`.** Well clear of the volume-space descriptors, and — the
load-bearing part — far from zero: `disc/bluray.rs:137` drops any clip extent
whose LBA is 0, so a file that landed at block 0 would vanish from the title
with no error.
**`MAX_AD_BYTES`.** The AD length field is 30 bits (the top two are the
ECMA-167 4/14.14.1.1 extent type), so 0x3FFF_FFFF is the arithmetic ceiling —
but a non-final extent must be a whole number of blocks, so the usable
ceiling is the largest multiple of 2048 below it. Files larger than this are
split across several ADs; `udf.rs` reads them back as a multi-extent file,
the same shape a dual-layer disc produces.
**`MAX_DEPTH`.** Matches `udf.rs`'s `MAX_DIR_DEPTH`: anything deeper would be
recorded but never descended into, so the files under it would be invisible
to every consumer. Refuse instead of silently dropping.
**`MAX_CS0_NAME_BYTES`.** The FID's name-length field is one byte, so 255 is
the ceiling; 254 leaves the encoder no way to produce a value that wraps to
zero.
**`MAX_SUBDIRS`.** A directory File Entry's link count is `u16` and counts
one per child directory plus one for its own entry in the parent, so the
last usable value is `u16::MAX - 1`. Lowered under `cfg(test)` ONLY so the
guard can actually be executed: building a folder with 65534 subdirectories
to reach the real cap is not a test anyone can run, so an earlier test
asserted arithmetic about the constant instead and would have passed with
the guard deleted. The production value is unchanged.
**Module-scope link-count assert.** Must live at module scope, not inside
`mod tests`: it previously sat inside `#[cfg(test)] mod tests` while also
carrying its own `#[cfg(not(test))]`, so it was compiled in NO configuration
and could never fire — exactly the dead gate the test above it was written
to replace.
**`MAX_IMAGE_SECTORS`.** A DVD title set records where its VOBS begins as an
offset in its own IFO, read verbatim out of a file in the folder. A
regenerated `.BUP`, a tool that rewrote an IFO, or a hand-assembled folder
can therefore name an offset far beyond the content — and the planner
honours it, because honouring it is what makes a real backup readable.
Without a ceiling the image grows to wherever that offset points: a `u32`
sector count reaches ~8.8 TB, and writing one to an `iso://` destination
would fill a disk with zeros before anything noticed. 128 GiB clears the
largest real medium (BD-100) with room to spare, so a genuine disc folder
never meets it.
**`MAX_META_BYTES`.** Every node costs a 2 KiB File Entry sector held for the
life of the image, so the entry cap alone permits ~205 MB of metadata for
content of no size at all — and the mux holds two of these at once while
probing. The module documents a budget of "a few MiB even for a large
Blu-ray"; this is what enforces it rather than merely asserting it.
**`is_excluded`.** Dot-files are host artefacts, never disc content: macOS
sprays `.DS_Store` and `._*` resource forks through any folder a Finder
window has touched, and including them would put files on the "disc" that
were never on the disc. `.partial` is freemkv's own in-flight extraction
suffix (see `disc/extract.rs`) — picking one up would mean planning an
extent over a file that is still being written.
**`read_head`.** Reads the first `n` bytes of an IFO so its placement
offsets can be resolved. Errors propagate: returning an empty buffer instead
would send every offset through `unwrap_or(0)`, recording NO placement
constraint, and a VOB placed without its constraint yields an image that
reads at the wrong offset with nothing reported. An IFO that cannot be read
is a folder that cannot be planned.
**`place_video_ts`.** DVD-Video records VOB positions INSIDE the IFOs, as
sector offsets from the IFO file's own first sector: `VIDEO_TS.IFO` + 0xC0
(`VMGM_VOBS`) → `VIDEO_TS.VOB`; `VTS_nn_0.IFO` + 0xC0 (`vtsm_vobs`) →
`VTS_nn_0.VOB` (the VTS menu); `VTS_nn_0.IFO` + 0xC4 (`vtstt_vobs`) →
`VTS_nn_1.VOB` (the title stream). `ifo.rs:554-556` re-derives every title
extent as `file_start_lba(IFO) + vtstt_vobs + cell.first_sector`, so only the
third of those is load-bearing for freemkv itself — but the other two are
load-bearing for DVD PLAYERS, which read the menus freemkv ignores. All
three are honoured. `VTS_nn_1.VOB … VTS_nn_9.VOB` are one logical stream
split at the 1 GB file-size limit, and the cell sector addresses run
continuously across the split, so they are placed back-to-back with no gap;
laying the group out in its canonical on-disc order gives exactly that for
free, so the constraint becomes a CHECK, not a search. The check can fail —
a regenerated `.BUP`, a tool that rewrote an IFO, a folder assembled by hand
— and when it does the required position lies below the end of the file
that must precede it. There is no placement that satisfies it, so it is a
typed error naming the file rather than a silent misplacement.
**`reject_ssif`.** `disc/bluray.rs:124-130` probes
`/BDMV/STREAM/SSIF/{clip}.ssif` and sets `is_3d` the moment one resolves —
unconditionally, before any capability check. On a real 3D disc the `.ssif`
and the base/dependent `.m2ts` files ALIAS the same sectors (the SSIF
interleave IS the two m2ts streams), which this planner cannot express: it
would allocate three disjoint copies, so the clip's extents and the m2ts's
extents would disagree and the rip would read the wrong bytes at exit 0.
Reject the folder instead.
**`plan` — capacity, and what it changes.** `total_sectors` becomes the
`Disc`'s `capacity_sectors` / `capacity_bytes`, and `capacity_bytes` is not
inert: `canonical_title_order`'s oversize gate (`disc/mod.rs:2039-2040`, body
`:2209-2210`) demotes any title whose `size_bytes` exceeds it, which decides
which title sorts first and therefore what `-t 1` selects. The capacity
reported here is **the synthesized image's own size** — what an ISO built
from this folder would report — and nothing else. It is NOT padded up to a
media tier (BD25/BD50/…): that was considered and does not work, since a
folder does not record the tier of the disc it came from, so a 22 GB folder
off a BD50 would pad to BD25 and diverge anyway, and the padding would
inflate any pre-sized output written from the image. The consequence, stated
plainly: **for a folder that is missing files the source disc had — a
selective MakeMKV backup being the normal case — the capacity is smaller
than the source disc's, the oversize gate is therefore stricter, and a
borderline "play-all" composite title that the ISO kept can be demoted here.
`-t 1` can select a different title from `dir://FOLDER` than from an
`iso://` of the same disc.** For a complete folder the two agree, because
the capacities agree. This is chosen over the alternatives because it is the
only one that is self-consistent: `dir://X` and an `iso://` built from `X`
describe the same image and must answer the same way.
**Tests.**
* `a_file_past_the_ad_ceiling_splits_on_a_block_boundary` — a file above the
30-bit AD ceiling must split into MULTIPLE descriptors, and every
non-final one must be a whole number of blocks, or the next extent's bytes
start mid-sector and the file reassembles wrong. Kills a mutant that emits
one oversized AD (whose length would collide with the extent-TYPE bits
`udf.rs:642` reads).
* `two_names_the_reader_cannot_tell_apart_are_refused` — audit finding: the
uniqueness check compared raw host names, while `parse_udf_name` trims
whitespace and drops code units `char::from_u32` rejects. So
" 00000.m2ts" and "00000.m2ts" were two distinct entries at plan time and
ONE name at read time; `find`/`read_file` take the first match, so a title
resolved to the wrong file's extents and muxed the wrong bytes at exit 0.
Calls `plan` on a real folder and fails if the round-trip key is reverted
to comparing host names.
* `an_over_long_name_is_refused_by_the_planner` — audit finding: the length
was narrowed with `as u8`, so a 255-byte ASCII name (POSIX NAME_MAX, legal
on ext4/APFS/NTFS) encoded to 256 bytes with the CS0 compression byte and
wrote a length of ZERO, making every later entry in that directory read
from the wrong offset. An earlier version of this test asserted arithmetic
about the constants and never called `plan`, so it would have passed with
the guard deleted.
* `the_subdir_cap_refuses_a_folder_with_too_many_subdirectories` — pins the
arithmetic relationship the guard relies on and executes the guard rather
than restating the constant: link count (child dirs + 1) is 16 bits, so
exceeding it wraps and the image lies about its own directory structure.
Creating 65,535 directories in a test is not reasonable, so this does NOT
exercise `walk`.
+462
View File
@@ -0,0 +1,462 @@
# src/disc/mod.rs — internal notes
Overflow rationale for internal (non-`pub`) items in `src/disc/mod.rs`,
relocated here by the comment-guard so the in-file comment can stay short.
Each section is pointed to by a one-line `// See docs/disc-mod.md — <topic>`
comment at the corresponding call site.
## aacs_dir_present
THE ONE definition of "is this disc structurally AACS-encrypted". Both
`Disc::identify` (fast path, filesystem only) and `Disc::scan_with` (full
scan) need this, and both used to spell out the same two `find_dir` calls by
hand. They agreed today; nothing made them agree tomorrow — adding a third
AACS location, or excluding an empty placeholder directory, to one copy and
not the other would silently desync the fast identify from the full scan (the
same disc reported encrypted by one and clear by the other).
This is STRUCTURAL, not cryptographic: it says the tree looks like an
encrypted disc, not that any sector actually is. A folder copied verbatim
from a decrypted disc keeps its `AACS/` and answers true here (see
`session::scan_dir`, which corrects for exactly that).
## correct_truehd_channels
Corrects a title's TrueHD audio-stream metadata by probing the first
decrypted access units — channel count, real sample rate, and Atmos detection
in a single major-sync read. The MPLS descriptors declare the BASE layout
(often 5.1, or a container-guessed rate) even for a 7.1/Atmos TrueHD track;
the truth is in the MLP major sync. `reader` must yield DECRYPTED sectors
(the m2ts is AACS-encrypted, so this can only run at mux time, not scan).
Reads a bounded window of the title's first extent.
Corrections, each individually guarded so a malformed field never writes a
wrong header:
- **Channels**: from the presentation channel masks (as before).
- **Sample rate**: from the whitelisted rate nibble; left untouched on an
unknown rate or no major sync.
- **Atmos**: when a 4th substream is detected AND the stream still carries
the basic descriptor label, the label is promoted to the Atmos form;
richer editorial labels (e.g. an existing "Dolby Atmos") are left intact.
## Resolution::pixels
`Unknown` deliberately does NOT fabricate a plausible 1920x1080. This is the
same trap already closed on `AudioChannels::count` (which used to return 6)
and `SampleRate::hz` (48000.0): a plausible wrong answer is indistinguishable
from a real one at every call site, so a caller can forget to check the
variant first — the `json://` sink had already walked into exactly that,
reporting confident dimensions for a stream whose neighbouring `resolution`
field said "unknown".
It also does not return `(0, 0)`: that shape looks like a usable pair, so the
MP4 sink stored it and serialised a 0x0 video track — a structurally complete
file no player can render, written with no error. Before that it returned a
fabricated 1920x1080, which was wrong but at least playable, so the sink had
never needed a guard and the absence of one was invisible. A doc comment
listing which callers are safe cannot hold this: one did not check, and a
third kept its own duplicate `Unknown` test long after the accessor took the
job over. `Option` forces every caller to make its OWN decision: Matroska
omits the optional PixelWidth/PixelHeight elements, the metadata sinks report
the dimensions as absent, and MP4 must refuse per ISO/IEC 14496-12 (width and
height are mandatory in both `tkhd` 8.3.2 and VisualSampleEntry 12.1.3).
## has_probable_video
The exposed `streams` come from the title's FIRST PlayItem only, so a
feature whose first PlayItem is a non-video bumper reports
`has_video() == false` and would be wrongly disqualified (audit R6). This
predicate is deliberately MORE permissive than `has_video` — it only ever
ADMITS a title the stricter gate would reject — so it cannot reopen the
decoy hole: a size-inflated streamless decoy is demoted STRUCTURALLY by
composite detection in `Disc::rank_titles`, not by this gate, while it still
rejects tiny streamless menu/metadata playlists. The size fallback requires
at least one clip, so a clip-less pure-metadata decoy (which cannot be a
real bumper-lead feature — that has the feature clip) is still rejected.
## image_crack_extents
The extents an image-time CSS crack scans, in the crate's CANONICAL order:
the main feature's own extents, in natural playback order. Both halves are
deferred to logic the crate already owns rather than re-derived here:
- WHICH title — `scan_with` has already sorted `titles` with
`Self::canonical_title_order`, so the canonical main feature is the first
title that actually has extents. No local pick.
- WHAT ORDER — playback order, i.e. the extent vector untouched, exactly as
`Self::decrypt_keys_for_title` hands `&title.extents` to
`css::crack_key_outcome`.
**Why this function exists (do not let the copy grow back):** `Disc::scan_image`
used to re-implement both halves inline — it picked the title with the
largest total sector count, then re-sorted that title's extents
LARGEST-CELL-FIRST before handing them to `crack_key_outcome`. Both had
drifted from the canonical rules:
- Largest-cell-first is the 1.5.1 garbage bug. A CSS DVD's biggest cell opens
with a long CLEAR run, and the crack's 50_000-sector budget is shared
across the whole extent list, so starting there can exhaust the budget
without ever reaching a scrambled sector — the crack reports `Unencrypted`
and the mux emits scrambled MPEG as plaintext. Playback order reaches the
scrambled feature body after only the small clear front matter that
precedes it on a real disc (CSS: the title key is recovered from the
scrambled data itself, so the scan must actually MEET scrambled data).
- The sector-count pick ignores `canonical_title_order`'s capacity gate, so
it selects the oversize "play-all" composite (whose declared cells
double-count data shared with other playlists) instead of the real feature
— i.e. a DIFFERENT title, and on a multi-VTS disc a different VTS, whose
CSS title key does not descramble the feature at all.
The result: the same disc cracked as an ISO could disagree with the same
disc cracked from the drive, which is precisely what the duplicate was free
to do. Keep the derivation here, shared, so it cannot recur.
## read_aacs_inputs_from_reader
Reads a disc's AACS key-input files from a sector source: returns
`(Unit_Key_RO.inf, MKB)` raw bytes. Shared body for `Disc::read_aacs_inputs`
(ISO) and `Disc::read_aacs_inputs_from_drive` (live drive). Prefers MKB_RO,
falls back to MKB_RW, then TRIMS to the real record length. Both files are
allocated to a fixed ~128 MiB and zero-padded, so reading either ships up to
~124 MiB of nothing — trim to the record stream so callers send/store a few
MB, not 128 MiB.
## read_aacs_version
AACS major version (`crate::aacs::mkb::AACS_MAJOR_BD` /
`crate::aacs::mkb::AACS_MAJOR_UHD`) from the content certificate. Drives the
`Unit_Key_RO.inf` parse stride (48-byte V10 vs 64-byte V20/V21), so the
out-of-band key-fetch path parses `enc_title_keys` at the right stride (a
server VUK then derives the correct unit keys).
When no content certificate is readable/parseable, defaults to **UHD (V20,
64-byte stride)** — the conservative choice the pre-1.2.0 fetch path
hardcoded — and logs it: a wrong stride here folds a server VUK against
mis-strided title keys (silent wrong unit keys), so a missing cert must not
quietly pick the V10 stride for a UHD disc.
## read_mkb_content
Reads the AACS MKB's real record stream — NOT its zero padding. `MKB_RO.inf`
/ `MKB_RW.inf` are allocated to a fixed ~128 MiB and zero-padded; the actual
record stream is a few MiB. Reads a bounded prefix, finds the record-stream
length via `crate::aacs::mkb::mkb_content_len` and returns exactly that,
growing the prefix if the records run past it. This avoids reading 100+ MiB
of padding on every scan AND avoids the `read_file` `MAX_FILE_BYTES` cap that
(since 0.31.0) rejected the padded 128 MiB MKB outright — which made
`read_aacs_inputs` fail and autorip report "could not read this disc's key
files" without ever contacting the keyserver.
## CANONICAL_TITLE_ORDER_KEYS
**Why not just sort by duration descending?** Branching UHDs (and some BD
authoring) ship a "play-all" virtual playlist that references the same
source clips multiple times for seamless alternate-angle / alternate-ending
playback. Those playlists report an inflated `duration_secs` (often 4+
hours) and an inflated `size_bytes` greater than the disc's physical
capacity. Example seen in the wild — *The Amateur (2025)* UHD, 58.5 GB
BD-100 disc:
| Title | Playlist | Duration | Size | Clips |
|-------|------------|----------|---------|-------|
| 1 | 00020.mpls | 4h 13m | 92.4 GB | 253 |
| 2 | 00800.mpls | 2h 02m | 57.2 GB | 1 |
Title 1's 92.4 GB cannot fit on a 58.5 GB disc unless the same clip data is
referenced multiple times — proof it's a virtual composite. A duration-only
sort would put it at `titles[0]`, so `freemkv -t 1`, `disc.titles.first()`,
and autorip's main-feature picker all grab the 4-hour composite instead of
the 2-hour movie that actually matches TMDB.
**Effect on non-branching discs:** unchanged — the main movie is already the
longest 1-clip title. **Effect on branching UHDs:** the virtual play-all
playlist is pushed to the back, the actual movie surfaces at index 0.
Defined immediately beside the comparator, so a diagnostic can NAME the
ordering instead of restating it. The `freemkv::diag` main-feature decision
row used to carry its own hand-written copy of this list, and it drifted: it
still advertised a `fewest-clips` key long after the comparator replaced
clip-count with largest-physical-size, so the `--log-level 3` bug-report log
explained freemkv's top-level pick with a rule freemkv does not apply. Any
change to the keys must change this list in the same edit.
## probe_forced_subtitles_for_bdts_titles
Pulled out of `scan_with` as its own callable predicate: the decision is
otherwise reachable only by driving the full scan pipeline (tree dispatch ->
title parsing -> this loop), so a test could not pin the gate without also
authoring an entire synthetic disc image. Running it against an HD-DVD/DVD
(`MpegPs`) title's extents would demux the wrong container, since only
`BdTs` titles carry PES-wrapped PGS the shared classifier
(`pgs_forced_probe`) understands.
## MAIN_FEATURE_ORDER_KEYS
- `nav-feature`: the title the disc's own HDMV navigation plays as the
feature, resolved by running First-Play in the `crate::bdnav` VM the way a
real player would. This is the authoritative, size-independent pick; it is
video-gated (a mis-resolve cannot select a streamless playlist) and simply
absent when the VM abstains (BD-J discs, or non-convergence), in which
case the lower keys decide.
- `authoring-feature`: a title the disc's own menu authoring designates as
the feature (has video, is not itself a composite, and is within a sane
fraction of the longest title's duration) outranks everything else — a
size-independent signal an obfuscation decoy cannot forge.
- `standalone`: a title that is a play-all / wrapper COMPOSITE — its clip
set properly contains another substantial video title's whole clip set (a
real UHD title's decoy playlist `00245` = `[bumper][00001 feature
clip][outro]` case) — is demoted below the standalone title it wraps. This
is the STRUCTURAL anti-decoy signal; it does not depend on stream counts,
physical size, or disc capacity, so it survives where those are spoofed or
unknown.
- `has-video`: a title with no plausible video content (see
`DiscTitle::has_probable_video`) is demoted below every title that has
some — a floor that drops streamless menu/metadata playlists.
## rank_titles
This catches both the wrapper decoy (`00245 ⊋ 00001`) and the oversize
play-all — independent of capacity, which the `fits-disc` gate needs and
which is `0`/unknown on drives that fail READ CAPACITY. The corpus
title-selection gate is the backstop across the disc hoard for the known
residual (a real feature with a separate "resume from the middle" branch
playlist reusing most of its clips).
## aligned_unit_keys_validate
Conservative: a sample that is not AACS-scrambled proves nothing, and with
no scrambled sample at all there is nothing to disprove against, so it
returns `true` (accept). It returns `false` when ANY scrambled sample cannot
be restored to clear MPEG-TS by ANY unit key in the set. That covers two
distinct failure shapes: (1) a wholly wrong key (a keydb VK that does not
match this disc) — no sample decrypts; and (2) a *partially* applicable key
set on a multi-CPS-unit disc — the resolved keys cover CPS unit 0 but not
CPS unit 1. Accepting on the first sample that decrypts (the old behaviour)
would commit such a set, after which CPS-unit-1 sectors pass through as raw
encrypted bytes into the ISO/MKV with no error surfaced anywhere. Requiring
every scrambled sample to decrypt rejects the incomplete set so the caller
falls through to the next candidate. Reuses the ecosystem's single
`is_clean` content-clarity predicate and the full (bus + AACS) unit decrypt,
so it agrees with the actual mux decrypt.
## ensure_decryptable
The verdict, in order:
- `raw == true` → `Ok(())`. `--raw` intentionally skips decryption and needs
no key (the caller wants an encrypted image).
- `self.css_error.is_some()` → `Err(Error::CssNoDiscKey)`. The scan saw
scrambled CSS sectors but recovered no title key (`self.css` is `None` yet
the content IS encrypted). Treating `css.is_none()` as "unencrypted" would
mux scrambled MPEG as plaintext garbage. A DISC-LEVEL verdict
(`error::is_disc_level_no_key`) — the main feature's crack failed, so
every title fails the same way and the rip loop must stop rather than
skip each title in turn.
- AACS-encrypted (`self.aacs.is_some()`) with no usable key
(`decrypt_keys()` is `None`) → `Err(Error::NoDiscKey { .. })`, naming the
disc by hash.
- CSS-encrypted (`self.css.is_some()`) with no usable key →
`Err(Error::CssKeyMissing)`. (The disc-wide `decrypt_keys()` yields
`Css{..}` whenever `css.is_some()`, so this is defensive; the live
multi-VTS case is gated by `Self::ensure_decryptable_keys`.)
- otherwise → `Ok(())`. A genuinely unencrypted disc has `None` keys
legitimately, and a CSS disc whose keyless crack succeeded has a key.
## decrypt_keys_for_title
For a DVD the CSS title key MUST be recovered before descrambling: a
scrambled sector without a recoverable crib cannot self-crack, and CSS
leaves the pack/PES header clear, so a sector left un-descrambled would mux
as a structurally-valid but corrupt PES packet with no loss reported. Two
ways to get it:
- **Fast path** — the scan already cracked a key whose LBA span
(`crate::css::CssState::crack_span`) covers this title's VTS: reuse it. No
re-read, and on a live drive no second CSS bus-auth round-trip. CSS title
keys are per-VTS, so an overlapping span is the same key.
- **Crack** — when up-front detection missed (`self.css == None`) or the
title lives in a different VTS: crack the key from this title's OWN
extents in a SINGLE scan, in natural PLAYBACK ORDER (never
largest-cell-first, which starved the crack in a big cell's clear prefix —
the 1.5.1 bug). Playback order reaches the scrambled feature body after
only the small clear front matter (logo / rating card) that precedes it.
One scan = one CSS-locked early-bail, so a locked title is not
re-hammered per cell against a live drive (hard rule #2); its 50k-sector
budget is the same accepted bound the disc-wide scan uses.
Outcomes (`batch_sectors` sizes the crack's batched reads):
- `(Css{title_key}, false)` — descramble with the recovered/reused key.
- `(None, true)` — a genuinely-clear title needs no key; the gate passes it.
- `(None, false)` — scrambled but no key recoverable → hard failure via
`Self::ensure_title_decryptable`, never a silent garbage mux.
## inject_unit_keys
Injects pre-resolved AACS unit keys into a scanned disc — the deferred-mux /
resume path. The keys come from the mapfile's `# freemkv-uk:` header
(persisted at sweep time when the disc was keyed), so the mux decrypts
directly with NO key-service round-trip. Populates `self.aacs.unit_keys` so
`Self::decrypt_keys` returns them and marks the source `ExternalUk`.
If the scan built no AACS state (`self.aacs == None`) — which happens when
the keydb was absent at scan time (`scan_aacs_no_keydb` ->
`aacs_error = KeydbLoad`) — this synthesizes a minimal `ExternalUk` state
for an encrypted AACS disc. A Unit Key is the FINAL per-title decryption
key; the keydb is only needed to *derive* it, and that derivation already
happened at sweep (the UK is in the mapfile). So a UK alone is sufficient to
decrypt the on-disk ISO — AACS 2.0 bus decryption was applied by the drive
at read time, so `read_data_key` is unused for file-backed mux. Without
this, a keyed disc swept without a keydb would recover its UK yet still
report E8005 (no usable `decrypt_keys`) at remux. No-op for an unencrypted
or CSS (DVD) disc.
## scan_with_passes_the_halt_flag_to_the_bluray_enumerator
Every BD and DVD cancellation test in this crate calls
`scan_bluray_titles` / `scan_dvd_titles` DIRECTLY, so the three-line wiring
in `scan_with` that connects the operator's Stop to them was covered by
nothing at all: replacing `opts.halt.as_ref()` with `None` on either branch
left the whole suite green while a Stop during a BD or DVD scan silently did
nothing. Only the HD-DVD branch was pinned.
The fixture is deliberately the smallest disc that takes the BD branch — a
bare `/BDMV` with no PLAYLIST — because the point under test is the
ARGUMENT, not the enumerator's own (separately tested) halt polling. Nothing
between `scan_with`'s entry and the enumerator reads the flag (the disc is
unencrypted, so the AACS path is skipped), so a green here can only mean the
flag arrived. Mutation: `Self::scan_bluray_titles(reader, &udf_fs, None)?`
fails here.
## halted_reads_do_not_report_the_hddvd_scan_as_successful
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The HD-DVD enumerator must not swallow that into a
successful scan. Measured before this was fixed: the scan returned `Ok`
with both titles present and ZERO streams on each — a cancelled scan
wearing the shape of a disc whose clips carry no video or audio. Downstream
that is a title list to cache, display and rip from.
## scan_with_capacity_bytes tests
`scan_with`'s `capacity_bytes = capacity as u64 * 2048` feeds
`canonical_title_order`'s "bigger than the whole disc = play-all composite"
threshold.
`..._not_addition`: chosen so `capacity * 2048` clears both titles (neither
is oversize, so the bigger one — OTHER — ranks first), but
`capacity + 2048` lands BETWEEN the two sizes: MAIN stays non-oversize while
OTHER flips to oversize and gets demoted, changing `titles[0]`. A `*` -> `+`
mutation is caught by this ranking flip, not merely a wrong numeric
threshold value.
`..._not_division`: same mechanism, tuned to catch a `*` -> `/` mutation
instead: `capacity * 2048` clears both titles, but `capacity / 2048`
collapses to a threshold far below MAIN's declared size while still leaving
MAIN under it, so only OTHER flips to oversize.
## read_mkb_content_grows_prefix_past_16mib_when_records_run_longer
This fixture's record stream is deliberately built so the first 16 records
land EXACTLY on the 16 MiB boundary (so the 16 MiB prefix alone parses as a
clean, complete record stream — the case a naive "n < buf.len() means done"
check would wrongly accept) and 4 more MiB of records follow, terminated by
the explicit `00 000000` end marker. Only a caller that actually grows past
16 MiB recovers the true 20 MiB content length.
## main_feature_order_demotes_wrapper_composite_decoy
The decoy `00245.mpls` is a WRAPPER COMPOSITE — `[00339 2s bumper][00001 the
whole feature clip][00336 outro]` — so its first-PlayItem STN reports v=0,
and it is the LONGEST and LARGEST title (8350s / 78.4 GB vs the feature's
8038s / 76.5 GB). Muxing its zero frames produced E6008 forever. It must be
demoted STRUCTURALLY (its clip set properly contains the feature's), not by
the fragile first-PlayItem stream count — this is what makes the fix robust
to the video-bearing-composite variant (see
`main_feature_order_demotes_video_composite`).
## key_source_failure_is_not_reported_as_a_missing_disc_key
Drives the real chain end to end — `KeySource` -> `resolve_and_apply_traced`
-> `Disc::aacs_error` -> `ensure_decryptable` — twice over the SAME disc,
and asserts the two operator-visible verdicts differ:
- source returns `Err(KeyServiceUnavailable)` (the HTTP-502 outage) → E7028
"the key service could not answer; retry later",
- source returns `Ok(vec![])` (the service answered, no entry) → E7022 "no
key source has a decryption key for this disc".
The credential and rate-limit verdicts are asserted alongside, since each is
a different operator action (fix the token / back off).
## css_disc_wide_no_key_is_disc_level_while_per_title_stays_skippable
The two CSS no-key conditions are NOT the same verdict and must classify
oppositely through the public predicates:
- **disc-wide** — `css_error` is set: the MAIN feature's crack failed, so
every title of this disc fails identically. Must be
`crate::error::is_disc_level_no_key` (the rip loop fail-fasts) and must
NOT be `crate::error::is_skippable_title_stub`. While both conditions
shared `E_CSS_KEY_MISSING`, an uncrackable CSS disc iterated all N titles
logging "title skipped" and exited 0 — a total failure reported as
success.
- **per-title** — one title's own re-crack failed on a multi-VTS disc
(`title_is_clear == false`, no key): skipping it and finishing the rest is
correct policy, so it must STAY skippable and must NOT be disc-level.
Pinned in both directions so a future change cannot silently flip either.
## correct_truehd_channels_full_correction_and_atmos_promotion
Kills the `was_basic ==`, the channels/rate `!=`/`&&` guards' "already
correct" branch, the `!matches!` per-stream skip, the `is_atmos ==
Some(true)` branch, and the whole-function no-op mutant.
## byte_offset_in_title_accumulates_across_extents
Kills: `lba >= start && lba < end` flipped to `||` (the first extent's
disjunction would trivially match almost any lba and return the wrong,
too-early offset); `cumulative +=` flipped to `*=` (cumulative is seeded at
0, so `*=` freezes it at 0 forever); and `sector_count * SECTOR_BYTES_U64`
flipped to `+` or `/` (wrong per-extent byte length folded into cumulative).
## chapter_at_offset_concrete_arithmetic
Kills: the `/` (time-fraction) flipped to `%` or `*`; the `*` (duration
scale) flipped to `+`; the `<=` chapter-scan comparison flipped to `>`; the
final `chapter_idx + 1` flipped to `-` or `*`; and every whole-function
fixed-tuple replacement (none produce `(2, 60.0)`).
## locate_ranges_positive_bps_duration_and_at_risk
Kills `bps > 0.0` flipped to `==`/`<` (both would take the `else 0.0`
branch here, wrongly reporting 0); `(*size as f64) / bps` flipped to `%` or
`*`; and the trailing `* MILLIS_PER_SEC` flipped to `+` or `/`.
## decrypt_keys_for_title_css_span_reuse_is_half_open
A CSS title key is per-VTS: reusing the scan's cracked key for a title that
lives OUTSIDE the cracked span descrambles that title with the wrong key,
and the mux emits garbage at exit 0. `crack_span` is documented as the
half-open LBA span `[start, end)`, so overlap is `extent.start < span.end &&
span.start < extent.end` — both comparisons STRICT. A title that merely
ABUTS the span (ends exactly where it begins, or begins exactly where it
ends) shares no sector with it and must NOT reuse the key.
The reader serves only clear (all-zero) sectors, so a title that falls
through to its own crack is reported unencrypted — distinguishable from the
reused-key answer both in the key and in the is-clear flag.
## merge_content_key_ranges
Merges per-title AACS key ranges into the sorted, disjoint set the
whole-disc map needs (`crate::decrypt::AacsKeyMap::entry_for` requires
disjoint ranges). Titles that share a clip resolve the SAME physical span
(same LBAs -> same CPS unit -> same key). When a later range overlaps a kept
one that carries the SAME key index and phase, the two are UNIONED (end
extended to the max) — this covers both the exact-duplicate (shared clip)
case and any partial overlap without ever dropping coverage, so no encrypted
LBA is left in no range (which would pass through as ciphertext). A real disc
never produces two DIFFERENT keys for one LBA; if that malformed case ever
appeared, the later range is dropped to keep the set disjoint rather than
extend one key over another key's LBAs.
+9
View File
@@ -0,0 +1,9 @@
# `disc::profile`
Everything in `DiscProfile` is DERIVED from the `Disc` model at construction:
the `Stream` enum is split into three typed vectors (audio/video/subtitle),
the `qualifier` / `purpose` enums are decomposed into booleans, and the
"default track" selection is precomputed so downstream never recomputes it.
Every field is always populated — a sensible default (`"und"` language, empty
`name`, `false` flag) stands in where the model has nothing, so consumers
never handle a bare `Option`.
+17
View File
@@ -0,0 +1,17 @@
# disc_tests: capacity/layer derivation
`scan_image_reports_capacity_in_bytes_and_the_layer_count` covers the scan
reporting the medium's size on both axes it exposes, derived from the one
sector count the caller hands in:
* `capacity_bytes` is that sector count times the 2048-byte logical sector
(ECMA-167 / BD-ROM logical block size). It is what sizes a full-disc image
read and what the progress percentage divides by, so a wrong scale is a
wrong ISO length, not a cosmetic number.
* `layers` distinguishes single- from dual-layer media. The threshold sits
between the two real capacities: a single-layer BD-25 is 12,219,392
sectors (25,025,314,816 bytes / 2048) and a dual-layer BD-50 is 24,438,784
sectors, so BD-25 must report 1 layer and BD-50 must report 2.
`scan_image` takes the sector count as a parameter, so this exercises the
real derivation without a 50 GB fixture.
+23 -15
View File
@@ -29,7 +29,6 @@ seam).
| `init()` | Route to the matching registered unlocker (if any), then prepare for reads |
| `probe_disc()` | Probe disc surface for optimal speeds |
| `read(lba, count, buf, recovery)` | Read sectors. Single-shot — no inline retries or reset. |
| `reset()` | Eject-cycle escape hatch. Caller-invoked only; not on the read path. |
| `lock_tray()` | Prevent tray ejection during rip |
| `unlock_tray()` | Allow tray ejection (also runs on Drop) |
| `eject()` | Eject disc tray |
@@ -104,14 +103,14 @@ descriptors or calls ioctls outside of a `ScsiTransport` implementation.
| Platform | Implementation | Device |
|----------|---------------|--------|
| Linux | `SgIoTransport` — async `write`/`poll`/`read` on `/dev/sg*` | `/dev/sg*` |
| Linux | `SgIoTransport` — synchronous `ioctl(fd, SG_IO, &hdr)` on `/dev/sg*` | `/dev/sg*` |
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
The Linux backend uses the sg driver's asynchronous interface: `write()` submits
the command, `poll()` waits with an enforceable wall-clock timeout, `read()`
retrieves the result. If `poll()` times out, the fd is abandoned (closed in a
background thread) and a fresh fd opened — the kernel's USB error recovery
cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
The Linux backend issues each command as one synchronous `ioctl(fd, SG_IO, &hdr)`
(`linux.rs`); the timeout is the kernel's, set via `hdr.timeout`, not a userspace
`poll()`. Opens with `O_RDWR | O_NONBLOCK`. (Pre-0.13.20 this backend ran an async
`write()` + `poll(1.5s)` + close-on-timeout + background-reopen pattern; it was
reverted in 0.13.20 — see the note at the top of `linux.rs`.)
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
The shim handles:
@@ -127,14 +126,13 @@ The shim handles:
On non-zero SCSI status, the transport parses sense key from the sense buffer
and returns `Error::ScsiError`.
`SgIoTransport::reset` (Linux) does pure userspace state cleanup: an open +
close pair to make the kernel cancel any SG_IO commands queued against a
previous fd, a 2 s sleep to let the kernel finish that cancellation, then a
fresh fd to send ALLOW MEDIUM REMOVAL to clear any stale tray lock. It does
NOT issue `SG_SCSI_RESET` or escalate via STOP+START UNIT. Both were tried
in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA bridge); both failed
to recover wedged drives and made the wedge worse. The macOS reset (which
had been a no-op) was removed entirely in 0.13.6, and the top-level
The Linux backend has no `reset`. `SG_SCSI_RESET` and STOP+START UNIT
escalation were tried in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA
bridge); both failed to recover wedged drives and made the wedge worse, so no
Linux transport-reset path exists today — recovery is just a fresh fd opened on
the next command. The only `reset` on a transport in the crate is
`SptiTransport::reset` on Windows (`windows.rs`). The macOS reset (which had
been a no-op) was removed entirely in 0.13.6, and the top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
removed at the same time (no callers).
@@ -241,3 +239,13 @@ Available speeds:
| Blu-ray | 1x (4,500 KB/s) through 12x (54,000 KB/s) |
| DVD | 1x (1,385 KB/s) through 16x (22,160 KB/s) |
| Max | 0xFFFF (drive decides) |
---
## Windows Path Normalization
`normalize_path` in `src/drive/windows.rs` converts a device path ("D:",
"D:\\", "\\.\D:", "\\.\CdRom0") to Windows `\\.\X:` form. A near-identical
`normalize_device_path` exists in `scsi::windows`; both are kept because
they live in separate `cfg(windows)` modules that cannot easily share a
helper without introducing cross-module coupling.
+15
View File
@@ -0,0 +1,15 @@
# Linux `sg*` device enumeration
`enumerate_sg_names` (in `src/drive/linux.rs`) lists `/dev/sgN` SCSI-generic
device names to probe for optical drives.
Linux assigns `/dev/sgN` sequentially across *all* SCSI-generic devices
(disks, tape, HBAs, optical), not just optical drives. That means a fixed
`sg0..15` range can miss an optical drive on a host with many other SCSI
targets ahead of it in enumeration order.
To avoid that, the function prefers the exact present-device list from
`/sys/class/scsi_generic/`, which enumerates only the `sg*` nodes that
actually exist. It falls back to a bounded `sg0..15` probe only when sysfs
itself is unreadable, which happens in some minimal containers that lack a
mounted `/sys`.
+142
View File
@@ -0,0 +1,142 @@
# src/drive/mod.rs — internal notes
Long-form rationale relocated from `src/drive/mod.rs` comments that exceeded
the comment-guard's internal-comment cap. Each section is pointed to by a
short `// See docs/drive-mod.md — <topic>` comment at its original site.
## Error-recovery mode page (PER bit)
`MODE_PAGE_ERROR_RECOVERY` (0x01) is the SBC/MMC Read-Write Error Recovery
mode page. We flip the `PER` bit to make the drive REPORT a recovered read
(via CHECK CONDITION + sense key RECOVERED ERROR) instead of silently
returning best-effort data as GOOD status. On marginal/dirty media that
silent-GOOD data can be mis-corrected — a rip that "passed clean" but decoded
with errors. With PER on, freemkv sees the marginal read and re-reads it in
Pass N (a loud miss, never a silent commit). See
`build_error_recovery_select_payload`.
## `sleep_until_halted` — why it exists and why it was dormant
This was `#[cfg(test)]` for two releases, kept alive only by its unit tests,
while the two production paths that actually sleep — the `wait_ready` poll
backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
`std::thread::sleep` and so were deaf to the operator's Stop for ~30 s and
~15 s respectively. Every other drive path returns `Halted` at the next
`checked_exec` boundary; a Stop pressed during spin-up simply did nothing
visible until the poll ran out. The primitive existed; the sleeping code just
did not call it. It is production code again.
## `Drive::read` — single-shot contract and the removed inline retries
`recovery=true` uses `crate::scsi::READ_RECOVERY_TIMEOUT_MS` (60 s, matches
sg_dd) for the `freemkv_engine::recovery::patch` pass; `recovery=false` uses
`crate::scsi::READ_TIMEOUT_MS` (10 s) for `freemkv_engine::recovery::copy`'s
fast skip-forward sweep. Both budgets are generous enough that the drive can
finish ECC recovery on a marginal sector — pre-0.13.21 this was 1.5 s on the
fast path which forced the kernel mid-layer to time out and escalate while we
waited anyway. On any failure this returns `Err(DiscRead)` immediately;
orchestration (`freemkv_engine::recovery::patch` multi-pass, `DiscStream`
adaptive batch halving) handles retry policy.
Inline retry phases (5x gentle + reset+reopen + 5x more) were removed in
0.13.6: on some USB-SATA bridges the inline reset wedged drive firmware
without ever recovering a sector. The remaining recovery layers
(`freemkv_engine::recovery::patch` multi-pass, `DiscStream` batch halving) do
not touch the wedge-prone reset path.
## `find_drive` — media-preference selection policy
On a multi-drive system (common on Windows, where an empty/not-ready drive
can enumerate first) returning the first drive blindly can pick a drive with
no disc, dooming the operation. So `find_drive` opens each candidate in
enumeration order, queries `Drive::drive_status` (GET EVENT STATUS, which
works regardless of firmware state), and returns the first drive reporting
`DriveStatus::DiscPresent`. If no drive reports a disc — or `drive_status()`
is unavailable/returns `Unknown` everywhere (single-drive or quirky bridges)
— it falls back to the first drive that opened, preserving historical
behavior so those setups don't regress. For just listing drives without
opening (e.g. UI sidebar), use `scsi::list_drives()` instead — it returns
`DriveInfo` (path + identity) without the cost of running every drive's
profile + identity probe.
## `open_block_device_for_sg` — why no `O_DIRECT`
Resolves a `/dev/sg*` path to the corresponding `/dev/sr*` block device by
walking sysfs, then opens it for read. No `O_DIRECT`:
`posix_fadvise(POSIX_FADV_DONTNEED)` flushes the cache before each pread,
which avoids buffer-alignment requirements while still forcing fresh device
reads. Returns `None` on any error (sysfs not present, no matching block
device, open failed); callers treat that as "no fallback available" and
propagate the original SCSI READ error.
## `resolve_device` — staged, not yet wired
The cross-platform dispatch is kept ready for the caller that will consume
it, so the per-platform implementations below it (and their tests) stay
live. `allow(dead_code)` marks that deliberately — this is not an accidental
orphan.
## Test rationale: recovery/halt/chunking coverage
The `halt_tests` and `command_tests` modules exercise mutation-sensitive
edges (halt-aware sleep timing, chunked-read offset arithmetic, drive-status
byte decoding, spin-cycle/wait-ready halt responsiveness). The following
notes preserve the "why this exact scenario" reasoning that the inline test
doc comments were trimmed from:
- **`disc_is_dvd_matches_only_dvd_profile_family`**: must match the DVD
profile family (0x0010..=0x001F) and ONLY that family. A false positive on
a BD/UHD profile (0x0040+) would skip the drive unlock UHD reads require; a
false negative on a DVD would re-introduce the CSS read failure. Widening
the range to `..=0x0040` makes the BD-ROM assert fire; a failed/short GET
CONFIGURATION must default to NOT-DVD so the unlock still runs.
- **`media_event_reply` / media-status decoding**: MMC-6 §6.7 — byte 5 of the
GET EVENT STATUS reply is a Media Status only when the Event Header says a
media event descriptor follows (NEA clear AND Notification Class == 4).
Decoding byte 5 regardless of those preconditions reads a reserved/zero
byte as Media Status 0 and reports NoDisc on a drive that has a disc loaded
— the classic "works on my drive, not theirs" firmware split. The drive is
untrusted input, so an event-less reply must fall back to the TEST UNIT
READY status instead.
- **Chunked-read tests** (`read_chunks_write_into_correctly_offset_buffer_regions`,
`an_undersized_buffer_errors_on_the_chunked_path_just_like_the_single_one`,
`chunk_lba_near_u32_max_errors_not_overflows`): the multi-chunk path slices
the caller's buffer by `count * 2048` and advances the per-chunk LBA with
`lba + done`, both unchecked. An undersized buffer PANICKED
("range end index out of range") out of the public `read`/`read_fua`
instead of returning `Err(DiscRead)` like the single-chunk path; an LBA
range crossing `u32::MAX` overflowed (debug panic, or release wrap to a low
LBA silently read). These tests pin both to error-not-panic/overflow, and
one writes a distinct marker per chunk to confirm the byte-offset
arithmetic (`done * 2048`, not `done + 2048`) is correct — no other mock
touches the buffer contents, so a `*` -> `+`/`/` mutation would otherwise go
unasserted.
- **`wait_ready`/`spin_cycle` halt tests**: `wait_ready` was the one drive
path that called `self.scsi.as_mut().execute(..)` instead of
`checked_exec`, and its 60 x 500 ms loop never read `self.halt` — a cancel
during spin-up was ignored for ~30 s. `spin_cycle` had the same gap across
two bare `execute` calls and two plain `std::thread::sleep` pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`), and it runs from the
recovery path — precisely when an operator is most likely to press Stop.
The tests pin: a Stop set before the call returns `Halted` immediately with
no command issued; a Stop set mid-poll (on a deterministic Nth call, no
wall-clock race) exits at the next `checked_exec` boundary; a Stop that
lands during the spin-down pause wakes the halt-aware sleep within
~100 ms instead of blocking for the full 5 s.
- **`read_logs_a_good_status_short_transfer`**: a READ(10) that completes
with GOOD status but a residual underrun is correctly refused, but the
refusal used to log nothing — so a drive that residual-underruns on GOOD
status produced the same silent journal as a scratched disc, and the
operator was sent after the wrong remedy (replace the drive vs. clean the
disc). The test pins that the refusal now warns with lba/transferred/
expected/code.
## `open_block_device_for_sg` doc contract (kept short in code)
See the section above for the `O_DIRECT` rationale; the pointer comment near
the function keeps only the API contract.
+69
View File
@@ -0,0 +1,69 @@
# `dropgate` — audio drop bookkeeping rationale
## Keep/drop rule
A clean mux keeps every frame it can and drops the ones it can't — video
always survives (it's inter-frame predicted; a per-frame drop would cascade,
so video resyncs/conceals instead), audio keeps every decodable access unit,
and a damaged audio AU is dropped rather than shipped as a decoder-choking
glitch.
## Detection is per-codec
The DETECTION is inherently per-codec — each format carries its own
authoritative corruption check (DTS: the core sync/header parse per ETSI TS
102 114; AC-3: the header CRC per ETSI TS 102 366; FLAC: the frame CRC-16;
…). `DropTally` only carries the UNIFORM response so every audio parser
behaves identically:
1. **Count** kept vs dropped AUs and the dropped duration.
2. **Log** every drop (fail-loud, never silent) — a per-drop trace plus a
once-per-track aggregate at `warn` so it surfaces without debug logging.
3. **Whole-track fallback**: once a track is judged mostly undecodable, latch
a poison flag so the remainder is dropped too (a track that damaged isn't
worth muxing).
## Sync preservation
**Sync preservation is the caller's responsibility**, not this type's: the
parser must advance its PTS clock across a dropped AU exactly as it would for
an emitted one, so a drop becomes a silence gap and never a shift of the
following audio. See `DtsParser`'s `stamp_pts` call ordering for the pattern.
## `verified_dropped` vs `dropped`
`verified_dropped` counts AUs dropped because they were INDIVIDUALLY verified
undecodable (a failed CRC/header/parity check); only these feed the
whole-track poison verdict. `dropped` also counts *collateral* drops — AUs
discarded as a consequence of one corruption (TrueHD's resync-forward run, or
a poisoned track) — which must NOT amplify a few real errors into a false
whole-track loss.
## `record_collateral_drop`
A collateral drop is caused by another AU's corruption (TrueHD's
resync-forward run to the next major sync, or an already-poisoned track), not
by being individually verified undecodable. It is counted and logged for the
drop report, but deliberately does NOT feed the poison verdict, so one
corruption event can't amplify into a false whole-track loss.
## `maybe_poison`
Whole-track fallback: after enough AUs to judge, if more than half were
dropped the track is too damaged to be worth muxing — latch `poisoned` and
log it loudly once. The minimum-sample gate keeps a short damaged burst from
poisoning an otherwise-good track. Judge on VERIFIED drops vs all AUs seen: a
track is only poisoned when a majority of its access units are individually
undecodable — not when a couple of corruption events forced long collateral
resync runs.
## Test: `interleaved_keeps_are_in_the_poison_denominator`
The poison verdict is a RATIO — verified drops against every AU seen — so
the kept count is half of it. `does_not_poison_a_mostly_good_track` records
its keeps AFTER the single drop, and `maybe_poison` only runs inside
`record_drop`, so the keeps are never in the denominator when the verdict is
actually computed: that test passes even with the kept count never
incremented. Interleaving them puts the kept count on the critical path,
where losing it turns the ratio into "verified drops vs verified drops" —
always >50% — and silently discards a healthy track.
+309
View File
@@ -0,0 +1,309 @@
# DTS / DTS-HD elementary stream parser (`src/mux/codec/dts.rs`)
Internal design notes moved out of source comments (see `ci/comment-guard.py`
prose caps). Each section is pointed to by a short `//` comment at the
corresponding site in `dts.rs`.
## `DTS_HD_EXT_SYNC`
An access unit is delimited by the next CORE sync; the parser locates and
exactly sizes each extension substream (via `exss_frame_size`) so a false
core sync inside the EXSS payload can't split the AU and truncate the
lossless extension.
## `emit_or_drop`
Gates an assembled access unit through the decodability check and either
pushes it or drops it. `au_pts`/`dur_ns` are already stamped on the shared
PTS clock (which the caller advances whether or not the AU survives), so a
drop leaves the following audio on its true timeline — a gap, not a shift.
Every drop is logged (fail-loud, never silent).
## `stamp_pts`
`front` is the AU's own core-PES PTS (from `front_pts`); `dur_ns` its
decoded duration.
The model matches the (correct) AC-3 path: **re-base to each PES's own
container timestamp, and advance by one frame duration ONLY within a run of
AUs that share the same PES.** A new PES (its `front` differs from the
previous AU's) trusts its own timestamp — so the emitted timeline tracks the
container and never drifts. Advancing within a PES is what fixes the DVD
case (several DTS core frames packed in one PES, which otherwise all
collided on that single PES timestamp → "non monotonically increasing dts").
A global running clock was WRONG here: once accumulated frame durations
exceeded the PES spacing it never re-based, drifting the track minutes past
the real length over a feature-long title.
## `CORE_HEADER_MIN_BYTES`
Number of leading bytes that must be buffered before the core `fsize` field
(bytes 5-7) can be decoded. This is a HEADER-LAYOUT minimum — "enough bytes
to read the size field" — and is deliberately distinct from
`MIN_CORE_FRAME_BYTES` (the decoded-frame-size validity floor). They must
not be conflated: this one gates buffer reads of the header, the other
rejects implausible decoded sizes.
## `MIN_CORE_FRAME_BYTES`
Minimum plausible decoded DTS core frame size, per ETSI TS 102 114: the
on-wire FSIZE floor is 95, so a real core frame is at least 96 bytes. A
decoded `core_size` below this means we matched a false/corrupt core sync
(a lucky 0x7FFE8001 in extension-substream payload whose 14-bit `fsize`
decoded to a tiny value) rather than a real frame, so we resync instead of
closing an access unit at a junk boundary and dropping the DTS-HD extension
tail.
## `PTS_UNSET`
Sentinel for "no valid PTS base captured yet". Real PTS-in-ns values are
non-negative (derived from the unsigned 90 kHz PES timestamp), so a
negative value can never collide with a genuine timestamp. Used to mark the
PTS base invalid after a forced flush so the next PES sets it regardless of
buffer state.
## `flush_tail`
Emits the final buffered access unit (the last core + its extension
substreams, which had no following core sync to close it during
streaming), gated through the decodability check. Requires a complete core
frame; drops a bare partial sync tail.
## `SYNCWORD_BYTES`
Byte length shared by the DTS core sync and the DTS-HD EXSS sync (both
32-bit words). `next_core_boundary` finds the next *valid* core sync after
the current core frame, to delimit the access unit: extension-substream
payload can contain byte sequences that match the core syncword, so each
candidate is validated by decoding its core size — a match whose decoded
size is implausible (`< MIN_CORE_FRAME_BYTES` or `> MAX_AU_BYTES`) is a
false sync and is skipped, continuing the search.
## EXSS header field bit widths
`EXSS_USER_DEFINED_BITS`, `EXSS_INDEX_BITS`, `EXSS_HEADER_SIZE_TYPE_BITS`,
etc. are the DTS-HD extension-substream (EXSS) header field bit widths
(ETSI TS 102 114, ExtSS header). `bHeaderSizeType` selects the short form
(`nuExtSSHeaderSize` 8 bits, `nuExtSSFsize` 16 bits) or, for larger
substreams, the long form (12 / 20 bits).
## `exss_frame_size`
DTS-HD extension substream (EXSS) total byte size — INCLUDING the
`0x64582025` syncword — read precisely from its header. `buf` must begin
with `DTS_HD_EXT_SYNC`. Returns `None` when the size fields aren't fully
buffered.
`nuExtSSFsize` is the total frame size in bytes minus one. Parsing it lets
the AU framer skip the extension by its exact length instead of scanning
its (arbitrary) payload for a core sync.
## `next_core_boundary`
Offset where the current access unit ends (the start of the next core
frame). The AU is the core frame plus its trailing DTS-HD extension
substreams, which are skipped PRECISELY by their declared size — so a
chance core syncword inside the XLL lossless payload can never be mistaken
for the next AU boundary (the bug that truncated the extension and produced
the "Failed to decode block code(s)" class). Falls back to the heuristic
core-sync scan only when an extension can't be sized (malformed / truncated
input).
## `scan_for_next_core`
Heuristic fallback (the pre-fix behaviour): scan forward for the next core
syncword whose decoded size is plausible. Used only when precise extension
skipping can't proceed; a chance core syncword in extension payload usually
decodes to an implausible size and is skipped.
## `dts_core_frame_size`
`fsize` is the 14-bit field at bits 46-59 of the header (bytes 5-7). On the
wire `fsize` is the frame length minus one, so this returns `fsize + 1`,
i.e. the core frame length in bytes (range 1..=16384). Callers treat the
result as the actual byte length and the MIN..=MAX range checks assume so.
Returns `0` when `data` is shorter than `CORE_HEADER_MIN_BYTES` — every call
site rejects that via the minimum-frame lower bound, so a `0` is never
mistaken for a valid tiny frame.
## `dts_core_samples`
Samples in one DTS core frame: `(NBLKS + 1) * 32`. `NBLKS` (7 bits) is the
core-header PCM-sample-block count (ETSI TS 102 114) that fixes the frame's
decoded sample count. Bit layout after the 32-bit sync: FTYPE(1) SHORT(5)
CPF(1) **NBLKS(7)** FSIZE(14) …, so NBLKS = byte4 bit0 + byte5 bits7-2.
## `DTS_PCMBLOCK_SAMPLES` / `DTS_SUBBAND_SAMPLES` (core-header validity)
DTS core-header validity constants (ETSI TS 102 114). For a NORMAL frame
`deficit_samples` must equal `DTS_PCMBLOCK_SAMPLES` — a termination frame
may carry fewer; `npcmblocks` must be a multiple of `DTS_SUBBAND_SAMPLES`;
`audio_mode` must be below `DTS_AMODE_COUNT`; `lfe_present ==
DTS_LFE_FLAG_INVALID` is rejected.
## `DTS_AMODE_COUNT`
Number of LEGAL `AMODE` (channel-arrangement) codes. The 6-bit AMODE field
(ETSI TS 102 114 §5.3.1) has 16 defined channel arrangements, codes 0-15;
only 16-63 are reserved/user-defined and undecodable. The per-AMODE channel
counts in ETSI TS 102 114 §5.3.1 cover all 16, confirming they are
decodable — codes 10-15 are the 6/7/8-channel layouts. A frame is dropped
only when `audio_mode >= DTS_AMODE_COUNT` (i.e. a truly reserved 16-63
code); dropping a legal 10-15 multichannel core would silence recoverable
audio.
## `DTS_CORE_SR_VALID`
Sample rate (Hz) per core `SFREQ` code (ETSI TS 102 114 Table 6-4); a `0`
entry marks a reserved code that fails header validation as an invalid
sample rate. Valid entries are locked to the spec by
`dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
{0, 4, 5, 9, 10}.
## `core_header_drop_reason`
Decodability gate: the core-frame header validity checks from ETSI TS 102
114. Returns `Some(reason)` when the DTS core-frame header is invalid — in
which case the packet is undecodable ("Invalid data found") and dropping it
loses nothing a decoder could have used. Returns `None` (keep) for a
decodable header OR if the header can't be fully read (never false-drop on
our own buffer underrun; the framer only emits AUs whose core is fully
buffered and ≥ 96 bytes).
The 4-byte core sync is already validated by the framer, so this reads the
header fields that follow it. The 16-bit CPF header CRC is not verified (we
skip past it) and the audio-header/side-info CRCs are likewise not checked,
because decoders treat those bytes as optional/ignored — verifying them
would drop frames that decode fine (false positives).
## Test: `stamp_pts_reuses_front_when_no_running_cursor_yet`
`stamp_pts`'s third arm (`else if front != PTS_UNSET`) is only reached once
the first arm's `front != self.last_front_pts` has already failed (i.e.
`front == self.last_front_pts`) and no within-PES running cursor is
available yet (`next_pts_ns == PTS_UNSET`). In that state the AU must still
be stamped with its own (repeated) front timestamp, not silently collapsed
to 0 — a `!=` -> `==` typo on the guard would only take this arm when
`front` IS the unset sentinel, producing 0 instead here and the sentinel
value itself if front really were unset.
## Test: `drain_front_collapses_offset_zero_markers_instead_of_leaking`
Every drained access unit that shares its PES with the previous one pushes
a new `(0, pts)` marker once rebased; `drain_front` must collapse those down
to the single most-recent one each time, or `pts_marks` grows once per
access unit for the life of the track — an unbounded allocation on a
multi-hour disc. `front_pts()` alone can't observe this: it always finds the
last offset-0 entry regardless of how many duplicates precede it, so the
leak is invisible unless something checks that the collapse actually ran.
## Test: `make_exss`
Builds a real DTS-HD EXSS substream of `total` bytes (short header form),
with an optional false DTS core syncword embedded in its payload (decoding
to a plausible core size) — to prove precise sizing, not a payload scan,
bounds the extension.
## Test: `make_dts_ext`
Builds a minimal DTS-HD extension substream of `size` bytes (just the sync
+ zero-padding). The parser delimits extensions by the next CORE sync, not
by the extension's own size header, so a valid header isn't required — only
that the bytes carry no spurious core sync.
## Test: `bogus_tiny_core_sync`
Builds 4 bytes that look like a DTS core sync but whose `fsize` field
decodes to a tiny `core_size` (< MIN_CORE_FRAME_BYTES). With the dead-code
guards this passed validation and could close an access unit at a junk
boundary; with the fix it must be drained and resynced past.
## Test: `next_core_boundary_exact_syncword_length_is_not_need_more`
Only the 4-byte EXSS syncword is buffered after the core frame — no header
fields at all. The guard `buf.len() < pos + SYNCWORD_BYTES` must be strict
`<`: at exactly `pos + SYNCWORD_BYTES` the sync bytes ARE fully present, so
the function proceeds to identify them as an extension sync (then fails to
size the header and falls back, ending in `NextCore::None` here since
nothing after it looks like a core sync either). A `<=` typo would instead
return `NeedMore` at this exact length without ever inspecting what the 4
buffered bytes are.
## Test: `make_bad_dts_core`
Builds a structurally-framed but UNDECODABLE core: a valid `make_dts_core`
whose LFE flag is set to the reserved value 3 (`DTS_LFE_FLAG_INVALID`). It
still sizes and syncs correctly (so the framer delimits it normally), but
the core-frame header validity check rejects it as an invalid LFE flag
(ETSI TS 102 114 §5.3.1: an LFF value of 3 is invalid). LFE is byte10
bits2-1, and does NOT feed the frame duration (NBLKS + SFREQ only), so a
dropped bad core still carries the same `DTS_CORE_DUR_NS` as its good peers.
## Test: `reparse_real_dts_file`
Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
through `DtsParser` and writes the emitted access units back out, so the
garbage-extension → core-only drop can be validated against an actual
damaged stream (e.g. an extracted DTS-HD MA track) end-to-end with an
external DTS decoder. Env: `DTS_IN` (input), `DTS_OUT` (output).
cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture
## Test: `find_sync_matches_a_buffer_that_is_exactly_the_syncword`
`find_sync` scans `0..=len-4`, so a buffer that is EXACTLY the syncword
still matches. The existing tests cover 0, 3 and "longer than 4", which
leaves the `len == 4` boundary open — and that is the case a syncword split
across PES packets lands on the moment its last byte arrives.
## Test: `a_core_header_of_exactly_the_minimum_length_is_decoded_not_deferred`
`CORE_HEADER_MIN_BYTES` is "enough bytes to DECODE the size field", so a
buffer holding exactly that many must be decoded, not deferred. The
distinction is visible precisely at the boundary: with 10 bytes of a core
sync whose decoded size is sub-spec, the parser must recognise the false
sync, drain past it and resync down to the 3-byte carry-over tail. Waiting
instead leaves the false sync sitting at the front of the buffer, where it
blocks every later real core behind it.
## Test: `flush_emits_a_core_that_exactly_fills_the_buffer`
The end-of-stream flush must emit a final access unit whose core is exactly
as long as the buffer — the ordinary case, since the last AU is closed by
end-of-stream rather than by a following core sync. Rejecting it at the
boundary silently drops the last frame of every DTS track.
## Test: `flush_discards_a_long_buffer_that_does_not_begin_with_a_core_sync`
The flush guard is a DISJUNCTION: a buffer that does not BEGIN with a core
sync is discarded whatever its length. Requiring both conditions instead
lets a long run of junk through to `dts_core_frame_size`, which happily
decodes a 14-bit size out of arbitrary bytes — and the flush then emits an
"access unit" that is not DTS at all.
The junk here is sized so that mis-decoding it yields a plausible core size
that the buffer fully covers, which is exactly when the wrong answer looks
like a right one.
## Test: `exss_frame_size_needs_the_worst_case_header_and_no_more`
`EXSS_HEADER_MIN_BYTES` is the WORST-CASE header length — the long form's
43 bits after the syncword, rounded up to 6 bytes, plus the 4-byte sync. It
gates whether an extension substream can be sized precisely, and that is
what keeps a chance core syncword inside XLL payload from being mistaken
for the next AU boundary; too large and every extension falls back to the
payload scan, too small and the reader runs off a truncated header.
Pinned at both sides of the boundary rather than by value, so the field
widths it is summed from stay honest.
## Test: `an_access_unit_carries_the_source_of_the_packet_its_core_arrived_in`
The whole point of the shared buffer: an access unit whose core arrived in
an EARLIER packet keeps that packet's source offset, not the offset of
whichever packet completed it. At a clip boundary the two belong to
different clips, and taking the later one puts the unit in the wrong clip —
which is what left nine audio and subtitle tracks unplaceable.
</content>
</invoke>
+149
View File
@@ -0,0 +1,149 @@
# DVD AC-3 physical sub-stream probing
## Why this exists (a real-disc wrong-substream bug)
A DVD VTS IFO declares its audio streams in a fixed table, and freemkv's
scan assigns each declared stream a `private_stream_1` sub-stream id purely
by per-codec ordinal — the first AC-3 stream becomes `0x80`, the second
`0x81`, and so on (`ifo::assign_audio_sub_stream_ids`). That assumes the
physical sub-stream order on the wire matches the IFO declaration order.
On some discs it does NOT. A real R2 PAL DVD release's feature declares
ONE AC-3 audio stream the IFO nibble marks as 5.1 (6 channels), but the
physical VOB carries the 5.1 main mix and a 2.0 down-mix on DIFFERENT
`0x8x` sub-stream ids, and the 2.0 is the one that happens to land at the
ordinal `0x80` slot. Routing the declared 5.1 stream to `0x80` by ordinal
therefore muxes the 2.0 down-mix while labelling it 5.1 — the wrong
physical track.
The robust fix is data-driven and codec/disc agnostic: read each physical
AC-3 sub-stream's REAL channel count from the VOB (the `acmod`/`lfeon` of
its first frame after the `0x0B77` sync) and route each IFO-declared AC-3
stream to the physical sub-stream whose actual channel count matches the
IFO's declared count — instead of trusting the ordinal. This never
re-reads the disc beyond a bounded head-of-feature probe and degrades to
the original ordinal mapping when the probe yields nothing
(unreadable/short VOB).
## `PROBE_SECTORS` sizing
How many 2048-byte sectors of the first feature extent to probe. The head
of a DVD feature opens with logos/warnings whose audio is frequently a
thin 2.0 bed on the FIRST sub-stream only — the other physical `0x8x`
sub-streams and the main 5.1 mix do not appear until a sector or two
further in. 512 sectors (1 MiB) was too short: on one real disc it saw
ONLY `0x80`, and only its opening 2.0 frames. 1024 sectors (2 MiB)
reliably contains at least one frame of every physical AC-3 sub-stream
AND enough of `0x80` to reach its 5.1 frames. Still bounded so a live
drive is never hammered (see the project "don't hammer the live drive"
rule).
## Why the maximum channel count, not the first frame
The first frame of a sub-stream at the head of a feature is NOT
representative. A DVD opens with logos/warnings, and the main `0x80`
sub-stream there frequently carries a thin 2.0 bed before transitioning
to its real 5.1 main mix a fraction of a second later (observed on a real
disc: `0x80`'s first frames are acmod=2 → 2 channels, then it becomes
acmod=7+lfe → 6 channels within the same 2 MiB window). Recording only
the FIRST frame read `0x80=2` and missed the 5.1 entirely, defeating the
channel-match routing. The 5.1 capability of a sub-stream is the
*maximum* channel count any of its frames carries, so
`probe_ac3_substream_channels` scans them all and keeps the max.
## `remap_audio_pids` conservatism
Conservative — it only ever REASSIGNS among the physical sub-streams the
probe actually saw, and only when a better (exact-channel) match exists
than the stream's current assignment. A stream whose current sub-stream
already matches is left alone; a stream with no matching physical
sub-stream keeps its ordinal assignment. So a normal disc (physical order
== IFO order) is a no-op.
## Test helper notes (`dvd_audio_probe::tests`)
- `ac3_frame`: builds a single, correctly-SIZED AC-3 frame whose
`acmod`/`lfeon` encode a known channel count. `byte4` is
`fscod=0 | frmsizecod=0`, so `ac3_frame_size` reports 128 bytes and the
frame is zero-padded to exactly that — this lets `max_substream_channels`
advance frame-by-frame over a multi-frame payload exactly as it does on
real VOB data. The BSI bits are laid down with a writer so the test
never hand-miscomputes the lfeon offset, matching `acmod_channels`'
reader.
- `ps_ac3_frames`: builds a minimal `private_stream_1` PES carrying
`frames` for `sub_id`, each preceded only by the 4-byte AC-3 sub-header
at the PES head. Mirrors the on-disc layout the PS demux expects: PES
start `0x000001BD`, length, PES header (no PTS), sub-header
`[sub_id, frame_count, ptr_hi, ptr_lo]`, then the concatenated AC-3
frames.
### `probe_reads_max_channels_no_cross_contamination`
Real-disc regression — the probe must read each sub-stream's TRUE
(max-mix) channel count, not be poisoned by an unrepresentative head
frame, and must NOT cross-contaminate between sub-streams. Mirrors the
real on-disc layout that caused the mis-read: the feature head carries
`0x80` opening with a 2.0 frame and THEN a 5.1 frame (its real main mix),
interleaved with `0x81` carrying only 2.0. The old first-frame probe read
`0x80=2` (the logo bed) and missed the 5.1; the max-over-frames probe
must report `0x80=6` and `0x81=2`.
### `max_substream_channels_locates_sync_after_leading_non_sync_bytes`
Regression guard for a hand-checked mutation (`+` → `-` at the
`pos + rel` offset computation): with `pos` starting at 0 and the first
sync found 3 bytes in, `pos - rel` would underflow a `usize` and panic,
or (if it somehow didn't) index the wrong start entirely. `pos + rel` is
the only computation that is always in-bounds, since `rel` is itself
bounded by the length of the slice searched from `pos`.
### Unmappable-size fallback tests
When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
`fscod == 3`), `max_substream_channels` must fall back to stepping
`start + 2` bytes past the sync to re-lock onto the next genuine sync,
and must keep making forward progress doing so (never revisit the same
sync, which would loop forever, and never jump so far that it skips the
very next real frame).
- `max_substream_channels_unmappable_size_steps_forward_by_two` lays a
bogus-sized header at absolute offset 4 (so `start == 4`,
`start + 2 == 6`) immediately followed, at offset 6, by a real, fully
decodable 2.0 frame — the position the `+ 2` fallback must land on
exactly.
- `max_substream_channels_unmappable_size_at_start_steps_forward_not_back`
puts the unmappable-size sync at absolute offset 0 (`start == 0`) so
that stepping backward instead of forward (`start - 2`) would underflow
rather than merely land on the wrong byte. Also proves the real frame is
still found 6 bytes further in, confirming forward progress past the
bogus header.
### `remap_reads_current_substream_via_and_not_or_or_xor`
`remap_audio_pids` must read a stream's CURRENT physical sub-stream id
from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
`0x00FF`, both of which force the low byte to `0xFF` regardless of the
real PID and so always miss the "already matches" shortcut. That matters
observably when TWO physical sub-streams share the same probed channel
count: with a correct read, a stream already sitting on a matching
sub-stream is left alone (conservative, per the module's documented
behaviour); with the low byte forced to `0xFF`, `probed.get(&0xFF)` is
always `None`, so the code falls through to the "find any unclaimed
match" path and picks the FIRST (lowest-keyed, BTreeMap-ordered) matching
physical sub-stream instead — which here is a *different* sub-stream
(0x80) than the one the PID already correctly names (0x81), producing a
spurious PID change.
### `probe_and_remap_reroutes_swapped_substream_scenario_end_to_end`
End-to-end `probe_and_remap`: a real-disc-shaped MpegPs title (one
declared 5.1 AC-3 stream ordinally assigned 0x80) whose physical VOB
bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on 0x81. This must
reach the `remap_audio_pids` call and re-route the stream to 0xBD81. It
also, by construction, proves each of the guards along the way lets a
real, positive case through: the content-format check must NOT bail on
`MpegPs` (only on non-`MpegPs`), the AC-3 presence check must NOT bail
when AC-3 IS present, and the sector-count check must NOT bail when the
count is nonzero — any one of those inverted would skip the probe
entirely and leave the PID at its untouched ordinal value (0xBD80), which
the test's assertion would catch.
+177
View File
@@ -0,0 +1,177 @@
# DVD Title Scan (src/disc/dvd.rs) — Internal Notes
Rationale, regression history, and test-fixture layout notes that were
originally inline comments/doc-comments on private items in
`src/disc/dvd.rs`, relocated here to satisfy the comment-guard's internal-
comment cap. Each pointed-to site in the source carries a short `// See
docs/dvd.md — <topic>` line.
## `Disc::scan_dvd_titles` halt handling
Cancellation: `halt` is polled before the IFO tree is read, and an IFO read
that fails with `Error::Halted` — how a live drive reports a Stop, since
`Drive::checked_exec` fails every command once its flag is set — is
propagated rather than swallowed. Every other IFO failure keeps its
best-effort `Ok(vec![])`.
It has to be an error and not an empty title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles. This one was the worst of the three enumerators — a bare
`Err(_) => return Vec::new()` turned an operator Stop into ZERO titles at
rc=0, a disc reported as carrying no video at all.
`scan_dvd_titles` returns the scanned titles plus, when the disc's own
First-Play navigation deterministically dispatches to a title, that title's
`playlist_id` — the nav-resolved main feature. The caller feeds it into
`sort_titles_by_main_feature` as the DVD `nav_feature`, the same seam the BD
path uses. It is `None` (existing ranking heuristics stand) whenever the
navigation is menu-only, malformed, or does not converge.
## DefaultDuration / cadence TODO
`measured_cicp` TODO(spec): populate `top_field_first` + measured colour
from `Mpeg2Parser` once tracks are built after frame parsing — needs a
`CodecParser`->title channel like `codec_private`. Until wired, `None`
falls back to TFF / PAL-NTSC guess.
DefaultDuration TODO(spec): comes from IFO `frame_rate`, so a
soft-telecined 23.976-in-29.97 DVD reports 29.97 instead of true film
rate. Cadence is detectable via `nb_fields` (2:3 pulldown); needs the same
channel as `top_field_first` above.
## Test fixture builders: `build_vmg` / `build_vts`
`build_vmg` builds a VIDEO_TS.IFO: magic "DVDVIDEO-VMG"@0, TT_SRPT sector
ptr@0xC4. TT_SRPT lives at `tt_srpt_sector*2048`: `num_titles(u16)`@0, then
12-byte entries from +8. Each entry: `num_chapters(u16)`@+2, `vts_number`@+6,
`vts_title_num`@+7.
`build_vts` builds a VTS_XX_0.IFO. Layout per ifo.rs:
- magic "DVDVIDEO-VTS"@0
- `vtstt_vobs` (Title VOBS start sector, u32 BE)@0xC4 — the production
`vob_start_sector` the cell sectors are relative to. (0xC0 is
`vtsm_vobs`, the menu VOBS, which the scan must NOT use.)
- VTS_PGCIT sector ptr(u32 BE)@0xCC
- video attr byte@0x200
- `num_audio(u16 BE)`@0x202, audio blocks (8B) @0x204
- `num_subs(u16 BE)`@0x254, subtitle blocks (6B) @0x256
PGCIT (at `pgcit_sector*2048`): `num_pgcs(u16)`@0, PGC info entries (8B)
from +8 with PGC byte offset(u32 BE)@+4. PGC: `nr_programs`@0x02,
`nr_cells`@0x03, BCD time@0x04, `pgm_map` ptr@0xE6, `cell_playback` ptr@0xE8
(both u16 BE rel to PGC start).
## `HaltingReader` test double
A `SectorSource` that fails every read at or above `halt_at` with
`Error::Halted` — how a LIVE DRIVE behaves once the operator presses Stop:
`Drive::checked_exec` fails every SCSI command with `Halted` from then on,
and `Drive::read` deliberately preserves the variant. Reads below the
threshold still succeed, so the scan gets far enough to have something to
truncate.
## Test: `halted_ifo_read_is_not_reported_as_a_shorter_disc`
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The DVD enumerator must not swallow that into a
successful scan.
This was the worst of the three enumerators. RED BEFORE GREEN, two
distinct swallows, both measured with the fix reverted:
- `ifo::parse_vmg` treats a failed title set as a placeholder entry and
continues, so a cancel landing on VTS_02's IFO returned
`Ok([VTS_01_1.VOB])` — one title from a two-title disc.
- `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel landing
on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc reported as
holding no video at all.
Both are indistinguishable from a real disc, and both are now
`Err(Error::Halted)`.
Fixture: two title sets, VTS_01's IFO at `PART_START+6000`, VTS_02's at
`PART_START+7000`. Both ICBs sit far below, so filesystem metadata
resolves fine and only the second title set's CONTENT read is cancelled —
the truncation case.
## Test: `scan_dvd_titles_uses_title_vobs_not_menu_vobs`
Regression (first-play menu prepended to the feature): `vob_start` must
come from the **Title** VOBS pointer `vtstt_vobs` (VTS_IFO 0xC4), NOT the
**menu** VOBS pointer `vtsm_vobs` (0xC0). On discs with a per-title menu —
e.g. the Universal "the parental level has been set, press yes" first-play
still — `vtsm_vobs` points at that menu VOB, which sits just before the
title VOB. Cell `first_sector` values are relative to `vtstt_vobs`; reading
0xC0 prepended the menu and shifted every extent back by
`vtstt_vobs - vtsm_vobs`, so the rip opened on the parental prompt instead
of the movie (one real NTSC R1 disc: vtsm=44, vtstt=3640).
Here `build_vts` stamps `vtstt_vobs = 3640` (0xC4); the test additionally
stamps a *different* `vtsm_vobs = 44` (0xC0). The extent must resolve
from 3640.
## Test: `scan_dvd_titles_extent_is_absolute_three_term_sum`
ABSOLUTE-REBASE regression (real-disc menu-at-start fix): `ifo::parse_vts`
now sets `vob_start_sector = file_start_lba(IFO) + vtstt_vobs`, so an
extent's `start_lba` must equal the sum of THREE independent terms — the
IFO file's absolute on-disc LBA, the IFO-relative `vtstt_vobs` (0xC4), and
the cell's `first_sector` — none of which may be dropped. The earlier code
used the bare relative `vtstt_vobs`, placing every extent `ifo_lba`
sectors too early (the rip opened in the VMGI/menu region before drifting
into the movie). The other tests fold two of the three terms together
(zero cell offset, or a single combined expectation); this one keeps all
three distinct and non-overlapping so a regression to ANY two-term
combination is caught.
## Test: `scan_dvd_titles_lpcm_routes_to_a0_pid_range`
LPCM SCAN ROUTING (audit §2 / §5 #6): the 0xA0..=0xA7 PID range was never
exercised in the dvd.rs scan. An LPCM stream (coding_mode 4) at audio
position 1 must get `sub_stream_id 0xA1` → PID `0xBDA1` via
`dvd_audio_pid`, distinct from the AC-3 `0xBD80` space, with its real
channel count preserved.
## Test: `scan_dvd_titles_multiple_vobsub_tracks_distinct_pids`
MULTI-VOBSUB SCAN (audit §2 / §5 #6): the single-subtitle test covered one
track; a multi-subtitle VTS must emit one `Stream::Subtitle` per entry
with distinct PIDs (`0x20 + ordinal`) and per-language tags, all sharing
the PGC palette `codec_data`.
## `stamp_first_play_jumptt` test helper
Stamp a First-Play PGC into a VMG whose pre-command list is a single
unconditional `JumpTT ttn`, so `dvdnav::resolve_main_title` resolves that
title through TT_SRPT. Placed in sector 0, clear of the magic / FP_PGC /
TT_SRPT pointers `build_vmg` already writes.
## Test: `scan_dvd_titles_nav_promotes_first_play_target`
The DVD First-Play nav promotion branch (issue #40): when
`resolve_main_title` returns a `(vtsn, vts_ttn)` target, `scan_dvd_titles`
must map it — via the 1-based `vts_ttn == vts_title_idx + 1` join within
the matching `vts_number` — to the running `title_number`, and surface it
as the `nav_feature`. TT_SRPT here maps title 2 → (VTS 2, title-in-set 1);
the First-Play unconditionally `JumpTT 2`, so the promoted feature must be
the SECOND scanned title (VTS_02 → title_number 2). An off-by-one on the
join or a `vtsn`/`vts_ttn` field swap would promote the wrong title or
none.
## Test: `scan_dvd_titles_drops_leading_scene_index_cell`
End-to-end bug-4 fix: a feature PGC that opens with a leading interleaved-
angle sub-block cell (category 0x90 = in-block cell of an angle block)
must have that cell DROPPED from the muxed extents, so the rip starts at
the real feature. Chapters shift earlier by the dropped duration.
## Test: `scan_dvd_titles_mp2_audio_pid_fallback_is_additive`
Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio stream
has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio, per
`ifo::assign_audio_sub_stream_ids` — the PID falls back to `0xBD00 + i`
where `i` is the stream's positional index in the IFO audio-attribute
table. Two MPEG-audio (coding_mode 2) streams must land on two DISTINCT,
correctly-offset PIDs: 0xBD00 and 0xBD01. This pins the `+` (not `-`/`*`)
so the second stream doesn't collide with, or wrap under, the first.
+23
View File
@@ -0,0 +1,23 @@
# src/dvdnav/mod.rs
## Module rationale
Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
PGC/cell tables, PCI/HLI button packets); the VM command decoder is
verified against real discs.
Current contents: `vmcmd` — the VM command decoder; `nav` — the
First-Play navigation executor that resolves which title the disc's own
navigation selects as the feature.
## `resolve_feature_start` — parked status (issue #40)
PARKED (issue #40, `USE_NAV_RESOLVER=false`): this is intentionally a stub,
not an accidental dead one. It unconditionally returns `None` and ignores
`reader`/`udf` on purpose — the IFO/PCI parsing + nav executor (built on
`vmcmd`) land incrementally behind that flag. Until wired in, calling this
is behaviour-neutral (the caller always falls back); improvements to the
resolver take effect here without touching the call site. Do not delete.
`reader`/`udf` are the seam inputs the nav executor will consume to read
VIDEO_TS.IFO + the VTS IFOs/menu VOBs. Reserved until that lands.
+28
View File
@@ -0,0 +1,28 @@
# dvdsub — VobSub SPU boundary rule
The presence of a PTS — not merely an open `pending` — is the authoritative
SPU-boundary signal, so a lost continuation or a corrupt SPU_size can't merge
the next subtitle into the stuck unit.
## ycbcr_to_rgb range convention (deliberate)
This uses the **full-range (JFIF) BT.601** coefficients with no 16/235 luma
scaling. DVD IFO palette YCbCr is nominally studio-swing BT.601, so
studio-swing math would be more colorimetrically "correct" in isolation. But
the output here is a VobSub `.idx` `palette:` line, and the entire VobSub
ecosystem (the original tooling, mkvtoolnix, players that read the `.idx`
palette) is built around this full-range formula — it is the de-facto
on-disk convention. Emitting studio-swing-scaled RGB here would make
freemkv's palettes inconsistent with every other tool and wrong in players
that assume the VobSub convention. Do NOT "fix" this to studio-swing without
changing the consuming side in lockstep.
## ycbcr_to_rgb_reads_byte2_as_cr_and_byte3_as_cb test fixture
A saturated RED entry appears on disc as Y=76, Cr=255, Cb=85 (full-range
BT.601 encoding of RGB #FF0000), i.e. bytes `[0x00, 76, 255, 85]`. Reading
byte 2 as Cb and byte 3 as Cr instead turns this entry BLUE, which is the
exact user-visible symptom. The fixture is deliberately NOT built from this
crate's own doc comments: those described the order wrongly for a long time,
and the previous version of this test inherited the error from them and
therefore could not detect it.
+62
View File
@@ -0,0 +1,62 @@
# src/mux/ebml.rs
## `fixed_width_vint8`
Encodes `data_size` as the FIXED-WIDTH 8-octet EBML VINT used to back-patch
a master element's size field: the `0x01` VINT_MARKER octet followed by the
56-bit VINT_DATA payload, big-endian (RFC 8794 section 4.4 — an 8-octet
VINT carries 7 octets of VINT_DATA).
Extracted so the full 56-bit payload can be exercised directly: reaching
the high payload bytes through `end_master` / `end_master_buf` would take
a multi-terabyte buffer, leaving them unconstrained by any test.
`data_size` must be below 2^56; both callers check that first.
## `start_master_buf` / `end_master_buf`
This is the seek-free twin of `start_master`/`end_master` for elements
small enough to assemble whole before hitting the file (a BlockGroup: one
Block plus a couple of tiny elements). Because `end_master` always
back-patches a FIXED-WIDTH 8-byte VINT (`0x01` + 7 payload bytes) rather
than a minimal-width one, the bytes produced by this pair are byte-for-byte
identical to the seek-and-back-patch pair — assembling in memory cannot
change the emitted Matroska.
## `read_exact_bounded`
Reads exactly `len` bytes WITHOUT trusting `len` to size the allocation.
`vec![0u8; len]` on an attacker-controlled EBML size would allocate
gigabytes before the read fails. Instead the reader is capped to `len`
and the buffer grows as bytes actually arrive: a malformed element that
claims a huge length but supplies few bytes allocates only what it
delivers, then errors on the short read.
## Test: `buffered_master_matches_seeking_master_byte_for_byte`
The buffer-based master helpers must produce byte-for-byte what the
seek-based ones produce. `write_block_group` was rewritten onto
`start_master_buf`/`end_master_buf` to eliminate ~4 seeks (and 4 MiB
BufWriter flushes) PER FRAME, which on an MPEG-2 title is ~350k frames — a
change only safe because the two encodings are identical.
Both write a FIXED-width 8-byte VINT placeholder (0x01 + 7 payload bytes)
and patch it in place; neither ever emits a minimal-width size. This test
pins that, so a future "optimisation" of either one to minimal-width sizes
cannot silently desync the two and change emitted Matroska.
## Test: `buffered_master_nests_correctly`
Nested masters must patch correctly too — the MVC BlockGroup nests
BlockAdditions > BlockMore inside the BlockGroup, and in-memory patching
works by index rather than by file offset, so nesting is where an
index-arithmetic slip would show up.
## Test: `fixed_width_vint8_is_big_endian_over_the_full_payload`
The fixed-width 8-octet VINT is `0x01` followed by the 56-bit VINT_DATA
payload in BIG-ENDIAN order (RFC 8794 section 4.4). Every payload octet is
distinct here, so a shifted, reversed or dropped octet is visible; the top
payload octets are unreachable through `end_master` without a
multi-terabyte buffer, which is why this is tested at the encoder.
+51
View File
@@ -0,0 +1,51 @@
# `dirimage::encode` — descriptor encoder notes
Overflow rationale for comments in `src/dirimage/encode.rs` that would
otherwise exceed the comment-guard caps. Each section below is pointed to
by a short `//` comment at the relevant spot in the code.
## Why a pure function
`encode` is a pure function from `Layout` to sectors — nothing here touches
the filesystem. That is what makes it testable against the production
parser in `udf.rs`: the same bytes this module writes are read back and
checked by the real UDF reader, so the encoder and the parser can't drift
apart silently.
## Why UDF 1.02 with a single Type-1 partition map
UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
the DVD-Video profile, it is the shape `read_filesystem` takes when
`num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
entirely. That also means a synthetic image never exercises the Metadata
Partition path in `udf.rs` (`:946-991`).
## `finish_tag`: why `tag_loc` correctness matters
Getting `tag_loc` wrong (absolute vs. partition-relative) is the classic
reason a hand-built volume mounts nowhere: a driver that validates the tag
location rejects the descriptor outright. `desc_len` is the descriptor's
total length including the tag; the CRC covers `buf[16..desc_len]`.
## `encode_cs0`: why ASCII, not Latin-1, for the 8-bit form
`parse_udf_name` (`udf.rs:1467`) decodes a compression-8 name with
`from_utf8_lossy`, so a 0x80-0xFF byte — legal CS0 under Latin-1 — would
come back as U+FFFD. Every character above 0x7F therefore takes the 16-bit
(UTF-16BE) form, which that parser decodes correctly.
## `file_entry`: why tag 261, not the Extended File Entry (266)
Real BD-ROMs use the Extended File Entry (tag 266), but an EFE requires
UDF 2.00+, and this image declares 1.02, so `file_entry` writes the plain
File Entry (tag 261) instead. `udf.rs` reads both — the 261 field offsets
it uses (`l_ea` 168, `l_ad` 172, ADs at `176 + l_ea`) are the ones written
here.
## `push_fid`: why FIDs may span logical blocks
FIDs are packed with no inter-descriptor padding beyond the 4-byte
alignment the spec mandates, and they are allowed to span logical blocks —
which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
directory extent as one flat byte run and stops at the first non-257 tag,
so any block-alignment gap would truncate the directory.
+151
View File
@@ -0,0 +1,151 @@
# encrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/encrypt.md — <topic>` pointer
left at the original site.
## HandshakeResult::drive_unlocked
True when the VID came from an unlocker (in `freemkv-unlock`) that
unlocked the drive. Such a drive serves CLEAR content, so AACS bus
encryption is already removed AT THE DRIVE — the same end state a
successful cert handshake's `read_data_key` provides, just via
firmware instead of the AKE. The bus-key gate MUST credit this as a
valid bus-removal: bus encryption is unremovable only when NEITHER the
firmware unlocked the drive NOR the cert handshake yielded a bus key.
Without this, a SUCCESSFUL unlock (VID present, `read_data_key: None`)
paradoxically trips the gate and blocks ALL key resolution (incl. the
online source).
## bus_encryption_removed
Single source of truth for "is AACS bus encryption gone for this
scan?". The gate asks ONLY this — `if !removed { error }` — never
enumerating cases. Bus encryption is gone when ANY of these holds:
the disc never had it; file/ISO reads (content already clear at read
time); an unlocker unlocked the drive (it serves clear content); or
the cert handshake produced the bus key. Add a NEW removal mechanism
HERE, never in the gate.
## AacsCertUnlocker
libfreemkv-side driver for the AACS cert route. It owns the host-cert
collection (a keysource concern that stays in libfreemkv) and then
dispatches the actual mutual-auth to the `freemkv-unlock` AACS
unlocker via the `crate::unlock_bridge`. The firmware (drive-prep) and
CSS routes dispatch the same way at their own call sites; this one
carries the host certs.
## AacsCertUnlocker::authenticate
Runs the host-certificate mutual-auth handshake: collect
non-compiled-in host certs from the key sources + credentials, then
hand them to the AACS unlocker (via the `freemkv-unlock` dispatch),
which tries each cert (wedge-guarded) and on success yields the
Volume ID + `read_data_key` (the AACS 2.0 bus key). Returns a
`CertUnlockFailure` on every no-VID outcome.
## unlock_error_to_error
Maps a `CertUnlockFailure` back to the `Error` variant
`do_handshake_cert` has always surfaced, so `scan_with`'s rendering
and the path-1 disc-hash → VUK fallback are byte-for-byte unchanged.
(`NoHostCert` keeps the `<no host cert>` sentinel.)
## handshake_has_volume_id
Did the cert handshake actually carry a Volume ID? Extracted so it can
be tested as a VALUE. It only ever reaches an operator as the
`has_volume_id` field of the `bus_key_unavailable` warn, and asserting
on a `tracing` field means installing a capturing subscriber — which
is thread-local, while `tracing`'s callsite-interest cache is global.
Those two facts race: the test failed roughly one run in ten under the
full parallel suite while passing every time in isolation, and
serialising the captures was not enough because the cache can be
re-evaluated against the process default rather than the thread-local
dispatch.
A predicate this small does not need a subscriber to verify. The
polarity is the whole point: an `==` here would tell an operator a VID
was absent on exactly the discs where one was present.
## Disc::do_handshake
SCSI handshake — drives the VID-acquisition flow and returns a
structured `HandshakeResult` for downstream key resolution.
VID acquisition runs through `do_handshake_cert`, which first uses the
OEM VID a firmware unlocker may have stashed at drive `init()` (a
drive-functionality capability decoupled from the host cert + HRL) and
falls back to the cert-based mutual-auth handshake (dispatched to the
`freemkv-unlock` AACS unlocker) when none is present. The cert path
also yields `read_data_key`, required for AACS 2.0 bus decryption.
Returns `(handshake, error)`:
- `(Some(_), None)` — VID acquired
- `(None, Some(_))` — specific failure mode (`AacsHostCertRejected` or
`AacsVidUnavailable`)
- `(None, None)` — handshake not attempted (no keydb; resolution will
proceed with VID=zero and rely on path 1 disc-hash → VUK lookup)
## Disc::do_handshake_cert
Cert-based AACS handshake — the cert route for VID acquisition.
Before running the cert mutual-auth, this checks for an OEM Volume ID
a firmware unlocker stashed at drive `init()`. Such an unlocker
unlocks *drive functionality*, not just the disc: VID retrieval via
the drive's OEM CDB is a capability separate from `unlock`. When one
served a VID, we use it and SKIP the cert handshake entirely — the
OEM path gets the VID *without* the host certificate + HRL, decoupling
VID from the cert chain. The OEM path yields no `read_data_key` (no
bus-key is derived); AACS 2.0 content needing read_data_key for bus
decryption must still use the cert path, so an unlocker with no OEM
VID capability returns `None` and we fall through to cert auth
unchanged.
## Disc::collect_host_certs
Collect every AACS host cert the caller carries, from BOTH the
explicit `DriveCredentials` and the key-source layer
(`crate::KeySource::host_certs` across each source), unioned. Host
certs are keysource-served, never compiled in; this is the one place
the OEM cert route gathers them. An empty result is the graceful
no-cert signal (the caller turns it into `Error::AacsNoHostCert`).
`mkb` is the disc's MKB generation when known, forwarded to each
source's `crate::KeySource::host_certs` so a source MAY return only
generation-appropriate certs (the default ignores it).
## Disc::resolve_vid_only
Build a keys-free AACS state that carries only the Volume ID (+
version metadata), for callers that resolve Unit Keys out-of-band and
have disabled the local keydb. The VID is on-disc content read during
the handshake; preserving it here lets the out-of-band path use it. No
keys are present (`unit_keys` empty, `vuk` None), so the disc reports
as "encrypted, no keys" until the caller re-scans with a resolved Unit
Key.
## build_mkb (test helper)
An MKB with one Type-and-Version record (type 0x10) carrying the
version as BE u32 at record offset 8, followed by a recorded EOF
record then trailing zero padding. mkb_content_len walks records and
stops at the first padding (type 0) byte (aacs/inf.rs).
## resolve_vid_only_no_cert_defaults_version_uhd (test)
No content cert at all → version defaults to UHD (major 2), matching
`read_aacs_version` so the scanned `AacsState.version` and the
out-of-band fetch agree on the Unit_Key_RO stride (audit #4: a wrong
BD-vs-UHD guess mis-parses unit keys). bus_encryption false
(unreadable → off).
## Skipped/deferred test coverage
Tests for `read_vid_oem` (response parsing) are skipped in this file
since `Drive` construction requires a live transport; parsing branches
are exercised through `read_vid_oem`'s callers in integration. The
`collect_host_certs` tests union `DriveCredentials` with the
key-source layer; empty means the route fails gracefully
(`AacsNoHostCert`).
+181
View File
@@ -0,0 +1,181 @@
# Error Codes
Overflow detail for `src/error.rs` doc comments that would otherwise exceed
the comment-guard caps. Each section below is pointed to by a short `//` or
`///` comment at the corresponding definition.
## Error Code Ranges
| Range | Category |
|-------|----------|
| E1xxx | Device errors |
| E2xxx | Profile errors |
| E3xxx | Unlock errors |
| E4xxx | SCSI errors |
| E5xxx | I/O errors |
| E6xxx | Disc format errors |
| E7xxx | AACS errors |
| E8xxx | Keydb errors |
| E9xxx | Stream/mux errors |
## E_UDF_NO_USABLE_EXTENT
A file that EXISTS and whose allocation descriptors resolved without error,
yet yields not one usable extent: an empty AD list, or a list every entry of
which is zero-length or points at LBA 0. Reported by the HD-DVD clip resolver
(`disc::hddvd`).
It has no `Error` variant on purpose. `UdfFs::file_extents` returns
`Ok(vec![])` here rather than failing — the emptiness is only a defect in the
eye of a caller that needs bytes — so the condition is DETECTED by the
caller, not returned to it. The code exists so that detection can be
accounted in the log with something other than a neighbouring error's code:
logging it as E6017 would file a zero-length AD list as an authoring hole
and send whoever triages it at the wrong population.
## E_CSS_NO_DISC_KEY / CssNoDiscKey history
While the disc-wide CSS failure and the per-title CSS failure both used
`E_CSS_KEY_MISSING`, an uncrackable CSS disc was iterated title by title,
each one logged as skipped, and the run exited reporting success — a total
failure reported as success. `E_CSS_NO_DISC_KEY` / `Error::CssNoDiscKey` was
split out so `is_disc_level_no_key` classifies it and a multi-title rip loop
fails fast on it instead.
## E_KEY_SERVICE_UNAVAILABLE
A seven-hour run of HTTP 502s reported as `E_NO_DISC_KEY` told operators
their disc was not in the key database and sent them hunting for a VUK that
was never missing; the correct action was to wait. `E_KEY_SERVICE_UNAVAILABLE`
exists so a key-source outage (transport error, DNS failure, timeout, TLS
failure, HTTP 5xx, unreadable reply) is distinguishable from a source that
answered and simply has no key.
## E_MKV_SOURCE_INVALID vs E_MKV_INVALID
`is_skippable_title_stub` classifies `E_MKV_INVALID` as a title that yielded
no muxable frames, which an all-titles rip may skip while finishing the
rest. A corrupt or truncated `mkv://` source file is a FAILURE, not a stub —
reporting it as skippable would let a broken source be silently passed over
by a run that then exits successfully. `E_MKV_SOURCE_INVALID` (and the
sibling codes `E_MUX_HEADER_BUFFER_EXCEEDED`, `E_MKV_LACING_INVALID`) exist
to keep that distinction, and are fatal.
## UdfEmbeddedData
A file that legitimately stores its data this way is tiny (an ICB caps it at
well under 2 KiB — the AACS `*.inf` key files are the usual case), and the
callers that expect one read it via `read_inline_data` long before extents
are ever requested. A stream file cannot be one. `read_directory` already
refuses the same shape for directories; `UdfEmbeddedData` is the file half
of that decision.
## MkvInvalid / MuxHeaderBufferExceeded history
`MuxHeaderBufferExceeded` exists because hundreds of megabytes of real
frames with unresolvable codec-init headers is NOT the same thing as
`MkvInvalid`'s canonical case (an empty nav/menu PGC stub). Reporting the
former as the latter would have `is_skippable_title_stub` silently drop a
main feature from a rip that then exits successfully.
## AacsBusKeyUnavailable
NOT raised for AACS 1.0 BD (no bus encryption, `read_data_key` legitimately
absent) nor for file-backed (ISO) scans, where bus encryption was already
removed at read time and no handshake runs. Only a live-drive AACS 2.0/UHD
scan that obtained the Volume ID but produced no bus key raises this.
## SourceTerminated
The alternative answer is a lie: a dead source that reports `Ok(0)` is
indistinguishable from end-of-stream, and `DiscStream::fill_extents`
legitimately reads a short count as a skippable hole — zero-filling and
advancing over every remaining sector of the title and still returning
success. Unlike a bad sector, this condition cannot be retried at a smaller
size or skipped past.
## error_code / From<Error> for io::Error round-trip
`mux_stream` hands back an `io::Error`, and a front-end reporting *why* a
title failed had no way to recover the code from it — the typed `Error` is
gone by then and only the `E<code>` string prefix survives. Parsing that
prefix is `error_code`'s job; every consumer re-implementing the parse is how
the string-matching this crate spent 1.5.x removing comes back.
`From<Error> for io::Error` boxes the typed value as the payload
(`io::Error::new(kind, e)`), whose `Display` is the same `E<code>[: …]`
string the stringifying version produced — so `error_code`'s parse is
unaffected, and `From<io::Error> for Error` can additionally `downcast` the
payload back to the exact typed error. Errors that did NOT come from this
crate carry no `E<code>` prefix and yield `None`.
## is_skippable_title_stub
This replaces the CLI's `E7023`/`E6008` string-match with a typed check on
the `io::Error` `mux_stream` returns.
`Error::MkvSourceInvalid`, `Error::MkvLacingInvalid`, `Error::MkvUnencodable`
all used to be raised as `Error::MkvInvalid` and therefore landed in the
skippable set, so a corrupt source was reported as a title worth silently
passing over by a run that then exited successfully. They now carry their
own codes and are fatal, as is `Error::MuxHeaderBufferExceeded`.
`Error::CssNoDiscKey` (the disc's CSS crack recovered no key at all) is the
same conflation on the decrypt axis: while it too was raised as
`Error::CssKeyMissing`, every title of an undecryptable disc classified as a
skippable stub, so the rip loop skipped all of them and exited successfully.
It belongs to `is_disc_level_no_key` instead, and is fatal here.
## is_disc_level_no_key
`E_CSS_NO_DISC_KEY` is the CSS side of the disc-level/per-title split, and it
exists because the disc-wide CSS failure used to be raised as the per-title
`E_CSS_KEY_MISSING`: an undecryptable CSS disc landed in
`is_skippable_title_stub`, so the rip loop skipped all N titles with an
"empty stub" notice and exited successfully.
The key-SOURCE failures (`E_KEY_SERVICE_UNAVAILABLE`,
`E_KEY_SERVICE_UNAUTHORIZED`, `E_KEY_SERVICE_RATE_LIMITED`) are classified
here for the same fail-fast reason and NOT because they mean "no key": a
service that is down, refusing the token, or throttling is down for every
title on the disc, so iterating N titles re-issues N doomed requests (and,
on 429, digs the rate-limit hole deeper). They are separate CODES precisely
so the front-end can say "retry later" / "fix the token" instead of
`E_NO_DISC_KEY`'s "no key source has a key for this disc".
## is_marginal_read
- MEDIUM ERROR (sense key 3) — canonical bad-sector signal.
- ABORTED COMMAND (sense key B) — transient; retry usually works.
- NOT READY (sense key 2) — the dominant bad-sector response on the BU40N
(ASC 0x04/ASCQ 0x3E); a pause + retry often recovers.
- RECOVERED ERROR (sense key 1) / NO SENSE (sense key 0) — not classified as
fatal; treat as recoverable.
Caller-agnostic predicate — describes a property of the *error*, not what
one specific call site should do with it. Used by
`freemkv_engine::recovery::copy`'s hysteresis dispatch.
## Test rationale: declared_error_codes / uniqueness
The uniqueness test used to carry a hand-maintained `vec![]` of the error
constants, with a doc comment claiming it "pins all code assignments". It did
not. At the time this was written the file declared 127 `pub const E_*` and
the vector named 109 of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
`E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`, `E_DIR_NAME_COLLISION`,
`E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`, `E_DIR_RAW_REJECTED`,
`E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`, `E_DRIVE_INQUIRY_SHORT`,
`E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`, `E_SEAM_PLAN_DROPPED_MOST`,
`E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`, `E_SOURCE_TERMINATED`,
`E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were outside the guarantee entirely,
so a new variant colliding with any of them passed green. Worse, an earlier
audit READ that doc comment and trusted it while assigning new codes.
The defect is not the eighteen omissions, it is that the list is
hand-maintained at all: adding a constant and forgetting the vector is a
silent no-op, which is the definition of a guarantee that decays. Deriving
the list from the declarations makes forgetting impossible — the only way to
escape the check is to stop declaring the constant. `include_str!` of the
same file is the cheapest seam that does that: no `build.rs`, no proc macro,
no dependency, and — being `#[cfg(test)]` — not one byte of the source
embedded in a release build.
+150
View File
@@ -0,0 +1,150 @@
# `src/disc/extract.rs` — decrypted file-tree extraction
## Relation to the ISO sweep/patch recovery passes
`extract_tree` is a sibling of the disc→ISO sector dump (the sweep/patch
recovery passes, which now live in the `freemkv-engine` crate), specialized
to write **per file** rather than a whole image, applying decryption on the
way out, and **without** any multipass / recovery orchestration — it is
1-shot, decrypt-only.
No new error codes are introduced: it reuses the existing `Error` set
(`error.rs` is numeric-only, no English).
## `extract_tree` contract details
`reader` is consumed for content reads (it is moved into a
`DecryptingSectorSource`); the structure + extent metadata are read first
with the raw reader. `dest` receives the tree STRAIGHT IN (no auto-named
subfolder). The caller must have run the pre-flight decrypt gate
(`Disc::ensure_decryptable`); this method resolves per-VTS CSS keys itself
for DVD.
Bad sectors become recorded zero-filled holes in the affected file (the run
does not abort); undecryptable units are counted as loss. Files are written
`<name>.partial` and renamed on success, so an interrupted run never leaves
a half-written file that looks complete.
## `resolve_vts_key` — why a failed crack must be a hard error
FALLIBLE, because the three outcomes of a crack are not two.
[`crate::css::CrackOutcome`] exists precisely to separate "no scrambled
sector was seen, so there is nothing to decrypt" from "scrambled sectors
were seen and no key came out", and its doc says callers MUST surface the
second as a hard error. An earlier version of this function used the
`Option`-returning `crack_key`, which collapses both into `None`, and then
fell back to the disc-wide key — descrambling this VTS with ANOTHER VTS's
title key. The hazard is described in the ordering note in the function
body, which was written about the same fallback; ordering makes the crack
far more likely to succeed, but it cannot make a failed crack safe.
## `Borrowed` — why a borrowing `SectorSource` wrapper
Lets the decrypting decorator "own" an inner source for its lifetime while
the caller keeps the underlying `&mut dyn SectorSource` (the decorator is a
`DecryptingSectorSource<S>` generic over `S`, so it does NOT require a
`'static` boxed inner — unlike the mux highway, which takes the reader by
value). The decorator is dropped before `extract_tree` returns, so the
borrow never escapes.
## `whole_unit_batch` — the trailing-partial rule
`remaining` is the sectors left in the current extent; the batch is capped
at `READ_BATCH_SECTORS` and rounded DOWN to a whole number of 3-sector units
UNLESS it is the extent's final (possibly short) tail — the tail always
begins on a unit boundary, so a 1-2 sector partial there is handled by
`decrypt_sectors`' trailing-partial contract. Used by `extract_one_file` so
this rounding rule lives in exactly one place.
## `available_space` (windows) — why the free-space gate matters
Windows has no `statvfs`. This once returned `None` unconditionally, which
does not merely skip a test — it skipped the free-space GATE, so a Windows
user extracting a disc to a full volume got a confusing failure part-way
through instead of a clear refusal up front. libfreemkv ships a Windows
GUI, so that is the platform where the friendly error matters most.
`FreeBytesAvailableToCaller` (not `TotalNumberOfFreeBytes`) is used because
it accounts for per-user quotas, matching what `statvfs`'s `f_bavail` gives
on the unix side.
## `sanitize_component` — why reserved names are substituted, not rejected
A name whose base matches a Windows reserved device name (`NUL`, `CON`,
`COM1`..) is **substituted** (prefixed with `_`) rather than rejected — a
Linux-authored disc may legally carry such a file and a single reserved
name must not abort the whole tree walk. An empty result after stripping
is an error.
## `names_differing_only_by_case_are_a_collision` — why this must be an error
Two disc names differing only by CASE are one host file on macOS APFS and
Windows NTFS, both case-insensitive by default. Keyed by the case-preserving
path, the collision map would see two entries, raise nothing, and the
second file extracted would overwrite the first — while both `PlannedFile`s
report `complete: true`. Silent data loss reported as a clean extract is
the one outcome this crate must never produce, so the host-equivalence key
has to model the host's namespace, not the disc's.
## `a_files_partial_path_colliding_with_another_files_final_name_is_an_error`
A disc carrying both `X` and `X.partial` plans two distinct final names, so
nothing collides in the final-name check — but extracting `X` writes
through `X.partial`, the same host path the other file owns. Whichever
lands second truncates the other, and both entries would still be reported
complete without this guard.
## `multi_extent_aacs_anchors_unit_base_per_extent` — the alignment bug
Regression (rc.6 audit, finding #449): a MULTI-EXTENT AACS file must
re-anchor the unit-alignment base PER extent, not once at the first extent.
The two extents in this test start at absolute LBAs whose difference is NOT
a multiple of 3 (`PART_START+5000` vs `PART_START+5004` — a 4-sector delta).
With a single first-extent unit base the second extent's first read is off
the unit grid (offset 4, `4 % 3 == 1`) → `is_unit_aligned` fails →
`DecryptFailed` → the whole extent becomes a zero-filled (false) hole. With
per-extent anchoring both extents read clean: `bytes_unreadable == 0` and
the file content survives. The fixture content is clear (no TS syncs) so
the decrypt step restores each unit verbatim — this isolates the GATE from
the cipher math.
## `extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it`
An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED: the
space belongs to the file and occupies its byte range, but nothing was ever
written there, and the standard defines its contents as zeros.
`read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
`read_file_limited` honours it by emitting zeros WITHOUT touching the
media. The tree extractor reads the same ICBs and must agree: reading those
sectors returns whatever the media happens to hold there — on an AACS disc,
ciphertext that decrypts to noise — and writes it into the extracted file
as if the disc had recorded it.
## `per_extent_base_is_aligned_first_extent_base_is_not`
Focused alignment-computation check underpinning the per-extent fix: when
each extent anchors its OWN start as the unit base, the extent's own batch
starts are always unit-aligned; anchoring a later extent against the FIRST
extent's base mis-aligns whenever the extents' starts differ by a
non-multiple of 3 sectors. This is the exact arithmetic the decrypt-on-read
gate (`aacs::content::is_unit_aligned`) performs.
## `css_two_vts_groups_do_not_cross_contaminate_keys`
Regression: the per-VTS key crack in `resolve_vts_key` must gather ONLY
this VTS's own title-VOB extents. Two VTS groups here carry DISTINCT
scrambling keys; if the group filter (`vts_group_of(..) == Some(vts) &&
is_title_vob(..)`) is loosened (`!=`, or `&&` -> `||`), one VTS's resolve
gathers the OTHER VTS's extents, cracks the wrong key, and that VTS's own
content silently fails to descramble to the expected plaintext.
## `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`
A VTS that IS scrambled but whose key could not be recovered must FAIL, not
borrow another VTS's key. `crack_key` returns `Option`, which collapses "no
scrambled sector was seen" with "scrambled sectors were seen and no key
came out"; the fallback then descrambled this VTS under the disc-wide key
and `extract_tree` reported `complete = true` at exit 0, because no read
had failed. `CrackOutcome` exists to keep those two apart, and every
sibling path in the crate already honours it. The fixture: VTS_01 is
crackable, VTS_02 is scrambled with NO periodic crib, so its crack
genuinely fails.
+36
View File
@@ -0,0 +1,36 @@
# `file_sector_source::linux` — read-side platform hints
## Why both `hint_sequential` and `drop_window`
`POSIX_FADV_SEQUENTIAL` at open widens the kernel's readahead window so
each pread aggregates into fewer NFS round-trips. `DONTNEED` on the
consumed window (called periodically by the caller) drops the
already-read pages from the page cache so an 85 GB streaming ISO read
doesn't fill memory and starve concurrent writes (the MKV output
during mux). Together they mirror the write-side WritebackPipeline's
policy.
## History
Pre-Phase-1 (0.20.7 baseline) had both. Phase 1's introduction of
`FileSectorSource` silently dropped the read-side DONTNEED, and
0.21.2's revert of `SEQUENTIAL` (mistakenly attributing a regression
to it) removed the hint. Net effect: 85 GB of ISO reads pinned in the
page cache + no readahead widening → mux throughput collapse from
18 MB/s historical to 2.7-8 MB/s on 0.21.x. Restored in 0.21.6.
## `prefetch`
Async-prefetch `len` bytes at `offset` into the page cache. The kernel
`readahead(2)` syscall queues the I/O and returns immediately — it
does NOT wait for completion. Called right after each consumed read
so the next batch's I/O overlaps with the caller's processing of the
current batch (decrypt + demux + mux).
Without this hint, with a synchronous demux consumer running at
~50 MB/s and a single-spindle disk capable of ~150 MB/s, the disk sits
idle ~70% of each iteration because kernel readahead alone (capped at
`/sys/block/<dev>/queue/read_ahead_kb`, default 128 KB) can only
pre-stage a tiny slice of the next batch. An explicit `readahead()` of
the same size as the current batch tells the kernel to queue the full
next-batch read now.
+29
View File
@@ -0,0 +1,29 @@
# file_sector_source::macos
macOS has no direct `POSIX_FADV_SEQUENTIAL` equivalent; the idiomatic hint is
`fcntl(F_RDADVISE, &radvisory)` describing the byte range you intend to read
soon. The open-time sequential hint points it at the whole file (clamped to
`RDADVISE_MAX_BYTES` so a multi-TB ISO doesn't ask the kernel to prefetch
everything at once).
## RDADVISE_MAX_BYTES
Cap on the byte length passed to `F_RDADVISE`. Asking for a multi-GB
readahead window is counterproductive — the OS doesn't have that much cache
to throw at one fd. 64 MiB is generous for our use case (sweep, mux) so the
kernel's prefetch keeps pace with our app-level pipeline depth.
## drop_window
macOS has no direct `POSIX_FADV_DONTNEED` equivalent for a byte range.
`fcntl(F_NOCACHE)` would disable caching globally on the fd (too coarse — we
want the unread region to still benefit). Best approximation: no-op.
macOS's unified buffer cache is generally less prone to the pin-everything
pathology that triggers the regression on Linux NFS clients.
## prefetch
Async-prefetch the byte range `[offset, offset+len)`. macOS uses the same
`fcntl(F_RDADVISE, &radvisory)` primitive as the open-time sequential hint,
just targeted at a moving window instead of the whole file. The kernel
queues I/O for the requested range and returns immediately.
+77
View File
@@ -0,0 +1,77 @@
# `FileSectorSource` design rationale
`FileSectorSource` reads 2048-byte sectors from an ISO file on disk via
direct `seek + read_exact` (`pread`-equivalent) calls, letting the kernel's
own readahead policy manage prefetch.
## Why no app-level buffer
Pre-0.21.3 this source held a 32 MiB (later 4 MiB) read-ahead buffer to
amortise per-sector NFS round-trips. Empirically that buffer hurt: 32 MiB
refills bursted the NFS TCP connection hard enough to starve the concurrent
writer, and even a 4 MiB window gave the kernel less freedom to pipeline
reads with writes. Direct pread per call lets Linux's readahead widen as it
detects the sequential pattern, and naturally interleaves with writeback.
## DONTNEED on the consumed window
Without page-cache eviction an 85 GB streaming ISO read pins the entire
file in memory, starves the concurrent writer, and collapses mux throughput
(observed: 2.7 MB/s mux on 0.21.5 vs. 70 MB/s isolated NFS reads). Every
`READ_DROP_CHUNK_BYTES_DEFAULT` of consumed bytes we call
`posix_fadvise(DONTNEED)` over that window, mirroring the write-side
`crate::io::writeback::WritebackPipeline` policy.
The drop window is accounted by a monotonic forward byte counter, which
matches the sequential streaming pattern the mux highway drives. Under
random or backward access the dropped range no longer lines up with the
bytes actually read — but `DONTNEED` is purely an advisory cache hint with
no correctness impact, so this degrades to a slightly imprecise hint rather
than a bug.
`READ_DROP_CHUNK_BYTES_DEFAULT` (32 MiB) is the empirically tuned value on
a 7200rpm HDD via SATA: smaller windows (8 / 16 MiB) shorten the
kernel-readahead overlap and slow the producer; larger windows (64 / 128
MiB) let the page cache pin enough of the ISO to pressure concurrent
writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
`READ_DROP_CHUNK_MIB_MAX` (64 GiB) bounds `FREEMKV_READ_DROP_CHUNK_MIB`
before the MiB→byte multiply, mirroring `WRITEBACK_CHUNK_MIB_MAX`, whose
identical multiply is bounded for exactly this reason: without the bound,
a value above 2^44 overflows — a panic on the first ISO open in an
overflow-checked build, and in release a wrap to a near-zero window that
fires `drop_window` on every read. Out-of-range values fall back to the
default.
## Platform open hint
On `open()` each platform issues its "sequential access expected" hint so
OS-level readahead widens. The hint and the DONTNEED call live in per-OS
sibling modules (`linux::hint_sequential` et al.) — no inline `#[cfg]` in
this file.
## Read-ahead prefetch
After every consumed read we issue an OS-level prefetch hint for the next
equivalent-sized window (`platform::prefetch`). The kernel queues that I/O
asynchronously and returns immediately, so the next batch's read overlaps
with the caller's processing of the current batch (decrypt + demux + mux).
Without this the disk sits idle ~70% of each iteration because kernel
SEQUENTIAL readahead alone (capped at `read_ahead_kb`, default 128 KB) is
far smaller than our 16 MiB app-level batch.
## Test notes
- `read_sectors_with_an_undersized_buffer_errors_rather_than_panicking`:
this is a public `SectorSource` impl, so buffer length is caller input,
and the guard used to be a `debug_assert!` — compiled out in release,
where the `out[..bytes]` slice then panicked with 'range end index out
of range'. `Drive::read_fua` already carries this exact guard with a
comment recording the same panic being fixed there, and
`PrefetchedSectorSource` has `direct_read_too_small_buffer_errors` for
the same case; this impl had neither.
- `dontneed_eviction_does_not_affect_data`: reads past the default 32 MiB
drop chunk (16384 sectors) so the eviction block fires at least once,
asserting every sector still reads correctly. Avoids mutating
`FREEMKV_READ_DROP_CHUNK_MIB` to sidestep a parallel-test env race with
`drop_chunk_size_env_override`.
+16
View File
@@ -0,0 +1,16 @@
# FLAC elementary-stream decodability gate (`src/mux/codec/flac.rs`)
FLAC frames carry no length field, so a raw stream is delimited only by
sync-scanning + CRC validation. In freemkv, though, FLAC never arrives raw:
it comes from mp4/mkv, where each packet is exactly one container-delimited
FLAC frame (a complete, pre-delimited frame per packet). So this parser
is a per-packet gate, not a framer: every FLAC frame ends with a 16-bit CRC
(poly 0x8005, init 0, non-reflected) computed so the residue over the whole
frame — footer CRC included — is zero (per the FLAC format specification,
RFC 9639, frame footer). A nonzero residue is definitive corruption → drop
the frame (a silence gap, never a shift — each packet keeps its own PTS),
logged via the shared tally.
A packet that does not begin with the FLAC frame sync is not a delimited
frame we can validate, so it is passed through unchanged (never
false-dropped).
+36
View File
@@ -0,0 +1,36 @@
# fMP4 muxer (`src/mux/fmp4/mod.rs`) — stub status and design notes
## Status
**STUB**. The muxer can emit the init segment (`ftyp` + a minimal HEVC
`moov` skeleton with one video track) via
[`Fmp4Mux::write_init_segment`], so the shape and call site are
validated, but media fragments (`moof`/`mdat`) are NOT emitted.
[`Fmp4Mux::write_video`] therefore returns
[`Error::Fmp4Unimplemented`](crate::error::Error::Fmp4Unimplemented)
rather than silently accepting and discarding frames. It buffers
nothing, so it cannot accumulate memory.
## Not yet implemented
- `moof` box: `mfhd` (sequence_number) + `traf` (`tfhd` + `tfdt`
+ `trun` with sample sizes, durations, flags, composition offsets).
- `mdat` box: concatenated sample data.
- Fragment cadence: one fragment per GOP or every N seconds,
whichever comes first.
- HEVC `hvcC` box inside `moov.trak.mdia.minf.stbl.stsd` so the
init segment is self-describing (`stsd` currently has zero entries).
- Sample-flags computation (sync vs. delta, depends_on, etc.).
- Edit lists / fragment_duration for accurate seeking.
Reference: ISO/IEC 14496-12 §8 (Movie Fragments).
## `wrap_box` size-cast rationale
All callers build tiny init-segment boxes (kilobytes at most), so the
`u32` size never overflows; the saturating cast plus the debug assert
documents and guards that invariant rather than silently emitting a
truncated, structurally corrupt size field. `body` is always
internally constructed here, never untrusted input — a future caller
feeding a multi-gigabyte body trips the debug assert instead of
writing a malformed box.
+70
View File
@@ -0,0 +1,70 @@
# Fox — `dcx.xml` label parser
Older Fox authoring (e.g. a real Fox release, `/BDMV/JAR/05001/dcx.xml`) ships a
human-readable XML manifest alongside the BD-J jar. Its root is `<dcx>`, and
under `<disc>` it lists every playlist the disc plays. The main-feature
playlists carry nested per-stream `<audio>`/`<subtitle>` elements naming
language, editorial purpose and forced/SDH state outright — everything a
label parser wants, in attributes, with no bytecode to walk.
NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this file
is one authoring house's internal metadata that happens to press onto the
disc. Every field meaning below was read off a real disc; treat an
unfamiliar value as unknown rather than guessing.
## Confirmed schema
A real Fox release, `/BDMV/JAR/05001/dcx.xml`:
```xml
<dcx>
<disc>
<playlist id="00001" lang="eng" name="topmenu"/>
...
<playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="eng" type="rnib"/>
<audio id="03" lang="spa" dial="lat" type="feature"/>
...
<subtitle id="01" lang="eng" type="feature" form="sdh"/>
<subtitle id="02" lang="spa" dial="lat" type="embed"/>
...
<subtitle id="11" lang="eng" type="text"/>
<properties> ...chapter marks... </properties>
</playlist>
<playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628"> ... </playlist>
</disc>
</dcx>
```
## Field meanings (read off the disc, not documented)
* `<audio type>`: `feature` = a normal program track; `rnib` = a
descriptive/narration track ("Royal National Institute of Blind People"
described-video), mapped to `LabelPurpose::Descriptive`. A `type`
containing "comment" maps to `LabelPurpose::Commentary`.
* `<subtitle form>`: `sdh` marks a subtitles-for-the-deaf-and-hard-of-
hearing track (`LabelQualifier::Sdh`).
* `<subtitle type>`: `embed` marks a dedicated forced/embedded-subtitle
track (`LabelQualifier::Forced`); `feature`/`text` are full tracks.
* `id` on a nested stream is its 1-based STN slot WITHIN ITS TYPE
(audio ids 01..N, subtitle ids 01..N independently), matching the vendor
`stream_number` convention the ordinal binder in `super::apply_labels`
reads.
The `<audio>`/`<subtitle>` elements are NESTED inside one feature playlist,
so extraction is scoped to a single `<playlist name="feature">` element —
never a document-wide `<audio>` scan, which would merge the regional
`00800` (eng) and `00801` (jpn) tables into one and collide their slots.
## `select_feature_playlist` scoping rationale
Among all `<playlist name="feature">` elements — Fox presses one per
regional variant (`00800` eng, `00801` jpn) — the one carrying the most
`<audio>`/`<subtitle>` streams wins, ties to the first.
Returning ONE element is the whole point: each feature playlist is its own
STN table with its own 1-based slot ids, so merging two would put two
different streams on slot `01`. The richest table is the fullest label set
and anchors to its matching title by language sequence in
`super::apply_labels`.
+13
View File
@@ -0,0 +1,13 @@
# Linux `f_type` classification
`classify_f_type` in `src/platform/fs_type/linux.rs` is the single source of
truth for the magic-number comparisons used by both the path-based
(`detect_impl`) and fd-based (`detect_fd_impl`) entry points.
## Why the `i64` cast
`statfs::f_type` is signed `__fsword_t` on glibc and unsigned `c_ulong` on
musl; a portable comparison needs a common type, so the value is cast to
`i64`. On glibc x86_64 both are already `i64` — clippy flags the cast as
unnecessary on that target only, but the cast is required for musl, hence
the `#[allow(clippy::unnecessary_cast)]`.
+12
View File
@@ -0,0 +1,12 @@
# fs_type module
The buffering architecture (Phase 2) selects different output sinks
for local vs network filesystems: NFS gets the adaptive
`WritebackFile` machinery on Linux; local disks get `LocalFileSink`
and rely on the kernel's default writeback policy. This module
provides the construction-site primitive that picks which one.
Per the per-OS file-split convention, the actual `statfs` call lives
in the matching platform file (`linux.rs`, `macos.rs`, `windows.rs`,
`other.rs`); this `mod.rs` exposes only the cross-platform enum and
the `detect` entry point.
+20
View File
@@ -0,0 +1,20 @@
# `io::fsync` — platform-aware crash-durability primitives
Two flush operations need OS-specific handling to make a write survive a
crash / power loss:
- **`dir`** — fsync a directory so a prior `rename(2)` into it is durable.
After a crash a renamed file's dirent can otherwise be lost even though
the rename returned, because it is still page-cache-only. This is a POSIX
concept: on Windows std cannot even open a directory as a `File` (it does
not set `FILE_FLAG_BACKUP_SEMANTICS`), and NTFS/ReFS commit the rename's
dirent without an explicit directory flush — so it is a no-op there
rather than a failed open that logs on every marker write.
- **`file_durable`** — fsync a file's contents + metadata. Opens the file
**read+write**: on Windows `File::sync_all` maps to `FlushFileBuffers`,
which requires a handle with write access and returns
`ERROR_ACCESS_DENIED` (os error 5) on a read-only handle. (A read-only
`File::open` + `sync_all` is legal on POSIX, which is why that bug only
bit Windows.) The open mode is platform-uniform, so this lives here with
no dispatch.
+37
View File
@@ -0,0 +1,37 @@
# `fvi://` sink notes
Rationale and design notes for `src/mux/fvi_sink.rs` that don't belong in the
public rustdoc contract.
## Relationship to `VideoMap`
The sink is a thin consumer of the reusable, pure-data
[`VideoMap`](crate::mux::videomap) model
([`MapHeader`](crate::mux::videomap::MapHeader) /
[`PictureRecord`](crate::mux::videomap::PictureRecord)): it builds the header
from the title, then writes one record per video
[`PesFrame`](crate::pes::PesFrame) straight to disk. Nothing here re-parses
the elementary stream, and the whole index is never buffered.
Serialization is inlined in the sink itself.
## Extensibility
A different output format would be a DIFFERENT sink reusing the same
`VideoMap` model (e.g. a future `fvi2://`), not a pluggable encoder —
extensibility is by adding a sink, like every other sink in this crate.
## `write_fvi_record`: coding-derived members (§7.1)
Coding-derived members (§7.1) are produced via the codec-agnostic
`PictureInfo` accessors, never the raw bitstream: `field_order` is emitted
only when measured (OMITTED on codec-type-only HEVC/H.264/VC-1),
`progressive` only when signalled.
## `write_fvi_record`: `dts` / future `recovered` member
`dts` is MAY per the format spec — the highway carries no DTS on a frame, so
it is always omitted.
TODO(provenance→recovery join): a `recovered` MAY member (sweep/patch mapfile
overlap with this AU's `src`) belongs here; unreachable at `PesFrame`.
+189
View File
@@ -0,0 +1,189 @@
# H.264 codec parser notes
## Slice-type mapping
`h264_slice_coding_type` maps an H.264 `slice_type` (Rec. ITU-T H.264 §7.4.3,
Table 7-6) to a coding type. Values 5..=9 repeat 0..=4 (the "all slices of
this type" forms), so `slice_type % 5`: 0 = P, 1 = B, 2 = I, 3 = SP
(predicted → P), 4 = SI (intra → I). Returns `None` for values outside 0..=9
(malformed header).
## `unescape_ebsp_prefix`
Copies the leading bytes of an EBSP with emulation-prevention bytes removed
(ISO/IEC 14496-10 §7.3.1: a 0x03 following 0x00 0x00 is not part of the
RBSP). Only a short prefix is produced, because the sole caller decodes two
ue(v) fields from the very start of a slice header. `first_mb_in_slice` and
`slice_type` are each at most 32 bits, so 16 octets is ample and keeps this
off the per-frame allocation path for anything larger.
## `unescape_ebsp`
Copies `ebsp` with emulation-prevention bytes removed, stopping after at most
`max_octets` output bytes (pass `ebsp.len()` for "no cap").
The zero run-length counter is cumulative across the whole scan and is reset
only when an escape byte is actually dropped — matching the reference
decoding process in ITU-T H.264 §7.3.1, which discards a 0x03 following ANY
run of two-or-more 0x00 bytes, not just an exact `00 00 03` sliding window. A
window-based scanner that advances by 3 on a match and by 1 otherwise
disagrees with this on a run of 3+ real zero bytes ahead of an 0x03:
`00 00 00 03` decodes here as `00 00 00` (the run is escaped, matching every
other H.264 decoder), where a window scan starting fresh after each
non-match would keep the 0x03 as real payload. Only one implementation of
this rule may exist in this module — see `parse_sps_high_profile_ext`, which
used to keep its own window-scan copy that disagreed with this one on
exactly that input.
## `handle_param_set`
Handles an SPS/PPS NAL (mirrors the HEVC fix). The strip/emit decision is
made against the currently-ACTIVE set `cur`, NOT the codecPrivate copy
`first`: a streaming MKV decoder applies avcC once at init and thereafter
updates a parameter set only from an in-band NAL, so a switch BACK to the
first-seen body (== codecPrivate) is still a change the decoder must be told
about. Stripping on `== first` silently dropped that revert.
- First of its type → seeds codecPrivate; stripped (decoder gets it from avcC).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Unconditional (not only when the active set differs from codecPrivate): a
streaming decoder applies the avcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IDRs
even though its body is unchanged; if the decoder then drops it (a reset
event), nothing re-sends it and every subsequent slice fails (param-set id
out of range) until the next genuine change. Re-asserting at EVERY keyframe
— what compliant muxers do at every IDR — makes streaming decode
self-healing. Re-sending an identical param set is benign; cost is a few
bytes per keyframe. This strictly supersets the change-only re-assert, so
the param-set-revert fix is unaffected.
## `parse_sps_high_profile_ext`
Parses `(chroma_format_idc, bit_depth_luma_minus8, bit_depth_chroma_minus8)`
from a High-Profile SPS NAL (profile_idc ∈ `HIGH_PROFILES` — the 14
chroma/bit-depth extended profiles `codec_private` invokes this for).
SPS RBSP layout (ITU-T H.264 §7.3.2.1.1) up to the fields we need:
```
byte 0 NAL header (already known to be type 7)
byte 1 profile_idc
byte 2 constraint_set_flags / reserved
byte 3 level_idc
ue(v) seq_parameter_set_id
— High-profile branch —
ue(v) chroma_format_idc
if chroma_format_idc == 3: u(1) separate_colour_plane_flag
ue(v) bit_depth_luma_minus8
ue(v) bit_depth_chroma_minus8
```
RBSP emulation-prevention bytes (0x00 0x00 0x03 → 0x00 0x00) are removed
before bit-parsing so the bit reader sees clean RBSP data.
Returns `None` if the SPS is too short or malformed (Exp-Golomb code
overflows 32 bits, leading-zero count > 31, etc.). The caller silently omits
the extension in that case.
## Test notes
### `open_gop_intra_access_unit_is_a_keyframe`
Open-GOP resync fix: a non-IDR intra-coded access unit MUST be flagged a
keyframe so the B1 resync gate (`mux/resync.rs`) can disarm on a BD
open-GOP tail instead of dropping every frame to EOF and truncating the
video. BD H.264 titles reach random access at a non-IDR I-frame (the
recovery point); HEVC gets this free from the CRA NAL type, H.264 does not,
so the measured intra coding type is the signal. Red before the intra
promotion in `parse` — a non-IDR I-slice used to yield `keyframe == false`.
### `inter_coded_access_unit_is_not_a_keyframe`
Guards the promotion's precision: a non-intra (P) access unit must NOT be
promoted to a keyframe, or the resync gate would resume on a frame that
carries inter-references to dropped data and the MKV cue index would gain
bogus random-access points.
### `slice_header_parse_removes_emulation_prevention_bytes`
The slice-header parse must remove emulation-prevention bytes first
(ISO/IEC 14496-10 §7.3.1/§7.4.1): the payload after the NAL header is EBSP,
and a 0x03 following 0x00 0x00 is not part of the RBSP.
Reachable when `first_mb_in_slice` is 65535 or more — a ue(v) that needs
sixteen leading zero bits, so the payload opens 0x00 0x00 and an encoder
must escape it. A UHD frame is ~32,400 macroblocks so conforming Blu-ray
never gets there, but 8K does, and the disc is untrusted input. Reading the
raw NAL decoded `slice_type` against the inserted byte and produced the
wrong picture type.
### `unescape_ebsp_drops_escape_after_a_run_of_three_zeros`
Regression: `parse_sps_high_profile_ext` used to re-derive the
emulation-prevention rule with its own window scanner (match `00 00 03` at
position i, advance by 3; else advance by 1) instead of calling the shared
`unescape_ebsp`. On a run of 3+ real 0x00 bytes ahead of an 0x03 —
non-conformant, but this is untrusted disc input, not a spec-clean encoder —
the two disagreed: `unescape_ebsp`'s cumulative zero counter (the rule the
H.264 reference decoding process in §7.3.1 specifies) drops the 0x03 as an
escape, while a fresh 3-byte window starting right after the non-matching
first byte kept it as real payload. Pin the shared function's behaviour
here so a second hand-rolled copy doesn't quietly reappear.
### `keyframe_param_reassert_emits_exact_bytes`
The keyframe SPS/PPS re-assert must produce EXACTLY these bytes: the active
SPS, then the active PPS, then the access unit's own NALs, each as a 4-byte
big-endian length prefix followed by the NAL body (ISO/IEC 14496-15
length-prefixed form, lengthSizeMinusOne = 3).
This pins the full byte string, not just its length, so the in-place
`splice` that replaced the build-a-new-buffer-and-copy approach is proven
byte-for-byte equivalent — including the ORDER (parameter sets ahead of the
slices, which is what makes the keyframe self-contained). The literals were
captured from the pre-splice implementation's output.
### `keyframe_param_reassert_does_not_reallocate_the_frame`
MEASURED: the keyframe parameter-set re-assert must be spliced into the
front of the already-assembled access unit IN PLACE, not built as a fresh
full-size buffer.
It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-hundred-KB allocation — memcpy'd the
whole frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a UHD title is
~200,000 frames of 150-400 KB with a keyframe every 1-2 s, i.e. thousands of
avoidable multi-hundred-KB copies per title, each large enough to go through
mmap. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates; this
counts the reallocations that happen, which must be zero. Mirrors the HEVC
parser's test of the same name.
### `avcc_profile_244_appends_extension_bytes`
ISO 14496-15 §5.3.3.1.2 regression: profile_idc=244 (High 4:4:4 Predictive)
ALSO mandates the chroma/bit-depth extension. It was missing from
HIGH_PROFILES, so a 244 stream took the Baseline/Main path and emitted an
avcC with NO extension bytes — non-conforming, and strict parsers then
assume 8-bit 4:2:0. The extension must be appended.
### `sps_reader_read_ue_thirty_one_leading_zeros_is_still_valid`
`SpsReader::read_ue`'s truncation guard is `leading_zeros > 31`: 31 leading
zero bits is the longest legal code and must decode, not abort. Mirrors the
equivalent guard in the shared `BitReader` in `startcode.rs`, kept here
because `SpsReader` is a separate, unshared implementation used only for SPS
parsing.
+16
View File
@@ -0,0 +1,16 @@
# halt
## Why `Ordering::Relaxed`
`Ordering::Relaxed` is sufficient on both load and store because this flag
is purely advisory — no other memory operations piggyback on it for
happens-before ordering. Callers that need to publish data across threads
do so via channels or other synchronization, not via this bit.
## Why 250ms for `POLL_INTERVAL`
250 ms is the sweet spot between responsiveness (operator presses Stop,
sees it take effect within ~quarter-second) and waste (atomic load + clock
read is cheap but not free at thousands of hertz). It's centralised here
so the half-dozen halt-polling loops across `io` can't drift apart
silently.
+35
View File
@@ -0,0 +1,35 @@
# Seeded robustness harness (`src/harness.rs`)
Every parser reached from here takes bytes that came off a disc, and this
crate's primary boundary is that the disc is untrusted: a malformed, damaged
or hostile image must never crash the library. These tests assert exactly
that one property — the parser returns `Ok` or `Err`, and never panics.
## Why this exists rather than `cargo-fuzz`
`cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
for libFuzzer's coverage feedback) and this project pins stable. So the
generator lives here instead. It gives up coverage-guided mutation — the real
loss — and keeps everything else: millions of cases, structure-aware input,
and a crash corpus. It also gains determinism, which a fuzzer does not have:
the same seed replays the same cases on any machine.
## Why no `proptest` or `arbitrary`
This crate has exactly one dev-dependency. That is a deliberate posture, and
a randomness crate is not worth ten transitive dependencies when the parsers
take plain `&[u8]` and a good enough generator is forty lines.
## Budget
`FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
enough that the per-commit gate stays under a second). The overnight run sets
it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
fixed so a failure in CI reproduces locally verbatim.
## On failure
The panic message carries the seed, generator and case index. Re-run with
`FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
into `tests/corpus/` as a permanent regression fixture — discovery happens
here, defence happens there.
+465
View File
@@ -0,0 +1,465 @@
# HD-DVD title scanning (`src/disc/hddvd.rs`)
HD-DVD is a **tree-level peer** of DVD and Blu-ray (not a stream variant like
FMTS): its content lives in `HVDVD_TS/` as `.evo` clips — Enhanced VOB, an
MPEG **program** stream — each with a small `.map` timemap sidecar, navigated
by `.xpl`/`.ifo` playlists in `HVDVD_TS/` and `ADV_OBJ/`. Because it is a
different tree with a different playlist format, it gets its OWN scanner
(this file), a peer to `Disc::scan_bluray_titles` — the two-format design
rule: a genuinely different format is a new enumerator, not an extension
bolted into the BD path.
## Title composition (authoritative path)
Title composition is authoritative, from the Advanced-Content playlist. HD-DVD
ships a real player playlist at `ADV_OBJ/VPLST000.XPL` (DVD-Forum
`HDDVDVideo/Playlist` XML). The scanner parses it (with a real XML parser,
`roxmltree`) into one `DiscTitle` per `<Title>`: its `<PrimaryAudioVideoClip>`
clips in playback order (each an EVO, referenced via its `.MAP` sidecar), the
`titleDuration`, the `displayName`, and the `<ChapterList>`. A layer-break
split (`FEATURE_1` + `FEATURE_2`, or `feature`/`feature_Divide`) is composed
into ONE title with the two parts as clips, carrying each clip's title-time
in/out points (45 kHz ticks) so a seamless join can be spliced onto one
timeline. Container is `ContentFormat::MpegPs`, so the existing PS mux path
handles it. Per-clip streams are enumerated by demuxing the clip head and
building one `Stream` per distinct elementary stream (video + DD+ audio),
codec sniffed from the ES bytes.
## Fallback path
When no playlist is present (or it fails to parse), the scanner falls back to
the older clip-name heuristic: parse the `HVA*.VTI` clip table, join the
`feature*`-named clips into one title, and emit every other clip on its own.
## Known gaps
Not parsed yet: subtitles (8-bit RLC on `0xBD` sub `0x20..=0x3F`) and per-track
audio languages (the XPL carries `<Audio description=...>` but they are not yet
wired onto the streams). Extents and size are real (the ripper images the
clips).
## VTI parsing
`parse_vti_clip_order` parses the `ADVANCED-VTS` VTI clip-name table. The
table is a run of `VTI_CLIP_ENTRY_STRIDE`-spaced records, each carrying a
NUL-terminated `<name>.EVO`. Rather than trust the (imprecise) header pointer,
it collects every NUL-terminated `*.EVO` name and keeps the largest group
sharing one residue modulo the stride — the clip table — in offset order.
Returns empty for a non-VTI blob or one with no recognizable table.
## Codec sniffing
`sniff_video_codec` reads MPEG / Annex-B start codes: `00 00 01 B3` → MPEG-2
(sequence_header), `00 00 01 0F` → VC-1 (BD/HD-DVD sequence-header BDU),
`00 00 01 [x7]` H.264 SPS NAL (type 7, forbidden_zero_bit clear) → H.264.
Returns `None` when no recognizable start code is present. The scan prefers
the unambiguous MPEG-2 / VC-1 sequence headers; H.264 is inferred from an SPS
NAL so a stray slice/picture code can't be mistaken for a different codec.
`sniff_audio_codec` recognizes only Dolby Digital Plus (E-AC-3) today, via its
`0x0B77` syncword — what real retail HD-DVD titles carry on sub-ids
`0xC0..=0xC7`. Returns `None` for an unrecognized sample so the caller drops
the stream rather than mislabeling it.
## probe_evo_streams
Demuxes the head of an `.evo` clip (through the disc's `SectorSource`) and
builds one `Stream` per distinct elementary stream found: the video track
(mapped to the canonical `DVD_VIDEO_PID` for plain 0xE0-0xEF video, or the
extended-stream-id PID for HD-DVD VC-1) and every DD+ audio sub-stream
(mapped via `dvd_audio_pid`). Codec is sniffed from the demuxed ES bytes.
Mirrors the stream construction in `Disc::scan_dvd_titles`; resolution /
language / channels use sane HD-DVD defaults (the muxer reads the true pixel
dimensions from the H.264 SPS, and E-AC-3 channel counts are not decoded
here). Returns an empty vec when the clip cannot be read or carries no
recognizable stream (e.g. an AACS-encrypted clip probed as ciphertext).
Cancellation is checked before every chunk read and is the ONE condition that
returns `Err` rather than an empty vec: a probe cut short by a Stop has not
established that the clip carries no streams, and reporting it as if it had
would enumerate a stream-less title as a scanned fact. A read that fails for
any other reason keeps the existing best-effort behaviour.
## collect_es
VC-1 video rides extended-stream-id `0xFD` with `stream_id_extension` `0x55`.
HD audio (MLP/TrueHD) can also use `0xFD` with other extensions — routing
those to their own audio tracks is deferred (see the HD-DVD program-chain
follow-up); until then only the VC-1 extension is treated as video, so an
audio `0xFD` sub-stream can never mis-stamp the video PID.
## XPL amplification bounds
The XPL parser guards against a crafted `ADV_OBJ/VPLST000.XPL` amplifying a
small file into an enormous amount of work or memory. Four separate axes are
bounded, each independently, because they are separate attacks:
- **`MAX_XPL_DEPTH` (32)** bounds element NESTING. A real `VPLST000.XPL` is
about six levels deep at its deepest
(`Playlist`/`TitleSet`/`Title`/`PrimaryAudioVideoClip`/`Video`, or
`.../ChapterList/Chapter`); 32 leaves a 5x margin for authoring tools that
wrap extra `ApplicationSegment`/`ObjectMappingList` layers, while staying
trivially stack-safe.
- **`MAX_XPL_TITLES` (512)** bounds how many `<Title>` elements one playlist
may declare — BREADTH, separate from the depth guard: a flat playlist
passes the depth check trivially and can still declare a title per 51
bytes, so a 64 MiB `VPLST000.XPL` (exactly `udf::MAX_FILE_BYTES`) reaches
~1.3 million of them. This matters here and not on the Blu-ray side because
of where the count comes from: `bluray.rs` learns its playlists from the
MPLS files actually present in the UDF directory, so the medium bounds it —
an XPL DECLARES its titles, so nothing does. The cost is drive time, not
memory: `compose_xpl_titles` runs `probe_evo_streams` once per title, and
each probe reads real sectors, so an uncapped count turns a `Disc::scan()`
the operator expects to take seconds into tens of terabytes of optical
reads on a disc that is only pretending to be large. The bound that matters
is `cap x EVO_PROBE_SECTORS`, since `compose_xpl_titles` memoizes on the
title's RESOLVED extent list, and distinct titles built from different
combinations of the same clips have different lists, so each one misses the
memo — at 4096 that product was 64 GiB of optical reads, the same worst
case, and the same contradiction with "seconds", as the sibling
`MAX_HDDVD_CLIPS` path. 512 keeps it at 8 GiB and remains far above any
real disc (retail HD-DVDs carry tens of titles).
- **`MAX_HDDVD_CLIPS` (512)** caps `.evo` clips taken from the `HVDVD_TS/`
directory listing. `MAX_XPL_TITLES` and `xpl_depth_within_limit` bound the
`ADV_OBJ/*.XPL` path, but a crafted disc simply omits `/ADV_OBJ`; control
then reaches the clip-name fallback in `Disc::scan_hddvd_titles`, which
emits one title per directory entry and probes each — the same unbounded-
probe cost, reached without the playlist. The directory is the only bound
the medium supplies, and `udf::MAX_DIR_BYTES` (1 MiB) still leaves room for
~24,000 FIDs. Memoizing the probe (`EvoProbeCache`) collapses entries that
resolve to the SAME extents, closing the many-names-one-File-Entry case —
but not this one: a File Entry is a single sector, so an attacker can give
every FID its own, each declaring its own extent, for ~48 MiB of image;
every probe then misses the memo and costs a full `EVO_PROBE_SECTORS`
(16 MiB) read, hundreds of GiB on a scan the operator expects to take
seconds. The cap's job is to bound `cap x EVO_PROBE_SECTORS`; it was 4096
(matching `MAX_XPL_TITLES`, which bounds a different, no-per-item-read-cost
path) — 4096 x 16 MiB is 64 GiB, more than dual-layer HD-DVD media
physically holds (reachable anyway, since distinct extent lists may overlap
on the medium). 512 keeps the product at 8 GiB and still leaves a wide
margin over any real disc (retail HD-DVDs carry TENS of `.evo` clips, ~10x
the real-world maximum, the sizing convention `MAX_XPL_CHAPTERS_PER_TITLE`
also uses against the 99-chapter authoring ceiling). Pinned by
`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`, which
asserts the PRODUCT rather than either constant, so raising one without
re-examining the other fails.
- **`MAX_XPL_CLIPS_PER_TITLE` (256)** bounds how many
`<PrimaryAudioVideoClip>` elements a SINGLE `<Title>` may contribute — the
fourth amplification axis, the one the other three guards leave open. A
64 MiB XPL holds ~1.7 million `<PrimaryAudioVideoClip src="A.EVO"/>`
elements at ~38 bytes each, and the collector is `descendants()`, not
`children()`, so a clip nested inside N ancestor `<Title>` elements is
collected by EVERY one of them — the depth cap still permits 32 such
ancestors (measured: a bare `<Title>` root nests 32 deep and still parses,
because the guard counts the self-closing clip element as non-nesting).
The product is ~1.7M x 32 ≈ 56 million heap-allocated `XplClip`s, each with
its own `String`, and then as many `Clip`s again in `compose_xpl_titles`.
Unlike the title cap, the cost here is MEMORY rather than drive time:
`compose_xpl_titles` de-duplicates by `.evo` name before probing
(`seen_evos`), so repeats cost no extra sector reads, but the `Vec`s are
built and held regardless. 256 is far above any real disc — a feature is
one clip, or two across a layer break; even seamless-branching/multi-angle
is tens — and keeps the aggregate bounded at `MAX_XPL_TITLES` x 256 clips
rather than the tens of millions above.
- **`MAX_XPL_CHAPTERS_PER_TITLE` (1024)** bounds `<Chapter>` elements per
`<Title>` — the same `descendants()` amplification, on the second unbounded
`collect()` in the same loop, bounded separately so capping clips doesn't
simply move the attack next door. A chapter is a bare `f64` rather than a
`String`-carrying struct so it's cheaper, but a
`<Chapter titleTimeBegin="…"/>` element is also smaller, so the element
count an XPL can reach is comparable. 1024 is far above any real disc: the
DVD/HD-DVD authoring convention tops out at 99 chapters per title, a 10x
margin.
## scan_hddvd_titles
Scans HD-DVD titles from the `HVDVD_TS/` `.evo` clips. The main feature is
authored as one or more `.evo` clips (a layer-break split —
`FEATURE_1`/`FEATURE_2` or `feature`/`feature_Divide`). The `HVA*.VTI`
navigation file names every clip in authored order; this parses it to
concatenate the feature clips into ONE title (so the largest-title pick gets
the whole movie, not just part 1), emitting every other clip as its own
title. Falls back to one title per clip when the VTI is absent or
unparseable, so a disc with no readable navigation still enumerates.
`chapters`/duration are left empty pending deeper VTI parsing.
Cancellation: `halt` is polled once per clip and again before every probe
chunk, and a read that fails with `Error::Halted` — how a live drive reports
a Stop, since `Drive::checked_exec` fails every command once its flag is set
— is propagated rather than swallowed. `Halted` is the only error this
returns; every other read failure keeps its best-effort behaviour. It has to
be an error and not a short title list: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it.
## xpl_depth_within_limit
Rejects a disc-supplied XML blob whose element nesting exceeds
`MAX_XPL_DEPTH`, BEFORE it reaches the parser. `roxmltree` is
recursive-descent and its depth-10 limit applies only to ENTITY expansion, so
element nesting is unbounded: a few hundred KB of well-formed XML (far under
the read cap) overflows the thread stack. That is a process ABORT, not an
`Err` and not an unwind, so neither the `Document::parse` fallback nor
`catch_unwind` can contain it; the only fix is to not hand the document over
at all. A byte-size cap is deliberately NOT added: file size is already
bounded by the UDF read path, and a size cap does not close this class (a
180 KB document already aborts) — depth is the bound that matters.
This is a conservative scan, not a validator: it tracks `<name …>` /
`</name>` while skipping comments, CDATA, processing instructions and
declarations, and treats `<… />` as non-nesting. A miscount can only cost a
pathological document its fast path, and the failure mode is the existing
one — fall back to the clip-name heuristic.
## Test rationale
Detailed rationale for tests in `src/disc/hddvd.rs`'s `#[cfg(test)]` module,
kept here so the in-code doc can stay within the internal-comment cap.
**`scan_hddvd_feature_composition_saturates_absurd_disc_declared_sizes`** —
The SECOND size accumulator. `compose_xpl_titles` sums an XPL's clip sizes;
this one sums the composed FEATURE title's parts, and it reads the same
disc-declared `u64` Information Length from the same `clip_extents` map. A
split feature whose two parts each declare a size near `u64::MAX` overflows
it exactly as the XPL path did. Nothing reconciles the declared size against
the extents that back it — here each part declares `u64::MAX` while
occupying ten sectors — because the allocation descriptor's length field is
only 32 bits wide, so the disc can always claim more than it holds.
Mutation: restore `size_bytes += *size` in the feature-composition path and
this goes red (debug: attempt to add with overflow).
**`scan_hddvd_does_not_compose_a_feature_over_an_unrecorded_part`** — A split
feature (FEATURE_1 + FEATURE_2) where ONE part carries an unrecorded extent
must not be composed into a FEATURE title at all. Composing around the
unusable part is the dangerous outcome: the title would play through as if
it were the whole movie while silently missing that part's runtime, and its
size still counts it. The other clips remain available as their own titles
— refusing the composition is not the same as refusing the disc.
**`scan_hddvd_logs_an_unrecorded_feature_part_with_its_own_code`** — The
refusal above must also be ACCOUNTED, with the unrecorded extent's own code.
This warn site carried a hardcoded `code = 6017` literal — in the very file
whose sibling arm was changed to stop doing exactly that — and nothing
tested it, so neither the literal nor its absence broke anything. Refusing a
feature and saying nothing leaves the operator with a disc that scanned
"fine" and is quietly missing its main title. Mutations: deleting the
`tracing::warn!` (no event to find); restoring the literal `6017` still
passes here BY VALUE, which is the point — what is pinned is
`E_UDF_UNRECORDED_EXTENT`'s value reaching the log, so a renumbering that
the literal could not follow goes red; logging a neighbouring code (e.g.
`E_UDF_NO_USABLE_EXTENT`) goes red immediately.
**`scan_hddvd_does_not_compose_a_feature_over_a_part_with_no_usable_extent`**
— The `Ok`-but-empty twin of the test above, and the one that actually bites
on an ordinary disc: a feature part whose `file_extents` call SUCCEEDS and
yields nothing usable — here a zero-byte `FEATURE_2.EVO`, no crafted ICB
required — must refuse the composition too, and must say so in the log.
Before this, the empty-`Ok` clip entered neither `clip_extents` nor
`unusable`, so the `any(|n| unusable.contains(..))` guard missed it and the
`filter(|n| clip_extents.contains_key(..))` beside it quietly DELETED the
part: the scan composed a `FEATURE` title out of part one alone, still
advertising itself as the feature, at rc=0, with no diagnostic anywhere.
Half a movie presented as a whole one — strictly worse than the `Err` case
the guard was built for, and reachable without a hostile disc. Mutations
this catches, all of which the pre-fix code exhibited: dropping the
`extents.is_empty()` branch entirely (a `FEATURE` title appears with one
extent); inserting into `clip_extents` anyway (same); marking the clip
unusable but NOT logging (the log assertion fails — absence of a log is
itself the defect); logging the neighbouring `E_UDF_UNRECORDED_EXTENT`
instead of `E_UDF_NO_USABLE_EXTENT` (the code assertion fails, because a
zero-length file is not an authoring hole and triaging the two together
sends whoever reads it at the wrong population).
**`parse_xpl_titles_caps_a_playlist_declaring_absurdly_many_titles`** — A
disc-supplied playlist nested far deeper than any real one must be REFUSED
BEFORE the XML parser sees it. `roxmltree` is recursive-descent and its
depth-10 limit covers only ENTITY expansion, so unbounded element nesting
blows the thread stack — an ABORT, which no `catch_unwind` and no `Err`
fallback can contain. ~50k levels is a few hundred KB of well-formed XML,
far under the read cap. The correct outcome is the same as any other
unusable playlist: an empty `Vec`, so the caller falls back to the clip-name
heuristic. Separately, this test also exercises BREADTH, not depth: a flat
playlist passes the nesting guard trivially and can still declare a title
per ~51 bytes, so a 64 MiB XPL (exactly `udf::MAX_FILE_BYTES`) reaches ~1.3
million of them. The cost is drive time, not memory: `compose_xpl_titles`
runs `probe_evo_streams` once per title and each probe reads real sectors,
so an uncapped count turns a scan the operator expects to take seconds into
tens of terabytes of optical reads. `bluray.rs` is not exposed to this
because its playlist count comes from the MPLS files present in the UDF
directory; an XPL DECLARES its titles, so nothing bounds them. Mutation:
delete the `.take(MAX_XPL_TITLES)` and this goes red at the declared count.
**`nested_title_xpl`** — Builds a playlist that nests `nesting` `<Title>`
elements inside one another and puts `clips` self-closing
`<PrimaryAudioVideoClip>` elements at the innermost level, plus `chapters`
`<Chapter>` elements. `parse_xpl_titles` collects a title's clips with
`descendants()`, not `children()`, so EVERY one of the `nesting` ancestors
collects the SAME innermost clip list — the amplification this exercises.
**`parse_xpl_titles_caps_clips_per_title_against_descendant_amplification`**
— CLIPS PER TITLE is the axis the title cap, the depth cap and the
directory cap all leave open. `MAX_XPL_TITLES` bounds how many titles a
playlist may declare and `MAX_XPL_DEPTH` bounds how deeply it may nest, but
NEITHER bounds how many `<PrimaryAudioVideoClip>` elements a single title
collects. Worse, the collector is `descendants()`, so one clip nested inside
N ancestor `<Title>`s is collected N times over — the clip count MULTIPLIES
by the nesting the depth guard still permits. A 64 MiB XPL
(`udf::MAX_FILE_BYTES`) holds ~1.7M clip elements at ~38 bytes each; with
the 32 `<Title>` ancestors the depth cap still permits, that is ~56 million
heap-allocated `XplClip`s, and then as many `Clip`s again in
`compose_xpl_titles`. This test uses small numbers that exercise the same
multiplication without a 64 MiB fixture: pre-fix it collected 756 clips per
title, 18,900 across the 25 nested titles, from a document declaring only
756 clip elements — an exact 25x, the nesting count. Mutation: delete the
`clips.len() >= MAX_XPL_CLIPS_PER_TITLE` break and this goes red at the
unamplified per-title count.
**`parse_xpl_titles_caps_chapters_per_title`** — CHAPTERS PER TITLE is the
same `descendants()` amplification on the second unbounded `collect()` in
the same loop. Bounded separately so closing the clip axis does not simply
move the attack next door. Mutation: delete the
`.take(MAX_XPL_CHAPTERS_PER_TITLE)` and this goes red at the declared count.
**`parse_xpl_titles_keeps_every_clip_of_a_realistic_title`** — The control: a
realistic multi-clip title still resolves EVERY one of its clips and
chapters. `SYNTH_XPL`'s main movie is a layer-break split — two clips on one
timeline — plus two chapters; losing either to the cap would cost a genuine
disc half its feature. Mutation: set `MAX_XPL_CLIPS_PER_TITLE` to 1 and this
goes red.
**`parse_xpl_titles_accepts_real_world_nesting_depth`** — The depth guard
must not reject real discs. A genuine `VPLST000.XPL` is a handful of levels
deep; this is the same document the parsing test uses, plus self-closing
tags, a comment and a processing instruction that the pre-parse scanner has
to account for without inflating its depth count.
**`scan_hddvd_composes_titles_from_xpl_playlist`** — The authoritative path:
when a VPLST000.XPL is present, titles come from the playlist — the
layer-break split (FEATURE_1 + FEATURE_2) is composed into ONE title with
the real duration, name, chapters, and per-clip title-time offsets — not the
clip-name heuristic.
**`compose_xpl_titles_saturates_absurd_disc_declared_clip_sizes`** — A
title's `size_bytes` is a running sum of DISC-DECLARED clip sizes: the `u64`
UDF File Entry Information Length, which nothing cross-checks against the
extent list that file actually occupies. Two clips can therefore each
declare a size near `u64::MAX`. Summed with a plain `+=`, that overflows: a
panic in a debug build — a crash on untrusted disc content, which this
library must never do — and a wrap to a small garbage total in release,
which misreports the title's size while looking entirely normal. Mutation:
restore `size_bytes += *size` in `compose_xpl_titles` and this goes red
(debug: attempt to add with overflow).
**`compose_xpl_titles_dedups_repeated_clip_references_by_evo`** — A crafted
Advanced-Content playlist can name the SAME `.evo` in
`<PrimaryAudioVideoClip>` many times over (the XML has no fixed element
count, unlike MPLS's binary PlayItem count). Each repeat must NOT push
another copy of that clip's extent list onto the title — mirroring
bluray.rs's `first_ref = seen_clips.insert(...)` gate, which pushes a clip's
extents only the first time its id is seen. Here the analogous key is the
`.evo` filename (this file has no clip_id; `.evo` is what `clip_extents` is
keyed by). Without the gate, extents grows by the referenced clip's full
extent list on EVERY repetition — unbounded by the number of on-disc files,
bounded only by playlist size.
**`scan_hddvd_titles_drops_a_split_feature_whose_part_cannot_be_read`** — A
clip whose extents cannot be resolved must drop the SPLIT FEATURE. Catches
reverting the `Err(e) => { warn; unusable.insert(..) }` arm in
`Disc::scan_hddvd_titles` back to the bare `Err(_) => {}` it replaced. RED
BEFORE GREEN, and TWO earlier attempts at this test did NOT go red. The
first asserted "no title names the broken clip" — that passes either way,
because a clip that resolves to no extents is never inserted into
`clip_extents` and so yields no per-clip title regardless. The second fixed
that but shipped no `.vti`: `order` is built solely from
`parse_vti_clip_order` of the navigation file, so with no `.vti` it is
EMPTY, `feature` is empty, and no composed title is ever built — the
assertion held vacuously with the fix reverted. Hence the synthetic
`HVA00001.VTI` fixture: it is what makes the composer run at all. The
defect lives one level up from the per-clip titles, in the composed
feature: `unusable` is what tells the composer that a part is MISSING
rather than merely absent, and the old bare `Err(_) => {}` populated it for
nothing but an unrecorded extent. So a scratched sector under FEATURE_2's
ICB (`Error::DiscRead`) left FEATURE_1 composing a title named "FEATURE" by
itself — half a movie offered as the whole one.
**`scan_hddvd_titles_excludes_a_clip_with_zero_sectors`** — A clip whose
file has a zero-byte size (a degenerate/empty allocation: its ICB's
allocation descriptor has `data_len == 0`, the UDF AD-list terminator, so
`file_extents` yields no extent at all) must not produce a title. NOTE: this
exercises the *upstream* `data_len == 0` terminator path in
`crate::udf::UdfFs::file_extents`, not the `sectors > 0 && lba > 0` guard in
`scan_hddvd_titles` itself — with this fixture (`file_extents` never returns
a `(lba, 0)` tuple, and `PART_START` unconditionally makes every resolved
`lba` positive) that guard is unreachable in a divergent way; kept here as a
regression check on the zero-byte-file behavior in its own right.
**`scan_hddvd_titles_probes_once_for_entries_resolving_to_identical_extents`**
— THE THIRD AMPLIFICATION AXIS on HD-DVD title composition, past both
existing guards. `xpl_depth_within_limit` bounds XML NESTING;
`MAX_XPL_TITLES` + the `.evo` de-dup bound what an XPL can DECLARE. Both
live on the playlist path — and an attacker simply omits `/ADV_OBJ`, so
`read_adv_obj_xpl` returns `None` and neither guard is ever consulted.
Control then reaches the clip-name FALLBACK, which emits one title per
`.evo` DIRECTORY ENTRY and probes each. Nothing de-duplicates a FID's ICB
LBA, so every name in a 1 MiB directory (`udf::MAX_DIR_BYTES`, ~24,000 FIDs)
can point at ONE File Entry. Each resolves to the SAME extents and each
costs a full `EVO_PROBE_SECTORS` (16 MiB) probe: ~375 GiB of optical reads
from a directory that describes a single file. Memoizing the probe on the
resolved extent list collapses that to ONE probe, because all of those
names resolve to one extent list. Mutation: drop the memo lookup in
`EvoProbeCache::streams` (always probe) and this goes red at `ENTRIES`
probes.
**`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`** —
The clip cap and the per-probe read budget are ONE bound, not two, and only
their product is meaningful: `MAX_HDDVD_CLIPS` probes each cost up to
`EVO_PROBE_SECTORS`, so the pair fixes the worst-case drive time of a
`Disc::scan()`. `EvoProbeCache` does not help against a crafted disc here:
it keys on the RESOLVED extent list, and distinct-but-overlapping extent
lists all reading one physical region miss the memo every time while every
read succeeds. So the product is genuinely reachable, and it is the number
that has to stay within what a scan can absorb. 8 GiB is the ceiling
asserted here. It is far above any genuine disc — retail HD-DVDs carry tens
of `.evo` clips, i.e. well under 1 GiB of probing — and far below the 64 GiB
the pair used to permit, which exceeded the capacity of the dual-layer media
this format tops out at and took tens of minutes of optical reads. This is
a relationship, not a magic number: raising EITHER constant without
re-examining the other trips it.
**`scan_hddvd_titles_still_probes_each_distinct_clip`** — CONTROL:
memoization must not silently collapse DISTINCT clips. A legitimate disc
with several different `.evo` files still probes each one, and each title
keeps the streams of ITS OWN clip. Mutation: make `EvoProbeCache`'s key a
constant (ignore the extents) and this goes red — the junk clip inherits
the first clip's H.264 streams.
**`scan_hddvd_titles_caps_a_directory_declaring_absurdly_many_clips`** —
Memoization alone is NOT the whole fix. Collapsing identical extent lists
bounds the one-File-Entry attack, but distinct extent lists are only as
bounded as the File Entries an attacker cares to lay down — and a File
Entry is ONE sector. A 1 MiB directory's ~24,000 FIDs can each point at
their own 1-sector File Entry (a ~48 MiB image) declaring its own huge
extent, so every probe misses the memo and the amplification is back.
`MAX_HDDVD_CLIPS` is the bound that closes that. Mutation: drop the
`clips.len() < MAX_HDDVD_CLIPS` gate in the directory scan and this goes
red at `ENTRIES`.
## EvoProbeCache
`EvoProbeCache` memoizes `probe_evo_streams`, keyed on the RESOLVED extent
list. Probing is the expensive half of title composition: each pass reads up
to `EVO_PROBE_SECTORS` (16 MiB) off the medium. Both composition paths can
reach the same physical clip many times over — an XPL naming one `.evo` from
many `<Title>`s, or a directory whose FIDs all point at one File Entry — and
a probe is a pure function of the extents it reads, so the second and later
passes over an identical extent list are re-reads of bytes already seen. The
key is the extent list itself, NOT the clip name or title: two names for one
File Entry share a key (one probe), while two genuinely different clips have
different `start_lba`s and so keep their own probes and streams. Cache size
is bounded by the caller's title/clip cap (`MAX_XPL_TITLES` /
`MAX_HDDVD_CLIPS`).
+99
View File
@@ -0,0 +1,99 @@
# HEVC Annex B muxer (`src/mux/hevc/mod.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## Module overview
Writes a raw `.hevc` / `.h265` Annex B byte stream:
`00 00 00 01 | NAL_unit | 00 00 00 01 | NAL_unit | …` with no container
framing. On the first frame the muxer emits the codec_private's VPS, SPS, PPS
(parsed from a `HEVCDecoderConfigurationRecord` in `length-prefixed-in-hvcC`
form), then converts each PES frame's length-prefixed NAL units to Annex B
and writes them.
## `write_frame`
Input may be either:
- Length-prefixed: `[u32-BE len][NAL bytes]` repeated. This is the form
emitted by libfreemkv's HEVC parser (the MKV-native layout). Converted to
Annex B.
- Already Annex B: a buffer beginning with a `00 00 00 01` or `00 00 01`
start code. Passed through unchanged.
## `hvcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §8.3.3.1.2):
- 22-byte fixed header
- byte 22 = `numOfArrays`
- each array: `array_completeness:1 | reserved:1 | NAL_unit_type:6`,
`numNalus:u16-BE`, then `numNalus` × `(nalUnitLength:u16-BE + NAL bytes)`.
We don't filter on NAL type — VPS (32), SPS (33), PPS (34), and any SEI
arrays included in hvcC all get the same Annex B treatment.
## `length_prefixed_to_annex_b`
Already-Annex-B input (a buffer beginning with a `00 00 00 01` or `00 00 01`
start code) is detected up front and passed through unchanged — some
upstream paths (raw HEVC ES from disc) hand Annex B straight through the
PES layer, and a genuine start code would otherwise be misread as a u32-BE
length prefix.
Truncation policy (single source of truth across all muxers): if a length
prefix runs past the end of the buffer (e.g. a NAL truncated by a bad disc
sector), the truncated trailing NAL is dropped and only the valid Annex-B
prefix accumulated so far is emitted. We never emit a half-NAL nor leak raw
length-prefixed bytes into the Annex-B stream.
## `nal_length_size`
Number of octets each NAL length prefix occupies in the elementary data of a
track described by `record` — the `lengthSizeMinusOne + 1` field of the
decoder configuration record (ISO/IEC 14496-15).
- avcC (`AVCDecoderConfigurationRecord`, §5.3.3.1.2): byte 4 is
`bit(6) reserved | unsigned int(2) lengthSizeMinusOne`.
- hvcC (`HEVCDecoderConfigurationRecord`, §8.3.3.1.2): byte 21 is
`constantFrameRate(2) | numTemporalLayers(3) | temporalIdNested(1) |
lengthSizeMinusOne(2)`.
The spec permits only 1, 2 or 4 octets (`lengthSizeMinusOne` of 0, 1 or 3);
a declared 3 is non-conformant but is decoded rather than rejected, since
reading N octets is the same operation for every N. A record too short to
carry the field, or a codec with no such record, falls back to
`DEFAULT_NAL_LENGTH_SIZE` — the width every freemkv parser emits.
This exists because assuming 4 is silent corruption for a legal source:
reading a 2-octet-prefixed frame as u32-BE yields an absurd first length,
the conversion loop bails with nothing parsed, and the raw length-prefixed
bytes are passed through as though they were already Annex B — a stream
with no start codes at all, and no error anywhere.
## `append_length_prefixed_as_annex_b`
Same conversion as `length_prefixed_to_annex_b` but writes directly into a
caller-owned buffer, avoiding an intermediate allocation on hot paths (e.g.
per-frame video muxing). If `data` doesn't parse as length-prefixed (no
NALs extracted), it's appended unchanged on the assumption it's already
Annex B.
## `append_length_prefixed_as_annex_b_sized`
For a source whose NAL length prefixes are `length_size` octets wide rather
than the 4 this crate's own parsers emit. Derive `length_size` from the
track's configuration record with `nal_length_size` — ISO/IEC 14496-15 lets
a legal avcC/hvcC declare 1 or 2 octet prefixes, and reading those as
u32-BE mangles the frame.
`length_size` outside `1..=4` is clamped to `DEFAULT_NAL_LENGTH_SIZE`; the
field it comes from is 2 bits wide, so that is unreachable from real input.
## `avcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §5.3.3.1.2):
- 5-byte fixed header
- byte 5 = `[reserved:3 | numOfSequenceParameterSets:5]`
- `numOfSPS` × `(sequenceParameterSetLength:u16-BE + SPS bytes)`
- 1 byte = `numOfPictureParameterSets`
- `numOfPPS` × `(pictureParameterSetLength:u16-BE + PPS bytes)`
+265
View File
@@ -0,0 +1,265 @@
# HEVC parser (`src/mux/codec/hevc.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## `hevc_num_extra_slice_header_bits`
`num_extra_slice_header_bits` from a HEVC PPS NAL (H.265 §7.3.2.3): after the
2-byte NAL header, skip `pps_pic_parameter_set_id` + `pps_seq_parameter_set_id`
(both `ue(v)`) and `dependent_slice_segments_enabled_flag` +
`output_flag_present_flag` (`u(1)` each), then read `u(3)`. `None` if the PPS
is too short to parse — the caller then declines to guess a slice type.
## `hevc_first_slice_coding_type`
Measures the coding type from the FIRST coded slice of an access unit
(H.265 §7.3.6.1 `slice_segment_header`). Reads only the leading fields of the
first slice segment: `first_slice_segment_in_pic_flag` u(1), the IRAP
`no_output_of_prior_pics_flag` u(1), `slice_pic_parameter_set_id` ue(v), the
`num_extra_slice_header_bits` reserved bits, then `slice_type` ue(v). Returns
`None` for a non-first slice or on truncation — never a guess. `num_extra`
MUST come from the active PPS so the bit offset to `slice_type` is exact.
## `HevcParser::hdr10`
Combines the accumulated mastering-display and content-light SEI into a
complete `Hdr10Metadata`, or `None` until BOTH HDR10 SEI messages have been
seen. Requiring both means an SDR / partially-signalled stream never emits a
half-populated (confidently-wrong) HDR10 record.
## `HevcParser::scan_sei`
Scans an SEI NAL (`[2-byte NAL header][RBSP]`) for the two HDR10 payload
types and captures each the FIRST time it appears (per-stream constants).
RBSP structure (Rec. ITU-T H.265 D.2 `sei_rbsp` / `sei_message`): a sequence
of messages, each `payloadType` then `payloadSize` encoded as a run of 0xFF
bytes plus a final <0xFF byte (the "ff-extension" coding), followed by
`payloadSize` payload bytes. Emulation-prevention (00 00 03) is stripped
before reading — unlike a slice header, an SEI payload can be deep enough
that an emulation byte falls inside the fields we read. Unknown payload
types are skipped by their size so a later HDR10 message in the same NAL is
still reached.
## `HevcParser::mark_clip_boundary`
MPLS `connection_condition` 0x05 and 0x06 are the non-seamless values (per
the BD-ROM spec: 0x01 = first item / seamless, 0x05/0x06 = non-seamless).
The first CRA at/after this call is rewritten CRA_NUT (21) → BLA_W_LP (16)
so a linear decoder sets NoRaslOutput and discards the now-dangling RASL
leading pictures with no "could not find ref" error.
It is a no-op for the rewrite unless a CRA actually follows: an
IDR/IDR_W_RADL boundary needs no fix (it carries no cross-splice
references), and the flag is cleared by the first IRAP-class NAL it reaches.
SAFETY: never call this for connection_condition 0x01 (seamless/first item)
or within a single-clip title — doing so could convert a legitimate
mid-content CRA to BLA. The default (never called) path leaves output
byte-identical.
## `handle_param_set`
Handles a VPS/SPS/PPS NAL: decides whether to strip it (the decoder already
has the value) or emit it in-band, and tracks the currently-active body.
The decision MUST be made against the currently-active set (`cur`), NOT the
codecPrivate copy (`first`). The two player behaviours for hvcC-in-MKV
diverge exactly here:
- A *seek-capable / Annex-B* player (one that converts hvcC to Annex-B by
inserting the parameter sets) re-applies the hvcC sets at every keyframe.
`reassert_active` handles it.
- A *streaming* decode (a decoder consuming the MKV directly — what most
integrity checkers do) applies hvcC ONCE at init and thereafter updates a
parameter set ONLY from an in-band NAL.
So when a title redefines a set mid-stream (id 0 body A → B) and later
switches BACK to A (== codecPrivate), the change to A must STILL be emitted
in-band: the streaming decoder is sitting on B and will never revert
otherwise, decoding the whole A-segment against B → CABAC/cu_qp_delta
desync. Stripping on `== first` (the old behaviour) dropped exactly that
revert and corrupted every "switch back to the first body" segment.
Rules:
- First of its type → seeds codecPrivate; stripped (the decoder gets it from
hvcC at init).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change, in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active VPS/SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Why unconditional (not only when the active set differs from codecPrivate):
a streaming decoder applies the hvcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IRAPs
even though its body is unchanged; if the decoder then drops it (a CRA reset
or SPS event), nothing re-sends it and every subsequent slice fails with
"PPS id out of range" until the next genuine change (observed as a ~24 min
corrupt band on one dual-layer UHD title). Re-asserting the active set at
EVERY keyframe — what compliant Matroska muxers do at every IRAP — makes
streaming decode self-healing. Re-sending an identical param set is benign
(decoders expect it at IRAPs); cost is a few hundred bytes per keyframe.
This strictly supersets the earlier change-only re-assert, so the
param-set-revert fix is unaffected.
## `parse_mastering_display`
Parses a Mastering Display Colour Volume SEI payload (Rec. ITU-T H.265
D.2.28 / semantics D.3.28). Layout — 24 bytes total, all big-endian:
`display_primaries_x[c]` u(16), `display_primaries_y[c]` u(16) for c=0,1,2
(SEI primary order is c=0 Green, c=1 Blue, c=2 Red); white_point_x u(16),
white_point_y u(16); max_display_mastering_luminance u(32);
min_display_mastering_luminance u(32). Returns `None` if the payload is
shorter than 24 bytes (malformed → ignored, never partially populated).
## Test notes
**`hevc_hdr10_sei_keeps_the_first_value_and_ignores_later_repeats`** — the
`scan_sei` match arms are guarded with `self.sei_mastering.is_none()` /
`self.sei_content_light.is_none()`, so the FIRST mastering-display and
content-light SEI a title carries wins and later repeats (every AU of a real
HDR10 stream repeats both) are ignored. Each AU below carries only ONE of
the two messages, so the whole-scan early return
(`if self.sei_mastering.is_some() && self.sei_content_light.is_some()`)
never fires early and the per-arm guards are actually exercised.
**`sei_nal` (test helper)** — wraps one or more SEI messages in a prefix-SEI
NAL (type 39) preceded by an Annex-B start code. The assembled message bytes
are emulation-prevented (as a conforming encoder would) so they never form a
false start code; the 0x80 RBSP trailing-bits byte is appended.
**`scan_sei_stops_copying_once_both_hdr10_messages_are_captured`** — MEASURED,
not reasoned: an HDR10 stream carries a prefix SEI per access unit, and
`scan_sei` used to allocate + byte-copy the whole SEI RBSP through
`strip_emulation_prevention` on EVERY one, including after both HDR10
messages were already captured. On a ~200,000-frame UHD title that is
~200,000 wasted allocations/copies. Counted at the single
`strip_emulation_prevention` call site.
**`nonzero_num_extra_slice_header_bits_shifts_the_slice_type_offset`** —
`num_extra_slice_header_bits` (H.265 §7.3.2.3) is a PPS field, and the
`slice_reserved_flag[i]` bits it counts sit BETWEEN
`slice_pic_parameter_set_id` and `slice_type` in the slice segment header
(§7.3.6.1). Every earlier fixture used a PPS with the field == 0, so the
skip was never exercised: a parser that ignored the field entirely agreed
with all of them, and would then read `slice_type` from the wrong bit offset
on any real stream that sets it — mislabelling every picture's coding type.
**`reasserts_active_pps_at_bare_keyframe`** — regression (UHD banded
corruption): a stream redefines PPS id 0 mid-title, then a later keyframe
arrives WITHOUT repeating it (the source relies on the decoder retaining the
redefinition — valid for a raw bitstream). An hvcC player re-applies the
FIRST (codecPrivate) PPS at every keyframe, so the active redefinition must
be re-asserted in-band at that bare keyframe or the whole segment decodes
against the wrong parameter set.
**`keyframe_param_reassert_does_not_reallocate_the_frame`** — MEASURED: the
keyframe parameter-set re-assert must be spliced into the front of the
already-assembled access unit IN PLACE, not built as a fresh full-size
buffer. It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-MB allocation — memcpy'd the whole
frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a 2 h UHD title at
24 fps with a 1 s GOP is ~7,200 keyframes, ~14-28 GB of avoidable memcpy per
title. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates;
this test counts the reallocations that happen, which must be zero.
**`emits_switch_back_to_codecprivate_pps`** — regression (a UHD title, the
real bug): id 0 is body A (→ hvcC), then redefined to B, then the title
switches BACK to A. A streaming decoder (hvcC at init, in-band updates only)
is sitting on B; the switch back to A must be emitted IN-BAND even though A
== codecPrivate, or the whole A-segment decodes against B (cu_qp_delta
desync). Stripping on `== hvcC` dropped this revert.
**`cra_at_auto_detected_pts_backstep_rewritten_to_bla`** — regression for a
UHD Dolby Vision profile 7 dual-layer title: a multi-clip title is read as
one concatenated stream and the mpls connection_condition is never plumbed
to the parser, so the splice CRA opening the next clip kept its dangling
RASL leading pictures and a linear decoder flooded "Could not find ref with
POC N". The parser must AUTO-DETECT the boundary from the backward PES-PTS
reset between clips (each .m2ts has its own PTS base) and rewrite that
splice CRA → BLA_W_LP with no explicit `mark_clip_boundary` call.
**`cra_after_33bit_pts_wrap_not_rewritten`** — regression for the 33-bit PTS
wraparound false-trigger (rc.5.2 audit #1): a SINGLE clip whose raw 90 kHz
PES PTS crosses the 2^33 counter wrap (~26.5 h) must NOT be mistaken for a
non-seamless clip join. Before the fix the raw 2^33→0 backward step armed
`pending_clip_boundary` and the next in-clip CRA was wrongly rewritten
CRA→BLA_W_LP (dropping valid RASL pictures — visible corruption). After
unwrapping onto a monotonic timeline the wrap is absorbed and the CRA stays
CRA.
**`cra_splice_detected_at_large_pts_magnitude_not_masked_by_wrap_logic`** —
the wrap-vs-backstep test above keeps `high_pts` near the very top of the
33-bit range, close enough to the `PTS_WRAP_PERIOD / 2` threshold that
`high - unwrapped` and a hand-flipped `high + unwrapped` land on the SAME
side of the threshold at every step — it doesn't actually distinguish the
two. This test uses PTS magnitudes around 3e9 (order 2^32, well below the
wrap threshold but reached in well under an hour), where the subtraction and
the addition diverge: ordinary forward progression keeps `high - unwrapped`
small, but `high + unwrapped` would already exceed `PTS_WRAP_PERIOD / 2` on
the very next frame, which would pollute `pts_wrap_offset` by a full period
and mask the genuine two-clip splice that follows.
**`non_cra_nals_never_rewritten_at_boundary`** — non-CRA NALs are never
rewritten even when a boundary IS marked. IDR (19), RASL (8/9), VPS/SPS/PPS,
and a trailing slice all pass through unmodified; the IDR clears the pending
boundary so no later CRA is wrongly converted.
**`no_boundary_marker_is_byte_identical`** — a frame stream with NO boundary
marker is BYTE-IDENTICAL to a parser that has no splice-rewrite field at all
(the UHD-safety guarantee): asserts byte-equality of every emitted frame
across a multi-AU stream containing CRAs, IDRs, RASLs, VPS/SPS/PPS, and
trailing slices — none of which is ever marked.
**`seamless_boundary_no_rewrite`** — a SEAMLESS boundary (connection_condition
0x05/0x06) is expressed by NOT calling `mark_clip_boundary`, so a CRA across
a seamless join is left unchanged. This encodes the contract: only
non-seamless joins call `mark_clip_boundary`; seamless ones never do, so no
rewrite occurs.
**`redefined_pps_emitted_inline`** — a parameter set REDEFINED mid-stream
(same id, different body) must be emitted INLINE so the decoder
re-activates it. Some discs redefine PPS id 0 partway through the title;
the old parser kept only the first PPS, so the second segment decoded
against the wrong PPS (CABAC desync).
**`make_sps_full` (test helper)** — builds a stored SPS NAL with sub-layers
and a conformance window, so the parser must skip sub-layer PTL and the 4
conformance-window ue(v) fields before reaching the bit depths.
`max_sub_layers_minus1` controls the sub-layer loop.
**`hvcc_array_length_round_trips_above_256_bytes`** — the hvcC array length
is a 16-bit big-endian field written as two separate `push`es: `(len >> 8)
as u8` then `len as u8`. Every VPS/SPS/PPS the rest of the suite feeds is
well under 256 bytes, so the high byte is always 0 and a `>>` -> `<<`
mutation (which also always truncates to 0 for those inputs) is
unobservable there. A real HEVC SPS with an extended VUI/HRD block can
exceed 256 bytes, so this test uses 300+ byte VPS/SPS/PPS and decodes the
16-bit length fields back to confirm they round-trip to the exact NAL
length, not merely a byte that happens to be 0.
**`hevc_ps_reorder_is_installed_and_flush_drains_its_real_frames`** — HEVC
counterpart of `h264_ps_reorder_reconstructs_distinct_display_pts`. A
DVD/HD-DVD program stream stamps a PTS only on each GOP anchor, so the
parser must reconstruct display-order timestamps for the rest. Two things
are pinned here that nothing else constrains: `with_ps_reorder(true)` must
actually INSTALL the reorderer (a builder that returned a
default-constructed parser would silently leave the transport-stream path
in place on every DVD title), and `flush()` must drain the reorderer's real
buffered frames at EOF — not nothing, and not a manufactured empty frame at
PTS 0 (a zero-length Block whose timestamp jumps back behind every cluster
already written, RFC 9559 §5.1.3.2). Slice bodies follow H.265 §7.3.6.1:
`first_slice_segment_in_pic_flag`, the IRAP-only
`no_output_of_prior_pics_flag`, `ue(pps_id)`, `ue(slice_type)` (Table 7-7:
0 = B, 1 = P, 2 = I). PPS body 0xC0 sets `num_extra_slice_header_bits = 0`.
+26
View File
@@ -0,0 +1,26 @@
# hex
The single hex → bytes parser for the whole workspace.
## Why one parser
Key material arrives as hex from three third-party sources — the keydb, an
online key service, and the mapfile's `# freemkv-vid:` comment — and each
used to parse it slightly differently (one stripped `0x`/`0X`, one stripped
nothing, one stripped `0x` only). A key written with a prefix one parser
didn't expect was silently dropped → "can't decrypt" with no error. This is
the one parser they all call, so the prefix/case/validation rules live in
exactly one place.
## Byte-oriented, not char-oriented
Operates on BYTES, not `&str` char indices: the inputs are untrusted, so a
multi-byte UTF-8 scalar must reject as malformed, never panic on a
mid-codepoint slice.
## `byte()` internals
Combines two ASCII hex-digit bytes into one byte. `as char` is intentional:
for a non-ASCII byte it produces a Latin-1 scalar that `to_digit(16)` then
rejects — so non-hex (incl. `+`/`-` sign chars) and multi-byte input fail
cleanly rather than slipping through `from_str_radix`'s sign handling.
+75
View File
@@ -0,0 +1,75 @@
# identity.rs — drive identity test notes
## `OversizedCountTransport` (test double)
Returns the requested data length but reports a `bytes_transferred` larger
than the caller's buffer — models a drive that lies about its transfer
count. The old slicing code panicked on this; the clamps in `from_drive`
must keep it from indexing out of range.
## `inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive`
`ascii_field` with a buffer shorter than `start` returns empty string
rather than panicking.
- Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer (e.g.
a device that reports fewer than 8 bytes) must not panic.
- Mutation: removing the `data.len() > start` guard makes it panic on
short inputs.
A drive that answers INQUIRY with GOOD status but a short or empty data
phase must fail the probe, not present as a blank drive.
The buffer is pre-zeroed, so decoding it unconditionally yielded empty
vendor/product/revision strings and a byte 0 of 0x00. Every platform
enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
and 0x00 is DIRECT ACCESS — so the drive silently vanished from the device
list rather than reporting that its identity probe failed. A USB-SATA
bridge mid-wedge does exactly this.
The two GET CONFIGURATION calls in the same function already clamped on
`bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
lines above them, discarded it.
## `ascii_field_boundary_len_equals_start_is_empty`
`ascii_field`'s guard is `data.len() > start` (strictly greater), not
`>=`: a buffer whose length is exactly `start` has NO byte at that offset,
so it must still yield empty, not attempt to slice.
Mutation: `>` -> `>=` would try to slice `data[start..]` when
`data.len() == start`, which panics (empty range at the very end is fine,
but the guard's job is the `< start` case below it — pinning the exact
boundary catches an off-by-one either direction).
## `display_formats_trimmed_fields_space_separated`
`Display` renders the four trimmed identity fields space-separated — the
human-readable counterpart of `match_key`'s pipe-separated form. Not
exercised anywhere else in this test module.
Mutation: replacing the `fmt` body with `Ok(Default::default())` writes
nothing at all, so formatting any `DriveId` yields "".
## `FixedGcCountTransport` (test double) / `from_drive_gc_failure_yields_empty_firmware_date`
GET CONFIGURATION failure (transport error) must not abort the identity
probe — firmware_date is empty, raw_gc_010c is empty. Mutation:
propagating the GET_CONFIGURATION error with `?` aborts `from_drive`.
`FixedGcCountTransport` is a transport whose GET CONFIGURATION responses
report an exact, caller-chosen `bytes_transferred` for each of the two GC
features (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
boundary guards can be pinned precisely. INQUIRY always succeeds.
## `from_drive_firmware_date_boundary_exactly_12_is_empty`
`end > 12` in the firmware-date branch (`from_drive`) is a strict
inequality: `bytes_transferred == 12` reports the field absent (offset 12
is the first byte of the 12-char date; a count of exactly 12 covers bytes
0..12, none of which is the date), so `firmware_date` must be empty, not
the mutant's off-by-one read.
Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an empty
but non-panicking slice — silently reporting "present" data that is
actually all outside the transferred count.
+168
View File
@@ -0,0 +1,168 @@
# IFO parser notes
Overflow rationale relocated from `src/ifo.rs` comments (see `// See docs/ifo.md`
pointers in that file for the anchor point of each section below).
## `DvdTitle::feature_start_cell` (Bug-4: scene-selection/logo at head of feature)
The main feature's PGC can open with leading cells that are NOT part of the
movie — a scene-index segment or an interleaved-angle sub-block. This walks
the leading run and returns the index of the first cell that is a plain
feature cell (category `0x00`-class).
Conservative by construction — it can NEVER truncate a normal feature:
- It only ever skips a *prefix*; the scan stops at the first plain-feature
cell and keeps everything from there on.
- A normal single-angle feature has category `0x00` on cell 0, so the scan
stops immediately at index 0 and drops nothing.
- It never drops on duration or any heuristic — only on the spec category
bits — and it never drops the FIRST cell of an angle block (the angle we
keep).
- As a final guard it never returns past the last cell, and never drops when
that would leave zero cells.
For a real disc where every feature cell category is `0x00` and chapter 1
sits at 00:00:00, this returns 0 — a no-op — which is the correct result: the
disc's scene-index lives in a separate menu/title PGC, not in leading cells
of the feature PGC, so there is nothing to drop here.
## `bcd_to_frames` / `bcd_to_secs` (dvd_time_t is a timecode, not wall-clock time)
Format: `[hours_bcd, minutes_bcd, seconds_bcd, rate_and_frames]`
- Byte 0: hours in BCD (e.g. 0x01 = 1 hour, 0x12 = 12 hours)
- Byte 1: minutes in BCD
- Byte 2: seconds in BCD
- Byte 3: bits 7-6 = frame rate flag (01=25fps, 11=29.97fps),
bits 5-0 = frame count in BCD
The seconds field advances once every [`DvdRate::nominal_fps`] frames. On
NTSC discs that is every 30 frames, but 30 frames of 30000/1001 fps video
last 1001/1000 s — so reading H:M:S as literal seconds under-reports real
time by exactly 0.1% (3.6 s per hour). Callers that need real seconds must go
through the frame count, which is what `bcd_to_secs` does.
## `MAX_TT_SRPT_TITLES` / `parse_tt_srpt`
DVD-Video caps a disc at 99 titles (VMGI TT_SRPT `TT_Ns`, and the 99-title /
99-title-set structure the format is built around), but the on-disc count is
an untrusted `u16`: a ~800 KB crafted IFO can declare 65535 entries, each
re-parsing a PGC into a full `DvdTitle` (~540 MB of `DvdInfo`). Headroom: this
IS the format maximum, so it clips no conformant disc — a real DVD cannot
address a 100th title through TT_SRPT.
`parse_tt_srpt` has two bounds on untrusted input: the declared entry count
is clamped to `MAX_TT_SRPT_TITLES`, and entries naming a `(vts_number,
vts_title_num)` pair already seen are dropped. De-duplication is a
correctness fix as well as a bound: two TT_SRPT entries pointing at the same
VTS title are the same title, and `parse_pgcit` would otherwise re-parse
that one PGC into a separate `DvdTitle` per entry.
## `parse_audio_attr` visibility
`pub(crate)` for the CROSS-MODULE tests only. The sole production caller is
`parse_vts_attributes` in this file; `src/mux/mkv.rs`'s `#[cfg(test)]` block
calls it directly so its language-mapping tests run the real parser over real
on-disc IFO bytes end to end, instead of a hand-built `DvdAudioAttr` that
could agree with the muxer while both disagree with the disc. Narrowing this
would mean either a `#[cfg(test)]`/`#[cfg(not(test))]` pair of signatures
that can drift apart, or moving those tests away from the code they exist to
pin — both worse than the widened crate-internal visibility, which reaches no
public API.
## `assign_audio_sub_stream_ids`
On DVD-Video the sub-id's low nibble is the audio-stream *number* (0-7),
shared across all codecs — the single stream index the PGC `audio_control`
table / navigation registers select — and the high nibble is the codec base.
So the sub-id is `codec_base | position`, where `position` is the stream's
index in the IFO audio-attribute table (NOT a per-codec running count):
- AC-3 → `0x80 | i`
- DTS → `0x88 | i`
- LPCM → `0xA0 | i`
- MP1/MP2 and anything else → `None` (regular MPEG-audio PES, not a
private-stream-1 sub-id).
A per-codec ordinal was wrong: it only coincides with the wire id when a
codec's first stream is also the disc's audio stream #0. Any codec that is
not the first audio stream (e.g. a DTS track after an AC-3 track) then got
a sub-id one-too-low, so the demux routing key (`0xBD00 | sub_id`) never
matched and the track muxed silent. The positional index is the real wire
number, so distinct positions still give distinct sub-ids (no collision).
Position saturates at 7 so a malformed over-count never produces an
out-of-range sub-id.
## `parse_raw_dvd_lang_bytes`
The salvage is deliberately not narrowed to a-z: whatever survives is only
ever a lookup key for `dvd_lang_to_iso639_2`, which degrades anything it does
not recognize to `und`, so a stray `X` or `5` costs nothing and cannot reach
an output stream as a language code.
## `dvd_lang_to_iso639_2`
Uses `labels::vocab::iso639_1_to_iso639_2`, which spans the WHOLE of ISO
639-1 (plus the withdrawn spellings `iw`/`in`/`ji` that DVD-Video's
frozen-1988 language list still puts on disc). The narrower `vocab::menu_lang`
table is deliberately NOT used here: it exists for Blu-ray menu-graphic
filename tokens and knows only 25 languages, so a Region-2 disc's Romanian,
Bulgarian, Croatian, Serbian, Slovak, Slovenian, Hebrew, Estonian, Latvian,
Lithuanian and Icelandic tracks would all fold onto `und` together. DVD
streams carry an empty `label`, so the language is the only thing
distinguishing one subtitle track from the next — a valid code that is
identical for six tracks is worse for the user than the invalid one it
replaced. Both tables normalize to ISO 639-2/T, so they agree wherever they
overlap.
An empty or unrecognized code degrades to `"und"` (ISO 639-2 / Matroska's own
"undetermined" value) — a valid element value — rather than passing through
an invalid 2-letter code or an empty string. Never guesses.
## `video_attr_absolute_bytes_pin_real_layout` test
ABSOLUTE-BYTE pin (audit §3 #2): the existing video-attr tests build the byte
via `v_atr_byte(...)`, which uses the SAME shift constants the parser reads
with — a co-edit of constant + helper would silently re-introduce the
PAL-as-NTSC bug and every test would still pass. This test feeds
`parse_video_attr` HARDCODED bytes captured from real DVD-Video layouts
(DVD-Video video attributes: mpeg_version[7-6] video_format[5-4]
display_aspect[3-2] permitted_df[1-0]) — no `v_atr_byte`. If the parser's bit
positions drift, these fail.
## `dts_after_ac3_uses_positional_substream_id` test
Regression (The Punisher 2004): audio[0]=AC-3 5.1, audio[1]=DTS 5.0. The DTS
track sits at audio position 1, so its wire sub-id is 0x89 (0x88 | 1), NOT
the per-codec 0x88. With the old per-codec ordinal it got 0x88 → demux
routing key 0xBD88 had no match → every DTS packet (which carries 0x89) was
dropped → the track muxed present-but-silent while the AC-3 (at position 0,
where ordinal and position coincide) played fine. Positional numbering fixes
it end-to-end.
## `pgc_chapter_times_ntsc_no_pulldown_drift` test
Regression test for issue freemkv#25 (NTSC chapter drift). Builds a PGC from
the real cell table of the disc in the bug report and checks the chapter
marks against the reference tool's values. Before the fix, chapter 14 landed
4.019 s early; the drift was proportional to elapsed time (0.1%), so a short
synthetic fixture would not have caught it.
## Test additions note
The block of tests starting at `cell_category_low_flags_are_bit_isolated`
covers cell-category bit isolation, palette/program-map arithmetic, and the
malformed-language salvage paths.
## `build_pgc` test fixture
Builds a standalone PGC starting at offset 0. Layout: 0xEA-byte header, then
the program map (one byte per program, the 1-based first cell number), then
the 24-byte cell playback table. `cells` are `(category, BCD time,
first_sector, last_sector)`.
## `build_pgcit` test fixture
Builds a VTS_PGCIT at offset 0: VTS_PGC_Ns(2) + reserved(2) +
VTS_PGCIT_EA(4), then one 8-byte VTS_PGCI_SRP per PGC (VTS_PGC_CAT(4) +
VTS_PGCI_SA(4), the PGC's byte offset from the VTS_PGCIT start). `pgcs` are
appended after the SRP table.
+35
View File
@@ -0,0 +1,35 @@
# `src/io/image_writer.rs`
## Why this is not `freemkv_engine::copy`
The engine's `copy` is the RECOVERY path — mapfile sidecar, `--multipass`
sweep/patch, damage-jump, ECC-aware batching, auto-resume. Every one of those
exists because an optical drive returns read errors on marginal media. A
file-backed or synthesized source has no marginal media: a read either
succeeds or the underlying file is broken, and retrying it is pointless.
Routing a non-drive source through the recovery path is not merely wasteful,
it is wrong. Its mapfile identity check compares AACS unit keys and the VID,
both of which are empty for an already-decrypted source, so identity passes
for ANY such source: a second run with a different input to the same output
path would resume over the previous image and produce wrong content at exit
zero. Keeping the two paths separate makes that unrepresentable.
So: drive sources get `freemkv_engine::copy`. Everything else gets this.
## `write_image` rationale
The output is a faithful image of whatever the source presents — decrypted
if the caller wrapped the source in a
[`DecryptingSectorSource`](../src/sector/decrypting.rs), ciphertext if it did
not. `write_image` performs no decryption itself and makes no decryption
decision; that belongs to the caller, which knows whether the run is
`--raw`.
On cancellation the partial file is left in place — the caller decides
whether a partial image is worth keeping, and deleting a multi-gigabyte file
the user may want to inspect is not this function's call to make.
A short read from the source is treated as an error rather than zero-filled:
silently padding a truncated source would produce an image that looks
complete and is not.
+10
View File
@@ -0,0 +1,10 @@
# `/BDMV/index.bdmv` layout
Binary layout of the BD navigation index (First Play, Top Menu, and the
title table), as parsed by `src/bdnav/index.rs`.
`"INDX"` + version(4) + `indexes_start`(u32 @8) + … ; at `indexes_start`:
`index_len`(u32), First-Play object(12), Top-Menu object(12),
`num_titles`(u16), then `num_titles` title objects(12 each). Every object's
`object_type` is the top two bits of its first byte; for an HDMV object the
`id_ref` into `MovieObject.bdmv` is a big-endian u16 at object offset 6.
+7
View File
@@ -0,0 +1,7 @@
# FMTS index selection
Where the resolved index comes from is a separate concern from disposition
classification (`resolve_disc_index` in `src/aacs/index_select.rs`): today it
is read off the index keys the key source handed us; when Processing Keys are
available it will come from the VK derivation instead. Either way the
disposition logic in `unit_disposition` is identical.
+100
View File
@@ -0,0 +1,100 @@
# `aacs::inf` — notes
## `UnitKeyFile`'s hand-written `Debug`
Redacting `Debug`, per the policy `aacs::types` documents: this struct holds
the disc's ENCRYPTED CPS unit keys — exactly the material a keydb entry stores
— plus the disc hash they are looked up by. A derived `Debug` printed every key
byte verbatim, so any `{:?}` (a downstream crate, an `assert_eq!` failure
message, a future `tracing::debug!` in this module) leaked them. Only
non-secret shape is printed. Guarded by `unit_key_file_debug_is_redacted`.
The disc hash is the public keydb lookup key, printed as hex the same way
`DiscEntry` prints its own — never as raw bytes.
## `parse_vtkf`
Layout — AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, a fixed
2480-byte file, verified byte-exact against real discs (Freedom `VTKF090`,
Dukes of Hazzard `VTKF000`):
```text
[0x00..0x0C] magic "DVD_HD_V_TKF"
[0x0C..0x10] BE32 HD_VTKF_SIZE (2480)
[0x10..0x1C] associated playlist name ("VPLST%%%.XPL")
[0x1C..0x80] reserved
[0x80..] 64 entries × 36 bytes:
BIFO (1) | reserved (3) | ENCRYPTED title key (16) | binding MAC (16)
BIFO bit 7 (AV_FLG) set = this slot holds a title key
(pre-recorded discs fill the binding MAC with 0xFF)
[0x9A0..2480] 16-byte TKF MAC (CMAC keyed by Kvu — NOT a key)
```
The slot index (1-based) is the CPS unit number, so an absent slot is
SKIPPED (not a terminator) — collapsing gaps would renumber later keys and
hand the wrong title key to CPS unit N+1. The title→CPS mapping is
playlist-driven (`VPLST%%%.XPL`) and owned by the HD DVD enumerator, so
`title_cps_unit` is left empty here.
The prior parser used a 32-byte stride (a 12-byte pad instead of the 16-byte
binding MAC). That reads entry #1 correctly but drifts +4 bytes per entry
after it, so it only decrypted single-CPS-unit discs; every multi-key VTKF
(real multi-CPS-unit HD-DVD titles) yielded garbage keys for CPS unit ≥2.
## `synth_vtkf` test helper
Build a synthetic `VTKF%%%.AACS` matching the real on-disc layout (AACS
HD DVD Book Table 3-8, verified against Freedom `VTKF090` and Dukes
`VTKF000`): magic, BE32 size, playlist name, reserved to 0x80, then 64
entry slots of 36 bytes (the first `keys.len()` present with `AV_FLG`
set, the rest empty), a reserved gap, and the 16-byte trailing TKF MAC.
## `unit_key_file_debug_is_redacted` test
`UnitKeyFile` holds the disc's ENCRYPTED CPS unit keys. A derived `Debug`
printed every byte; the hand-written impl must not. Sentinel key byte
0xD5 = decimal 213 (a derived `Debug` renders `[u8; 16]` in decimal), the
same probe `aacs::types::redaction_tests` uses. Mutation guard: putting
`#[derive(Debug)]` back fails this.
## `read_mkb_from_drive_returns_the_concatenated_pack_payload` test
`read_mkb_from_drive` is the in-drive MKB source: every AACS derivation
downstream (`mkb_find_mk_dv`, the subset-difference walk, the whole
Media Key ladder) consumes exactly what it returns. An empty return is
not a benign "no MKB" — it is a total read failure reported as success,
and every derivation then fails with a key-not-found code that points
the operator at their keydb rather than at the drive.
This pins the CONTENT: the concatenated payload of all packs, in pack
order, byte for byte.
## `read_mkb_from_drive_issues_the_exact_mmc_cdb_for_each_pack` test
The CDB is what the drive actually acts on, and every byte of it is
load-bearing: a wrong format code returns a different disc structure
entirely, and a wrong allocation length truncates the pack. The existing
test above pins the opcode, the format code and the pack number; this
pins the WHOLE 12-byte CDB, so no field can drift unnoticed.
Expected layout (MMC-6 READ DISC STRUCTURE, AACS MKB format):
`[0]` opcode, `[1]` media type 0x01, `[2..6]` address = pack number
(BE32), `[6]` layer 0, `[7]` format 0x83, `[8..10]` allocation length
BE16 = 32772 = `0x80 0x04`, `[10..12]` reserved/control.
## `read_mkb_from_drive_accepts_a_full_size_pack` test
A pack payload filling the FULL 32768-byte window must come back whole.
The `len > 0 && len <= 32768` bound is what stands between a maximal
pack and a silently dropped one, and the small payloads used elsewhere
in this module never reach it.
## `read_mkb_from_drive_ignores_a_pack_declaring_more_than_the_buffer_holds` test
A drive that DECLARES more payload than it returned must not be
believed. The BE16 length in the response header is drive-supplied data:
a firmware bug, a short transfer, or a hostile device can put a value in
it that runs past the 32772-byte buffer. Copying `len` bytes on that word
alone panics the rip thread mid-scan.
Both the first-pack read and the per-pack loop carry the same bound, so
both are exercised here: the over-declared pack contributes nothing and
the honest pack still comes through.
+34
View File
@@ -0,0 +1,34 @@
# `src/io/mod.rs` — background
## `WritebackFile`
Drop-in wrapper around `std::fs::File` for any call site that performs
large sequential writes (sweep, patch, mux, etc.). It implements `Write`
and `Seek` so existing code paths can swap `File` for `WritebackFile`
with no body changes. Internally it drives a `WritebackPipeline` that,
on Linux, drains dirty pages continuously at 32 MB granularity to avoid
the kernel's accumulate-then-burst flush behaviour. macOS and Windows
use a no-op pipeline — their default cache policies have not been shown
to exhibit the same pathology for this access pattern.
## `FileSectorSource`
The read-side dual of `WritebackFile` — it implements
[`crate::sector::SectorSource`] for an ISO file using direct
`pread`-equivalent calls so the kernel's own readahead policy runs
(which interleaves naturally with the concurrent writeback). It pairs
that with periodic `posix_fadvise(DONTNEED)` drops on the consumed
window so an 85 GB streaming ISO read doesn't fill the page cache and
starve the concurrent MKV write.
## `Pipeline` + `Sink`
The generic producer/consumer primitive used by sweep, patch, and mux
to overlap reads with writes via a bounded channel + dedicated consumer
thread.
## `byte_prefetcher`
The read-ahead producer feeding the mux pipeline for `io::Read`-backed
sources: a worker thread fills a recycled pool of buffers and ships
them through a channel, exposing `BytePrefetcher` / `PrefetchShell`.
+25
View File
@@ -0,0 +1,25 @@
# src/labels/jar.rs — relocated comment prose
## `MAX_CLASS_BYTES` rationale
A real BD-J `.class` is far under this ceiling (64 MiB); a lying header
simply gets truncated and the class fails to parse, which is skipped like
any other bad entry. The buffer is grown incrementally and the read is
capped here rather than pre-sized from the declared size.
## `for_each_jar` "top-level" vendor examples
"Top-level" means entries directly under `/BDMV/JAR/`, not nested under a
subdir. Pixelogic, Criterion, Paramount, etc. put their data files inside
`/BDMV/JAR/<x>/`; dbp and Deluxe put their jar directly at
`/BDMV/JAR/<name>.jar`.
## `read_is_bounded_by_cap` test rationale
The read is bounded by `MAX_CLASS_BYTES`: a stored entry whose real payload
exceeds the cap yields only the first `MAX_CLASS_BYTES` bytes to the
parser, never the full (unbounded) entry. Verified here on a small cap via
the entry-count path: the truncated bytes still parse a valid class
prefix, but no read beyond the cap occurs. The test asserts the entry is
still surfaced exactly once (the cap does not drop legitimate entries) and
that the call returns.
+68
View File
@@ -0,0 +1,68 @@
# KeyProvider (src/aacs/provider.rs)
Long-form notes for `src/aacs/provider.rs`, relocated from doc comments to
stay under the comment-guard caps. See the module's own (trimmed) `//!` doc
for the short version.
## Method flavors
`KeyProvider` methods come in two flavors:
- **Bulk material** (`device_keys`, `processing_keys`, `media_keys`) — the
resolver unions (and dedups) results across all providers and tries each
candidate.
- **Disc-keyed lookup** (`lookup_disc_by_hash`, `lookup_disc_by_vid`) — the
resolver short-circuits on the first hit, so providers are queried in
array order with fastest/closest first.
`host_certs` is a sixth method but is NOT consumed by the resolver chain:
the SCSI handshake reads host certs directly from the caller-supplied
credentials, not from the provider array. A provider that overrides
`host_certs` today has no effect on the handshake; the method is retained
as a forward-looking extension point only. Correspondingly,
`Providers::host_certs` (the union helper) is `#[allow(dead_code)]` and
unused by the resolver chain today.
Default impls return empty / `None` so backends only override the methods
they actually support — an external key service might implement only
`lookup_disc_by_hash`, while a local file might implement all six.
Calls may block (disk I/O, network round-trips). The resolver invokes each
method at most a handful of times per scan; for per-disc memoization,
implementations should cache internally.
## `Providers` (resolver-side aggregation helper)
The resolver wraps `ctx.providers` (`&[&dyn KeyProvider]`) in `Providers`;
its methods apply the union-vs-short-circuit policy described above per
method. The bulk unions dedup so overlapping providers don't make the
resolver re-walk/re-validate identical material.
## `SuppliedKey` (bridge for `Disc::decrypt_with`)
A `KeyProvider` backed by a single caller-supplied key's raw material — the
bridge for `crate::disc::Disc::decrypt_with`.
The application's key source did the lookup and handed in material at one
level (DK / PK / MK / VUK). This exposes exactly that material to the
version-dispatched resolver, which owns ALL derivation — so a source never
derives, and the lib remains the single home for the AACS chain across
1.0 / 2.0 / 2.1 / 2.x.
Each level fills only its own field; the rest stay empty, so the resolver
naturally runs the matching path (DK→…, PK→…, MK-pool brute, or a
disc-keyed VUK hit). `decrypt_with` already knows the disc, so the
`lookup_disc_by_*` hash/VID arguments are irrelevant — a present
`disc_entry` is returned for any query.
## Test: `providers_host_certs_unions_every_providers_certs_in_array_order`
`Providers::host_certs` is the union across the provider array. It is not
wired into the handshake today (see above), so nothing else in the crate
would notice a body that dropped every cert on the floor — and the day it
IS wired in, a silently-empty cert list means the drive AACS authentication
finds no host certificate to present and every disc fails to open, with no
indication that the caller's certs were discarded.
Unlike the bulk key unions this one does NOT dedup (`HostCert` is not
Ord/Hash), so the assertion is on the full concatenation in array order.
+254
View File
@@ -0,0 +1,254 @@
# keysource.rs — extended notes
Overflow rationale relocated from `src/keysource.rs` doc comments by the
comment-guard (prose caps: 8 lines for pub-item rustdoc, 3 lines for internal
comments). Each section below is pointed to by a short `//`/`///` comment at
the corresponding site in the source.
## Module overview — ownership split
libfreemkv still OWNS the crypto: the boil-down primitives and the AES live
here. A source owns only PATH ORCHESTRATION — deciding which primitive to
call with what input for the material it happens to hold. Source
implementations are published in the companion `freemkv-keysources` crate,
keeping key *policy* (which store, which order, online vs local) out of the
library. It knows what material it holds (a DK / MK / VUK / pre-decrypted UK)
and what it must fetch from the disc (VID, MKB, encrypted title keys, content
samples) to get there.
## `MIN_SAMPLE_UNITS` — why this many, why here
The key service identifies a key by which of the submitted units it decrypts,
so too few samples — especially on FMTS, where a segment interleaves several
variants at the unit level — can return a key that matches an incidental unit
rather than the one asked about (a false positive). This many distinct units
make the request unambiguous.
Canonical here (the base crate) so BOTH consumers agree on one value: the
online source in `freemkv-keysources` (which refuses to send an under-sampled
request) re-exports it, and libfreemkv's own FMTS forensic query
([`crate::mux`]) sizes its per-segment batch by it. Layering forbids the
reverse import (keysources depends on libfreemkv, not vice versa), so the
value lives at the lower layer both share.
## `DecodeSampleSet` — parse, don't validate
"Parse, don't validate": the only constructor, `DecodeSampleSet::new`, returns
`None` for an under-sized slice, so an online key request simply *cannot be
built* from too few samples. The runtime `len() < MIN_SAMPLE_UNITS` check that
used to live at the request site (and was silently forgotten by an
under-sampling caller, reading as "key service down") becomes a compile-time
obligation: a request builder that takes `&DecodeSampleSet` can never receive
an unchecked `Vec`.
The *count* enforced here is a runtime property of the disc (how many
encrypted units it yields); the *requested* count is a caller-side
compile-time constant that callers pin to `MIN_SAMPLE_UNITS` (see e.g.
autorip's `SAMPLE_UNITS`). Together the two make under-sampling
unrepresentable at the request boundary.
The wrapped samples are on-disc AACS ciphertext — the same bytes the sibling
`DiscInputs::samples` redacts as key MATERIAL — so `Debug` is hand-written and
redacting.
## `DecodeSampleSet`'s hand-written `Debug`
A derived `Debug` dumped every wrapped sample verbatim: a `DecodeSampleSet`
carries at least `MIN_SAMPLE_UNITS` 6144-byte aligned units (>= 49 KiB, in
practice multi-MB) of AACS ciphertext plus each unit's clear 16-byte
derivation seed, so one `tracing::debug!("{set:?}")` on a failed `/decode`
request — or an `assert_eq!` whose panic message formats it — wrote all of it
to the log that gets attached to a bug report. Same policy and same shape as
`DiscInputs`'s impl.
## `DiscInputs`'s hand-written `Debug`
Redacting `Debug`, per the policy `aacs::types` documents (and which
`aacs::types::Vid` already applies to this very Volume ID). `DiscInputs` is
public and returned by `crate::Disc::inputs`, so a consumer's
`tracing::debug!("{inputs:?}")` used to print the Volume ID, the whole
`Unit_Key_RO.inf` (the encrypted title keys), the entire MKB and every
ciphertext sample verbatim into a log that ends up attached to a bug report.
Only non-secret identity and shape (presence, lengths) is printed.
## `DiscInputsCtx::new` — parse-failure policy
A present-but-malformed `unit_key_ro` (truncated / wrong magic / wrong
stride) parses to an empty key set, so a later `enc_title_keys` returns
`Ok(&[])` indistinguishably from a disc that legitimately has no title
keys — the parse failure is swallowed here, not surfaced as an error.
## `KeySource` trait — the two resolve operations
Returning an empty `Vec` means "no key for this disc from this source"; an
`Err` means the source itself failed (I/O, parse, network). The caller
(`resolve_and_apply`) tries each source in order and validates the returned
keys against real ciphertext before committing them, so a wrong key from one
source transparently falls through to the next.
Two explicit resolve operations, one per key kind — never one overloaded call
whose meaning depends on how many keys came back:
* `get_unit_keys` — the disc's base per-CPS-unit Unit Keys (index space =
CPS-unit number). The common path for every disc.
* `get_fmts_indexes` — the AACS 2.1 forensic index keys (index space =
forensic index 1..N). Defaults to empty: a source with no forensic material
opts out, and only an FMTS disc ever asks.
What each source must do to answer is the source's own business: a keydb keys
on `disc_hash` and reads no samples; the online source submits the ctx's
content samples (a base batch for `get_unit_keys`, an index-1 anchor batch
for `get_fmts_indexes`) to the key service.
## `resolve_and_apply_traced` — one-shot semantics and CPS-unit numbering
One-shot per source: each source's `KeySource::get_unit_keys` is called
exactly once with a `DiscInputsCtx` over `inputs`. Non-empty Unit Keys are
mapped to terminal `Key::Unit`s and applied via `crate::Disc::decrypt_with`,
which validates them against `inputs.samples` and only mutates the disc on
success — so a wrong/partial key set is rejected and the loop continues.
CPS-unit numbering: a source returns Unit Keys carrying the POSITIONAL index
from `crate::aacs::derive::decrypt_unit_key`; the library's canonical
CPS-unit number is `position + 1` (matching
`crate::aacs::inf::parse_unit_key_ro`'s `(i + 1)`), so the committed
`AacsState.unit_keys` is byte-identical to the library-resolved path.
The NUMBER itself is not what descramble indexes by — but the ORDER is
load-bearing, so a source must return its keys in CPS-unit order. Trial
decrypt-and-check was deliberately deleted (see `crate::decrypt::AacsKeyMap`:
decryption is driven by the disc's CPS-unit / FMTS-segment structure, "never
by trial-decrypt-and-check per unit"), and `decrypt_sectors_mapped` indexes
the committed pool POSITIONALLY — `unit_keys[key_idx].1`, where `key_idx` is
a POSITION in the Vec a source returned, recorded by
`resolve_mux_key_map_cached` / `resolve_fmts_key_map`. Return the same keys
in a different order and every `AacsKeyMap` points at the wrong key: the
whole title decrypts under a neighbour's key, or a forensic range trips the
`is_clean` net into `DecryptFailed`. (The doc used to say the number "is
cosmetic for descramble (the decrypt path strips it and tries every key)",
which is what the DELETED trial-decrypt path did; the only place that still
tries every key is `Disc::decrypt_with`'s sample VALIDATION, which does not
descramble content.)
## `fetch_unit_keys` — the two callers
This is exactly what both paths do — only the samples differ:
* at disc open, `ctx` carries reachable-content samples → resolves the
up-front CPS units (the common one),
* in the read, on a decrypt miss, `ctx` carries the FAILING unit's ciphertext
→ resolves the CPS unit that wasn't sampled up front.
Same sources, same call; there is no separate "fetch". Unlike
`resolve_and_apply` this does not validate/commit to a disc — the read's
decorator re-decrypts with the returned keys, which is the validation.
## `FetchOutcome` — the error-signal struct
Whether a driver run resolved keys, and — when it did NOT — whether the miss
was a genuine "no source holds this key" (`errored == false`) or at least one
source FAILED (`errored == true`, e.g. a network source was unreachable). The
distinction gates negative-result memoization: an empty-because-absent
result is safe to cache, an empty-because-a-source-was-down result is
transient and must NOT be cached (the key may resolve once the source
recovers).
## `drive_unit_keys`
`fetch_unit_keys` plus the error signal: drive `sources` in order, return the
first source's non-empty Unit Keys, and flag whether any source that failed
to answer did so with an `Err` (a source failure) rather than an empty `Ok`
(genuine absence — see `KeySource::get_unit_keys`).
## `key_fetch` — one builder, memoization policy
One builder, used by every read path (sweep / patch / mux) and every
consumer (CLI, autorip) — neither application contains the fetch logic, only
its key-source config. Cheap to clone; build once, clone into each read
path. `make_sources` is invoked per fetch (the cold path) so the resolver
stays `Send + Sync` without requiring `KeySource: Send`.
Each operation is memoized per sample-batch fingerprint (keys are disc-level,
so repeats hit cache). A genuinely-empty reply is cached, but a source
FAILURE is not — a transient miss (see `FetchOutcome::errored`).
## `read_encrypted_units` — why here, and the CPI selection rule
Lives in the library, not a key-source crate: reading the disc and carving
AACS units is decryption *mechanism* (unit geometry anchored at each
extent's `start_lba`), which the library owns. A key source is *handed*
these bytes via `DiscInputs.samples`; it never reads the disc itself.
"Encrypted" is decided by `crate::aacs::content::aacs_unit_encrypted` — the
AACS Copy Permission Indicator (CPI) in the top 2 bits of byte 0, the
spec-correct signal (`buf[0] & 0xc0`). NOT the `is_clean` TS-sync heuristic:
a unit lacking clean TS syncs does not imply encryption (an FMTS variant
frame or an odd clear unit can lack syncs yet be unencrypted), and a clear
unit sent to a key server yields nothing to validate against — the "0
encrypted units" rejection. A clip opens with clear navigation units
(PAT/PMT, menus) whose CPI is clear; only CPI-flagged content units are
collected — probing several points spread across EACH extent so a title
whose encrypted body starts late (or whose midpoint lands in clear nav)
still yields samples. CPI is read at each extent's `start_lba`
(clip-file-anchored), so byte 0 is a real unit start and the flag is
meaningful.
## Test doc overflow
### `key_source_host_certs_defaults_to_empty`
Spec: a source holding no cert returns the empty vec; the `mkb` param is
forward-looking and the default ignores it. Mutation: a default returning a
non-empty vec would inject phantom certs into the OEM handshake.
### `errored_empty_is_not_cached_and_retries_when_source_recovers`
A transient source outage must NOT be memoized as a permanent "no key": a
fingerprint whose first fetch failed because the source errored must be
re-asked, and once the source recovers the key resolves. Regression guard
for the negative-result memoization fix — caching the errored empty would
permanently drop a recoverable unit for the rest of the op.
### `key_fetch_routes_unit_and_fmts_to_distinct_source_methods`
The two `KeyFetch` operations route to the two DISTINCT trait methods:
`unit_keys` drives `get_unit_keys`, `fmts_indexes` drives
`get_fmts_indexes`. A source that returns different keys per method proves
the seam no longer collapses "1 base key" and "the forensic set" into one
overloaded call — the operation, not the return length, decides which.
### `read_encrypted_units_finds_scrambled_content_off_the_midpoint`
#4 regression: encrypted content NOT at the extent midpoint (a
late-starting feature, or a midpoint landing in clear nav) must still be
sampled — empty samples make `decrypt_with` skip wrong-key validation. The
old midpoint-and-forward sampler returned empty; the probe-spread finds the
early scrambled band.
### `read_encrypted_units_selects_by_cpi_not_ts_sync`
DISCRIMINATING: selection is by the AACS CPI (byte 0), NOT the TS-sync
clarity heuristic. Half the units lack TS syncs but are CPI-CLEAR
(`byte0 & 0xC0 == 0`) — genuinely UNencrypted units that merely lack TS
syncs; the old sampler collected these and the key server rejected the POST
as "0 encrypted units". `read_encrypted_units` must skip them and return
ONLY CPI-flagged units. A regression to selecting by TS-sync clarity would
collect the CPI-clear units too and fail the `& 0xC0` assertion.
### `disc_inputs_ctx_parses_unit_keys_at_the_version_stride`
Audit #5 — a DISCRIMINATING test for the version→stride fix. A 2-key
`Unit_Key_RO.inf` whose SECOND key sits at the V20 (64-byte) offset; a V10
(48-byte) parse reads a DIFFERENT region. Confirms `DiscInputsCtx` parses at
the stride for `inputs.version` — a swapped `from_major` branch or a
hardcoded stride (the exact bug 1.2.0 fixes) would fail this, where the
prior single-key fixtures passed regardless of stride.
### `disc_inputs_debug_is_redacted`
`DiscInputs` is public and returned by `Disc::inputs`, so any consumer's
`tracing::debug!("{inputs:?}")` prints it. A derived `Debug` printed the
Volume ID (the value `aacs::types::Vid` deliberately renders as
`Vid(<redacted>)`), the whole `Unit_Key_RO.inf` (the encrypted title keys),
the entire MKB and every ciphertext sample verbatim. Sentinel byte 0xD5 =
decimal 213, matching `aacs::types::redaction_tests`. Mutation guard:
restoring `#[derive(Debug)]` fails this.
### `decode_sample_set_debug_is_redacted`
`DecodeSampleSet` is public and wraps the SAME on-disc ciphertext the
sibling `DiscInputs` redacts, so a derived `Debug` dumped >= MIN_SAMPLE_UNITS
x 6144 bytes of verbatim AACS ciphertext (plus every unit's clear 16-byte
derivation seed) into any log that formatted it. Sentinel byte 0xD5 =
decimal 213, matching `aacs::types::redaction_tests` and the `DiscInputs`
test above. Mutation guard: restoring `#[derive(Debug)]` fails this.
+556
View File
@@ -0,0 +1,556 @@
# src/labels/mod.rs — design notes
Long-form rationale relocated from doc comments per the comment-guard
audience cap (internal items cap at 3 lines; see `ci/comment-guard.py`).
## `label_at`
The VENDOR label occupying 1-based STN slot `n` of `stream_type`, if any.
Slot lookup is restricted to labels with no `StreamId`, and that
restriction is the whole point. `stream_number` is not one coordinate
system, it is two that share a name:
* on a vendor label it is a slot in the one stream table the config blob
describes, which is what this function's callers count against; while
* on a derived label it is that stream's slot in *its own* playlist's
table, which says nothing about any other playlist's numbering.
Reading a derived label out of a slot lookup therefore answers a question
about table A with a fact about table B. A label that carries a `StreamId`
names the stream it describes outright and is bound by that id instead —
see `apply_labels`.
## `find_anchor`
Find the title the label list is actually describing, for one stream type.
A vendor label list is a single stream table — one playlist's STN slots —
but it is handed to every title on the disc. When sibling playlists cover
the same clip with different stream subsets, per-title ordinal numbering
puts the same label on different physical PIDs in each, and the flags
contradict each other. The list itself carries no playlist id, but it does
carry a language per slot, and that sequence is a fingerprint: on the
corpus, exactly one title's per-type language sequence reproduces the label
list position for position, and content confirms that title's binding is
the correct one.
So: the anchor is the title that `anchor_score` admits and that confirms
the most of the list. `None` means no title matches — the list describes a
stream table this disc scan cannot see, and nothing may be bound
authoritatively.
Only VENDOR slots take part. Derived labels (MPLS / CLPI) name their own
stream and bind directly, so admitting them here would be asking a title to
agree with a numbering that describes a different playlist — the mistake
that made this gate a coin flip while the merged list was the only list
there was.
## `anchor_score`
How strongly this title's stream sequence matches the vendor label list, or
`None` if the title contradicts it and cannot be the table it describes.
A stream disqualifies the title when the vendor label on its slot states a
different language. A slot the vendor list does not name constrains nothing:
under-yield is the normal shape of these blobs — an authoring layer ships
editorial labels for the streams it considers interesting and leaves the
rest as bare slots — so a hole is the list's silence, not a disagreement.
The score is how much of the list the title positively CONFIRMS: slots where
both sides state a language and state the same one. A list that names no
languages at all scores zero everywhere and the ranking falls through to
table size, which is what it did before this became a fingerprint.
## `apply_labels`
Apply a pre-extracted set of labels to titles' streams.
Every label either NAMES the stream it describes or it does not, and that
split — not a slot number — decides how it binds. Four tiers, most certain
first:
1. **This title is the anchor.** `find_anchor` identifies the title
whose stream table the vendor list is describing, so its slots are the
list's own slots and bind directly.
2. **An anchor proved this PID.** Every slot of the anchor yields a
`(clip, PID) -> label` fact, and a PID is the same physical stream in
every playlist that plays that clip. Sibling playlists bind through
that map, so a vendor label lands on the same elementary stream no
matter which playlist enumerates it, and a slot the anchor never showed
us is not bound at all.
3. **The label names this stream.** A derived label carries a
`StreamId` read out of the very table the stream itself was built
from, so `(clip, PID)` equality is identity, not inference. This is the
floor that gives every stream its language and codec.
4. **Ordinal, language-checked.** A vendor label with no anchor behind it
falls back to the 1-based per-type STN slot — but only where tier 3 had
nothing to say, and only when the languages do not contradict.
Tier 3 outranking tier 4 is the substance of this ordering: a label that
states which stream it belongs to beats a guess about which stream it might
belong to, even when the guess is the richer label. The policy where a title
cannot be bound confidently is to leave the stream alone: an unlabelled track
is a much smaller harm than a full-dialogue track wearing `forced`, which
presents to the user as a duplicate of the track they wanted, and which the
muxer can only undo on discs that state `forced_on_flag`. A subtitle label
carries nothing but the qualifier, so an unverifiable one has no upside at
all to trade against that risk, and the ordinal path applies one only when
the label and the stream both state a language and it is the same one.
Audio streams update `purpose` + `label` (codec/variant info; never
English purpose text). Subtitle streams update `qualifier` and the
`forced` flag.
Extracted from `apply()` so the matching logic is unit-testable without
needing a SectorSource / UdfFs.
## `merge_mpls_floor`
Merge the MPLS-derived floor into a framework parser's label list.
The two lists do not share a coordinate system, and the merge must not
pretend they do. A framework `stream_number` is a slot in the one stream
table the vendor blob describes; an MPLS label's is that stream's slot in
its own playlist's table. Matching them by number — which is what this
function used to do — merges by an equality that means nothing, and the
merged entries then land on whatever stream happens to sit at that ordinal
in each title.
So nothing is merged BY slot. Every MPLS label whose stream is not already
named in `framework` is appended, keeping its own `StreamId`, and
`apply_labels` decides per stream which of the two reaches it: the
vendor's editorial label where an anchor puts it there, the floor
everywhere else. Framework labels stay richer and stay ahead — they are
simply no longer competing for a slot number.
Sorted at the end (vendor slots first, in slot order, then the named
streams) so callers see a deterministic list.
## `append_clpi_orphans`
Append CLPI ProgramInfo streams that no existing label already names.
These are "orphan" streams — physically present in the .m2ts per CLPI's
clip-authoritative view, but no MPLS playlist references them, so the
framework and the MPLS floor both missed them. Empirically they are
commentary or alternate-version streams the authoring tool left out of the
published playlist. Returns the number appended.
Each carries the `(clip, PID)` it was read under, so it binds to that
stream and to nothing else. It used to be given `stream_number =
max(existing per type) + 1, +2, …`, an ordinal invented here and shared
with the slot numbering the vendor list uses — which meant a title with
more streams than the list had slots could reach an orphan by counting, and
be labelled from a stream no playlist even plays. There is no ordinal now:
a stream that is in no playlist is in no title, so an orphan label binds to
nothing, which is exactly right and is now structural rather than lucky.
## `select_result`
Pick the winning parser result from `results` (built in PARSERS order):
highest `Confidence` among non-empty results, with the earliest array
position winning on a tie — matching `extract()`'s strict-`>` first-wins
scan.
`Iterator::max_by_key` returns the LAST maximal element, so the key is
`(confidence, Reverse(index))`: among equal-confidence entries the one
with the smallest index has the largest `Reverse(index)` and is selected,
i.e. first wins.
## `collect_chapter_summary`
Scan `/BDMV/PLAYLIST/*.mpls`, parse each, return a row per playlist with
chapter count (entry marks only) and total duration. Sorted by playlist
filename. Skipped entries (read error, parse error, no marks) silently
dropped — this is a diagnostic field, not a correctness-critical one.
## `jar_inventory`
List filenames found under any `/BDMV/JAR/<x>/` subdirectory of the disc.
Deduped, sorted. Returns an empty vec if no JAR dir is present.
`pub(crate)` so filename-based parsers (e.g. `png_filenames`) can scan
menu-asset names without a reader.
## `jar_inventory_from`
The body of `jar_inventory`, over the `/BDMV/JAR` children directly, so
it is unit-testable without a `UdfFs`.
A `BTreeSet`, not `Vec::contains`: the entry names come from the disc's own
UDF directory records, so both the file count and the name lengths are
attacker-controlled, and a linear `contains` doing a full `String` compare
per candidate is quadratic in the number of files. The set also subsumes
the trailing sort — it yields sorted, deduplicated output directly.
## test: `jar_inventory_dedup_does_not_hang_on_a_hostile_directory`
`jar_inventory` deduplicated with a linear `Vec::contains`, doing a full
`String` comparison per candidate — quadratic in a file count taken
straight from the disc's UDF directory records, with attacker-chosen
name lengths to inflate each comparison.
This is a HANG GUARD, and the name says so: a return to the linear scan
makes this fixture run for minutes (120 000² / 2 comparisons over a
180-byte shared prefix), which without the deadline would wedge CI
rather than fail it. It is not a complexity proof — no assertion here
can distinguish `BTreeSet` from any other sub-quadratic dedup, and the
clock-free half of the claim (dedup, sort, directory exclusion) belongs
to `jar_inventory_dedups_sorts_and_skips_dirs` below.
The deadline is a real margin, unlike the 6x one that made
`paramount.rs`'s wall-clock test flake under a loaded CI box: measured
at 0.14 s debug / 0.07 s release against 10 s, so ~70x. A shared CPU
does not close that; a quadratic dedup does not survive it.
## test: `parsers_registry_order_locked`
Lock the parser roster + order. If someone reorders the array or
adds/removes a parser, this test forces them to update the expectation
explicitly. The order is load-bearing: first matching `parse()` wins, so
reordering changes which parser claims a disc on overlapping detect
signals.
dbp + deluxe MUST stay at the end (their detect triggers on "any BD-J
disc"; placing them earlier would short-circuit the stricter parsers
above them).
## test: `select_result_first_wins_on_tie`
`select_result` must pick the highest-confidence non-empty result and,
on a confidence tie, the FIRST in array order (regression for the old
`analyze()` `max_by(...then(Equal))` no-op that picked the LAST).
`extract()` — the path that actually ships — used to re-derive this
same rule with its own inline `>` scan and had no test at all, so the
bug this test guards against could have recurred there unnoticed. It
now calls `select_result`, so this test covers both.
## test: `suppression_is_by_named_stream_not_by_slot_number`
Spec: the merge suppresses a floor entry only when the list already
NAMES that stream. A framework label occupying the same slot number is
not the same fact — a vendor slot and a playlist STN slot are different
coordinate systems — so it suppresses nothing.
This is the merge rule inverted from what it used to be: the old key was
`(stream_type, stream_number)`, which dropped a floor entry whenever
some unrelated vendor slot happened to share its number, and kept one
whenever it did not. Both outcomes were decided by an accident of
counting.
## test: log-injection guard on playlist-name logging
A playlist name is a raw UDF directory entry — disc-controlled bytes,
validated no further than a lossy UTF-8 decode. Logging it through
tracing's `%` (Display) sigil writes those bytes VERBATIM, so a crafted
`.mpls` filename carrying ANSI escapes or control characters forges
terminal output and log structure in any consumer rendering the event
(CWE-117). `?` (Debug) escapes them, and `str`'s Debug is exactly the
escaping this needs.
`info!` is not covered by the debug/trace-logging exemption: this fires
on an ordinary rip of an ordinary disc.
Mutation: put `%` back on `playlist` in `apply_labels` and this goes red.
## test: `forced_label_follows_the_pid_not_the_ordinal_across_sibling_playlists`
The cross-playlist mis-binding this module's two-tier binding exists to
stop, in its measured shape: two playlists cover the identical feature
clip, one of them enumerates one extra subtitle ahead of the forced
slots, and the vendor label list describes the shorter of the two.
Numbering the same list from 1 inside each title puts the `Forced`
label on a different PID in each — in the longer playlist, on the full
dialogue track. The user then sees two identical-looking English
subtitle tracks, one of them wrongly flagged forced.
Binding by the PID an anchored title proved, rather than by ordinal,
puts `forced` on the same two physical streams in both playlists and
leaves the extra one alone.
## test: `ordinal_binding_drops_a_subtitle_label_that_contradicts_the_stream_language`
No anchor to work from (nothing on the disc reproduces the label list's
language sequence), so binding falls back to the STN ordinal — and a
label that names a different language than the stream it would land on
is evidence the list is describing some other stream table. Drop it:
unlabelled beats mislabelled, because the payload here is `forced`.
## test: `generate_video_label_secondary_dolby_vision_el`
Spec: a secondary (dependent-view) video stream with Dolby Vision
enhancement layer gets the brand string "Dolby Vision EL"; every other
HDR format on a secondary stream gets no label at all (that wording is a
CLI concern). Mutation: delete the `HdrFormat::DolbyVision` arm so it
falls through to the `_ => String::new()` catch-all, losing the brand.
## test: `generate_video_label_480_boundary`
Spec: 480 lines is the SD floor — a stream with height exactly 480 must
get the "480p"/"480i" token (BD spec height boundary), not fall through
to the empty-resolution case. Mutation: `h >= 480` -> `h < 480` inverts
the boundary so a legitimate 480-line stream (h == 480) produces no
resolution token at all.
## test: `generate_video_label_sdr_produces_no_hdr_token`
Spec: SDR is the unmarked default — it must never appear as a token in
the generated label (only non-SDR formats get an explicit tag).
Mutation: delete the `HdrFormat::Sdr` arm so it falls through to
`_ => parts.push(hdr.name())`, appending a spurious "SDR" token.
## test: `generate_audio_label_atmos_folds_brand`
Spec: the Atmos-aware variant folds "Atmos" into the codec brand name
for TrueHD/DD+ carriers, distinct from the plain wrapper. Mutation: stub
the whole function to `String::new()` / a constant literal — either way
it stops reflecting the codec/channel inputs.
## test: `generate_audio_label_covers_pc_container_codecs`
Spec: every disc-audio codec in the enum has a full marketing name,
including the lossy PC-container codecs (AAC/MP2/MP3/FLAC/Opus) that
`generate_audio_label_all_codecs` above doesn't cover. Mutation: delete
any one of these match arms — the codec falls through to
`_ => return String::new()`, silently losing its label.
## `codec_hint_consistent`: chained-OR boundary hardening
The family-detection booleans chain `h.contains(..) || ...` synonym
checks. Each test below isolates ONE clause so a `||` -> `&&` flip there
changes the verdict.
### `codec_hint_consistent_truehd_space_synonym`
Isolates the `"true hd"` (space form) synonym in `says_truehd`, which
mutant testing hit at 396:44's `||`. If that `||` is weakened to `&&`,
"True HD" alone (no "truehd" substring) no longer sets `says_truehd`,
`names_family` goes false entirely (no other family clause matches), and
the function takes the "no family named" early-return path — turning a
should-be-`false` verdict for a mismatched codec into `true`.
### `codec_hint_consistent_ddp_ac3_plus_no_hyphen_synonym`
Isolates the `"ac3+"` (no-hyphen) synonym in `says_ddp` (398:9's `||`).
A hint matching only this clause must still classify as DD+, not fall
through to the plain-AC3 `says_ac3` check.
### `codec_hint_consistent_ddp_eac3_synonym`
Isolates the `"eac3"` synonym in `says_ddp` (401:9's `||`), the last
clause before the chain moves to "digital plus"/"dd+".
### `codec_hint_consistent_lpcm_bare_pcm_synonym`
Isolates the `"pcm"` (no "lpcm") synonym in `says_lpcm` (409:40's `||`).
A bare "PCM" hint on a non-LPCM stream must still be judged
inconsistent — if the `||` were `&&`, "PCM" alone would fail to set
`says_lpcm`, `names_family` would go false, and the function would take
the "no family named" path, wrongly returning `true` for ANY codec.
### `codec_hint_consistent_names_family_dts_ma_alone`
Isolates the `says_dts_ma || says_dts_hr` disjunction inside the
`names_family` chain (418:60). A hint that sets `says_dts_ma` alone
(e.g. "Master Audio", without "hd ma") must still make `names_family`
true; weakening that `||` to `&&` requires both clauses at once, so
`names_family` goes false and the function wrongly reports "consistent"
for a codec the hint never named.
### `codec_hint_consistent_truehd_arm_atmos_alone`
Isolates the `Codec::TrueHd => says_truehd || says_atmos` arm (433:38).
An Atmos-tagged hint that names a DIFFERENT lossless carrier by name
(DD+) must still be judged consistent with a TrueHd stream purely on the
Atmos marker — `||` -> `&&` would require the hint to ALSO say "truehd",
which an Atmos-only marker doesn't.
### `codec_hint_consistent_dts_arm_not_bypassed`
Spec: `Codec::Dts` is consistent ONLY when the hint's DTS-family
bookkeeping (`says_dts`) is true, not just because `names_family` is
true via some other carrier. Mutation: delete the `Codec::Dts =>
says_dts` arm (438:9) — it falls to `_ => true`, so ANY named family is
(wrongly) "consistent" with a Dts stream.
### `codec_hint_consistent_lpcm_arm_not_bypassed`
Spec: `Codec::Lpcm` is consistent ONLY when `says_lpcm` is true.
Mutation: delete the `Codec::Lpcm => says_lpcm` arm (439:9) — same
bypass-to-`_ => true` failure mode as the Dts arm above.
## test: `slot_lookup_never_returns_a_label_that_names_its_own_stream`
Spec: a label that names its own stream is never reachable through the
slot lookup, whatever number it carries.
The two numbers are different coordinate systems. A derived label's
`stream_number` is its slot in ITS OWN playlist's table; the vendor's is
a slot in the one table the config blob describes. `label_at` answers
questions about the second, so it must not return the first.
Mutation: drop the `l.stream_id.is_none()` term from `label_at` — the
derived label at slot 1 is found first and shadows the vendor's.
## test: `the_derived_floor_cannot_veto_the_anchor`
Spec: the derived floor does not vote on which title anchors the vendor
list — and in particular cannot VETO the title that does.
The vendor list names two subtitle slots, eng then fra, and the feature
reproduces them. The feature has a third subtitle the vendor said
nothing about; the merge dropped a derived label into "slot 3", numbered
in a different playlist's coordinate system, and it says Spanish. Read
as part of the vendor sequence that is a contradiction, so the feature
is rejected, no title anchors, and every flag the list carries is lost.
This is the measured cost of a whole-sequence gate without provenance:
the vendor's own slots stay correctly positioned while the merged ones
break the sequence between them.
Mutation: drop the `l.stream_id.is_none()` term from `label_at` — the
derived "spa" is read as vendor slot 3, `anchor_score` returns `None`,
and there is no anchor.
## test: `the_anchor_is_the_title_that_confirms_most_of_the_list`
Spec: among titles the vendor list admits, the one that CONFIRMS more of
it wins — a slot where both sides state a language and state the same
one is evidence; a slot where either is silent is not.
Here the shorter title matches both named slots outright while the
longer one states no language at all, so it merely fails to contradict.
Size alone would hand the anchor to the title that proved nothing.
Mutation: rank on `n` alone (drop `score` from the comparison) — the
three-stream title with no languages wins.
## test: `a_vendor_qualifier_does_not_leak_onto_a_featurettes_own_stream`
Spec: an editorial qualifier the vendor list states for ONE stream table
does not leak onto a different physical stream in a title that merely
counts to the same ordinal.
Measured shape: the feature carries eleven subtitles whose first is SDH;
a dozen featurettes each carry a single English subtitle, on a different
clip and a different PID. Both are "subtitle number 1", so the ordinal
fallback put SDH on all of them — and the languages agree, so the
language gate could not catch it. What separates them is that the
featurette's stream is named by the derived floor, and a label that
names a stream outranks a label guessed onto it.
Mutation: move the `by_id` lookup in `resolve` after the ordinal
fallback — the guess wins again and the featurette is SDH.
## test: `an_anchor_proves_pids_only_for_the_clip_its_table_came_from`
Spec: the anchor proves a `(clip, PID)` fact only for the clip its
stream table was READ FROM — the first play item — never for every clip
the anchor happens to play.
`disc::bluray` builds a title's stream list from `play_items[0]`'s STN
table, and tier 3 fifty lines below says exactly that by keying its
derived ids on `clip0`. Tier 2's harvest contradicted it: it recorded
the anchor's slot PIDs against EVERY clip the anchor plays, so a sibling
playlist that plays a LATER clip of the anchor bound the anchor's
editorial label onto whatever stream in that clip happens to reuse the
PID — a different physical stream, in a different clip, whose own
language says so.
Mutation: harvest over `&title.clips` instead of its first clip — the
featurette wears the feature's SDH again.
## test: `a_vendor_codec_claim_does_not_follow_the_ordinal_onto_a_bonus_clip`
Spec: a vendor codec/variant claim does not follow the ordinal onto a
bonus clip that carries a different codec.
Measured shape: the feature's first audio is object audio; a dozen menu
and bonus titles each carry one plain stereo track. All of them are
"audio number 1", and the hint names the same codec family as the stream
it lands on, so the consistency guard passes it through and every bonus
clip advertises the feature's format.
Mutation: as above — move `by_id` after the ordinal fallback.
## test: `a_title_shorter_than_the_vendor_list_cannot_anchor_it`
Spec: a title whose stream count is smaller than the vendor list's
highest slot cannot be the table that list describes.
Mutation: drop the `n < extent` term from `find_anchor` — the one-stream
title becomes eligible, and on a disc where it sorts first it takes the
anchor away from the title that actually has the slots.
## test: `an_unnamed_slot_does_not_disqualify_a_title`
Spec: a slot the vendor list never names constrains nothing.
These blobs under-yield by design — the authoring layer ships editorial
labels for the streams it finds interesting and leaves the rest as bare
slots. Treating a hole as a disagreement rejects the very title the list
describes. (Before provenance the holes were filled with labels numbered
in an unrelated coordinate system, so the gate neither rejected nor
admitted on evidence.)
Mutation: make `anchor_score` return `None` for an unnamed slot — no
title anchors and the forced flag is never delivered.
## test: `fill_gaps_leaves_order_untouched_when_nothing_added`
Spec: the sort-by-(type, number) pass only runs when the merge actually
added something (`added > 0`); when MPLS contributed nothing new,
`framework`'s existing order (however the caller built it) must be left
untouched. Mutation: `added > 0` -> `added >= 0` is always true, so the
sort runs unconditionally, silently reordering a framework list that
wasn't already in (type, number) order even on a no-op merge.
## `build_program_info` / `build_clpi` (test fixtures)
`build_program_info`: builds a CLPI ProgramInfo section for one program
with the given (pid, stream_coding_info) pairs. Layout mirrors
`crate::clpi::parse_program_info`'s expectations: length(4) + reserved(1)
+ num_programs(1), then per-program spn(4)+pmt_pid(2)+num_streams(1)
+num_groups(1), then per-stream pid(2)+sci_len(1)+sci.
`build_clpi`: builds a full CLPI byte buffer (HDMV header + ProgramInfo)
declaring the given (pid, coding_type, lang) streams. `sci` layout
follows `crate::clpi::parse_program_info`'s per-coding-type match arms:
PG/IG = coding_type + 3-byte lang; audio (primary or secondary) =
coding_type + format/rate byte + 3-byte lang.
## test: `orphans_state_no_stn_slot_and_name_their_stream`
(d) An orphan states NO STN slot, and is identified by the stream it was
read from instead.
This test used to assert the opposite — that orphans continue the slot
numbering from `max(existing) + 1`. That number was invented here and
shared with the coordinate system `label_at` counts vendor slots in, so
a title with more streams of a type than the vendor list had slots could
reach an orphan by counting and be labelled from a stream no playlist
plays. An orphan is by definition in no playlist, hence in no title, so
there is no ordinal to give it.
## `codec_hint_consistent`
Does the parser's `codec_hint` name a codec consistent with the stream's
actual `codec`? `apply_labels` uses this to keep richer-but-consistent
hints (e.g. "Dolby Atmos" on a TrueHD stream — Atmos is a TrueHD extension
the raw spec codec can't express) while rejecting mis-bound ones (e.g.
"AC-3 2.0" on a TrueHD stream, the shuffled-label bug). Matching is by codec
FAMILY parsed out of the hint string. "Atmos" with no carrier named is
treated as compatible with its lossless carriers (TrueHD / E-AC-3). A hint
naming no recognizable codec family (pure editorial, e.g. "Commentary") is
consistent — it isn't asserting a codec.
## `generate_audio_label_atmos`
Atmos-aware variant: same codec/channel string as `generate_audio_label`
with the object-audio marker folded into the codec brand (e.g. "Dolby
TrueHD Atmos 7.1"). The "Atmos" string lives here in the label layer, not
in the core parser. Used when a bitstream probe detected an Atmos
substream and the stream still carries the basic (non-editorial) label.
+26
View File
@@ -0,0 +1,26 @@
# src/labels/xml.rs
Tolerant XML scraping helpers — promoted from two near-duplicate hand-rolls
in `paramount.rs` (attribute extraction) and `criterion.rs` (tag-text
extraction).
These are NOT a full XML parser. They handle the subset of XML the BD-J
authoring tools we've seen actually emit: ASCII tag/attr names, no entity
references inside label strings, optional XML namespaces. Hardening goals
over the prior `find("<tag>")` / `find(r#"name=""#)` matchers:
1. **Case-insensitive** tag and attribute names — vendors casing is
inconsistent across authoring-tool revisions.
2. **Namespace-aware** — strip an optional `ns:` prefix so `<ns:playlist>`
and `<playlist>` both match.
3. **Whitespace-tolerant** — multiple/tab/newline characters around `=`
between attribute name and value; whitespace inside the opening tag.
4. **Quote-style tolerant** — both `"value"` and `'value'`.
5. **Self-closing tag handling** — `<tag />` and `<tag/>` both work;
`text()` returns `Some("")` for empty content.
Out of scope (intentionally simple): XML entity decoding (`&amp;`, `&lt;`,
etc.), CDATA sections, comments, processing instructions, DTD
declarations. None of the BD-J authored disc data we've observed
exercises any of those — labels are plain ASCII/Latin-1 in attribute
values.

Some files were not shown because too many files have changed in this diff Show More