Commit Graph
22 Commits
Author SHA1 Message Date
Matthew Jackson fb321f51eb Reject a short READ CAPACITY reply, and count only entry marks as chapters
Two cases of the same shape: one policy implemented twice, with only one
copy hardened.

Disc::read_capacity decoded buf[0..4] from READ CAPACITY (10) without
checking that the transport actually delivered four bytes, even though its
comment claims to mirror decode_read_capacity — which has exactly that
check, and documents why. A drive answering GOOD with an empty data phase
leaves the buffer zeroed, so last_lba decodes to 0 and the probe reports a
one-sector disc instead of an error. It now calls the shared decoder rather
than re-deriving it.

collect_chapter_summary filtered chapters on mark_type <= 1, counting the
reserved type 0. PlaylistMark's own doc says filters must test == 1, and
disc/bluray.rs did; the labels path did not, inflating the public
chapter_count and letting a playlist whose only marks are reserved pass the
chapter_count == 0 skip. Both sites now share PlaylistMark::is_chapter_mark
so the copies cannot drift again.

Both fixes were confirmed red before green.
2026-08-01 11:00:49 -07:00
Matthew Jackson 71686f1407 Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv,
the crate the other seven build against and the one held up as the
reference workflow, was the last one still linting the library only — so
its ~3,000 tests, by far the largest body of test code in the project,
had never been linted at all. Turning the flag on surfaced 74 findings.

Most were mechanical and applied with clippy --fix. The rest, by hand:

- Four discarded Results in decrypt.rs. css::descramble_region returns a
  Result and four CSS tests threw it away, so a descramble that FAILED
  would have surfaced as a confusing buffer-comparison mismatch instead
  of the actual error. They expect() now.
- A dead `kp` field on the PlantedWalk fixture. The test deliberately
  asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4
  rather than against a stored value — its doc comment says so — which
  makes the field not just unused but a trap: the obvious "fix" of
  asserting against it would quietly weaken the test to comparing the
  fixture with itself. Removed.
- Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut,
  three vec!s that only ever needed arrays, a filter_map whose every arm
  was Some, and a Vec::new()+push chain.
- Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering
  in the generated docs.
- A five-[u8; 16]-tuple return type named FourLevelParts.

Three lints are allowed at the specific sites, with reasons, because
they are wrong for this domain: the underscores in the bitstream-header
literals mark BITFIELD boundaries, not digit groups, so regrouping them
uniformly would satisfy the lint by destroying the only thing they
encode; and in three table-validation loops the loop variable is the
domain value under test (a DTS SFREQ code, an AMODE value, a palette
entry number), which is what the assertion messages name.
2026-07-31 15:08:37 -07:00
Matthew Jackson b2e1982051 fix(clpi): resolve out_time past the last EP entry to the end of the clip
ClipInfo::get_extents fell back to `last EP SPN + 1` whenever out_time
lay past the last entry-point. EP entries mark I-frames (BD-ROM Part 3,
CPI / EP map) and a clip's final GOP lies after the last one, so a
PlayItem covering a whole clip — whose OUT_time is the presentation end
— always lands in that arm. The extent then stopped one source packet
after the last I-frame.

Measured on a fixture with 200,000 source packets and the last EP at
SPN 131,072: sector_count came back 12,289 where covering the clip
needs 18,750. Everything from the last entry point to EOF is outside
the returned extent.

Scope, stated plainly: get_extents has NO callers anywhere in the
ecosystem today — it is #[allow(dead_code)] and documented as reserved
for the timestamp-range read path. Nothing ships this loss. It is
fixed now because a latent truncation in extent arithmetic is far
cheaper to correct before it has callers than after.

The SPN at-or-after an out-of-range out_time is the end of the clip,
source_packet_count, with .max(last + 1) so a disc that under-declares
its own packet count against its own EP map still yields a sane bound.

Also 174 mutants killed across clpi, mpls, ifo and ebml — the first
time any of these four files has been examined. And ebml's 8-byte VINT
back-patch was duplicated verbatim in end_master and end_master_buf
with its top four payload octets unreachable through either (they need
a 16 MiB..256 TiB buffer); extracted to fixed_width_vint8 and tested
across the full 56-bit payload, no behaviour change.

38 of ebml's 46 survivors are one equivalence cluster: every | in
write_size / read_id / read_size / read_uint_val ORs into disjoint bit
lanes, where ^ is the identical operation. Applied all 38 at once —
green — then spot-checked four individually.
2026-07-30 14:24:57 -07:00
Matthew Jackson 5f8dc392c0 Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest
under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the
minimum, this goes to current stable and fixes what that costs.

The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54
clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy
collapsible_if lint in between. Stopping at the minimum would have been the most
expensive choice available.

Roughly 47 lints across the eight repos, the large majority auto-fixed:
libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6,
freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to
sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced
by enumerate, and a loop whose first let-else became a while-let.

Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only
appear once the crate DECLARES 1.89 or later, because let-chains become
available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87
reports clean and is meaningless — gate with the real precommit script, which is
also the only thing that covers build scripts.

The pin still sits below the Mac default, so it keeps doing its job: catching
lint drift locally before CI sees it.
2026-07-29 21:00:55 -07:00
Matthew Jackson d8c323bf9f Magic-number/taxonomy pass: central wire-format + sector + unit consts
- libfreemkv::consts: coding_type::* (ES coding-type bytes), pes_stream_id::*
  + PAYLOAD_RANGE, SECTOR_BYTES (usize) + SECTOR_BYTES_U64 (offset math)
- replace bare wire-code/sector literals across disc, mpls, clpi, labels,
  m2ts_mux, ps, tsmux, file_sector_source, extract
- remove two unreachable secondary-stream match arms in mpls parse_stream_entry
2026-06-26 13:20:21 -07:00
Matthew Jackson decb87a250 AACS pipeline reshape + TrueHD metadata + central consts + clippy/fmt clean
- AACS: delete in-lib keydb parser (Step 3); boil-down primitives
  (mk_from_dk/vuk_from_mk/uk_from_vuk) + newtypes; KeySource->get_uk(ctx)+
  ResolveCtx; Unlocker->unlock()->Result<Vid,UnlockError> + AacsCertUnlocker;
  OEM bus-key gate (AacsBusKeyUnavailable); structured ResolutionTrace (Step 4).
- TrueHD: sample-rate from major-sync, Atmos label, 44.1k AU duration.
- consts: central media/format constants module; 17 duplicate const-defs
  centralized (sector/TS-packet/source-packet); mpls stream-entry + category
  codes named.
- clippy --all-targets -D warnings clean (1.86); fmt clean; 2199 lib tests.
2026-06-26 12:19:24 -07:00
Matthew Jackson f79c2a0aa9 libfreemkv 0.31.4: prune 144 vacuous tests (keep spec-grounded subset) 2026-06-08 07:28:55 -07:00
Matthew Jackson 8000bae177 libfreemkv 0.31.2: comprehensive spec-grounded test suite (~950 tests)
Test-hardening release, no runtime changes. Adds spec-grounded unit tests
across the silent-corruption surfaces — UDF/MPLS/CLPI/IFO parsing, BD/DVD
title + extent assembly, AACS/CSS key handling, TS/PS demux + codec parsers,
MKV/EBML container output, the mux pipeline, sector prefetch + decrypt
decorator, drive/SCSI sense decoding, label extraction, and core I/O. Each
test is grounded in the format spec or real on-disc behavior and verified to
fail under a targeted source mutation. No behavior changed.
2026-06-07 22:28:29 -07:00
Matthew Jackson 061f68594a 0.31.0: hardening and correctness pass across mux, codec, AACS/CSS, UDF/MPLS/CLPI, recovery, drive/SCSI, labels, and I/O
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
2026-06-07 17:37:38 -07:00
MattJackson 0a2bab5789 mpls: resolve PID for sub-path / DV-EL stream entries (types 2/3/4)
parse_stream_entry only read the PID for stream-entry type 1 (a stream
in the PlayItem's own clip), returning PID 0 for every other type. The
Dolby Vision enhancement layer uses type 4 (verified on Wicked + Dune:
se_len=9 type=0x04 [00 10 15] -> PID 0x1015), so the EL fell through to
PID 0x0000 and was silently dropped by the demux/mux — Dolby Vision lost.

Now the PID offset is keyed off the entry type per the BD stream_entry()
layout: type 1 -> +2, type 2 -> +4, type 3/4 -> +3. The DV EL now
resolves to its real PID (0x1015) so it is demuxed and carried as a
stream through the PES layer to every writer (M2TS carries dual-PID DV
natively; the MKV writer's DV signaling is the format-specific piece).
2026-06-04 19:28:34 -07:00
Matt Jackson 80fcffd190 v0.11.5: MKV container fixes — timestamps, frame rate, HDR, chapters, disposition 2026-04-18 16:29:21 +00:00
MattJackson f48b4925c1 Zero clippy warnings: fix all 32 remaining
- Iterator::find() replaces manual loops (6 sites)
- Index-only loops → iterators (4 sites)
- Identical if-blocks merged
- Box large MkvStream WriteState enum variant
- Vec macro initializers, late init fixes
- Unused fields prefixed with underscore (format spec fields)
- Dead code removed or documented

0 clippy warnings. 319 tests passing.
2026-04-11 19:33:13 +00:00
MattJackson ffa0eaba4d cargo fmt + clippy --fix: 104 format violations fixed, 8 clippy auto-fixes 2026-04-11 19:10:20 +00:00
MattJackson 96a65de3ff Chapters, DVD subtitle palette, MKV track flags, progress total_bytes
Chapters:
- MPLS PlayList marks parsed (mark_type 1 = chapter)
- Chapter struct on DiscTitle (time_secs, name)
- MKV Chapters element with EditionEntry/ChapterAtom per mark
- 3 MPLS mark tests + 2 MKV chapter tests

DVD subtitle palette:
- IFO palette extraction (PGC offset 0xA4, 16 × YCbCr colors)
- YCbCr→RGB conversion for VobSub .idx format
- DvdSubParser codec_private returns formatted palette
- codec_data field on SubtitleStream flows through pipeline
- 5 palette tests (YCbCr conversion, formatting, overflow)

MKV track flags:
- FlagDefault: primary video/audio = 1, secondary = 0
- FlagForced: forced subtitles = 1
- Language: set from stream language code
- Already implemented, verified with 4 new tests

Progress total_bytes:
- IOStream trait: total_bytes() -> Option<u64>
- DiscStream, IsoStream: from disc_title.size_bytes
- M2tsStream, MkvStream: from file metadata on open
- NetworkStream, StdioStream, NullStream: None

316 tests total, all passing.
2026-04-11 17:43:47 +00:00
MattJackson ff5547363b Audit fixes + DVD support foundation (IFO, PS demux, MPEG-2, CSS crack)
Audit fixes (14 critical, 22 warnings):
- UDF: bounds checks on all ICB/FID parsing from disc data
- SCSI Linux: saturating_sub on residual, CDB length guard, buffer size guard
- SCSI macOS: SCSITaskStatus u32 (was u8 — stack corruption)
- AACS: EC mod_inv returns infinity instead of panic, key reduced mod n
- AACS: do_handshake tries all host certs (was returning on first failure)
- H.264: bounds check on SPS < 4 bytes
- ContentReader: error on missing unit key (was zero-fill)
- KEYDB: flat redirect loop (was recursive), 100MB response limit, Windows HOME fallback
- ISO writer: AVDP extent order, partition length, allocation cap
- Network: removed TCP_NODELAY on bulk stream
- MKV: guard on u64::MAX seek
- disc.rs: saturating_sub on extent offset, simplified dead region code
- cargo fmt (610 violations), cargo clippy --fix (55 auto-fixes)

DVD support (new files):
- src/ifo.rs — IFO parser (VIDEO_TS.IFO, VTS_XX_0.IFO, PGC chains, cells, streams) — 13 tests
- src/mux/ps.rs — MPEG-2 Program Stream demuxer (pack headers, PES, private stream 1) — 12 tests
- src/mux/codec/mpeg2.rs — MPEG-2 video parser (sequence headers, I-frame detection) — 15 tests
- src/css/crack.rs — split-attack algorithm (LFSR cipher needs verification — test ignored)

226 tests total (was 186), 1 ignored (CSS crack needs cipher verification).
2026-04-11 16:52:22 +00:00
MattJackson 995525d3ff Add 113 tests, update CI to checkout@v5, add FEATURES.md
Test suite: 64 → 177 tests
- MPLS parser: 6 tests (synthetic binary, streams, errors)
- CLPI parser: 6 tests (EP map, PTS/SPN math, errors)
- H.264: 12 tests (NAL parsing, SPS/PPS, keyframes)
- HEVC: 13 tests (VPS/SPS/PPS, IRAP range, codec private)
- AC3: 12 tests (syncword, frame extraction)
- VC1: 15 tests (BITMAPINFOHEADER, start codes)
- DTS: 5, TrueHD: 4, PGS: 4 tests
- EBML: 6 tests (size/ID/string/float roundtrips)
- UDF: 10 tests (MockSectorReader, filesystem parsing, error paths)
- Disc: 8 tests (scan_image, DiscTitle helpers)
- Streams: 5 new (meta roundtrip, MkvStream)
- NullStream: 4, StdioStream: 2, IsoSectorReader: 2

CI: actions/checkout@v4 → v5 (all workflows)
FEATURES.md: created for v0.7.1
2026-04-11 16:02:49 +00:00
MattJackson 074f21ba58 Refactor error types: replace generic AacsError/DiscError with typed variants
- Split AacsError { detail } into 13 specific error variants (AacsCertShort,
  AacsAgidAlloc, AacsCertRejected, etc.) with unique error codes E7001-E7012
- Split DiscError { detail } into 7 specific variants (DiscRead, MplsParse,
  ClpiParse, UdfNotFound, DiscNoTitles, DiscTitleRange, DiscNoExtents)
- Add WriteError (E5001), KeydbLoad (E8005), MuxLookahead (E9000), MuxWrite (E9001)
- Add OpenDisc API for single-call open+scan+rip workflow
- Remove all English text from error Display impl (code-only output)
- Normalize doc comments to use -- instead of em dash for ASCII consistency
2026-04-10 08:19:28 -07:00
MattJackson 5e3157b18d Fix subtitle language parsing: PGS in audio slots, secondary PG entries
Two MPLS parsing issues:

1. PGS subtitle (0x90) appearing in audio stream slot: language was read
 from audio offset sa[2..5] instead of PG offset sa[1..4], causing
 truncation ("ng " instead of "eng"). Fixed by detecting PGS coding
 type in audio slots and using correct offset.

2. Secondary PG (PiP subtitle) entries were not consumed, causing
 position tracking to drift. Added missing n_pip_pg loop.

3. Added stream_type 5/6/7 attribute parsing for secondary audio/video.
2026-04-08 22:41:07 -07:00
MattJackson d9c5f7a0d2 Fix read speed: detect kernel max transfer size, fix phantom streams
Root cause: READ_10 requested 510 sectors (1MB) but kernel sg driver
limits to max_hw_sectors_kb=120 (60 sectors). All bulk reads errored,
error handler fell to 3-sector reads → 4.8 MB/s instead of 12+ MB/s.

- detect_max_batch_sectors() now resolves sg→block device via sysfs
 and uses 80% of kernel limit for safety margin
- Default fallback reduced from 510 to 48 sectors
- ContentReader uses per-device detected max, not hardcoded constant
- Filter out coding_type 0x00 (empty/padding) stream entries
- Add MPLS stream_type 5/6/7 attribute parsing (secondary audio/video)

Tested on BU40N: 4.8 → 12.5 MB/s sustained, 23.7 MB/s peak.
2026-04-08 22:30:08 -07:00
MattJackson fd443e3a2f Add disc title, format, streams to Disc::scan() — move logic from CLI to lib
- Disc.volume_id: UDF Volume ID from PVD (always present)
- Disc.meta_title: from META/DL/bdmt_eng.xml (falls back to other languages)
- Disc.format: UHD/BluRay/DVD detected from video codec
- Disc.capacity_bytes, Disc.layers
- Disc.jar_labels: extracted from BDMV/JAR
- Fixed MPLS STN parsing: 16-byte header (was 8), proper stream entry offsets
- Streams now include: HDR, color space, Dolby Vision EL, secondary audio/video
- parse_dstring() for UDF d-string fields
- wait_ready() polls TEST UNIT READY before unlock

Tested on 12 disc captures — all return correct titles, streams, format.
2026-04-07 15:36:05 -07:00
MattJackson bac2e39ee5 AACS 100%: device key tree, MKB SCSI read, AACS2 detection, STN streams
- Device key subset-difference tree: aesg3 key derivation, v_mask calc,
 tree traversal from device key to processing key to media key
- MKB SCSI read: REPORT DISC STRUCTURE 0x83 with multi-pack support
- resolve_keys now has 4 paths:
 1. disc hash → KEYDB → VUK
 2. KEYDB media key + VID → VUK
 3. MKB + processing keys → media key → VUK
 4. MKB + device keys → processing key → media key → VUK
- setup_aacs reads MKB from drive (not just from file)
- AACS 2.0 detection: drive cert type 0x11 detected, falls back to
 AACS 1.0 handshake (P-256 crypto path prepared but not yet built)
- STN table parsing in mpls.rs: video format/rate, audio format/rate/lang,
 subtitle lang, coding type — all streamed into Disc title streams
- 31 tests passing
2026-04-07 11:19:34 -07:00
MattJackson 7db4606038 Add disc format parsers: UDF, MPLS, CLPI
- udf.rs: read files from Blu-ray disc filesystem
- mpls.rs: parse playlists → titles with clips and timestamps
- clpi.rs: parse clip info → EP map with coarse/fine SPN entries
- disc.rs: high-level title scanning, sector extent mapping
- drive.rs: add read_disc() for unencrypted reads, scsi_execute()
- error.rs: add E6000 DiscError

Stage 1 of freemkv rip: identify titles and their sector ranges.
2026-04-06 14:27:48 -07:00