B-frame video PTS is legitimately non-monotonic in decode/storage order; the
audio-oriented monotonic nudge was clobbering it to prev+1ms, which decoders
flagged as non-monotonic DTS (thousands per title). Apply the nudge to
audio/subtitle only; video keeps its true PES PTS. + regression test.
Test-hardening release, no runtime changes. Adds spec-grounded unit tests
across the silent-corruption surfaces — UDF/MPLS/CLPI/IFO parsing, BD/DVD
title + extent assembly, AACS/CSS key handling, TS/PS demux + codec parsers,
MKV/EBML container output, the mux pipeline, sector prefetch + decrypt
decorator, drive/SCSI sense decoding, label extraction, and core I/O. Each
test is grounded in the format spec or real on-disc behavior and verified to
fail under a targeted source mutation. No behavior changed.
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
Add `DiscInputs.volume_label: Option<String>` — the disc's human title (UDF/ISO
volume identifier, else BDMV <di:name>), populated by Disc::inputs() from the
scan. Identity only, no secret, not used in any AACS derivation; lets a key
source forward the title so a key service can catalog disc_hash → title.
Audit-driven fixes (rounds 1–3):
- hevc: correct hvcC profile/level SPS offsets (HEVC has a 2-byte NAL header)
- mkv: map all DTS variants to the registered A_DTS codec id; force a new
cluster before the i16 cluster-relative timestamp can overflow
- ebml/mkvstream: bound untrusted EBML sizes (no multi-GB allocs); reject
uint>8 (was an OOB panic) and non-{0,4,8} float widths (were a desync)
- ts: skip PES-header bytes that span a TS packet boundary; add the PMT
section_len/prog_info_len bounds the PAT parser already had
- ac3: preserve a 0x0B77 syncword split across a PES boundary; cap buffer
- dts: validate each next-core boundary by decoded core size (a 0x7FFE8001
pattern inside XLL payload no longer false-splits/drops the lossless
extension); reject sub-minimum core frames; fix forced-emit PTS base
- lpcm: DVD program-stream PCM no longer double-strips the BD LPCM header
- vc1/mpeg2: do not emit a parameter-set-only PES as a standalone frame
- pgs/truehd: cap the pending reassembly buffer (parity with ac3/dts)
- aacs: ts_syncs_intact uses the exact packet count
- prefetched: capacity-guard the recycled-buffer set_len
- Cargo.toml: exclude project docs from the published crate
Convergence: a third independent audit pass found no remaining material
(CRITICAL/HIGH/MEDIUM) issues. Full precommit (fmt + clippy -D + tests,
Rust 1.86) green.
scan now stashes the raw Unit_Key_RO.inf + MKB bytes on AacsState (via
resolve_vid_only, the disable_keydb path), so an external key-resolver can
derive unit keys from a resolved VUK without re-reading the disc — the
foundation for moving lookup/derivation out of libfreemkv. Additive: the keydb
path is untouched, all existing constructors default the new fields empty.
584 lib tests green. Builds on the KeyOrigin rename + the Key/decrypt_with API.
Add libfreemkv::Key (non_exhaustive; Unit wired) + Disc::decrypt_with(Key) — the
caller resolves a key from any source and hands it in; the library decrypts with
no keydb/network of its own. inject_unit_keys now synthesizes a minimal
ExternalUk AACS state when a scan built none (no keydb at scan time), so a disc
keyed at sweep but scanned without a keydb still decrypts from its mapfile UK
instead of reporting E8005 and deferring the mux forever. Tests cover the
synthesize, replace, empty->None, and decrypt_with paths.
Rename is_unit_encrypted -> is_aacs_scrambled and decide encryption from the unit's MPEG-TS sync bytes (destroyed by the encrypted body) instead of the TP_extra copy-control (byte 0) or TS scrambling-control (byte 7) flags, which discs do not set reliably. One shared predicate now backs the decrypt gate and out-of-band key validation, so callers agree on what 'encrypted' means. Decryption restores the syncs, so a decrypted unit reads as clear and there is no flag to clear.
is_unit_encrypted read the TP_extra copy-control bits (byte 0), which are a
copy-permission flag, not an encryption flag. On discs whose sampled units are
clear navigation packets (PAT/PMT) those bits can be set while the unit is not
scrambled, so a correct Unit Key was used to 'decrypt' already-plaintext data,
produced garbage, and the key was wrongly treated as failing.
Read the actual flag instead: the TS transport_scrambling_control bits (top two
of TS-header byte 3 = byte 7 of the aligned unit, inside the clear seed). AACS
encrypts whole aligned units, so this one packet's TSC reflects the unit.
decrypt_unit now clears the TSC bits of every packet on the way out so the
result is valid unscrambled TS. Tests updated to the TSC flag.
Adds a 5th key-resolution path that consumes pre-decrypted unit keys
directly from KEYDB when the entry has no VUK field. Covers ~4,572
entries in the public keydb (~2.5%), heavily skewed toward MKBv76+ UHD
discs where DVDFab/FindVUK can no longer extract a VUK but does extract
unit keys. Partial CPS-unit coverage is rejected so a disc is never
half-decrypted.
Resolver path order reordered root-to-leaf: DK (1) → PK (2) →
KEYDB-derived MK+VID (3) → KEYDB VUK (4) → KEYDB unit keys (5).
Previous order was leaf-first.
API:
- AacsState::vuk is now Option<[u8; 16]> (was [u8; 16])
- ResolvedKeys::vuk is now Option<[u8; 16]> (was [u8; 16])
- KeySource variants reordered + new KeyDbUnitKeys variant
3 new resolver tests (path 4 still works without VID; path 5 succeeds
with pre-decrypted unit keys; path 5 rejects partial CPS coverage).
- Introduce DrmScheme enum (Css/Aacs10/Aacs20/Aacs21) + drm module with
uniform detect/load dispatch across all four protection schemes.
- Land AACS 2.1 Media Key Variant framework in aacs::variants: chain
derivation, MKB record types 0x82/0x83, bit-0x02 SoftKCD and bit-0x04
online-challenge detection. Aacs21 dispatcher arm wired but commented
out pending validation against a Variant-scheme disc.
- Replace aacs2: bool with AacsVersion enum across ContentCertificate,
UnitKeyFile, ResolvedKeys. resolve_keys splits into _v1/_v2/_v21.
- Delete the libredrive raw-read VID shortcut from do_handshake; the
drive enforces the AGID requirement regardless of firmware-upload
state, so the shortcut spuriously dispatched E7017 instead of
surfacing the real downstream walls.
Two changes that make AACS 1.0 / DVD self-sufficient:
1. MKB record-type identification bug fix. `mkb_find_mk_dv` was
searching for type 0x10 (which is Type-and-Version, 12 bytes)
when the Verify Media Key Record is actually type 0x81 for
AACS 1.0 or type 0x86 for AACS 2.0/2.1. `mkb_version` had the
inverse bug. PK and DK derivation paths therefore silently
failed on every disc, masking how often the fallback paths
could have worked. Fix searches the correct types; tests added
covering both the 0x81 and 0x86 verify-record forms and the
0x10 version record at offset 8 of the body.
2. Built-in AACS keys + operator plugin slot. Four device keys
(covering MKB v01-v82+) and three processing keys (covering
v63-v68) compiled directly into the library. Combined with the
31 CSS player keys already in css/auth.rs, DVDs and Blu-rays
(AACS 1.0) now decrypt with zero external files. New plugin
path at ~/.config/freemkv/local_keys.cfg (same syntax as
keydb.cfg) layered additively on top of built-ins and main
keydb. `Disc::scan` no longer errors when keydb.cfg is absent;
AACS 2.0 / UHD still surfaces a specific error when the disc
needs keys none of the layers provide.
Public docstrings in project docs + README updated to describe the
three additive layers (built-ins → keydb.cfg → local_keys.cfg).
Pre-0.25.7 the AACS authenticate loop fired up to 16 host-cert
attempts back-to-back with no pause. Each attempt is 5-10 SCSI
REPORT_KEY/SEND_KEY exchanges, so on a disc whose host cert isn't
in our KEYDB (or one the drive rejects), the drive saw 80-160 SCSI
commands in a few hundred ms and entered a fast-fail firmware
wedge state where every subsequent CDB returns sense 05/24 until
power-cycled.
Three defences:
- MAX_CERT_ATTEMPTS capped at 3 (was 16)
- 1-second sleep between attempts
- Bail immediately on any sense_key == 0x05 (ILLEGAL_REQUEST) so
the loop can't deepen the wedge if a regression undoes the
attempt cap.
- MkvTrack::audio emits A_DTS/MA, A_DTS/HR, A_DTS per the DTS family
instead of mislabelling everything as A_DTS. Plex transcoder and
strict hardware decoders reject DTS-HD MA payload under a plain
A_DTS track.
- PgsParser is now stateful: pairs display PCS with the following
empty PCS to compute a duration. Frame::duration_ns + PesFrame::duration_ns
carry it through; MkvMuxer::write_frame gains a final Option<u64>
parameter that emits BlockGroup + BlockDuration when set. Fixes
subtitle bitmaps lingering past their intended end-time.
Introduces the freemkv mux throughput highway: a three-stage thread
pipeline that replaces the inline single-thread read path for any
file-backed source (ISO and m2ts file URLs both route through it).
Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher)
Thread B: M2TS demux (DemuxThread)
Thread C: codec parse (PipelinedPesStream, on caller thread)
Each handoff uses a bounded crossbeam channel with a recycled buffer
pool — no allocations or memcpys in the steady-state hot loop.
Component map:
* io/byte_prefetcher.rs (new) — std::io::Read producer thread with
recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector
side) so demux_thread::spawn_zero_copy can wire either upstream.
* sector/prefetched.rs — recycled buffer pool added; into_channels()
peels off the rx/recycle_tx/shell triple for zero-copy demux.
* mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs
feed() on its thread, ships Vec<PesPacket> batches.
* mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls
packets from the demux thread and runs codec parse on the caller.
* mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline
(private) assemble the three stages; iso:// and m2ts:// both
return PipelinedPesStream.
* mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted;
the read direction lives on the highway now).
* mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find.
* mux/codec/hevc.rs — tightened frame_data initial capacity.
* mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB
remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page
first-touch fault tax that the previous 256 KiB pre-alloc paid on
every PES boundary.
* mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as
the legacy autorip ingress (drive + multipass paths still need
on_event / skip_errors before they migrate to the highway).
* io/file_sector_source/* — per-OS prefetch() syscall hook
(Linux readahead, macOS F_RDADVISE, Windows/other no-op).
* decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS;
pool sized to all cores by default.
Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://):
60 → 322 MB/s warm cache (old new_pipeline path)
60 → 660 MB/s warm cache (highway path, this commit)
60 → 126 MB/s sustained disk-bound
The IsoSectorReader baseline reader was deleted in favour of
FileSectorSource so the freemkv CLI and autorip exercise the same
read path.
Matthew Jackson