Round 4: fix 26 defects across crypto, resource use and codec paths

Twenty-six confirmed findings from the fourth audit round, landed as one
cluster because they were found by agents working over disjoint file sets.

The one worth calling out is a pair of AACS tests that could not fail.
Both asserted CBC behaviour against a hand-rolled expectation that
happened to be IV-independent, so replacing AACS_IV with sixteen zero
bytes left them passing — they were pinning the code's own arithmetic,
not the published constant. Replaced with a literal witness of the
published IV plus the NIST SP 800-38A F.2.2 CBC-AES128 vector, and
verified the other way round: zeroing AACS_IV now fails three tests.

The rest are allocation and correctness work on hot paths: the Annex-B
writer in demux_sink allocated and freed a whole-frame Vec per frame,
which for a UHD title is ~200,000 allocations over the mmap threshold
plus the page faults to first-touch each one; it now reuses a buffer on
the writer, and still takes the NAL prefix width from the configuration
record rather than assuming four.

Six findings whose real fix lives in a consumer crate are recorded for
re-filing rather than patched here.
This commit is contained in:
Matthew Jackson
2026-07-29 22:09:52 -07:00
parent 4fcd28b487
commit 0bbceed985
19 changed files with 1342 additions and 126 deletions
+207 -11
View File
@@ -138,13 +138,28 @@ pub struct HevcParser {
// is consumed (cleared) by that first CRA so only ONE CRA per boundary is
// touched — never a mid-stream CRA, never an IDR, never a non-CRA NAL.
//
// SAFETY: defaults to `false` and is ONLY ever set through
// `mark_clip_boundary`, which the caller invokes ONLY for a non-seamless
// (0x05/0x06) join. connection_condition 0x01 is the first-item/seamless
// case and must NOT trigger this flag. A stream with no boundary marker
// (single-clip title, or seamless-joined 0x01 UHD/BD) never has this set,
// so the rewrite branch is never reached and output is byte-identical to a
// parser without this field.
// ARMED BY TWO PATHS — do not read this flag as caller-driven only:
//
// 1. `mark_clip_boundary`, which a caller invokes only for a non-seamless
// (0x05/0x06) join. connection_condition 0x01 is the first-item/seamless
// case and must NOT trigger this flag. In practice NO caller wires this
// up: the mpls connection_condition is not plumbed through the threaded
// mux pipeline (see the note at the auto-detect site in `parse`).
// 2. The in-parser PTS-backstep AUTO-DETECTION in `parse` — a backward PES
// PTS step beyond `BACKSTEP_TICKS` sets it with no caller involvement.
// This is the path that actually fires in production, and it is the one
// the CRA→BLA rewrite exists for.
//
// So the rewrite branch is NOT dead, and output is NOT byte-identical to a
// parser without this field: any stream whose PES PTS steps backward by more
// than `BACKSTEP_TICKS` — including a damaged/rewritten PTS field on an
// untrusted disc — has its next CRA_NUT rewritten to BLA_W_LP, which makes a
// decoder discard that CRA's valid RASL leading pictures. That false-arming
// risk is held down by the `PTS_WRAP_PERIOD` unwrapping and the high-water
// watermark, not by the flag being unreachable: REMOVING either guard on the
// strength of "only `mark_clip_boundary` sets this" is a live corruption bug.
// Pinned by `cra_at_auto_detected_pts_backstep_rewritten_to_bla` and
// `cra_after_33bit_pts_wrap_not_rewritten`.
pending_clip_boundary: bool,
// Highest PES PTS seen on this video stream so far, on a MONOTONIC 64-bit
// timeline (raw 33-bit PTS unwrapped across 2^33 wraparounds — see
@@ -216,6 +231,24 @@ const _: () = assert!(
"HEVC BACKSTEP_TICKS must mirror mux::timeline::DISCONTINUITY_BACKSTEP_NS"
);
// Bytes reserved at the front of every assembled access unit so the keyframe
// parameter-set re-assert can be spliced in without reallocating. A VPS + SPS +
// PPS re-assert is a few hundred bytes (each a 4-byte length prefix plus a NAL
// that is tens to low hundreds of bytes on real BD/UHD streams); 1 KiB covers it
// with margin, and costs 1 KiB of slack per in-flight frame. If a stream's
// parameter sets ever exceed this the splice still produces correct output — it
// just reallocates once, exactly as it always did.
const PARAM_REASSERT_HEADROOM: usize = 1024;
// Per-thread count of keyframe re-asserts that had to reallocate the frame buffer.
// Test-only instrumentation: the whole point of `PARAM_REASSERT_HEADROOM` is that
// the splice is in-place, so that is MEASURED rather than reasoned about. See
// `keyframe_param_reassert_does_not_reallocate_the_frame`.
#[cfg(test)]
thread_local! {
static PARAM_REASSERT_REALLOCS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
// The 33-bit 90 kHz PES PTS counter wraps at 2^33 ticks (~26.5 h). When the raw
// PTS steps backward by approximately a full period — i.e. it landed just past
// the wrap — it is a counter wraparound, NOT a clip reset: unwrap it (add 2^33)
@@ -302,6 +335,16 @@ impl HevcParser {
/// Unknown payload types are skipped by their size so a later HDR10 message
/// in the same NAL is still reached.
fn scan_sei(&mut self, nal: &[u8]) {
// Both HDR10 messages are per-stream constants and STICKY (first seen
// wins), so once both are captured every remaining match arm below
// declines and the whole scan is a guaranteed no-op. Return BEFORE
// `strip_emulation_prevention`, which allocates and byte-copies the entire
// SEI RBSP: an HDR10 UHD stream carries a prefix SEI per access unit, so
// without this the other ~200,000 access units of a title each paid one
// allocation and one copy for a result that is discarded.
if self.sei_mastering.is_some() && self.sei_content_light.is_some() {
return;
}
let Some(raw) = nal.get(2..) else {
return;
};
@@ -532,7 +575,11 @@ impl CodecParser for HevcParser {
// Pre-size: output is ~input bytes with a few 4-byte length
// prefixes added. UHD frames are 150-300 KB; the unsized Vec
// growth chain otherwise reallocs 5-7× per frame.
let mut frame_data = Vec::with_capacity(data.len() + 64);
//
// Plus `PARAM_REASSERT_HEADROOM` so the keyframe parameter-set re-assert
// below can be spliced in FRONT of the frame without reallocating. See
// that site.
let mut frame_data = Vec::with_capacity(data.len() + 64 + PARAM_REASSERT_HEADROOM);
// Single-pass NAL scan: extract params, detect keyframes, build length-prefixed output
let mut pos = 0;
@@ -670,13 +717,34 @@ impl CodecParser for HevcParser {
// when active == codecPrivate) so each keyframe is self-contained and a
// decoder that dropped the set (CRA reset / SPS event) self-heals.
if keyframe {
let mut prefix = Vec::new();
let mut prefix = Vec::with_capacity(PARAM_REASSERT_HEADROOM);
reassert_active(&mut prefix, &self.cur_vps, emitted_vps);
reassert_active(&mut prefix, &self.cur_sps, emitted_sps);
reassert_active(&mut prefix, &self.cur_pps, emitted_pps);
if !prefix.is_empty() {
prefix.extend_from_slice(&frame_data);
frame_data = prefix;
// SPLICE the few hundred prefix bytes into the front of the
// already-assembled frame, in place.
//
// This used to be `prefix.extend_from_slice(&frame_data)` followed
// by `frame_data = prefix`: that grew `prefix` from a few hundred
// bytes to the FULL access-unit size (a fresh multi-MB allocation),
// memcpy'd the whole frame into it, and dropped the presized
// `frame_data` buffer — one extra whole-frame allocation plus one
// extra whole-frame copy per keyframe. A 2 h UHD title at 24 fps
// with a 1 s GOP is ~7,200 keyframes, i.e. ~7,200 multi-MB
// allocations and ~14-28 GB of avoidable memcpy per title.
//
// `frame_data` was reserved with `PARAM_REASSERT_HEADROOM` to spare
// precisely so this splice fits without reallocating; what remains
// is one in-place memmove inside the existing buffer. Byte-identical
// output either way.
#[cfg(test)]
let cap_before = frame_data.capacity();
frame_data.splice(0..0, prefix);
#[cfg(test)]
if frame_data.capacity() != cap_before {
PARAM_REASSERT_REALLOCS.with(|c| c.set(c.get() + 1));
}
}
}
@@ -839,9 +907,22 @@ struct SpsChroma {
temporal_id_nesting_flag: u8,
}
// Per-thread count of `strip_emulation_prevention` calls. Test-only
// instrumentation: the function allocates and byte-copies a whole RBSP, and
// `scan_sei` used to run it for every SEI NAL of every access unit, so "how many
// copies did a stream actually cost" is worth MEASURING rather than reasoning
// about. Thread-local, not a global atomic, because `cargo test` runs tests
// concurrently. See `scan_sei_stops_copying_once_both_hdr10_messages_are_captured`.
#[cfg(test)]
thread_local! {
static RBSP_COPIES: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// Strip HEVC/H.264 emulation-prevention bytes (00 00 03 → 00 00) from a NAL
/// RBSP so a bit reader sees the true coded values.
fn strip_emulation_prevention(rbsp: &[u8]) -> Vec<u8> {
#[cfg(test)]
RBSP_COPIES.with(|c| c.set(c.get() + 1));
let mut out = Vec::with_capacity(rbsp.len());
let mut zeros = 0usize;
for &b in rbsp {
@@ -1169,6 +1250,67 @@ mod tests {
assert_eq!(h.max_pic_average_light_level, maxfall);
}
/// MEASURED, not reasoned: an HDR10 stream carries a prefix SEI per access
/// unit, and `scan_sei` allocated + byte-copied the whole SEI RBSP through
/// `strip_emulation_prevention` on EVERY one — including after both HDR10
/// messages were already captured and every match arm was guaranteed to
/// decline. On a ~200,000-frame UHD title that is ~200,000 allocations and
/// copies for a discarded result. Counted at the single
/// `strip_emulation_prevention` site.
#[test]
fn scan_sei_stops_copying_once_both_hdr10_messages_are_captured() {
let pps = {
let mut v = vec![0x00, 0x00, 0x01];
v.extend_from_slice(&hevc_nal_header(NAL_PPS));
v.push(0xC0);
v
};
let idr = {
let mut v = vec![0x00, 0x00, 0x01];
v.extend_from_slice(&hevc_nal_header(19));
v.push(0xEC);
v
};
// Every AU carries BOTH HDR10 SEI messages, as a real HDR10 stream does.
let au = || {
let mut data = pps.clone();
data.extend_from_slice(&sei_nal(&[
sei_message(
SEI_MASTERING_DISPLAY_COLOUR_VOLUME,
&mastering_payload([1, 2, 3], [4, 5, 6], 7, 8, 9, 10),
),
sei_message(SEI_CONTENT_LIGHT_LEVEL_INFO, &cll_payload(1000, 400)),
]));
data.extend_from_slice(&idr);
data
};
let mut parser = HevcParser::new();
// First AU: both messages captured, so this one legitimately copies.
parser.parse(&make_pes(au(), Some(0)));
assert!(
parser.sei_mastering.is_some() && parser.sei_content_light.is_some(),
"first AU must capture both HDR10 messages"
);
// Now measure the next 50 AUs, whose SEI scan is a guaranteed no-op.
RBSP_COPIES.with(|c| c.set(0));
for i in 0..50 {
parser.parse(&make_pes(au(), Some(3750 * (i + 1))));
}
let copies = RBSP_COPIES.with(|c| c.get());
assert_eq!(
copies, 0,
"SEI RBSP must not be copied once both HDR10 messages are captured; \
{copies} copies over 50 access units"
);
// And the captured metadata is still surfaced on those later frames.
let f = parser.parse(&make_pes(au(), Some(3750 * 51)));
assert!(
f[0].coding.unwrap().hdr10().is_some(),
"the sticky HDR10 metadata must still ride every later frame"
);
}
/// Only the mastering-display SEI (no content-light SEI) → metadata is NOT
/// surfaced. HDR10 requires BOTH; a half-populated record is never emitted.
#[test]
@@ -1510,6 +1652,60 @@ mod tests {
);
}
/// MEASURED: the keyframe parameter-set re-assert must be spliced into the
/// front of the already-assembled access unit IN PLACE, not built as a fresh
/// full-size buffer.
///
/// It used to `prefix.extend_from_slice(&frame_data)` and then replace
/// `frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
/// FULL access-unit size — a fresh multi-MB allocation — memcpy'd the whole
/// frame into it, and dropped the presized buffer. One extra whole-frame
/// allocation plus one extra whole-frame copy per keyframe: a 2 h UHD title at
/// 24 fps with a 1 s GOP is ~7,200 keyframes, ~14-28 GB of avoidable memcpy per
/// title. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates;
/// this counts the reallocations that happen, which must be zero.
#[test]
fn keyframe_param_reassert_does_not_reallocate_the_frame() {
fn nal(t: u8, body: &[u8]) -> Vec<u8> {
let mut v = vec![0x00, 0x00, 0x01];
v.extend_from_slice(&hevc_nal_header(t));
v.extend_from_slice(body);
v
}
let sps_body = [0x01u8; 24];
let pps_body = [0xA1u8, 0xA2, 0xA3];
let mut parser = HevcParser::new();
// AU1 seeds the active VPS/SPS/PPS.
let au1 = [
nal(32, &[0xAA; 12]),
nal(33, &sps_body),
nal(34, &pps_body),
nal(19, &[0x10; 4096]),
]
.concat();
parser.parse(&make_pes(au1, Some(0)));
// A run of BARE keyframes (source omits the parameter sets), each of which
// takes the re-assert path. Payload sized like a real coded picture so a
// reallocation would be the expensive one.
PARAM_REASSERT_REALLOCS.with(|c| c.set(0));
for i in 0..30i64 {
let au = nal(19, &vec![0x11u8; 300_000]);
let f = parser.parse(&make_pes(au, Some(3600 * (i + 1))));
// The re-assert really happened (otherwise the count is vacuously 0).
assert!(
f[0].data.len() > 300_000,
"keyframe {i} must carry the re-asserted parameter sets"
);
}
let reallocs = PARAM_REASSERT_REALLOCS.with(|c| c.get());
assert_eq!(
reallocs, 0,
"the parameter-set splice must fit in the reserved headroom; \
{reallocs} of 30 keyframes reallocated the whole frame"
);
}
/// Regression (Fight Club UHD, the real bug): id 0 is body A (→ hvcC), then
/// redefined to B, then the title switches BACK to A. A streaming decoder
/// (hvcC at init, in-band updates only) is sitting on B; the switch back to
+94 -12
View File
@@ -17,6 +17,32 @@ use super::dropgate::DropTally;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
use crate::mux::timeline::DISCONTINUITY_BACKSTEP_NS;
/// Is `w` an MLP-family major sync — a random-access / decoder re-init point?
///
/// The 24-bit signature is 0xF8726F; the following byte is the STREAM TYPE:
/// 0xBA = Dolby TrueHD (the only one Blu-ray carries), 0xBB = MLP. Both are
/// restart points, so the keyframe / re-sync decision accepts either.
fn is_mlp_major_sync(w: u32) -> bool {
(w & 0xFFFF_FFFE) == 0xF872_6FBA
}
/// Is `w` specifically the TrueHD major sync (stream type 0xBA)?
///
/// The 32-bit word that FOLLOWS the sync is laid out per stream type: TrueHD's
/// `format_info` carries [31..28] audio_sampling_frequency, the 5-bit 6-channel
/// and 13-bit 8-channel presentation channel-assignment masks; MLP's (0xBB) same
/// word carries quantization word lengths and the MLP group sample-rate fields
/// instead. So every site that DECODES `format_info` with the TrueHD layout must
/// require 0xBA exactly — masking the low sync bit there read a quantization code
/// as the rate nibble and pulled channel masks out of unrelated bits, giving the
/// track header a wrong `SamplingFrequency` and `truehd_au_duration_ns` a wrong
/// per-AU increment (audio drifting against video for the whole track).
/// `None` from these helpers is the safe outcome: the caller falls back to its
/// container-derived rate/channel count.
fn is_truehd_major_sync(w: u32) -> bool {
w == 0xF872_6FBA
}
/// Duration of one TrueHD access unit in nanoseconds for the 48 kHz family
/// (48 / 96 / 192 kHz). `access_unit_size = 40 << (ratebits & 7)` and
/// `sample_rate = 48000 << (ratebits & 7)`; the shared shift cancels in
@@ -130,7 +156,13 @@ impl TrueHdParser {
// The rate nibble is only trustworthy once the major sync's CRC has
// validated (above), so capture format_info here and refine the PTS
// cadence from it ONLY on this validated path.
if au.len() >= 12 {
// ONLY for stream type 0xBA. An MLP (0xBB) major sync's following word
// is not the TrueHD `format_info` layout, so decoding it as one gave a
// wrong rate and channel count; leaving it `None` keeps the
// container-derived rate, which is the honest answer.
if au.len() >= 12
&& is_truehd_major_sync(u32::from_be_bytes([au[4], au[5], au[6], au[7]]))
{
format_info = Some(u32::from_be_bytes([au[8], au[9], au[10], au[11]]));
}
}
@@ -445,10 +477,17 @@ impl CodecParser for TrueHdParser {
break; // incomplete access unit, wait for more data
}
// Restart-point question — either stream type (0xBA TrueHD, 0xBB MLP)
// is a decoder re-init point, so both count as a major sync here.
// DECODING format_info with the TrueHD layout is a separate question,
// gated on 0xBA alone in `au_check`.
let is_major_sync = unit_bytes >= 8
&& (u32::from_be_bytes([self.buf[4], self.buf[5], self.buf[6], self.buf[7]])
& 0xFFFF_FFFE)
== 0xF872_6FBA;
&& is_mlp_major_sync(u32::from_be_bytes([
self.buf[4],
self.buf[5],
self.buf[6],
self.buf[7],
]));
// Decodability gate. MLP/TrueHD decode state persists across access
// units, so a corrupt AU is dropped FORWARD to the next VALIDATED
@@ -597,7 +636,9 @@ pub fn truehd_channels_from_stream(data: &[u8]) -> Option<u8> {
let mut p = 0;
while p + 8 <= data.len() {
let w = u32::from_be_bytes([data[p], data[p + 1], data[p + 2], data[p + 3]]);
if (w & 0xFFFF_FFFE) == 0xF872_6FBA {
// 0xBA only: `truehd_channels` reads the TrueHD `format_info` channel
// masks, which an MLP (0xBB) major sync does not carry.
if is_truehd_major_sync(w) {
let fi = u32::from_be_bytes([data[p + 4], data[p + 5], data[p + 6], data[p + 7]]);
return truehd_channels(fi);
}
@@ -663,7 +704,9 @@ pub fn truehd_sync_info_from_stream(data: &[u8]) -> Option<TrueHdSyncInfo> {
let mut p = 0;
while p + 8 <= data.len() {
let w = u32::from_be_bytes([data[p], data[p + 1], data[p + 2], data[p + 3]]);
if (w & 0xFFFF_FFFE) == 0xF872_6FBA {
// 0xBA only: `format_info` (and the num_substreams/Atmos nibble) are the
// TrueHD layout, not MLP's.
if is_truehd_major_sync(w) {
let format_info =
u32::from_be_bytes([data[p + 4], data[p + 5], data[p + 6], data[p + 7]]);
// num_substreams is the top nibble of the 17th sync byte (p + 16).
@@ -1447,16 +1490,53 @@ mod tests {
// --- truehd_channels_from_stream: major-sync variant bit + scan ---
#[test]
fn channels_from_stream_matches_variant_sync_0xfb() {
// The sync match masks the low bit: 0xF8726FBA & 0xFFFFFFFE == base, and
// 0xF8726FBB (the +1 variant) matches the same masked pattern. A stream
// carrying 0xF8726FBB must still be recognised.
fn channels_from_stream_rejects_mlp_sync_0xfb() {
// 0xF8726FBB is the MLP stream type, NOT TrueHD (0xF8726FBA). The word
// after an MLP major sync holds quantization word lengths and the MLP
// group sample-rate fields, not TrueHD's rate nibble plus the 6ch/8ch
// presentation channel-assignment masks. Decoding it with the TrueHD
// layout (the scan used to mask the low sync bit) reported channels out of
// unrelated bits; the honest answer is None so the caller keeps its
// container-derived count. Byte pattern below decodes as 8 channels ONLY
// under the TrueHD layout, so this test fails if the mask comes back.
let mut data = vec![0x00];
data.extend_from_slice(&0xF872_6FBBu32.to_be_bytes());
data.extend_from_slice(&0x0000_001Fu32.to_be_bytes());
assert_eq!(
truehd_channels_from_stream(&data),
None,
"an MLP (0xBB) major sync must not be decoded as TrueHD format_info"
);
// The same bytes under the TrueHD stream type DO decode.
let mut data = vec![0x00];
data.extend_from_slice(&0xF872_6FBAu32.to_be_bytes());
data.extend_from_slice(&0x0000_001Fu32.to_be_bytes());
assert_eq!(truehd_channels_from_stream(&data), Some(8));
}
#[test]
fn mlp_sync_0xfb_yields_no_sample_rate_or_atmos() {
// Same split for the shared scan: an MLP major sync must not produce a
// TrueHD rate (bits 31..28 of an MLP header are a quantization code, not
// the rate) nor an Atmos verdict.
let mut data = vec![0x00];
data.extend_from_slice(&0xF872_6FBBu32.to_be_bytes());
// ratebits nibble 0x1 would decode as 96 kHz under the TrueHD layout.
data.extend_from_slice(&0x1000_001Fu32.to_be_bytes());
data.extend_from_slice(&[0x00; 12]);
assert!(
truehd_sync_info_from_stream(&data).is_none(),
"no TrueHD sync info from an MLP major sync"
);
assert_eq!(truehd_sample_rate_from_stream(&data), None);
// TrueHD stream type, identical trailing bytes → the rate IS decoded.
let mut data = vec![0x00];
data.extend_from_slice(&0xF872_6FBAu32.to_be_bytes());
data.extend_from_slice(&0x1000_001Fu32.to_be_bytes());
data.extend_from_slice(&[0x00; 12]);
assert_eq!(truehd_sample_rate_from_stream(&data), Some(96000));
}
#[test]
fn channels_from_stream_none_without_major_sync() {
// No major sync anywhere → None, no panic, scan terminates.
@@ -1527,8 +1607,10 @@ mod tests {
#[test]
fn major_sync_variant_bit_also_keyframe() {
// The keyframe check masks the low bit (0xFFFF_FFFE), so the 0xF8726FBB
// variant must also be detected as a major sync.
// The RESTART-POINT check masks the low sync bit, so 0xF8726FBB (MLP)
// counts as a major sync too — both stream types re-init the decoder, so
// both are keyframes. (Only the format_info DECODE is 0xBA-only; see
// `channels_from_stream_rejects_mlp_sync_0xfb`.)
let mut parser = TrueHdParser::new();
let mut unit = make_truehd_unit(200);
unit[4..8].copy_from_slice(&0xF872_6FBBu32.to_be_bytes());