Fix fifteen defects across perf, resource, panics and key hygiene
All 21 findings held up under verification; 15 fixed here, 6 deferred to files another agent held this round, 0 rejected. **A defect in my own round-2 probe fix.** CHUNK_SECTORS was 1024, and 1024 % 3 == 1 — verified — so every chunk after the first was misaligned against the 6144-byte AACS aligned unit and would be REJECTED by DecryptingSectorSource's alignment gate. On an encrypted disc the forced-subtitle probe I added last round would have read almost nothing past its first chunk. Now 1023 sectors (341 aligned units) with a const assertion that fails the build if it stops dividing, plus set_unit_base per extent so the source's gate is anchored where the extent actually starts. **The same probe skipped sectors on a short read**, advancing by the REQUESTED count rather than the bytes actually returned, so a partial read silently left a gap in the middle of the evidence. It now advances by n/SECTOR_BYTES and clamps n to the buffer. **Its cache key omitted the PGS PID set**, so a playlist declaring an extra subtitle PID got another playlist's verdict for a track that had never been probed. And the key was the whole extent list, so partial clip sharing missed entirely. Both fixed by keying (start_lba, sector_count, pid) — and per-extent keying was shown SOUND rather than assumed: ForcedTracker is two monotone booleans, so per-extent evidence composes by field-wise OR, order- and grouping-independently. Making that honest required per-extent demux state, so an extent's evidence comes only from its own bytes, and memoising only extents whose read reached a designed stop. **A reachable panic in the timeline.** mkvstream::parse_block accepts a TimestampScale up to i64::MAX, so a video frame can set high_ns = i64::MAX and the next passive frame panicked adding the backstep. In release it wrapped negative instead, firing the straggler clamp for essentially every passive frame — audio and subtitles rewritten onto the wrong point of the output timeline. All four sites saturate. **A public constructor divided by zero**: PrefetchedSectorSource::new_with_events with unit_align == 0. Now InvalidInput, matching its batch_sectors sibling. **Two Debug impls printed key material.** DiscInputs (volume_id, mkb, unit_key_ro, samples) and UnitKeyFile both derived Debug. Nothing logs them today — fixed as prevention, because the next tracing::debug! someone adds is the leak. A doc claim that DiscInputs "contains no secrets" was false and is corrected. **An env-var multiply could overflow** in file_sector_source; now bounded at 64 GiB like its writeback sibling, with the parse split out so the bound is testable without touching process env. **The mp4 demuxer allowed one sample per file byte** — ~64x RAM amplification. Now file_len/16, since only vide/soun tracks are indexed and the shortest legal AC-3 frame is 128 bytes. **Two pipeline concurrency defects**: a consumer apply() error was invisible to the producer, and abandon/finalise had a TOCTOU where a caller could report an unfinalised output. Both fixed with compare-exchange state rather than a bool. **Two per-frame copies removed**, both MEASURED rather than reasoned: the AU assembler now hands its allocation to the frame (same pointer, unchanged capacity, proven by asserting the pointer) and tsmux reuses one Annex-B buffer across frames. Both keep capacity deliberately — a naive split_off would have cost more than it saved. **A comment pointed at the wrong file** for a mirrored constant; the mirror is now compiler-enforced with a const assertion converting 90 kHz ticks to ns, so drift fails the build. Deferred to another agent's files, all confirmed: detect_rate's fractional-twin snap, the mp4 reserve's u32 truncation, round_up_grain's overflow, the quadratic base-key gap fill, and MkvStream's frame cap counting frames rather than bytes. Every fix verified red by reverting it. Also noted for later: DecodeSampleSet still derives Debug over multi-MB of on-disc ciphertext.
This commit is contained in:
+88
-16
@@ -70,6 +70,15 @@ pub struct TsMuxer<W: Write> {
|
||||
/// so a header-only `m2ts://` output can't be reported as success —
|
||||
/// mirroring `MkvMuxer.frame_count`.
|
||||
frame_count: u64,
|
||||
/// Reusable Annex-B conversion buffer for NAL video, kept across frames so
|
||||
/// the conversion does not allocate (and free) a whole-frame buffer per
|
||||
/// coded picture — the same reason `MkvMuxer` keeps its `block_group_buf`.
|
||||
/// A UHD frame is ~310 KB, i.e. an mmap + first-touch page faults + munmap
|
||||
/// per frame, ~200k times per feature. Cleared (never shrunk) per use, so it
|
||||
/// settles at the largest frame's size. Taken out of `self` while in use, so
|
||||
/// the borrow of the converted bytes does not conflict with the `&mut self`
|
||||
/// the writer needs.
|
||||
annex_b: Vec<u8>,
|
||||
}
|
||||
|
||||
impl<W: Write> TsMuxer<W> {
|
||||
@@ -84,6 +93,7 @@ impl<W: Write> TsMuxer<W> {
|
||||
video_codec: vec![Codec::Hevc; n],
|
||||
base_pts_ns: None,
|
||||
frame_count: 0,
|
||||
annex_b: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -180,11 +190,20 @@ impl<W: Write> TsMuxer<W> {
|
||||
// start-code ES, never length-prefixed) the bytes pass through
|
||||
// unchanged, so borrow `data` directly rather than copying it; only
|
||||
// NAL video needs an owned Annex-B conversion buffer.
|
||||
let es_data: std::borrow::Cow<'_, [u8]> = if is_video && self.is_nal_video(track) {
|
||||
// Size the buffer once for the whole frame. `Vec::new()` re-grew from
|
||||
// zero capacity on every frame, reallocating repeatedly inside a single
|
||||
// ~310 KB conversion. The slack covers any prepended parameter sets.
|
||||
let mut annex_b = Vec::with_capacity(data.len() + 1024);
|
||||
// Take the reusable conversion buffer out of `self` for the duration of
|
||||
// this frame: that frees the `&mut self` the writer needs below while the
|
||||
// converted bytes are still borrowed, and keeps the allocation across
|
||||
// frames instead of making (and dropping) a whole-frame one per picture.
|
||||
// It is put back at the end of the function, so a mid-frame error path
|
||||
// costs only the buffer's capacity, never correctness.
|
||||
let mut annex_b = std::mem::take(&mut self.annex_b);
|
||||
let convert = is_video && self.is_nal_video(track);
|
||||
if convert {
|
||||
annex_b.clear();
|
||||
// Size once for the whole frame; the slack covers any prepended
|
||||
// parameter sets. `reserve` is a no-op once the buffer has settled at
|
||||
// the largest frame's size.
|
||||
annex_b.reserve(data.len() + 1024);
|
||||
if keyframe && !self.params_written[track] {
|
||||
if let Some(ref cp) = self.codec_privates[track] {
|
||||
// avcC and hvcC are DIFFERENT box layouts; parsing one with
|
||||
@@ -221,13 +240,12 @@ impl<W: Write> TsMuxer<W> {
|
||||
// allocations and two full-frame copies. At ~200k frames averaging
|
||||
// ~310 KB of ES on a UHD, that is ~124 GB of pointless memcpy.
|
||||
append_length_prefixed_as_annex_b(&mut annex_b, data);
|
||||
std::borrow::Cow::Owned(annex_b)
|
||||
} else {
|
||||
if is_video {
|
||||
self.params_written[track] = true;
|
||||
}
|
||||
std::borrow::Cow::Borrowed(data)
|
||||
};
|
||||
} else if is_video {
|
||||
self.params_written[track] = true;
|
||||
}
|
||||
// Borrowed from a LOCAL (the taken-out buffer), never from `self`, so the
|
||||
// `&mut self` writes below are free of it.
|
||||
let es_data: &[u8] = if convert { &annex_b } else { data };
|
||||
|
||||
let pts_90k = if pts_ns >= 0 {
|
||||
(pts_ns as u64).saturating_mul(9) / 100_000
|
||||
@@ -240,15 +258,32 @@ impl<W: Write> TsMuxer<W> {
|
||||
// access units into multiple PES packets. Each emitted PES carries
|
||||
// the same PTS and starts on its own PUSI packet (only the keyframe
|
||||
// RAI rides the first packet of the first PES).
|
||||
if is_video || es_data.len() <= MAX_BD_PES_PAYLOAD {
|
||||
self.write_pes_chain(track, pid, pts_90k, is_video, keyframe, &es_data)?;
|
||||
//
|
||||
// The write result is held rather than `?`-propagated so the conversion
|
||||
// buffer goes back into `self` on every path.
|
||||
let res = if is_video || es_data.len() <= MAX_BD_PES_PAYLOAD {
|
||||
self.write_pes_chain(track, pid, pts_90k, is_video, keyframe, es_data)
|
||||
} else {
|
||||
let mut first_pes = true;
|
||||
let mut res = Ok(());
|
||||
for chunk in es_data.chunks(MAX_BD_PES_PAYLOAD) {
|
||||
self.write_pes_chain(track, pid, pts_90k, is_video, keyframe && first_pes, chunk)?;
|
||||
res = self.write_pes_chain(
|
||||
track,
|
||||
pid,
|
||||
pts_90k,
|
||||
is_video,
|
||||
keyframe && first_pes,
|
||||
chunk,
|
||||
);
|
||||
if res.is_err() {
|
||||
break;
|
||||
}
|
||||
first_pes = false;
|
||||
}
|
||||
}
|
||||
res
|
||||
};
|
||||
self.annex_b = annex_b;
|
||||
res?;
|
||||
// A frame that survived the pre-keyframe drop guard above and reached
|
||||
// the writer counts as emitted. `finish()` checks this so a zero-frame
|
||||
// mux fails loudly instead of producing a header-only "success".
|
||||
@@ -1133,4 +1168,41 @@ mod tests {
|
||||
assert_eq!(got.len(), big.len(), "no bytes lost in the PES split");
|
||||
assert_eq!(got, big, "split audio reassembles byte-for-byte");
|
||||
}
|
||||
|
||||
/// MEASURED: the Annex-B conversion buffer must be REUSED across video
|
||||
/// frames, not allocated per frame. Both the allocation's address and its
|
||||
/// capacity are unchanged after the second and third same-sized frames — if
|
||||
/// the conversion allocated a fresh `Vec` per frame (the old
|
||||
/// `Vec::with_capacity(data.len() + 1024)`), the buffer left on the muxer
|
||||
/// would be empty with zero capacity, and each frame would pay an
|
||||
/// allocate/first-touch/free cycle over the whole ~310 KB frame.
|
||||
#[test]
|
||||
fn annex_b_conversion_buffer_is_reused_across_frames() {
|
||||
let mut sink: Vec<u8> = Vec::new();
|
||||
let mut mux = TsMuxer::new(&mut sink, &[VIDEO_PID]);
|
||||
let idr = fake_hevc_nal(19, 300_000);
|
||||
mux.write_frame(0, 0, true, &idr).unwrap();
|
||||
let cap = mux.annex_b.capacity();
|
||||
let ptr = mux.annex_b.as_ptr();
|
||||
assert!(
|
||||
cap >= idr.len(),
|
||||
"buffer survives the frame with the frame's capacity, got {cap}"
|
||||
);
|
||||
|
||||
for i in 1..4 {
|
||||
let p = fake_hevc_nal(1, 300_000);
|
||||
mux.write_frame(0, i * 41_000_000, false, &p).unwrap();
|
||||
assert_eq!(
|
||||
mux.annex_b.as_ptr(),
|
||||
ptr,
|
||||
"frame {i}: conversion buffer must be the same allocation"
|
||||
);
|
||||
assert_eq!(
|
||||
mux.annex_b.capacity(),
|
||||
cap,
|
||||
"frame {i}: no re-grow once the buffer has settled"
|
||||
);
|
||||
}
|
||||
mux.finish().unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user