Parse MKV lacing, route by real TrackNumber, honour NAL length size and edit lists

Four conformance defects in the read paths, two of them silent corruption.

**Lacing was ignored entirely.** RFC 9559 §10.2 defines Xiph, EBML and
fixed-size lacing, where one Block carries several frames; the reader took the
Block payload verbatim, so a laced Block became a single "frame" consisting of a
lacing header followed by concatenated frames — garbage to the codec parser, no
error. Audio tracks from other muxers commonly use lacing, so an ordinary
foreign MKV was silently mangled.

All three modes are now parsed: Xiph 255-run sizes including the trailing-zero
rule for exact multiples of 255, EBML unsigned first size plus SIGNED VINT deltas
with the 2^((7*n)-1)-1 bias of §10.3.3, and fixed-size even division, with the
last frame's size deduced from the remainder. Laced timestamps follow §10.3.5:
the first frame takes the Block timestamp and the rest are spaced by the track's
DefaultDuration, else BlockDuration/count, else shared with a warn.

Parsing was chosen over refusing because refusal would leave freemkv unable to
remux common foreign audio at all, and each mode is about fifteen lines.

A malformed lacing header now raises a NEW code, E_MKV_LACING_INVALID = 9052,
deliberately NOT MkvInvalid — because is_skippable_title_stub classifies
MkvInvalid as a skippable nav stub, so reusing it would have recreated the exact
conflation that is still open as a separate finding. A test asserts the new code
is not skippable.

**TrackNumber was assumed to be 1..N in TrackEntry order.** RFC 9559 §5.1.4.1.1
only requires it to be non-zero and unique, so sparse or unordered numbers are
legal. Block routing and codec_private both computed track + 1. A real
TrackNumber map is now built, recorded only for TrackEntries that yield a stream
so dropped track types no longer shift the mapping.

Verified red here independently, and the failure mode is worse than mis-routing:
with track + 1 restored, a buttons track's payload was attributed to the AUDIO
stream — wrong payload into the wrong codec parser.

**The NAL length prefix was hardcoded to 4 bytes.** lengthSizeMinusOne lives in
avcC byte 4 and hvcC byte 21 (ISO/IEC 14496-15 §5.3.3.1.2, §8.3.3.1.2) and was
never read, so a source declaring 1- or 2-byte prefixes had its raw prefixed
bytes emitted verbatim with no start codes. All four conversion sites now derive
the width from the track's own configuration record.

**Edit lists were ignored.** No edts/elst was parsed, so the presentation
timeline an edit list defines (ISO/IEC 14496-12 §8.6.5/§8.6.6) was dropped —
which is how encoder delay is normally expressed. Leading empty edits and the
first media edit's media_time are now applied to both dts and pts, with the movie
vs media timescale distinction respected. A list needing more than a constant
shift applies the leading edit and warns rather than presenting the result as
faithful.

17 tests. I reproduced the lacing mutant independently: returning the body whole
kills five of them, including the exact-payload and malformed-header cases.

Still open and deliberately untouched: the MkvInvalid / is_skippable_title_stub
conflation across ~20 reader raise sites. It is a cross-cutting error.rs change
and E_MKV_LACING_INVALID is the template for it.
This commit is contained in:
Matthew Jackson
2026-07-29 21:53:19 -07:00
parent 9527bc1e13
commit 4fcd28b487
7 changed files with 1347 additions and 88 deletions
+19
View File
@@ -156,6 +156,15 @@ pub const E_MUX_EMPTY: u16 = 9023;
/// [`is_skippable_title_stub`] treats as a skippable empty nav/menu stub — a /// [`is_skippable_title_stub`] treats as a skippable empty nav/menu stub — a
/// cap-overflow is a real title and must never be silently skipped. /// cap-overflow is a real title and must never be silently skipped.
pub const E_MUX_HEADER_BUFFER_EXCEEDED: u16 = 9051; pub const E_MUX_HEADER_BUFFER_EXCEEDED: u16 = 9051;
/// An `mkv://` SOURCE Block declared lacing (RFC 9559 §10.3) whose header does
/// not describe its own payload, so the frame boundaries inside the Block are
/// unknowable. Deliberately NOT [`E_MKV_INVALID`], which
/// [`is_skippable_title_stub`] treats as a skippable empty nav/menu stub: a
/// laced Block belongs to a track with real media in it, and mis-reporting the
/// rejection as a stub would drop that media from a run that then exits
/// successfully — the same conflation [`E_MUX_HEADER_BUFFER_EXCEEDED`] exists to
/// avoid.
pub const E_MKV_LACING_INVALID: u16 = 9052;
pub const E_EXTENT_NOT_UNIT_ALIGNED: u16 = 9030; pub const E_EXTENT_NOT_UNIT_ALIGNED: u16 = 9030;
/// `mp4://` output but the title has no (primary) video track to carry. /// `mp4://` output but the title has no (primary) video track to carry.
pub const E_MP4_NO_VIDEO_TRACK: u16 = 9048; pub const E_MP4_NO_VIDEO_TRACK: u16 = 9048;
@@ -297,6 +306,10 @@ pub enum Error {
}, },
IfoParse, IfoParse,
MkvInvalid, MkvInvalid,
/// An `mkv://` source Block's lacing header does not describe its payload —
/// the frames packed into that Block cannot be separated. NOT
/// [`Error::MkvInvalid`]: see [`E_MKV_LACING_INVALID`].
MkvLacingInvalid,
NoStreams, NoStreams,
/// A [`crate::StreamSelection`] listed a PID that does not exist in the /// A [`crate::StreamSelection`] listed a PID that does not exist in the
/// title's declared streams — a caller bug (e.g. a stale scan), reported /// title's declared streams — a caller bug (e.g. a stale scan), reported
@@ -604,6 +617,7 @@ impl Error {
Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE, Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE,
Error::IfoParse => E_IFO_PARSE, Error::IfoParse => E_IFO_PARSE,
Error::MkvInvalid => E_MKV_INVALID, Error::MkvInvalid => E_MKV_INVALID,
Error::MkvLacingInvalid => E_MKV_LACING_INVALID,
Error::NoStreams => E_NO_STREAMS, Error::NoStreams => E_NO_STREAMS,
Error::SelectionPidUnknown { .. } => E_SELECTION_PID_UNKNOWN, Error::SelectionPidUnknown { .. } => E_SELECTION_PID_UNKNOWN,
Error::MapfileInvalid { .. } => E_MAPFILE_INVALID, Error::MapfileInvalid { .. } => E_MAPFILE_INVALID,
@@ -895,6 +909,9 @@ impl From<Error> for std::io::Error {
// 9051 MuxHeaderBufferExceeded: the source kept yielding frames but // 9051 MuxHeaderBufferExceeded: the source kept yielding frames but
// never its codec init data — the input is unusable as declared. // never its codec init data — the input is unusable as declared.
E_MUX_HEADER_BUFFER_EXCEEDED => std::io::ErrorKind::InvalidData, E_MUX_HEADER_BUFFER_EXCEEDED => std::io::ErrorKind::InvalidData,
// 9052 MkvLacingInvalid: a source Block's lacing header does not
// describe its own payload — malformed input data.
E_MKV_LACING_INVALID => std::io::ErrorKind::InvalidData,
// mp4:// demux errors: a malformed/truncated source file // mp4:// demux errors: a malformed/truncated source file
// (E_MP4_INVALID), or a source whose tracks the mux can't use — no // (E_MP4_INVALID), or a source whose tracks the mux can't use — no
// video track / missing codec-private config. All are invalid data. // video track / missing codec-private config. All are invalid data.
@@ -1391,6 +1408,7 @@ mod tests {
E_NETWORK_ADDR_BLOCKED, E_NETWORK_ADDR_BLOCKED,
E_MUX_EMPTY, E_MUX_EMPTY,
E_MUX_HEADER_BUFFER_EXCEEDED, E_MUX_HEADER_BUFFER_EXCEEDED,
E_MKV_LACING_INVALID,
E_MP4_NO_VIDEO_TRACK, E_MP4_NO_VIDEO_TRACK,
E_MP4_INVALID, E_MP4_INVALID,
E_MP4_MISSING_CODEC_PRIVATE, E_MP4_MISSING_CODEC_PRIVATE,
@@ -1485,6 +1503,7 @@ mod tests {
Error::MuxHeaderBufferExceeded { bytes: 0 }, Error::MuxHeaderBufferExceeded { bytes: 0 },
E_MUX_HEADER_BUFFER_EXCEEDED, E_MUX_HEADER_BUFFER_EXCEEDED,
), ),
(Error::MkvLacingInvalid, E_MKV_LACING_INVALID),
(Error::Mp4NoVideoTrack, E_MP4_NO_VIDEO_TRACK), (Error::Mp4NoVideoTrack, E_MP4_NO_VIDEO_TRACK),
(Error::Mp4Invalid, E_MP4_INVALID), (Error::Mp4Invalid, E_MP4_INVALID),
(Error::Mp4MissingCodecPrivate, E_MP4_MISSING_CODEC_PRIVATE), (Error::Mp4MissingCodecPrivate, E_MP4_MISSING_CODEC_PRIVATE),
+51 -4
View File
@@ -23,7 +23,9 @@
//! The sink does NOT touch the MKV mux path; it is purely additive. //! The sink does NOT touch the MKV mux path; it is purely additive.
use crate::disc::{Chapter, Codec, DiscTitle, Stream as DiscStream}; use crate::disc::{Chapter, Codec, DiscTitle, Stream as DiscStream};
use crate::mux::hevc::{append_length_prefixed_as_annex_b, avcc_to_annex_b, hvcc_to_annex_b}; use crate::mux::hevc::{
append_length_prefixed_as_annex_b_sized, avcc_to_annex_b, hvcc_to_annex_b, nal_length_size,
};
use crate::mux::timeline::TimelineContinuity; use crate::mux::timeline::TimelineContinuity;
use crate::pes::{PesFrame, Stream}; use crate::pes::{PesFrame, Stream};
use std::fs::File; use std::fs::File;
@@ -207,12 +209,17 @@ impl EsWriter for PassthroughWriter {
} }
} }
/// HEVC/H.264 writer: reframes 4-byte-length-prefixed NALs (the hvcC/avcC form /// HEVC/H.264 writer: reframes length-prefixed NALs (the hvcC/avcC form the
/// the parsers emit) into Annex-B, prepending the parameter sets once. /// parsers emit) into Annex-B, prepending the parameter sets once.
struct AnnexBWriter { struct AnnexBWriter {
/// Annex-B-framed VPS/SPS/PPS (or SPS/PPS), parsed from the hvcC/avcC. /// Annex-B-framed VPS/SPS/PPS (or SPS/PPS), parsed from the hvcC/avcC.
params: Vec<u8>, params: Vec<u8>,
wrote_params: bool, wrote_params: bool,
/// Octets per NAL length prefix, from the configuration record's
/// `lengthSizeMinusOne` (ISO/IEC 14496-15). NOT assumed to be 4: a legal
/// avcC/hvcC may declare 1 or 2, and reading those as u32-BE parses no NALs
/// at all, so the raw prefixed bytes would be emitted as if already Annex B.
length_size: usize,
} }
impl AnnexBWriter { impl AnnexBWriter {
@@ -223,6 +230,7 @@ impl AnnexBWriter {
Self { Self {
params, params,
wrote_params: false, wrote_params: false,
length_size: nal_length_size(codec, codec_private),
} }
} }
} }
@@ -244,7 +252,7 @@ impl EsWriter for AnnexBWriter {
// zero-length NALs and drops a truncated trailing NAL without panicking, // zero-length NALs and drops a truncated trailing NAL without panicking,
// rather than `break`ing on the first zero-length NAL. // rather than `break`ing on the first zero-length NAL.
let mut scratch = Vec::with_capacity(f.data.len() + (f.data.len() / 32) + 4); let mut scratch = Vec::with_capacity(f.data.len() + (f.data.len() / 32) + 4);
append_length_prefixed_as_annex_b(&mut scratch, &f.data); append_length_prefixed_as_annex_b_sized(&mut scratch, &f.data, self.length_size);
w.write_all(&scratch)?; w.write_all(&scratch)?;
n += scratch.len(); n += scratch.len();
Ok(n) Ok(n)
@@ -982,6 +990,45 @@ mod tests {
); );
} }
/// Regression (silent corruption): the sink reframed every frame as if the
/// NAL length prefixes were 4 octets wide. ISO/IEC 14496-15 §5.3.3.1.2 lets
/// an avcC declare `lengthSizeMinusOne = 1` (2-octet prefixes); reading those
/// as one u32-BE gives an absurd length, so nothing parsed and the raw
/// prefixed bytes were written to the `demux://` ES verbatim — no start
/// codes, undecodable video, and no error anywhere.
#[test]
fn annexb_writer_honours_the_records_declared_nal_length_size() {
// avcC with byte 4 = 0xFD → lengthSizeMinusOne 1 → 2-octet prefixes.
// numSPS = 1 (0xE1), SPS len 2 = [0x67 0x42], numPPS = 1, PPS len 1.
let rec = [
1, 0x42, 0x00, 0x1F, 0xFD, 0xE1, 0, 2, 0x67, 0x42, 1, 0, 1, 0x68,
];
assert_eq!(nal_length_size(Codec::H264, Some(&rec)), 2);
let mut w = AnnexBWriter::new(Codec::H264, Some(&rec));
let mut out = Vec::new();
let f = PesFrame {
coding: None,
source: None,
track: 0,
pts: 0,
keyframe: true,
// Two NALs with 2-octet length prefixes.
data: vec![0, 2, 0xAA, 0xBB, 0, 3, 0x01, 0x02, 0x03],
duration_ns: None,
};
w.write_frame(&mut out, &f, 0).unwrap();
assert_eq!(
out,
vec![
0, 0, 0, 1, 0x67, 0x42, // SPS
0, 0, 0, 1, 0x68, // PPS
0, 0, 0, 1, 0xAA, 0xBB, // frame NAL #1
0, 0, 0, 1, 0x01, 0x02, 0x03, // frame NAL #2
],
"2-octet-prefixed NALs must reach the ES as Annex B"
);
}
#[test] #[test]
fn annexb_writer_prepends_params_once() { fn annexb_writer_prepends_params_once() {
let rec = [ let rec = [
+172 -9
View File
@@ -96,7 +96,17 @@ impl<W: Write> HevcMux<W> {
} }
} }
} }
let annex_b = length_prefixed_to_annex_b(data); // The hvcC declares the NAL length-prefix width (ISO/IEC 14496-15
// §8.3.3.1.2 `lengthSizeMinusOne + 1`). Assuming 4 for a source that
// declares 1 or 2 emits the raw prefixed bytes with no start codes.
let length_size = nal_length_size(crate::disc::Codec::Hevc, self.codec_private.as_deref());
let annex_b = if starts_with_start_code(data) {
data.to_vec()
} else {
let mut out = Vec::with_capacity(data.len() + (data.len() / 32));
append_length_prefixed_as_annex_b_sized(&mut out, data, length_size);
out
};
self.writer.write_all(&annex_b) self.writer.write_all(&annex_b)
} }
@@ -190,6 +200,44 @@ pub(crate) fn length_prefixed_to_annex_b(data: &[u8]) -> Vec<u8> {
out out
} }
/// The NAL length-prefix width this crate's own parsers emit, and the width
/// ISO/IEC 14496-15 records declare as `lengthSizeMinusOne = 3`.
pub(crate) const DEFAULT_NAL_LENGTH_SIZE: usize = 4;
/// Number of octets each NAL length prefix occupies in the elementary data of a
/// track described by `record` — the `lengthSizeMinusOne + 1` field of the
/// decoder configuration record (ISO/IEC 14496-15).
///
/// * avcC (`AVCDecoderConfigurationRecord`, §5.3.3.1.2): byte 4 is
/// `bit(6) reserved | unsigned int(2) lengthSizeMinusOne`.
/// * hvcC (`HEVCDecoderConfigurationRecord`, §8.3.3.1.2): byte 21 is
/// `constantFrameRate(2) | numTemporalLayers(3) | temporalIdNested(1) |
/// lengthSizeMinusOne(2)`.
///
/// The spec permits only 1, 2 or 4 octets (`lengthSizeMinusOne` of 0, 1 or 3);
/// a declared 3 is non-conformant but is decoded rather than rejected, since
/// reading N octets is the same operation for every N. A record too short to
/// carry the field, or a codec with no such record, falls back to
/// [`DEFAULT_NAL_LENGTH_SIZE`] — the width every freemkv parser emits.
///
/// This exists because assuming 4 is silent corruption for a legal source:
/// reading a 2-octet-prefixed frame as u32-BE yields an absurd first length, the
/// conversion loop bails with nothing parsed, and the raw length-prefixed bytes
/// are passed through as though they were already Annex B — a stream with no
/// start codes at all, and no error anywhere.
pub(crate) fn nal_length_size(codec: crate::disc::Codec, record: Option<&[u8]>) -> usize {
use crate::disc::Codec;
let field_offset = match codec {
Codec::H264 => 4,
Codec::Hevc => 21,
_ => return DEFAULT_NAL_LENGTH_SIZE,
};
match record.and_then(|r| r.get(field_offset)) {
Some(&b) => (b & 0x03) as usize + 1,
None => DEFAULT_NAL_LENGTH_SIZE,
}
}
/// Append the Annex B form of `data` (length-prefixed NALs) into `out`. /// Append the Annex B form of `data` (length-prefixed NALs) into `out`.
/// ///
/// Same conversion as [`length_prefixed_to_annex_b`] but writes directly /// Same conversion as [`length_prefixed_to_annex_b`] but writes directly
@@ -198,20 +246,40 @@ pub(crate) fn length_prefixed_to_annex_b(data: &[u8]) -> Vec<u8> {
/// length-prefixed (no NALs extracted), it's appended unchanged on the /// length-prefixed (no NALs extracted), it's appended unchanged on the
/// assumption it's already Annex B. /// assumption it's already Annex B.
pub(crate) fn append_length_prefixed_as_annex_b(out: &mut Vec<u8>, data: &[u8]) { pub(crate) fn append_length_prefixed_as_annex_b(out: &mut Vec<u8>, data: &[u8]) {
append_length_prefixed_as_annex_b_sized(out, data, DEFAULT_NAL_LENGTH_SIZE);
}
/// [`append_length_prefixed_as_annex_b`] for a source whose NAL length prefixes
/// are `length_size` octets wide rather than the 4 this crate's own parsers
/// emit. Derive `length_size` from the track's configuration record with
/// [`nal_length_size`] — ISO/IEC 14496-15 lets a legal avcC/hvcC declare 1 or 2
/// octet prefixes, and reading those as u32-BE mangles the frame.
///
/// `length_size` outside `1..=4` is clamped to [`DEFAULT_NAL_LENGTH_SIZE`]; the
/// field it comes from is 2 bits wide, so that is unreachable from real input.
pub(crate) fn append_length_prefixed_as_annex_b_sized(
out: &mut Vec<u8>,
data: &[u8],
length_size: usize,
) {
let length_size = if (1..=4).contains(&length_size) {
length_size
} else {
DEFAULT_NAL_LENGTH_SIZE
};
let mut offset = 0; let mut offset = 0;
// True once we've consumed at least one well-formed length prefix // True once we've consumed at least one well-formed length prefix
// (even a zero-length one). Distinguishes "parsed as length-prefixed, // (even a zero-length one). Distinguishes "parsed as length-prefixed,
// all NALs empty" (emit nothing) from "not length-prefixed at all" // all NALs empty" (emit nothing) from "not length-prefixed at all"
// (pass through as already-Annex B). // (pass through as already-Annex B).
let mut parsed_any = false; let mut parsed_any = false;
while offset + 4 <= data.len() { while offset + length_size <= data.len() {
let len = u32::from_be_bytes([ // Big-endian over exactly `length_size` octets (ISO/IEC 14496-15: the
data[offset], // prefix is an unsigned integer of `lengthSizeMinusOne + 1` bytes).
data[offset + 1], let len = data[offset..offset + length_size]
data[offset + 2], .iter()
data[offset + 3], .fold(0usize, |acc, &b| (acc << 8) | b as usize);
]) as usize; offset += length_size;
offset += 4;
if offset + len > data.len() { if offset + len > data.len() {
// Mid-NAL truncation (e.g. a NAL cut by a bad disc sector) — // Mid-NAL truncation (e.g. a NAL cut by a bad disc sector) —
// drop the truncated trailing NAL and emit only the valid // drop the truncated trailing NAL and emit only the valid
@@ -637,6 +705,101 @@ mod tests {
assert_eq!(out, want); assert_eq!(out, want);
} }
/// ISO/IEC 14496-15 §5.3.3.1.2 (avcC byte 4) and §8.3.3.1.2 (hvcC byte 21)
/// each carry `lengthSizeMinusOne` in the low 2 bits. Nothing in the crate
/// read it, so every conversion assumed a 4-octet prefix.
#[test]
fn nal_length_size_is_read_from_the_configuration_record() {
use crate::disc::Codec;
// avcC: byte 4 = 0xFF → lengthSizeMinusOne 3 → 4-octet prefixes.
let mut avcc = vec![0x01, 0x64, 0x00, 0x28, 0xFF, 0xE1];
assert_eq!(nal_length_size(Codec::H264, Some(&avcc)), 4);
// 0xFD → lengthSizeMinusOne 1 → 2-octet prefixes (legal per §5.3.3.1.2).
avcc[4] = 0xFD;
assert_eq!(nal_length_size(Codec::H264, Some(&avcc)), 2);
// 0xFC → lengthSizeMinusOne 0 → 1-octet prefixes.
avcc[4] = 0xFC;
assert_eq!(nal_length_size(Codec::H264, Some(&avcc)), 1);
// hvcC: the field is byte 21, not byte 4.
let mut hvcc = vec![0u8; 23];
hvcc[21] = 0xFF;
assert_eq!(nal_length_size(Codec::Hevc, Some(&hvcc)), 4);
hvcc[21] = 0xFD;
assert_eq!(nal_length_size(Codec::Hevc, Some(&hvcc)), 2);
// Absent / too-short record, or a non-NAL codec → the crate's own width.
assert_eq!(nal_length_size(Codec::Hevc, None), DEFAULT_NAL_LENGTH_SIZE);
assert_eq!(
nal_length_size(Codec::Hevc, Some(&hvcc[..8])),
DEFAULT_NAL_LENGTH_SIZE
);
assert_eq!(
nal_length_size(Codec::Mpeg2, Some(&avcc)),
DEFAULT_NAL_LENGTH_SIZE
);
}
/// Regression (silent corruption): a source whose avcC declares 2-octet NAL
/// lengths was reframed by reading the first FOUR octets as one u32-BE
/// length. That value is absurd, the loop breaks with `parsed_any == false`,
/// and the whole frame is passed through verbatim — raw length-prefixed
/// bytes in a stream that is supposed to be Annex B, with no start codes,
/// no NALs and no error.
#[test]
fn two_octet_length_prefixes_convert_instead_of_leaking_raw_bytes() {
// Two NALs with 2-octet prefixes: [0x00 0x03][3 bytes][0x00 0x02][2 bytes]
let data = [
0x00, 0x03, 0x67, 0x42, 0x1E, // NAL 1
0x00, 0x02, 0x68, 0xCE, // NAL 2
];
let mut want = START_CODE.to_vec();
want.extend_from_slice(&[0x67, 0x42, 0x1E]);
want.extend_from_slice(&START_CODE);
want.extend_from_slice(&[0x68, 0xCE]);
let mut got = Vec::new();
append_length_prefixed_as_annex_b_sized(&mut got, &data, 2);
assert_eq!(got, want, "2-octet prefixes must be reframed to Annex B");
// What the 4-octet assumption produced: the raw bytes, verbatim, with no
// start code anywhere.
let mut assumed_four = Vec::new();
append_length_prefixed_as_annex_b(&mut assumed_four, &data);
assert_eq!(
assumed_four,
data.to_vec(),
"the 4-octet assumption leaks the source bytes unconverted"
);
assert!(
!assumed_four.starts_with(&START_CODE),
"no start code at all — the video cannot decode"
);
}
/// A 1-octet prefix width works the same way, and an out-of-range width
/// falls back to the crate's own 4 rather than panicking or looping.
#[test]
fn one_octet_length_prefixes_and_out_of_range_width() {
let data = [0x02, 0x40, 0x01, 0x01, 0x09];
let mut got = Vec::new();
append_length_prefixed_as_annex_b_sized(&mut got, &data, 1);
let mut want = START_CODE.to_vec();
want.extend_from_slice(&[0x40, 0x01]);
want.extend_from_slice(&START_CODE);
want.extend_from_slice(&[0x09]);
assert_eq!(got, want);
// width 0 and width 9 both clamp to DEFAULT_NAL_LENGTH_SIZE.
let mut four = Vec::new();
append_length_prefixed_as_annex_b(&mut four, &data);
for bad in [0usize, 9] {
let mut clamped = Vec::new();
append_length_prefixed_as_annex_b_sized(&mut clamped, &data, bad);
assert_eq!(clamped, four, "an impossible width clamps to 4");
}
}
#[test] #[test]
fn starts_with_start_code_detects_both_forms() { fn starts_with_start_code_detects_both_forms() {
assert!(starts_with_start_code(&[0x00, 0x00, 0x00, 0x01, 0x42])); assert!(starts_with_start_code(&[0x00, 0x00, 0x00, 0x01, 0x42]));
+8 -1
View File
@@ -242,7 +242,14 @@ impl<W: Write> M2tsMux<W> {
} }
// Append the Annex-B form directly into the pre-sized `es` // Append the Annex-B form directly into the pre-sized `es`
// buffer rather than materializing an intermediate Vec. // buffer rather than materializing an intermediate Vec.
super::hevc::append_length_prefixed_as_annex_b(&mut es, data); // The source's hvcC declares the NAL length-prefix width (ISO/IEC
// 14496-15 §8.3.3.1.2 `lengthSizeMinusOne + 1`); assuming 4 mangles a
// legal 1- or 2-octet-prefixed source into a start-code-free stream.
let length_size = super::hevc::nal_length_size(
crate::disc::Codec::Hevc,
self.video_codec_private.as_deref(),
);
super::hevc::append_length_prefixed_as_annex_b_sized(&mut es, data, length_size);
let pes = build_video_pes(pts_90k, &es); let pes = build_video_pes(pts_90k, &es);
self.write_pes(PID_VIDEO, &pes, Some(pcr), keyframe) self.write_pes(PID_VIDEO, &pes, Some(pcr), keyframe)
+778 -70
View File
File diff suppressed because it is too large Load Diff
+307 -2
View File
@@ -99,6 +99,13 @@ impl<R: Read + Seek> Mp4Reader<R> {
let mut title = DiscTitle::empty(); let mut title = DiscTitle::empty();
title.playlist = name; title.playlist = name;
// Movie timescale (ISO/IEC 14496-12 §8.2.2). An edit list's
// `segment_duration` is expressed in it, while its `media_time` is in the
// track's own media timescale, so both are needed to place an edit.
let movie_timescale = find_box(&moov, b"mvhd")
.and_then(mvhd_timescale)
.filter(|&t| t != 0);
let mut samples: Vec<SampleRef> = Vec::new(); let mut samples: Vec<SampleRef> = Vec::new();
let mut codec_privates: Vec<Option<Vec<u8>>> = Vec::new(); let mut codec_privates: Vec<Option<Vec<u8>>> = Vec::new();
let mut track_idx = 0usize; let mut track_idx = 0usize;
@@ -288,12 +295,27 @@ impl<R: Read + Seek> Mp4Reader<R> {
(ticks as i128 * NS / timescale as i128).clamp(i64::MIN as i128, i64::MAX as i128) (ticks as i128 * NS / timescale as i128).clamp(i64::MIN as i128, i64::MAX as i128)
as i64 as i64
}; };
// Edit list (ISO/IEC 14496-12 §8.6.5 `edts` / §8.6.6 `elst`): the
// presentation timeline is NOT the media timeline. Ignoring it — which
// this reader did — starts every track at media time 0, so a track
// carrying the standard encoder-delay/A-V-offset edit ends up shifted
// against its siblings for the whole title, silently.
let edit_offset_ticks = find_box(trak, b"edts")
.and_then(|edts| find_box(edts, b"elst"))
.map(|elst| {
let entries = parse_elst(elst);
elst_offset_ticks(&entries, movie_timescale, timescale, track_idx)
})
.unwrap_or(0);
let mut decode_ticks: i64 = 0; let mut decode_ticks: i64 = 0;
for (i, &size) in sizes.iter().enumerate() { for (i, &size) in sizes.iter().enumerate() {
let dur = durations.get(i).copied().unwrap_or(0); let dur = durations.get(i).copied().unwrap_or(0);
let comp = ctts.get(i).copied().unwrap_or(0); let comp = ctts.get(i).copied().unwrap_or(0);
let dts_ns = to_ns(decode_ticks); let dts_ns = to_ns(decode_ticks.saturating_add(edit_offset_ticks));
let pts_ticks = decode_ticks.saturating_add(comp as i64); let pts_ticks = decode_ticks
.saturating_add(comp as i64)
.saturating_add(edit_offset_ticks);
let pts_ns = to_ns(pts_ticks); let pts_ns = to_ns(pts_ticks);
decode_ticks = decode_ticks.saturating_add(dur as i64); decode_ticks = decode_ticks.saturating_add(dur as i64);
let keyframe = match &sync { let keyframe = match &sync {
@@ -474,6 +496,152 @@ fn be16(b: &[u8], o: usize) -> u16 {
u16::from_be_bytes([b[o], b[o + 1]]) u16::from_be_bytes([b[o], b[o + 1]])
} }
/// mvhd (version 0/1) → movie timescale (ISO/IEC 14496-12 §8.2.2).
fn mvhd_timescale(b: &[u8]) -> Option<u32> {
let version = b.first().copied()?;
if version == 1 {
// version(1)+flags(3) creation(8) modification(8) timescale(4) ...
(b.len() >= 24).then(|| be32(b, 20))
} else {
// version(1)+flags(3) creation(4) modification(4) timescale(4) ...
(b.len() >= 16).then(|| be32(b, 12))
}
}
/// Upper bound on parsed `elst` entries. Only the leading empty edits and the
/// FIRST non-empty edit shape the offset applied below, so a longer list buys
/// nothing but allocation; the cap keeps a crafted 256 MiB `moov` from turning
/// a box into a larger Vec than the box itself.
const MAX_ELST_ENTRIES: usize = 1024;
/// One `elst` entry: `(segment_duration, media_time, media_rate_integer)`.
type EditListEntry = (u64, i64, i16);
/// Parse an `elst` payload (ISO/IEC 14496-12 §8.6.6). Entry count is clamped
/// both by the box's own bytes and by [`MAX_ELST_ENTRIES`].
///
/// Version 1 entries are `segment_duration:u64, media_time:i64,
/// media_rate_integer:i16, media_rate_fraction:i16` (20 bytes); version 0 uses
/// 32-bit duration/time (12 bytes).
fn parse_elst(b: &[u8]) -> Vec<EditListEntry> {
if b.len() < 8 {
return Vec::new();
}
let version = b[0];
let entry_size = if version == 1 { 20 } else { 12 };
let declared = be32(b, 4) as usize;
let available = (b.len() - 8) / entry_size;
let n = declared.min(available).min(MAX_ELST_ENTRIES);
let mut out = Vec::with_capacity(n);
for i in 0..n {
let o = 8 + i * entry_size;
let (seg, media_time, rate_off) = if version == 1 {
let seg = u64::from_be_bytes([
b[o],
b[o + 1],
b[o + 2],
b[o + 3],
b[o + 4],
b[o + 5],
b[o + 6],
b[o + 7],
]);
let mt = i64::from_be_bytes([
b[o + 8],
b[o + 9],
b[o + 10],
b[o + 11],
b[o + 12],
b[o + 13],
b[o + 14],
b[o + 15],
]);
(seg, mt, 16)
} else {
(be32(b, o) as u64, be32(b, o + 4) as i32 as i64, 8)
};
let rate = be16(b, o + rate_off) as i16;
out.push((seg, media_time, rate));
}
out
}
/// Presentation-time offset an edit list imposes on a track's samples, in the
/// track's MEDIA timescale ticks (ISO/IEC 14496-12 §8.6.5-§8.6.6).
///
/// Two constructs cover essentially every real edit list, and both reduce to a
/// constant shift of the whole track:
/// * an EMPTY edit (`media_time == -1`) before the media edit, whose
/// `segment_duration` — in MOVIE timescale ticks — delays presentation;
/// * a non-empty edit whose `media_time` trims that much media off the front.
///
/// So the offset is `(sum of leading empty segment_durations) - media_time`.
/// A list with several non-empty edits, or a non-empty edit at a rate other than
/// 1, describes a timeline this frame model cannot express (it would need samples
/// dropped, reordered or repeated); the leading edit is still honoured, and the
/// part that is not is LOGGED rather than passed off as a faithful copy.
fn elst_offset_ticks(
entries: &[EditListEntry],
movie_timescale: Option<u32>,
media_timescale: u32,
track_idx: usize,
) -> i64 {
let mut empty_movie_ticks: u64 = 0;
let mut trim_media_ticks: i64 = 0;
let mut media_edits = 0usize;
let mut odd_rate = false;
for &(segment_duration, media_time, rate) in entries {
if media_time < 0 {
// Empty edit: blank presentation time. Only the ones BEFORE the first
// media edit shift this track's start.
if media_edits == 0 {
empty_movie_ticks = empty_movie_ticks.saturating_add(segment_duration);
}
continue;
}
media_edits += 1;
if media_edits == 1 {
trim_media_ticks = media_time;
odd_rate = rate != 1;
}
}
if media_edits > 1 || odd_rate {
tracing::warn!(
track = track_idx,
media_edits,
odd_rate,
"mp4: edit list describes a timeline richer than a constant shift \
(several media edits, or a rate other than 1); only the leading edit \
is applied and the remainder of the presentation timeline is not"
);
}
// An empty edit's duration is in MOVIE ticks; convert to media ticks before
// subtracting the media-timescale trim. i128 so neither product overflows.
let delay_media_ticks = match movie_timescale {
Some(mts) if empty_movie_ticks > 0 => {
((empty_movie_ticks as i128 * media_timescale as i128) / mts as i128)
.clamp(0, i64::MAX as i128) as i64
}
Some(_) => 0,
None => {
if empty_movie_ticks > 0 {
tracing::warn!(
track = track_idx,
"mp4: edit list has an empty edit but the movie timescale is \
absent or zero, so its delay cannot be converted to media \
ticks; the delay is not applied"
);
}
0
}
};
delay_media_ticks.saturating_sub(trim_media_ticks)
}
/// mdhd (version 0/1) → media timescale. /// mdhd (version 0/1) → media timescale.
fn mdhd_timescale(b: &[u8]) -> Option<u32> { fn mdhd_timescale(b: &[u8]) -> Option<u32> {
let version = b.first().copied()?; let version = b.first().copied()?;
@@ -1329,6 +1497,143 @@ mod tests {
); );
} }
// ============================================================
// Edit lists — ISO/IEC 14496-12 §8.6.5 (`edts`) / §8.6.6 (`elst`).
// ============================================================
/// A `mvhd` payload declaring the movie timescale (ISO/IEC 14496-12 §8.2.2).
fn mvhd_box(timescale: u32) -> Vec<u8> {
// v0: version+flags(4) creation(4) modification(4) timescale(4) duration(4) …
let mut p = vec![0u8; 100];
p[12..16].copy_from_slice(&timescale.to_be_bytes());
mp4_box(b"mvhd", &p)
}
/// A version-0 `elst` payload: `(segment_duration, media_time)` per entry,
/// each at media_rate 1.
fn elst_v0(entries: &[(u32, i32)]) -> Vec<u8> {
let mut p = vec![0u8, 0, 0, 0]; // version 0 + flags
p.extend_from_slice(&(entries.len() as u32).to_be_bytes());
for &(seg, media_time) in entries {
p.extend_from_slice(&seg.to_be_bytes());
p.extend_from_slice(&media_time.to_be_bytes());
p.extend_from_slice(&1i16.to_be_bytes()); // media_rate_integer
p.extend_from_slice(&0i16.to_be_bytes()); // media_rate_fraction
}
p
}
/// Insert an `edts > elst` into an existing `trak` box.
fn trak_with_elst(trak: &[u8], elst_payload: &[u8]) -> Vec<u8> {
let mut payload = mp4_box(b"edts", &mp4_box(b"elst", elst_payload));
payload.extend_from_slice(&trak[8..]); // the original trak's children
mp4_box(b"trak", &payload)
}
/// Regression (silent A/V desync): the sample timeline was built purely from
/// stts/ctts starting at tick 0 and no `edts`/`elst` was ever parsed, so the
/// presentation timeline an edit list defines was discarded. A non-empty edit
/// with `media_time = 1024` — the standard way encoder delay is expressed —
/// must move the track's presentation, not be ignored.
#[test]
fn edit_list_media_time_shifts_the_presentation_timeline() {
use std::io::Cursor;
let trak = trak_with_elst(&audio_trak(48_000), &elst_v0(&[(0, 1024)]));
let moov = mp4_box(b"moov", &trak);
let rd = Mp4Reader::from_reader(Cursor::new(moov), "elst".into()).unwrap();
assert_eq!(rd.samples.len(), 1);
// media_time 1024 at 48 kHz trims 1024 ticks off the front, so the first
// sample sits 1024 ticks BEFORE the presentation origin.
let want = -(1024i128 * NS / 48_000) as i64;
assert_eq!(rd.samples[0].pts_ns, want, "media_time must shift the pts");
assert_eq!(rd.samples[0].dts_ns, want, "and the dts with it");
assert_ne!(want, 0, "the shift is observable");
}
/// An EMPTY edit (`media_time == -1`) delays presentation by its
/// `segment_duration`, which is in MOVIE timescale ticks and must be
/// converted to the track's media timescale before it is applied.
#[test]
fn empty_edit_delays_presentation_in_movie_timescale() {
use std::io::Cursor;
// Movie timescale 1000 → segment_duration 40 = 40 ms of blank leader,
// then the media edit itself.
let trak = trak_with_elst(&audio_trak(48_000), &elst_v0(&[(40, -1), (0, 0)]));
let mut moov_payload = mvhd_box(1000);
moov_payload.extend_from_slice(&trak);
let moov = mp4_box(b"moov", &moov_payload);
let rd = Mp4Reader::from_reader(Cursor::new(moov), "empty-edit".into()).unwrap();
assert_eq!(rd.samples.len(), 1);
assert_eq!(
rd.samples[0].pts_ns, 40_000_000,
"a 40 ms empty edit delays the track by 40 ms"
);
}
/// A track with no `edts` is untouched — the shift only ever comes from a
/// declared edit list.
#[test]
fn no_edit_list_leaves_the_timeline_at_zero() {
use std::io::Cursor;
let moov = mp4_box(b"moov", &audio_trak(48_000));
let rd = Mp4Reader::from_reader(Cursor::new(moov), "no-elst".into()).unwrap();
assert_eq!(rd.samples[0].pts_ns, 0);
assert_eq!(rd.samples[0].dts_ns, 0);
}
/// `elst` decoding: both versions, the entry count bounded by the box's own
/// bytes, and the offset arithmetic in isolation.
#[test]
fn parse_elst_and_offset_arithmetic() {
// Version 0, two entries: an empty edit then a media edit.
let v0 = elst_v0(&[(40, -1), (0, 1024)]);
let entries = parse_elst(&v0);
assert_eq!(entries, vec![(40, -1, 1), (0, 1024, 1)]);
// Version 1: 64-bit segment_duration and media_time.
let mut v1 = vec![1u8, 0, 0, 0];
v1.extend_from_slice(&1u32.to_be_bytes());
v1.extend_from_slice(&5_000u64.to_be_bytes());
v1.extend_from_slice(&(-1i64).to_be_bytes());
v1.extend_from_slice(&1i16.to_be_bytes());
v1.extend_from_slice(&0i16.to_be_bytes());
assert_eq!(parse_elst(&v1), vec![(5_000, -1, 1)]);
// A declared count larger than the box can hold is clamped by the bytes.
let mut lying = elst_v0(&[(0, 0)]);
lying[4..8].copy_from_slice(&9_999u32.to_be_bytes());
assert_eq!(parse_elst(&lying).len(), 1, "bounded by the box bytes");
// Too short to hold even the header → no entries, no panic.
assert!(parse_elst(&[0, 0, 0, 0]).is_empty());
// Offset: the empty edit's 40 movie ticks at movie timescale 1000 is
// 40 ms = 1920 ticks at 48 kHz, minus a media_time trim of 1024.
assert_eq!(
elst_offset_ticks(&[(40, -1, 1), (0, 1024, 1)], Some(1000), 48_000, 0),
1920 - 1024
);
// No movie timescale → the empty edit's delay cannot be converted, so
// only the trim applies (and it is logged, not silently dropped).
assert_eq!(
elst_offset_ticks(&[(40, -1, 1), (0, 1024, 1)], None, 48_000, 0),
-1024
);
// An empty list, or a single identity edit, shifts nothing.
assert_eq!(elst_offset_ticks(&[], Some(1000), 48_000, 0), 0);
assert_eq!(elst_offset_ticks(&[(1000, 0, 1)], Some(1000), 48_000, 0), 0);
// Only the FIRST media edit's media_time is applied; trailing empty
// edits do not add to the leading delay.
assert_eq!(
elst_offset_ticks(&[(0, 512, 1), (40, -1, 1)], Some(1000), 48_000, 0),
-512
);
// A hostile segment_duration cannot overflow the tick conversion.
assert_eq!(
elst_offset_ticks(&[(u64::MAX, -1, 1)], Some(1), 48_000, 0),
i64::MAX,
);
}
#[test] #[test]
fn trak_loop_stops_at_max_tracks() { fn trak_loop_stops_at_max_tracks() {
use std::io::Cursor; use std::io::Cursor;
+12 -2
View File
@@ -4,7 +4,9 @@
//! packets. Each frame is wrapped in a PES header, split into TS packets, //! packets. Each frame is wrapped in a PES header, split into TS packets,
//! and prepended with the 4-byte TP_extra_header. //! and prepended with the 4-byte TP_extra_header.
use super::hevc::{append_length_prefixed_as_annex_b, avcc_to_annex_b, hvcc_to_annex_b}; use super::hevc::{
append_length_prefixed_as_annex_b_sized, avcc_to_annex_b, hvcc_to_annex_b, nal_length_size,
};
use crate::disc::Codec; use crate::disc::Codec;
use std::io::{self, Write}; use std::io::{self, Write};
@@ -239,7 +241,15 @@ impl<W: Write> TsMuxer<W> {
// and we then copied it in, so every video frame cost two full-frame // and we then copied it in, so every video frame cost two full-frame
// allocations and two full-frame copies. At ~200k frames averaging // allocations and two full-frame copies. At ~200k frames averaging
// ~310 KB of ES on a UHD, that is ~124 GB of pointless memcpy. // ~310 KB of ES on a UHD, that is ~124 GB of pointless memcpy.
append_length_prefixed_as_annex_b(&mut annex_b, data); // The prefix width is whatever the source's avcC/hvcC declares
// (ISO/IEC 14496-15 `lengthSizeMinusOne + 1`), NOT an assumed 4:
// a 1- or 2-octet-prefixed source read as u32-BE parses no NALs at
// all and its raw bytes are passed through with no start codes.
let length_size = nal_length_size(
self.video_codec[track],
self.codec_privates[track].as_deref(),
);
append_length_prefixed_as_annex_b_sized(&mut annex_b, data, length_size);
} else if is_video { } else if is_video {
self.params_written[track] = true; self.params_written[track] = true;
} }