pq/libfreemkv
1
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

mux: fix keyframe signalling for BlockGroup frames

Browse Source 
Inside a Matroska BlockGroup the SimpleBlock 0x80 keyframe bit is
reserved and is always written as 0; keyframe-ness is carried only by
the presence or absence of a ReferenceBlock child. Both halves of the
round-trip got this wrong:

  - the reader skipped past ReferenceBlock and read the reserved bit,
    so every BlockGroup frame came back as a non-keyframe;
  - write_block_group discarded its `keyframe` argument and never
    emitted a ReferenceBlock, on the assumption that only intra frames
    (PGS subtitles) reached that path.

The MPEG-2 parser stamps a per-frame duration on I, P and B pictures
alike, so all MPEG-2 video is written as a BlockGroup. That makes the
assumption false and left no video frame looking like a keyframe on
read-back. Downstream, mkv:// -> m2ts:// dropped every video frame (the
TS muxer discards non-key video until the first keyframe) while still
reporting success, and mkv:// -> mkv:// and the stdio round-trip failed
E6008, because the MKV muxer opens a cluster only on a track-0 video
keyframe and so wrote nothing at all. HEVC was unaffected: it carries no
per-frame duration, so it takes the SimpleBlock path where the flag bit
is authoritative.

Verified on a real CSS DVD: 841 keyframes out of 11440 video packets
survive a re-mux, matching the I-picture count in the source bitstream.

Also stop running non-NAL video through the Annex-B converter. MPEG-2
and VC-1 elementary streams are already start-code framed, so
length-prefix conversion corrupts them. TsMuxer takes a per-track
nal_video flag, defaulting to the previous behaviour, which M2tsStream
sets from each video stream's codec.
This commit is contained in:
Matthew Jackson
2026-07-29 15:29:48 -07:00
parent eedd27e352
commit c812a32f3d
4 changed files with 395 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/mux/m2ts.rs
+11 -1
View File
@@ -8,7 +8,7 @@
//! write-only.
use super::meta;
use crate::disc::{DiscTitle, Stream as DiscStream};
use crate::disc::{Codec, DiscTitle, Stream as DiscStream};
use std::io::{self, Write};
/// BD transport stream write sink with embedded FMKV metadata
@@ -40,6 +40,16 @@ impl M2tsStream {
.collect();
let boxed: Box<dyn Write + Send> = Box::new(writer);
let mut muxer = super::tsmux::TsMuxer::new(boxed, &pids);
// Only HEVC/H.264 arrive length-prefixed (MKV/PES NALU convention);
// MPEG-2 and VC-1 are already start-code ES and must NOT go through
// Annex-B conversion (see `TsMuxer::set_nal_video`) or the frame is
// silently mangled while the mux still reports success.
for (i, s) in title.streams.iter().enumerate() {
if let DiscStream::Video(v) = s {
let is_nal = matches!(v.codec, Codec::Hevc | Codec::H264);
muxer.set_nal_video(i, is_nal)?;
}
}
for (i, cp) in title.codec_privates.iter().enumerate() {
// codec_privates is parallel to streams/pids; ignore any
// trailing entries that exceed the track count rather than
src/mux/mkv.rs
+238 -22
View File
@@ -1469,27 +1469,36 @@ impl<W: Write + Seek> MkvMuxer<W> {
Some(_) => None,
None => None,
};
match block_additional {
// MVC: base view Block + dependent-view BlockAdditional, always a
// BlockGroup. Non-keyframe base frames get a ReferenceBlock to the
// last keyframe (a keyframe carries none), so a player never treats a
// P/B frame as a seek point.
Some(additional) => {
let reference = if keyframe {
// Offset (ticks) of the referenced keyframe relative to this block, for
// any BlockGroup Block that is NOT a keyframe. Inside a BlockGroup the
// SimpleBlock 0x80 keyframe bit is reserved and MUST be 0, so
// keyframe-ness is carried ONLY by the presence/absence of a
// ReferenceBlock. A non-keyframe that omits it is indistinguishable from
// an intra frame — which is how every MPEG-2 P/B frame used to be written
// (the MPEG-2 parser stamps a per-frame duration, so ALL its frames take
// the BlockGroup path, not just the intra ones this path was written for).
// Gated to video: audio/subtitle frames on this path are self-contained
// (keyframe==true), and referencing a video keyframe from a non-video
// track would be a bogus cross-track reference.
//
// Fall back to 0 (self-relative) in the pre-first-keyframe corner
// (unreachable in practice — such frames are dropped before a cluster
// opens) so the marker is never absent.
let reference = if keyframe || !is_video {
None
} else {
// Offset (ticks) of the referenced keyframe relative to this
// block. A non-keyframe MUST carry a ReferenceBlock or a reader
// treats it as a seek point; fall back to 0 (self-relative) in
// the pre-first-keyframe corner (unreachable in practice — such
// frames are dropped before a cluster opens) so the marker is
// never absent.
Some(
self.last_video_keyframe_ticks
.map(|kf| kf - pts_ticks)
.unwrap_or(0),
)
};
match block_additional {
// MVC: base view Block + dependent-view BlockAdditional, always a
// BlockGroup. Non-keyframe base frames get a ReferenceBlock to the
// last keyframe (a keyframe carries none), so a player never treats a
// P/B frame as a seek point.
Some(additional) => {
self.write_block_group_mvc(
track_idx + 1,
relative_ts,
@@ -1500,9 +1509,10 @@ impl<W: Write + Seek> MkvMuxer<W> {
)?;
}
None => match duration_ticks {
// BlockDuration present (PGS subtitles) → BlockGroup.
// BlockDuration present (PGS subtitles, AC-3 audio, and EVERY
// MPEG-2 video frame) → BlockGroup.
Some(dt) => {
self.write_block_group(track_idx + 1, relative_ts, keyframe, data, dt)?;
self.write_block_group(track_idx + 1, relative_ts, data, reference, dt)?;
}
None => {
self.write_simple_block(track_idx + 1, relative_ts, keyframe, data)?;
@@ -1797,22 +1807,25 @@ impl<W: Write + Seek> MkvMuxer<W> {
Ok(())
}
/// Write a BlockGroup (Block + BlockDuration, plus a ReferenceBlock when the
/// frame is not a keyframe).
///
/// `reference` is `Some(offset_ticks)` for a non-keyframe and `None` for a
/// keyframe. Inside a BlockGroup the SimpleBlock `0x80` keyframe bit is
/// reserved and MUST be 0, so a non-keyframe that omits ReferenceBlock is
/// indistinguishable from an intra frame. This path is NOT subtitle-only:
/// the MPEG-2 parser stamps a per-frame duration, so every MPEG-2 video
/// frame (I, P and B) arrives here.
fn write_block_group(
&mut self,
track_num: usize,
relative_ts: i16,
keyframe: bool,
data: &[u8],
reference: Option<i64>,
duration_ticks: u64,
) -> io::Result<()> {
let (tv, tv_len) = track_vint(track_num);
let track_vint = &tv[..tv_len];
// The 0x80 Keyframe flag is defined only for SimpleBlock; inside a
// Block within a BlockGroup that high bit is reserved and MUST be 0
// (keyframe-ness is signalled by the absence of a ReferenceBlock
// child). `keyframe` is intentionally unused here — every Block this
// path emits is intra (PGS subtitle frames carrying a duration).
let _ = keyframe;
let flags: u8 = 0x00;
let block_size = track_vint.len() + 2 + 1 + data.len();
@@ -1824,6 +1837,9 @@ impl<W: Write + Seek> MkvMuxer<W> {
self.writer.write_all(&[flags])?;
self.writer.write_all(data)?;
ebml::write_uint(&mut self.writer, ebml::BLOCK_DURATION, duration_ticks)?;
if let Some(ref_off) = reference {
ebml::write_int(&mut self.writer, ebml::REFERENCE_BLOCK, ref_off)?;
}
ebml::end_master(&mut self.writer, bg_pos)?;
Ok(())
}
@@ -1887,6 +1903,206 @@ mod tests {
use super::*;
use std::io::Cursor;
/// Keyframe flags must survive a write→read round-trip for frames that
/// carry a per-frame DURATION, i.e. the BlockGroup path.
///
/// Regression: every MPEG-2 frame carries a duration (the parser stamps one
/// on I, P and B alike), so ALL MPEG-2 video is written as BlockGroup, not
/// SimpleBlock. Inside a BlockGroup the SimpleBlock `0x80` keyframe bit is
/// reserved and the writer emits 0 — keyframe-ness lives ONLY in the
/// presence/absence of a ReferenceBlock. Two halves were broken:
/// - the writer discarded `keyframe` on this path (no ReferenceBlock ever),
/// so a shipped DVD rip marked every P/B frame as a seek point;
/// - the reader ignored ReferenceBlock and read the always-0 reserved bit,
/// so EVERY BlockGroup frame read back as a non-keyframe.
/// Downstream that silently dropped all video on `mkv://`(MPEG-2)→`m2ts://`
/// (TsMuxer drops non-key video until the first keyframe) and made
/// `mkv://`→`mkv://` / `stdio://` fail E6008 (MkvMuxer opens a cluster only
/// on a track-0 video keyframe → zero frames written).
///
/// A SimpleBlock case (duration `None`) is asserted alongside so a fix that
/// regresses the non-duration path is caught too.
#[test]
fn keyframe_survives_roundtrip_for_duration_bearing_frames() {
let v = VideoStream {
pid: 0xE0,
codec: Codec::Mpeg2,
resolution: Resolution::R480i,
frame_rate: crate::disc::FrameRate::F29_97,
hdr: HdrFormat::Sdr,
color_space: ColorSpace::Bt470bg,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
};
// ── BlockGroup path: every frame carries a duration (the MPEG-2 shape).
let t = MkvTrack::video(&v);
let mut muxer = MkvMuxer::new(Cursor::new(Vec::new()), &[t], None, 0.0, &[]).unwrap();
let dur = Some(33_366_667u64);
// I, then P, then B — decode order, all duration-bearing.
muxer
.write_frame(0, 0, true, &vec![0xAA; 188_459], dur, None)
.unwrap();
muxer
.write_frame(0, 33_366_667, false, &vec![0xBB; 27_053], dur, None)
.unwrap();
muxer
.write_frame(0, 66_733_334, false, &vec![0xCC; 4_096], dur, None)
.unwrap();
let data = muxer.writer.into_inner();
// The non-keyframes MUST have emitted a ReferenceBlock; the file is
// otherwise structurally unable to express "not a seek point".
assert!(
find_id(&data, ebml::REFERENCE_BLOCK).is_some(),
"a non-keyframe BlockGroup must carry a ReferenceBlock"
);
let mut s = crate::mux::mkvstream::MkvStream::open(Cursor::new(data)).unwrap();
let f0 = crate::pes::Stream::read(&mut s).unwrap().unwrap();
assert!(
f0.keyframe,
"BlockGroup I-frame must read back keyframe=true"
);
let f1 = crate::pes::Stream::read(&mut s).unwrap().unwrap();
assert!(
!f1.keyframe,
"BlockGroup P-frame must read back keyframe=false"
);
let f2 = crate::pes::Stream::read(&mut s).unwrap().unwrap();
assert!(
!f2.keyframe,
"BlockGroup B-frame must read back keyframe=false"
);
}
/// SimpleBlock path (no per-frame duration) — the keyframe bit is
/// authoritative there and must keep round-tripping.
#[test]
fn keyframe_survives_roundtrip_for_simple_blocks() {
let v = VideoStream {
pid: 0xE0,
codec: Codec::Mpeg2,
resolution: Resolution::R480i,
frame_rate: crate::disc::FrameRate::F29_97,
hdr: HdrFormat::Sdr,
color_space: ColorSpace::Bt470bg,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
};
let t = MkvTrack::video(&v);
let mut muxer = MkvMuxer::new(Cursor::new(Vec::new()), &[t], None, 0.0, &[]).unwrap();
muxer
.write_frame(0, 0, true, &vec![0xAA; 188_459], None, None)
.unwrap();
muxer
.write_frame(0, 33_366_667, false, &vec![0xBB; 27_053], None, None)
.unwrap();
let data = muxer.writer.into_inner();
let mut s = crate::mux::mkvstream::MkvStream::open(Cursor::new(data)).unwrap();
let f0 = crate::pes::Stream::read(&mut s).unwrap().unwrap();
assert!(
f0.keyframe,
"SimpleBlock keyframe must read back keyframe=true"
);
let f1 = crate::pes::Stream::read(&mut s).unwrap().unwrap();
assert!(
!f1.keyframe,
"SimpleBlock non-keyframe must read back keyframe=false"
);
}
/// End-to-end through the REAL `MkvStream::create`/`write`/`finish` path
/// (deferred Pending→activate buffering), in the shape the DVD pipeline
/// actually produces: several AC-3 audio frames arrive BEFORE the first
/// video frame (audio emits per-PES immediately, while MPEG-2 holds its
/// first GOP), and the video frame carries a per-frame DURATION so it is
/// written as a BlockGroup.
///
/// Guards the same regression as
/// `keyframe_survives_roundtrip_for_duration_bearing_frames`, but across the
/// buffering machinery rather than the bare muxer — the video keyframe must
/// still be a keyframe after being buffered and replayed on activation.
#[test]
fn mkvstream_preserves_video_keyframe_after_audio_preroll() {
let v = VideoStream {
pid: 0xE0,
codec: Codec::Mpeg2,
resolution: Resolution::R480i,
frame_rate: crate::disc::FrameRate::F29_97,
hdr: HdrFormat::Sdr,
color_space: ColorSpace::Bt470bg,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
};
let a = crate::disc::AudioStream {
pid: 0xBD,
codec: Codec::Ac3,
channels: crate::disc::AudioChannels::Stereo,
language: String::new(),
sample_rate: crate::disc::SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
};
let mut title = crate::disc::DiscTitle::empty();
title.streams.push(crate::disc::Stream::Video(v));
title.streams.push(crate::disc::Stream::Audio(a));
let dir = std::env::temp_dir()
.join("fmkv-test-preroll")
.join(format!("{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("preroll.mkv");
let writer: Box<dyn crate::mux::WriteSeek + Send> =
Box::new(std::fs::File::create(&path).unwrap());
let mut s = crate::mux::mkvstream::MkvStream::create(writer, &title, None).unwrap();
// 5 audio frames arrive first (realistic pre-video buffering).
for i in 0..5u32 {
let f = crate::pes::PesFrame {
coding: None,
source: None,
track: 1,
pts: i as i64 * 32_000_000,
keyframe: false,
data: vec![0xCC; 768],
duration_ns: None,
};
crate::pes::Stream::write(&mut s, &f).unwrap();
}
// Then the true video keyframe — duration-bearing, as MPEG-2 always is,
// so it is written as a BlockGroup (where the keyframe bit is reserved).
let vf = crate::pes::PesFrame {
coding: None,
source: None,
track: 0,
pts: 0,
keyframe: true,
data: vec![0xAA; 188_459],
duration_ns: Some(33_366_667),
};
crate::pes::Stream::write(&mut s, &vf).unwrap();
crate::pes::Stream::finish(&mut s).unwrap();
drop(s);
let f = std::fs::File::open(&path).unwrap();
let mut r = crate::mux::mkvstream::MkvStream::open(f).unwrap();
let f0 = crate::pes::Stream::read(&mut r).unwrap().unwrap();
assert_eq!(f0.track, 0, "first readback frame must be track 0 (video)");
assert!(
f0.keyframe,
"video keyframe written with 5 audio frames ahead of it must read back keyframe=true"
);
std::fs::remove_dir_all(&dir).ok();
}
/// Anamorphic DVD: a 720x576 (R576i) PAL stream flagged 16:9 must write a
/// DisplayWidth/Height carrying the 16:9 DAR (1024x576), NOT the square-pixel
/// 720x576 (which players show as ~5:4). Square-pixel video
src/mux/mkvstream.rs
+103 -7
View File
@@ -677,6 +677,15 @@ impl crate::pes::Stream for MkvStream {
let mut remaining = size;
let mut block: Option<Vec<u8>> = None;
let mut duration_ms: Option<u64> = None;
// Keyframe-ness of a BlockGroup is carried ONLY by the
// presence/absence of a ReferenceBlock child: inside a
// BlockGroup the SimpleBlock 0x80 keyframe bit is reserved
// and the writer always emits it as 0. Reading that bit
// (as this arm used to) makes EVERY BlockGroup frame look
// like a non-keyframe — which silently broke every re-mux
// of MPEG-2 video, whose parser stamps a per-frame duration
// so all of its frames take the BlockGroup path.
let mut has_reference = false;
while remaining > 0 {
let (cid, cs, hlen) = ebml::read_element_header(&mut rs.reader)?;
if cs == u64::MAX {
@@ -700,6 +709,13 @@ impl crate::pes::Stream for MkvStream {
ebml::BLOCK_DURATION => {
duration_ms = Some(read_uint_bounded(&mut rs.reader, cs)?);
}
ebml::REFERENCE_BLOCK => {
// Presence alone is the signal — this Block
// references another, so it is not a keyframe.
// The offset value itself is not needed here.
has_reference = true;
skip_bytes(&mut rs.reader, cs)?;
}
_ => skip_bytes(&mut rs.reader, cs)?,
}
}
@@ -710,13 +726,17 @@ impl crate::pes::Stream for MkvStream {
// in foreign MKVs) — same scaling PTS uses.
let dur_ns =
duration_ms.map(|ticks| ticks.saturating_mul(rs.ts_scale_ns as u64));
if let Some(frame) = parse_block(
if let Some(mut frame) = parse_block(
&block,
rs.cluster_ts_ticks,
rs.ts_scale_ns,
streams_len,
dur_ns,
) {
// Override the flag-bit guess from `parse_block`
// (meaningful for SimpleBlock only) with the
// BlockGroup's authoritative signal.
frame.keyframe = !has_reference;
return Ok(Some(frame));
}
}
@@ -1803,11 +1823,20 @@ mod tests {
#[test]
fn block_group_frame_round_trips_with_duration() {
// MkvMuxer emits AC3/PGS frames as a BlockGroup (BLOCK + BLOCK_DURATION).
// The reader must descend into the group and yield the frame (with its
// duration) rather than skipping it — otherwise every AC3/PGS frame this
// muxer writes is lost on read-back.
let block = [0x82u8, 0x00, 0x05, 0x00, 0x11, 0x22, 0x33]; // track 2, rel 5, not-kf, 3 data
// MkvMuxer emits AC3/PGS frames — and every MPEG-2 video frame — as a
// BlockGroup (BLOCK + BLOCK_DURATION [+ REFERENCE_BLOCK]). The reader
// must descend into the group and yield the frame (with its duration)
// rather than skipping it — otherwise every such frame this muxer writes
// is lost on read-back.
//
// Keyframe-ness: inside a BlockGroup the SimpleBlock 0x80 bit is
// RESERVED (always 0 here); a Block with NO ReferenceBlock child is a
// keyframe. This group has none, so the frame is a keyframe — which is
// also the truth for the AC-3/PGS frames this path was written for
// (they are self-contained). The `!keyframe` this test used to assert
// came from reading the reserved bit; see
// `reference_block_marks_block_group_frame_as_non_keyframe`.
let block = [0x82u8, 0x00, 0x05, 0x00, 0x11, 0x22, 0x33]; // track 2, rel 5, reserved bit 0, 3 data
let mut bg_body = Vec::new();
ebml::write_id(&mut bg_body, ebml::BLOCK).unwrap();
ebml::write_size(&mut bg_body, block.len() as u64).unwrap();
@@ -1853,12 +1882,79 @@ mod tests {
.unwrap()
.expect("BlockGroup frame must be read");
assert_eq!(frame.track, 1, "track 2 → index 1");
assert!(!frame.keyframe);
assert!(
frame.keyframe,
"a BlockGroup with no ReferenceBlock is a keyframe (the 0x80 bit is reserved here)"
);
assert_eq!(frame.data, vec![0x11, 0x22, 0x33]);
assert_eq!(frame.pts, 105 * 1_000_000, "pts = (cluster 100 + rel 5) ms");
assert_eq!(frame.duration_ns, Some(40 * 1_000_000));
}
/// A BlockGroup carrying a ReferenceBlock is NOT a keyframe — that element's
/// presence is the only non-keyframe signal a BlockGroup has (the
/// SimpleBlock 0x80 flag bit is reserved and always 0 inside one).
///
/// Regression: the reader used to `skip_bytes` past REFERENCE_BLOCK and read
/// the reserved bit instead, so EVERY BlockGroup frame came back as a
/// non-keyframe. Since the MPEG-2 parser stamps a per-frame duration, all
/// MPEG-2 video takes the BlockGroup path — so no video frame ever looked
/// like a keyframe on re-mux. That silently dropped all video on
/// `mkv://`→`m2ts://` and failed `mkv://`→`mkv://` with E6008.
#[test]
fn reference_block_marks_block_group_frame_as_non_keyframe() {
// Same construction as the test above, plus a ReferenceBlock child.
let block = [0x82u8, 0x00, 0x05, 0x00, 0x11, 0x22, 0x33];
let mut bg_body = Vec::new();
ebml::write_id(&mut bg_body, ebml::BLOCK).unwrap();
ebml::write_size(&mut bg_body, block.len() as u64).unwrap();
bg_body.extend_from_slice(&block);
ebml::write_uint(&mut bg_body, ebml::BLOCK_DURATION, 40).unwrap();
// References a keyframe 40 ms earlier ⇒ this Block is not a seek point.
ebml::write_int(&mut bg_body, ebml::REFERENCE_BLOCK, -40).unwrap();
let mut cluster = Vec::new();
ebml::write_id(&mut cluster, ebml::CLUSTER).unwrap();
ebml::write_unknown_size(&mut cluster).unwrap();
ebml::write_uint(&mut cluster, ebml::CLUSTER_TIMESTAMP, 100).unwrap();
ebml::write_id(&mut cluster, ebml::BLOCK_GROUP).unwrap();
ebml::write_size(&mut cluster, bg_body.len() as u64).unwrap();
cluster.extend_from_slice(&bg_body);
let mut out = Vec::new();
ebml::write_id(&mut out, ebml::EBML).unwrap();
ebml::write_size(&mut out, 0).unwrap();
ebml::write_id(&mut out, ebml::SEGMENT).unwrap();
ebml::write_unknown_size(&mut out).unwrap();
ebml::write_id(&mut out, ebml::INFO).unwrap();
ebml::write_size(&mut out, 0).unwrap();
let mut tracks = Vec::new();
for (n, t) in [(1u64, 1u64), (2u64, 2u64)] {
let mut entry = Vec::new();
ebml::write_uint(&mut entry, ebml::TRACK_NUMBER, n).unwrap();
ebml::write_uint(&mut entry, ebml::TRACK_TYPE, t).unwrap();
ebml::write_id(&mut tracks, ebml::TRACK_ENTRY).unwrap();
ebml::write_size(&mut tracks, entry.len() as u64).unwrap();
tracks.extend_from_slice(&entry);
}
ebml::write_id(&mut out, ebml::TRACKS).unwrap();
ebml::write_size(&mut out, tracks.len() as u64).unwrap();
out.extend_from_slice(&tracks);
out.extend_from_slice(&cluster);
let mut stream = MkvStream::open(Cursor::new(out)).unwrap();
let frame = stream
.read()
.unwrap()
.expect("BlockGroup frame must be read");
assert!(
!frame.keyframe,
"a BlockGroup WITH a ReferenceBlock must read back as a non-keyframe"
);
assert_eq!(frame.data, vec![0x11, 0x22, 0x33]);
assert_eq!(frame.duration_ns, Some(40 * 1_000_000));
}
#[test]
fn track_number_zero_is_rejected() {
// A TRACK_ENTRY with TRACK_NUMBER 0 must be rejected (the ts_pid
src/mux/tsmux.rs
+41 -6
View File
@@ -42,6 +42,16 @@ pub struct TsMuxer<W: Write> {
continuity: Vec<u8>, // per-PID continuity counter (0-15)
codec_privates: Vec<Option<Vec<u8>>>, // per-track codec_private (for video parameter sets)
params_written: Vec<bool>, // per-track: have we written parameter sets?
/// Per-track: does this video track's ES arrive length-prefixed (MKV/PES
/// NALU convention — HEVC, H.264) and need Annex-B conversion? Defaults
/// to `true` (the prior, only behavior) so every existing call site is
/// unaffected; a track carrying MPEG-2 or VC-1 — neither is NAL-based,
/// both already arrive as plain start-code ES — must opt OUT via
/// [`TsMuxer::set_nal_video`] or `length_prefixed_to_annex_b` mangles the
/// frame into empty/garbage output (frame_count still increments, so the
/// mux "succeeds" while silently producing a video-less file). Ignored
/// for non-video tracks.
nal_video: Vec<bool>,
/// Global PTS origin (nanoseconds), seeded by the FIRST video frame so
/// the audio/video offset is preserved. Frames that arrive before it
/// is set saturate to 0.
@@ -63,6 +73,7 @@ impl<W: Write> TsMuxer<W> {
continuity: vec![0u8; n],
codec_privates: vec![None; n],
params_written: vec![false; n],
nal_video: vec![true; n],
base_pts_ns: None,
frame_count: 0,
}
@@ -86,6 +97,25 @@ impl<W: Write> TsMuxer<W> {
Ok(())
}
/// Mark whether a video track's ES arrives length-prefixed (MKV/PES NALU
/// convention) and needs Annex-B conversion. Call with `false` for MPEG-2
/// or VC-1 tracks — neither is NAL-based, so the ES already IS the wire
/// format and must pass through unconverted (see [`Self::nal_video`]).
/// Ignored (harmlessly) for a non-video track. Returns
/// [`Error::MuxTrackRange`](crate::error::Error::MuxTrackRange) for an
/// out-of-range index.
pub fn set_nal_video(&mut self, track: usize, is_nal: bool) -> io::Result<()> {
if track >= self.nal_video.len() {
return Err(crate::error::Error::MuxTrackRange {
track,
tracks: self.nal_video.len(),
}
.into());
}
self.nal_video[track] = is_nal;
Ok(())
}
/// Write a PES frame as BD-TS packets.
/// Video frame data is expected as length-prefixed NALUs (MKV/PES format)
/// and is converted to Annex B for transport stream.
@@ -125,17 +155,19 @@ impl<W: Write> TsMuxer<W> {
let base = self.base_pts_ns.unwrap_or(pts_ns);
let pts_ns = pts_ns.saturating_sub(base);
// For video: convert length-prefixed NALUs to Annex B (start codes).
// Prepend codec_private parameter sets on the FIRST keyframe only.
// For NAL-based video (HEVC, H.264): convert length-prefixed NALUs to
// Annex B (start codes) and prepend codec_private parameter sets on
// the FIRST keyframe only.
//
// Arm `params_written` on the first video keyframe regardless of
// whether it carries data: an empty-data keyframe still anchors
// the stream, and leaving the flag unset would make every later
// non-key frame fail the drop guard above and silently vanish.
// For non-video the ES bytes pass through unchanged, so borrow
// `data` directly rather than copying it; only video needs an
// owned Annex-B conversion buffer.
let es_data: std::borrow::Cow<'_, [u8]> = if is_video {
// For non-video AND for non-NAL video (MPEG-2, VC-1 — already
// start-code ES, never length-prefixed) the bytes pass through
// unchanged, so borrow `data` directly rather than copying it; only
// NAL video needs an owned Annex-B conversion buffer.
let es_data: std::borrow::Cow<'_, [u8]> = if is_video && self.nal_video[track] {
let mut annex_b = Vec::new();
if keyframe && !self.params_written[track] {
if let Some(ref cp) = self.codec_privates[track] {
@@ -148,6 +180,9 @@ impl<W: Write> TsMuxer<W> {
annex_b.extend_from_slice(&length_prefixed_to_annex_b(data));
std::borrow::Cow::Owned(annex_b)
} else {
if is_video {
self.params_written[track] = true;
}
std::borrow::Cow::Borrowed(data)
};