pq/libfreemkv
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

mux: pipelined PES highway — read+decrypt → demux → parse on 3 threads

Browse Source 
Introduces the freemkv mux throughput highway: a three-stage thread
pipeline that replaces the inline single-thread read path for any
file-backed source (ISO and m2ts file URLs both route through it).

  Thread A: read + decrypt  (PrefetchedSectorSource / BytePrefetcher)
  Thread B: M2TS demux      (DemuxThread)
  Thread C: codec parse     (PipelinedPesStream, on caller thread)

Each handoff uses a bounded crossbeam channel with a recycled buffer
pool — no allocations or memcpys in the steady-state hot loop.

Component map:

* io/byte_prefetcher.rs (new) — std::io::Read producer thread with
  recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector
  side) so demux_thread::spawn_zero_copy can wire either upstream.
* sector/prefetched.rs — recycled buffer pool added; into_channels()
  peels off the rx/recycle_tx/shell triple for zero-copy demux.
* mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs
  feed() on its thread, ships Vec<PesPacket> batches.
* mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls
  packets from the demux thread and runs codec parse on the caller.
* mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline
  (private) assemble the three stages; iso:// and m2ts:// both
  return PipelinedPesStream.
* mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted;
  the read direction lives on the highway now).
* mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find.
* mux/codec/hevc.rs — tightened frame_data initial capacity.
* mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB
  remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page
  first-touch fault tax that the previous 256 KiB pre-alloc paid on
  every PES boundary.
* mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as
  the legacy autorip ingress (drive + multipass paths still need
  on_event / skip_errors before they migrate to the highway).
* io/file_sector_source/* — per-OS prefetch() syscall hook
  (Linux readahead, macOS F_RDADVISE, Windows/other no-op).
* decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS;
  pool sized to all cores by default.

Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://):

  60 → 322 MB/s warm cache (old new_pipeline path)
  60 → 660 MB/s warm cache (highway path, this commit)
  60 → 126 MB/s sustained disk-bound

The IsoSectorReader baseline reader was deleted in favour of
FileSectorSource so the freemkv CLI and autorip exercise the same
read path.
This commit is contained in:
MattJackson
2026-05-19 13:35:32 -07:00
parent 2a31a47434
commit c51b3181f2
32 changed files with 2238 additions and 771 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/mux/codec/h264.rs
+6 -4
View File
@@ -176,14 +176,16 @@ impl<'a> Iterator for NalIterator<'a> {
}
/// Find the position of the next start code (00 00 01) at or after `from`.
///
/// Backed by `memchr::memmem::find` for SIMD-accelerated bytestring
/// search. On AVX2-capable x86_64 this runs ~5–10× the byte-by-byte
/// scan that preceded it; on a 200 KB UHD HEVC frame the saving is
/// in the hundreds of microseconds per call.
pub fn find_start_code(data: &[u8], from: usize) -> Option<usize> {
if data.len() < from + 3 {
return None;
}
// Range excludes last 2 bytes since we read 3 bytes at each position.
// data.len()-2 as exclusive upper bound means last checked index is data.len()-3,
// which accesses data[len-3], data[len-2], data[len-1] — all valid.
(from..data.len() - 2).find(|&i| data[i] == 0x00 && data[i + 1] == 0x00 && data[i + 2] == 0x01)
memchr::memmem::find(&data[from..], b"\x00\x00\x01").map(|rel| from + rel)
}
/// Skip past the start code at position `pos`, returning the first byte after it.
src/mux/codec/hevc.rs
+4 -1
View File
@@ -52,7 +52,10 @@ impl CodecParser for HevcParser {
let pts_ns = pes.dts.or(pes.pts).map(pts_to_ns).unwrap_or(0);
let data = &pes.data;
let mut keyframe = false;
let mut frame_data = Vec::new();
// Pre-size: output is ~input bytes with a few 4-byte length
// prefixes added. UHD frames are 150-300 KB; the unsized Vec
// growth chain otherwise reallocs 5-7× per frame.
let mut frame_data = Vec::with_capacity(data.len() + 64);
// Single-pass NAL scan: extract params, detect keyframes, build length-prefixed output
let mut pos = 0;
src/mux/demux_thread.rs
+350
View File
@@ -0,0 +1,350 @@
//! `DemuxThread` — runs the read+decrypt+demux pipeline on a
//! dedicated thread, feeding completed `PesPacket` batches to the
//! caller via a bounded channel.
//!
//! ## Why a second worker thread
//!
//! With [`crate::sector::PrefetchedSectorSource`] alone, read+decrypt
//! already runs on a producer thread; the *consumer* (main) thread
//! still serialises `ts_demuxer.feed` (M2TS parsing) with the codec
//! parsers. Profiling on the rip1 testbed showed feed at ~37 % and
//! codec parse at ~44 % of consumer wall time — i.e. feed is heavy
//! enough that pipelining it with parse pays for itself.
//!
//! Splitting them: feed runs in [`DemuxThread`]; the consumer thread
//! receives `Vec<PesPacket>` batches and runs codec parse + frame
//! emission only. Total throughput becomes `1/max(feed, parse)`
//! instead of `1/(feed + parse)`.
//!
//! ## Lifecycle
//!
//! [`DemuxThread::spawn`] takes ownership of the inner reader and the
//! demuxer state, returns a handle plus a `Receiver<DemuxBatch>`.
//! Dropping the handle closes the channel which signals the thread
//! to exit; the join in `Drop::drop` is bounded.
use crate::halt::Halt;
use crate::sector::SectorSource;
use crossbeam_channel::{Receiver, Sender, bounded};
use std::thread::JoinHandle;
/// Output channel depth. Two batches in flight keeps the consumer
/// (codec parser) busy without piling up demuxed bytes if it stalls.
const DEMUX_CHANNEL_DEPTH: usize = 2;
/// One demuxed batch flowing from the demux thread to the consumer.
pub enum DemuxBatch {
/// Successfully demuxed PesPackets — non-empty.
Ts(Vec<super::ts::PesPacket>),
Ps(Vec<super::ps::PsPacket>),
/// Underlying reader returned an error. Terminal.
Err(std::io::Error),
}
/// Spawned demux thread. Drop joins.
///
/// In zero-copy mode the thread also owns an opaque
/// `producer_shell: Option<Box<dyn Send>>` — the join handle of the
/// upstream producer (sector or byte prefetcher). Dropping the
/// `DemuxThread` runs the shell's `Drop`, which joins the producer.
/// `Box<dyn Send>` rather than a concrete type so the same demux
/// worker can be wired behind either prefetcher kind.
pub struct DemuxThread {
handle: Option<JoinHandle<()>>,
#[allow(dead_code)]
producer_shell: Option<Box<dyn Send>>,
}
impl DemuxThread {
/// Spawn the demux thread. Returns the thread handle and a
/// receiver for [`DemuxBatch`] items.
///
/// `reader` is the fully-composed read+decrypt stack (e.g.
/// [`PrefetchedSectorSource`](crate::sector::PrefetchedSectorSource)
/// wrapping
/// [`DecryptingSectorSource`](crate::sector::DecryptingSectorSource)).
/// `extents` is what the thread walks; it issues one
/// `read_sectors` per batch of `batch_sectors` sectors (aligned
/// to 3-sector AACS units when possible).
pub fn spawn<S: SectorSource + Send + 'static>(
mut reader: S,
extents: Vec<crate::disc::Extent>,
batch_sectors: u16,
halt: Option<Halt>,
ts: Option<super::ts::TsDemuxer>,
ps: Option<super::ps::PsDemuxer>,
) -> (Self, Receiver<DemuxBatch>) {
let (tx, rx) = bounded::<DemuxBatch>(DEMUX_CHANNEL_DEPTH);
let mut ts = ts;
let mut ps = ps;
let handle = std::thread::Builder::new()
.name("freemkv-demux".into())
.spawn(move || {
let mut buf = vec![0u8; batch_sectors as usize * 2048];
let mut ext_idx = 0usize;
let mut offset: u32 = 0;
let prof = std::env::var_os("FREEMKV_PROFILE").is_some();
let mut prof_started = std::time::Instant::now();
let mut prof_last_dump = prof_started;
let mut prof_read_ns: u128 = 0;
let mut prof_feed_ns: u128 = 0;
let mut prof_send_ns: u128 = 0;
let mut prof_bytes: u64 = 0;
while ext_idx < extents.len() {
if halt.as_ref().map(|h| h.is_cancelled()).unwrap_or(false) {
return;
}
let ext = &extents[ext_idx];
let remaining = ext.sector_count.saturating_sub(offset);
if remaining == 0 {
ext_idx += 1;
offset = 0;
continue;
}
let mut sectors = remaining.min(batch_sectors as u32) as u16;
if sectors >= 3 {
sectors -= sectors % 3;
}
let bytes = sectors as usize * 2048;
if buf.len() < bytes {
buf.resize(bytes, 0);
}
let lba = ext.start_lba + offset;
let t0 = if prof {
Some(std::time::Instant::now())
} else {
None
};
let n = match reader.read_sectors(lba, sectors, &mut buf[..bytes], false) {
Ok(n) => n,
Err(e) => {
let _ = tx.send(DemuxBatch::Err(e.into()));
return;
}
};
let t1 = if prof {
Some(std::time::Instant::now())
} else {
None
};
offset += sectors as u32;
// Demux this batch immediately so the channel
// carries already-parsed PesPackets, not raw
// sector bytes.
if let Some(ref mut d) = ts {
let pkts = d.feed(&buf[..n]);
let t2 = if prof {
Some(std::time::Instant::now())
} else {
None
};
if !pkts.is_empty() && tx.send(DemuxBatch::Ts(pkts)).is_err() {
return; // consumer dropped
}
let t3 = if prof {
Some(std::time::Instant::now())
} else {
None
};
if prof {
prof_read_ns += t1.unwrap().duration_since(t0.unwrap()).as_nanos();
prof_feed_ns += t2.unwrap().duration_since(t1.unwrap()).as_nanos();
prof_send_ns += t3.unwrap().duration_since(t2.unwrap()).as_nanos();
prof_bytes += n as u64;
let now = t3.unwrap();
if now.duration_since(prof_last_dump)
>= std::time::Duration::from_secs(5)
{
let el = now.duration_since(prof_started).as_millis().max(1);
let mbps = prof_bytes as u128 * 1000 / 1_000_000 / el;
eprintln!(
"[demux] elapsed={}ms in={}MB/s read={}% feed={}% send={}%",
el,
mbps,
prof_read_ns / 10_000 / el,
prof_feed_ns / 10_000 / el,
prof_send_ns / 10_000 / el,
);
prof_last_dump = now;
prof_started = now;
prof_read_ns = 0;
prof_feed_ns = 0;
prof_send_ns = 0;
prof_bytes = 0;
}
}
} else if let Some(ref mut d) = ps {
let pkts = d.feed(&buf[..n]);
if !pkts.is_empty() && tx.send(DemuxBatch::Ps(pkts)).is_err() {
return;
}
}
}
// EOF — emit any flushed packets too.
if let Some(ref mut d) = ts {
let tail = d.flush();
if !tail.is_empty() {
let _ = tx.send(DemuxBatch::Ts(tail));
}
} else if let Some(ref mut d) = ps {
let tail = d.flush();
if !tail.is_empty() {
let _ = tx.send(DemuxBatch::Ps(tail));
}
}
// Sender drops here -> consumer sees RecvError → EOF.
})
.expect("freemkv-demux thread spawn failed");
(
Self {
handle: Some(handle),
producer_shell: None,
},
rx,
)
}
/// Zero-copy variant. Instead of taking a `SectorSource` and
/// memcpy-ing through its `read_sectors` API, this constructor
/// consumes the prefetch channels directly: filled buffers come
/// in via `prefetch_rx`, the demux thread feeds them, then
/// returns them to `recycle_tx` for the producer to re-fill.
/// Eliminates the 16 MiB memcpy per batch that the SectorSource
/// adapter incurred (and, with the producer-side recycling pool,
/// also eliminates the per-batch heap alloc / cross-thread free
/// that was costing 40 %+ of demux-thread time before).
///
/// `producer_shell` is an opaque handle whose only purpose is to
/// outlive the demux thread and join the upstream producer when
/// dropped. Both
/// [`crate::sector::PrefetchedSectorSource::into_channels`] and
/// [`crate::io::byte_prefetcher::BytePrefetcher::into_channels`]
/// hand back a shell that fits — pass either.
pub fn spawn_zero_copy<S: Send + 'static>(
prefetch_rx: Receiver<std::io::Result<Vec<u8>>>,
recycle_tx: Sender<Vec<u8>>,
producer_shell: S,
halt: Option<Halt>,
ts: Option<super::ts::TsDemuxer>,
ps: Option<super::ps::PsDemuxer>,
) -> (Self, Receiver<DemuxBatch>) {
let (tx, rx) = bounded::<DemuxBatch>(DEMUX_CHANNEL_DEPTH);
let mut ts = ts;
let mut ps = ps;
let handle = std::thread::Builder::new()
.name("freemkv-demux".into())
.spawn(move || {
let prof = std::env::var_os("FREEMKV_PROFILE").is_some();
let mut prof_started = std::time::Instant::now();
let mut prof_last_dump = prof_started;
let mut prof_read_ns: u128 = 0;
let mut prof_feed_ns: u128 = 0;
let mut prof_bytes: u64 = 0;
loop {
if halt.as_ref().map(|h| h.is_cancelled()).unwrap_or(false) {
return;
}
let t0 = if prof {
Some(std::time::Instant::now())
} else {
None
};
let buf = match prefetch_rx.recv() {
Ok(Ok(b)) => b,
Ok(Err(e)) => {
let _ = tx.send(DemuxBatch::Err(e));
return;
}
Err(_) => break, // producer done → EOF
};
let t1 = if prof {
Some(std::time::Instant::now())
} else {
None
};
let n = buf.len();
if let Some(ref mut d) = ts {
let pkts = d.feed(&buf);
let t2 = if prof {
Some(std::time::Instant::now())
} else {
None
};
// Recycle the buffer back to the producer
// before pushing the demuxed packets. If the
// recycle channel is closed the producer has
// exited; we drop the buffer and continue.
let _ = recycle_tx.send(buf);
if !pkts.is_empty() && tx.send(DemuxBatch::Ts(pkts)).is_err() {
return;
}
if prof {
prof_read_ns += t1.unwrap().duration_since(t0.unwrap()).as_nanos();
prof_feed_ns += t2.unwrap().duration_since(t1.unwrap()).as_nanos();
prof_bytes += n as u64;
let now = std::time::Instant::now();
if now.duration_since(prof_last_dump)
>= std::time::Duration::from_secs(5)
{
let el = now.duration_since(prof_started).as_millis().max(1);
let mbps = prof_bytes as u128 * 1000 / 1_000_000 / el;
eprintln!(
"[demux] elapsed={}ms in={}MB/s read={}% feed={}%",
el,
mbps,
prof_read_ns / 10_000 / el,
prof_feed_ns / 10_000 / el,
);
prof_last_dump = now;
prof_started = now;
prof_read_ns = 0;
prof_feed_ns = 0;
prof_bytes = 0;
}
}
} else if let Some(ref mut d) = ps {
let pkts = d.feed(&buf);
let _ = recycle_tx.send(buf);
if !pkts.is_empty() && tx.send(DemuxBatch::Ps(pkts)).is_err() {
return;
}
} else {
let _ = recycle_tx.send(buf);
}
}
// Flush tail packets at EOF.
if let Some(ref mut d) = ts {
let tail = d.flush();
if !tail.is_empty() {
let _ = tx.send(DemuxBatch::Ts(tail));
}
} else if let Some(ref mut d) = ps {
let tail = d.flush();
if !tail.is_empty() {
let _ = tx.send(DemuxBatch::Ps(tail));
}
}
})
.expect("freemkv-demux thread spawn failed");
(
Self {
handle: Some(handle),
producer_shell: Some(Box::new(producer_shell)),
},
rx,
)
}
}
impl Drop for DemuxThread {
fn drop(&mut self) {
if let Some(h) = self.handle.take() {
let _ = h.join();
}
}
}
src/mux/disc.rs
+326 -8
View File
@@ -1,7 +1,7 @@
//! DiscStream — read any disc (physical drive or ISO file) → PES frames.
//!
//! One stream type for all disc sources. The source is a SectorSource —
//! Drive (hardware) or IsoSectorReader (file). DiscStream doesn't care.
//! Drive (hardware) or FileSectorSource (file). DiscStream doesn't care.
//!
//! Read-only. For disc→ISO (raw sector copy), use `Disc::copy()`.
@@ -149,12 +149,28 @@ pub struct DiscStream {
// a percent without a separate API call.
bytes_total_extents: u64,
// PES output
// PES output. `ts_demuxer` and `ps_demuxer` are `None` when the
// stream is in pipeline mode — the demux state lives inside a
// [`super::demux_thread::DemuxThread`] and PesPackets arrive
// through `demux_rx` already parsed.
ts_demuxer: Option<super::ts::TsDemuxer>,
ps_demuxer: Option<super::ps::PsDemuxer>,
parsers: Vec<(u16, Box<dyn super::codec::CodecParser>)>,
pending_frames: std::collections::VecDeque<crate::pes::PesFrame>,
pid_to_track: Vec<(u16, usize)>,
// Pipeline mode: when `Some`, the read+decrypt+demux pipeline
// runs on a dedicated thread; this stream's `read()` just pulls
// PesPacket batches from `demux_rx` and runs codec parse on the
// caller thread. See [`super::demux_thread`].
//
// `demux_thread` is kept solely so that `Drop` joins the worker
// before this stream is dropped — direct reads happen through
// `demux_rx`. The `allow(dead_code)` keeps the optimizer happy
// since the field is only used at drop time.
#[allow(dead_code)]
demux_thread: Option<super::demux_thread::DemuxThread>,
demux_rx: Option<crossbeam_channel::Receiver<super::demux_thread::DemuxBatch>>,
}
impl DiscStream {
@@ -235,6 +251,118 @@ impl DiscStream {
parsers,
pending_frames: std::collections::VecDeque::new(),
pid_to_track,
demux_thread: None,
demux_rx: None,
}
}
/// Pipeline-mode constructor. Moves the read+decrypt+demux work
/// onto a [`super::demux_thread::DemuxThread`] so the caller's
/// `read()` thread only does codec parse + frame emission.
///
/// `reader` is a [`crate::sector::PrefetchedSectorSource`] — the
/// prefetched producer thread already runs read+decrypt on its
/// own thread; this constructor peels off its channels for the
/// demux thread to consume in zero-copy mode (no buffer memcpy
/// across thread boundary, recycled-pool of two buffers, no
/// allocator activity in the hot loop).
///
/// Pipeline mode is the preferred wiring for ISO file mux on a
/// multi-core host; it gives a ~2× consumer throughput in the
/// `null://` benchmark vs the single-thread inline path.
pub fn new_pipeline(
reader: crate::sector::PrefetchedSectorSource,
title: DiscTitle,
decrypt_keys: crate::decrypt::DecryptKeys,
batch_sectors: u16,
content_format: crate::disc::ContentFormat,
halt: Option<Halt>,
) -> Self {
let extents = title.extents.clone();
let bytes_total_extents: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum();
let mut pids = Vec::new();
let mut parsers = Vec::new();
let mut pid_to_track = Vec::new();
for (idx, s) in title.streams.iter().enumerate() {
let (pid, codec) = match s {
crate::disc::Stream::Video(v) => (v.pid, v.codec),
crate::disc::Stream::Audio(a) => (a.pid, a.codec),
crate::disc::Stream::Subtitle(s) => (s.pid, s.codec),
};
pids.push(pid);
pid_to_track.push((pid, idx));
parsers.push((pid, super::codec::parser_for_codec(codec, None)));
}
let (ts, ps) = match content_format {
crate::disc::ContentFormat::MpegPs => (None, Some(super::ps::PsDemuxer::new())),
crate::disc::ContentFormat::BdTs => {
let ts_pids: Vec<u16> = pids.clone();
if ts_pids.is_empty() {
(None, None)
} else {
(Some(super::ts::TsDemuxer::new(&ts_pids)), None)
}
}
};
let (prefetch_rx, recycle_tx, shell) = reader.into_channels();
let (handle, rx) = super::demux_thread::DemuxThread::spawn_zero_copy(
prefetch_rx,
recycle_tx,
shell,
halt.clone(),
ts,
ps,
);
// The DiscStream's own reader is a no-op pass-through — the
// real reader lives inside the demux thread. We need *some*
// Box<dyn SectorSource> to satisfy the field type; use a
// tiny stub. fill_extents won't be called in pipeline mode.
struct NullSource;
impl SectorSource for NullSource {
fn capacity_sectors(&self) -> u32 {
0
}
fn read_sectors(
&mut self,
_: u32,
_: u16,
_: &mut [u8],
_: bool,
) -> crate::error::Result<usize> {
Ok(0)
}
}
let dummy: Box<dyn SectorSource> = Box::new(NullSource);
Self {
reader: DecryptingSectorSource::new(dummy, crate::decrypt::DecryptKeys::None),
title,
disc: None,
decrypt_keys,
extents,
current_extent: 0,
current_offset: 0,
read_buf: Vec::new(),
buf_valid: 0,
adaptive: AdaptiveBatch::new(batch_sectors),
errors: 0,
skip_errors: false,
halt,
event_fn: None,
eof: false,
bytes_read_total: 0,
bytes_total_extents,
ts_demuxer: None,
ps_demuxer: None,
parsers,
pending_frames: std::collections::VecDeque::new(),
pid_to_track,
demux_thread: Some(handle),
demux_rx: Some(rx),
}
}
@@ -403,6 +531,153 @@ impl DiscStream {
}
}
/// Per-stage profiling state — populated only when `FREEMKV_PROFILE`
/// is set. Dumps a percentage breakdown to stderr every
/// [`PROFILE_INTERVAL`]. Zero overhead in normal runs (Option check
/// is the only added cost).
struct StageProf {
started: std::time::Instant,
last_dump: std::time::Instant,
fill_ns: u128,
feed_ns: u128,
consume_ns: u128,
bytes_in: u64,
}
const PROFILE_INTERVAL: std::time::Duration = std::time::Duration::from_secs(5);
thread_local! {
static STAGE_PROF: std::cell::RefCell<Option<StageProf>> = const { std::cell::RefCell::new(None) };
}
fn prof_active() -> bool {
std::env::var_os("FREEMKV_PROFILE").is_some()
}
fn prof_tick(stage: &str, ns: u128, bytes: u64) {
STAGE_PROF.with(|cell| {
let mut slot = cell.borrow_mut();
if slot.is_none() {
if !prof_active() {
return;
}
let now = std::time::Instant::now();
*slot = Some(StageProf {
started: now,
last_dump: now,
fill_ns: 0,
feed_ns: 0,
consume_ns: 0,
bytes_in: 0,
});
}
let p = slot.as_mut().unwrap();
match stage {
"fill" => p.fill_ns += ns,
"feed" => p.feed_ns += ns,
"consume" => p.consume_ns += ns,
_ => {}
}
p.bytes_in += bytes;
let now = std::time::Instant::now();
if now.duration_since(p.last_dump) < PROFILE_INTERVAL {
return;
}
let elapsed_ms = now.duration_since(p.started).as_millis().max(1);
let fill_pct = p.fill_ns / 10_000 / elapsed_ms;
let feed_pct = p.feed_ns / 10_000 / elapsed_ms;
let consume_pct = p.consume_ns / 10_000 / elapsed_ms;
let mbps = p.bytes_in as u128 * 1000 / 1_000_000 / elapsed_ms;
eprintln!(
"[profile] elapsed={}ms in={}MB/s fill={}% feed={}% consume={}%",
elapsed_ms, mbps, fill_pct, feed_pct, consume_pct,
);
p.last_dump = now;
});
}
impl DiscStream {
/// Pipeline-mode `read()` helper: pull one PesPacket batch from
/// the demux thread, run codec parse on each PES, enqueue the
/// resulting PesFrames, return the first one.
fn read_pipeline(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
use super::demux_thread::DemuxBatch;
let rx = self.demux_rx.as_ref().expect("read_pipeline without rx");
match rx.recv() {
Ok(DemuxBatch::Ts(packets)) => {
let skip_parse = std::env::var_os("FREEMKV_SKIP_PARSE").is_some();
for pes in packets {
if let Some((_, track)) = self
.pid_to_track
.iter()
.find(|(pid, _)| *pid == pes.pid)
.copied()
{
if skip_parse {
self.pending_frames.push_back(crate::pes::PesFrame {
track,
pts: pes.pts.map(super::codec::pts_to_ns).unwrap_or(0),
keyframe: false,
data: pes.data,
});
} else if let Some((_, parser)) =
self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid)
{
for frame in parser.parse(&pes) {
self.pending_frames.push_back(
crate::pes::PesFrame::from_codec_frame(track, frame),
);
}
}
}
}
Ok(self.pending_frames.pop_front())
}
Ok(DemuxBatch::Ps(packets)) => {
for ps in packets {
let track = match ps.stream_id {
0xE0..=0xEF => 0,
0xC0..=0xDF => 1,
0xBD => ps
.sub_stream_id
.map(|s| (s & 0x1F) as usize + 1)
.unwrap_or(1),
_ => continue,
};
if track >= self.title.streams.len() {
continue;
}
let pid = self
.pid_to_track
.iter()
.find(|(_, idx)| *idx == track)
.map(|(p, _)| *p)
.unwrap_or(0);
let pes = super::ts::PesPacket {
pid,
pts: ps.pts.map(|p| p as i64),
dts: ps.dts.map(|d| d as i64),
data: ps.data,
};
if let Some((_, parser)) = self.parsers.iter_mut().find(|(p, _)| *p == pid) {
for frame in parser.parse(&pes) {
self.pending_frames
.push_back(crate::pes::PesFrame::from_codec_frame(track, frame));
}
}
}
Ok(self.pending_frames.pop_front())
}
Ok(DemuxBatch::Err(e)) => Err(e),
Err(_) => {
// Channel closed → demux thread finished. EOF.
self.eof = true;
Ok(None)
}
}
}
}
impl crate::pes::Stream for DiscStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
if let Some(frame) = self.pending_frames.pop_front() {
@@ -413,7 +688,21 @@ impl crate::pes::Stream for DiscStream {
return Ok(None);
}
// Pipeline mode: read+decrypt+demux all happen on the demux
// thread. Pull a batch of PesPackets and run codec parse on
// this thread; loop until we have at least one frame OR EOF.
if self.demux_rx.is_some() {
loop {
match self.read_pipeline()? {
Some(f) => return Ok(Some(f)),
None if self.eof => return Ok(None),
None => continue,
}
}
}
loop {
let t0 = std::time::Instant::now();
if !self.fill_extents()? {
self.eof = true;
// Flush demuxer — last PES packet may still be in the assembler
@@ -475,6 +764,8 @@ impl crate::pes::Stream for DiscStream {
}
let bytes = self.buf_valid;
let t1 = std::time::Instant::now();
prof_tick("fill", t1.duration_since(t0).as_nanos(), bytes as u64);
// Plaintext: the wrapped reader (DecryptingSectorSource)
// applied AACS / CSS in-place during fill_extents'
// read_sectors call. The pre-0.18 inline decrypt step
@@ -482,21 +773,41 @@ impl crate::pes::Stream for DiscStream {
if let Some(ref mut demuxer) = self.ts_demuxer {
let packets = demuxer.feed(&self.read_buf[..bytes]);
for pes in &packets {
if let Some((_, track)) =
self.pid_to_track.iter().find(|(pid, _)| *pid == pes.pid)
let t2 = std::time::Instant::now();
prof_tick("feed", t2.duration_since(t1).as_nanos(), 0);
let skip_parse = std::env::var_os("FREEMKV_SKIP_PARSE").is_some();
for pes in packets {
if let Some((_, track)) = self
.pid_to_track
.iter()
.find(|(pid, _)| *pid == pes.pid)
.copied()
{
if let Some((_, parser)) =
if skip_parse {
// Profiling escape hatch — bypass the codec
// parser and pass the raw PES bytes straight
// through as a single PesFrame. Lets us
// attribute consumer-thread time to
// "demux + framing" vs "codec parse".
self.pending_frames.push_back(crate::pes::PesFrame {
track,
pts: pes.pts.map(super::codec::pts_to_ns).unwrap_or(0),
keyframe: false,
data: pes.data,
});
} else if let Some((_, parser)) =
self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid)
{
for frame in parser.parse(pes) {
for frame in parser.parse(&pes) {
self.pending_frames.push_back(
crate::pes::PesFrame::from_codec_frame(*track, frame),
crate::pes::PesFrame::from_codec_frame(track, frame),
);
}
}
}
}
let t3 = std::time::Instant::now();
prof_tick("consume", t3.duration_since(t2).as_nanos(), 0);
} else if let Some(ref mut demuxer) = self.ps_demuxer {
let packets = demuxer.feed(&self.read_buf[..bytes]);
for ps in &packets {
@@ -570,6 +881,13 @@ impl crate::pes::Stream for DiscStream {
}
fn headers_ready(&self) -> bool {
// FREEMKV_SKIP_PARSE bypasses codec parsers entirely for
// bottleneck profiling, so codec_private is never populated.
// Pretend headers are ready immediately in that mode so the
// CLI loop doesn't hang waiting for them.
if std::env::var_os("FREEMKV_SKIP_PARSE").is_some() {
return true;
}
for (idx, s) in self.title.streams.iter().enumerate() {
if let crate::disc::Stream::Video(v) = s {
if !v.secondary && self.codec_private(idx).is_none() {
src/mux/iso.rs
-101
View File
@@ -1,101 +0,0 @@
//! ISO sector reader — file-backed SectorSource for Blu-ray ISO images.
//!
//! An ISO is a flat image of 2048-byte sectors. Sector N starts at byte offset N * 2048.
//! Used by DiscStream::open_iso() and Disc::scan_image().
use crate::error::{Error, Result};
use crate::sector::SectorSource;
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
use std::path::Path;
const SECTOR_SIZE: u64 = 2048;
/// File-backed sector reader for ISO images.
pub struct IsoSectorReader {
file: File,
capacity: u32,
}
impl IsoSectorReader {
pub fn open(path: &str) -> std::io::Result<Self> {
let file = File::open(Path::new(path))?;
let size = file.metadata()?.len();
let sectors = size / SECTOR_SIZE;
if sectors > u32::MAX as u64 {
return Err(crate::error::Error::IsoTooLarge {
path: path.to_string(),
}
.into());
}
let capacity = sectors as u32;
Ok(Self { file, capacity })
}
pub fn capacity_sectors(&self) -> u32 {
self.capacity
}
}
impl SectorSource for IsoSectorReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let bytes = count as usize * SECTOR_SIZE as usize;
self.file
.seek(SeekFrom::Start(lba as u64 * SECTOR_SIZE))
.map_err(|e| Error::IoError { source: e })?;
self.file
.read_exact(&mut buf[..bytes])
.map_err(|e| Error::IoError { source: e })?;
Ok(bytes)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn iso_reader_read_sectors() {
let mut data = vec![0u8; 4 * SECTOR_SIZE as usize];
for i in 0..4u8 {
let offset = i as usize * SECTOR_SIZE as usize;
data[offset] = i + 1;
data[offset + 2047] = i + 100;
}
let dir = std::env::temp_dir().join("freemkv_test_iso_read");
std::fs::write(&dir, &data).unwrap();
let mut reader = IsoSectorReader::open(dir.to_str().unwrap()).unwrap();
assert_eq!(reader.capacity_sectors(), 4);
let mut buf = [0u8; 2048];
reader.read_sectors(0, 1, &mut buf, true).unwrap();
assert_eq!(buf[0], 1);
assert_eq!(buf[2047], 100);
reader.read_sectors(2, 1, &mut buf, true).unwrap();
assert_eq!(buf[0], 3);
assert_eq!(buf[2047], 102);
std::fs::remove_file(&dir).ok();
}
#[test]
fn iso_reader_capacity() {
let data = vec![0u8; 10 * SECTOR_SIZE as usize];
let dir = std::env::temp_dir().join("freemkv_test_iso_cap");
std::fs::write(&dir, &data).unwrap();
let reader = IsoSectorReader::open(dir.to_str().unwrap()).unwrap();
assert_eq!(reader.capacity_sectors(), 10);
std::fs::remove_file(&dir).ok();
}
}
src/mux/m2ts.rs
+26 -232
View File
@@ -1,57 +1,21 @@
//! M2tsStream — BD transport stream with embedded metadata header.
//! M2tsStream — BD transport stream write sink.
//!
//! Write: prepends FMKV metadata header, then muxes PES frames into BD-TS.
//! Read: extracts metadata header (or scans PMT), then demuxes BD-TS into PES frames.
//! Write: prepends FMKV metadata header, then muxes PES frames into
//! BD-TS. The read direction lives on the pipeline highway —
//! `m2ts://` URLs route through
//! [`super::resolve::input`] → `build_m2ts_pipeline` →
//! [`super::pipelined_stream::PipelinedPesStream`], so this type is
//! write-only.
use super::{meta, ts};
use super::meta;
use crate::disc::{DiscTitle, Stream as DiscStream};
use std::io::{self, Read, Write};
use std::io::{self, Write};
type PesSetup = (
Vec<u16>,
Vec<(u16, Box<dyn super::codec::CodecParser>)>,
Vec<(u16, usize)>,
);
/// Size of initial scan buffer for PMT/stream detection.
const SCAN_SIZE: usize = 1024 * 1024;
enum Mode {
Write {
muxer: super::tsmux::TsMuxer<Box<dyn Write + Send>>,
},
Read {
reader: Box<dyn Read + Send>,
},
}
/// Read as many bytes as possible into buf (multiple read calls if needed).
/// Bounded by buf.len() — caller controls max bytes read.
fn read_fill(r: &mut impl Read, buf: &mut [u8]) -> io::Result<usize> {
let mut total = 0;
while total < buf.len() {
match r.read(&mut buf[total..]) {
Ok(0) => break,
Ok(n) => total += n,
Err(e) if e.kind() == io::ErrorKind::Interrupted => continue,
Err(e) => return Err(e),
}
}
Ok(total)
}
/// BD transport stream with embedded metadata.
/// BD transport stream write sink with embedded FMKV metadata
/// header.
pub struct M2tsStream {
disc_title: DiscTitle,
mode: Mode,
// PES support
demuxer: Option<ts::TsDemuxer>,
parsers: Vec<(u16, Box<dyn super::codec::CodecParser>)>,
pending_frames: std::collections::VecDeque<crate::pes::PesFrame>,
pid_to_track: Vec<(u16, usize)>,
pes_eof: bool,
/// Codec private data per stream (from FMKV header).
stored_codec_privates: Vec<Option<Vec<u8>>>,
muxer: super::tsmux::TsMuxer<Box<dyn Write + Send>>,
}
impl M2tsStream {
@@ -81,208 +45,38 @@ impl M2tsStream {
}
Ok(Self {
disc_title: title.clone(),
mode: Mode::Write { muxer },
demuxer: None,
parsers: Vec::new(),
pending_frames: std::collections::VecDeque::new(),
pid_to_track: Vec::new(),
pes_eof: false,
stored_codec_privates: Vec::new(),
})
}
fn setup_pes(streams: &[DiscStream]) -> PesSetup {
let mut pids = Vec::new();
let mut parsers: Vec<(u16, Box<dyn super::codec::CodecParser>)> = Vec::new();
let mut pid_to_track = Vec::new();
for (i, s) in streams.iter().enumerate() {
let (pid, codec) = match s {
DiscStream::Video(v) => (v.pid, v.codec),
DiscStream::Audio(a) => (a.pid, a.codec),
DiscStream::Subtitle(s) => (s.pid, s.codec),
};
pids.push(pid);
pid_to_track.push((pid, i));
parsers.push((pid, super::codec::parser_for_codec(codec, None)));
}
(pids, parsers, pid_to_track)
}
/// Open an M2TS stream for reading. Takes any Read source — file, pipe, socket.
///
/// Tries FMKV metadata header first. Falls back to PMT scan of first 1 MB.
pub fn open(mut reader: impl Read + Send + 'static) -> io::Result<Self> {
// Read first chunk — enough for FMKV header or PMT scan
let mut head = vec![0u8; SCAN_SIZE];
let head_len = read_fill(&mut reader, &mut head)?;
head.truncate(head_len);
// Try FMKV metadata header from the buffered head
let mut cursor = io::Cursor::new(&head);
if let Ok(Some(m)) = meta::read_header(&mut cursor) {
let header_end = cursor.position() as usize;
let title = m.to_title();
let (pids, parsers, pid_to_track) = Self::setup_pes(&title.streams);
// Chain: remaining head bytes + rest of reader
let remaining_head = &head[header_end..];
let chain: Box<dyn Read + Send> =
Box::new(io::Cursor::new(remaining_head.to_vec()).chain(reader));
return Ok(Self {
disc_title: title.clone(),
mode: Mode::Read { reader: chain },
demuxer: if pids.is_empty() {
None
} else {
Some(ts::TsDemuxer::new(&pids))
},
parsers,
pending_frames: std::collections::VecDeque::new(),
pid_to_track,
pes_eof: false,
stored_codec_privates: title.codec_privates,
});
}
// No FMKV header — scan head for PMT
let streams = ts::scan_streams(&head)
.ok_or_else(|| -> io::Error { crate::error::Error::NoStreams.into() })?;
let (pids, parsers, pid_to_track) = Self::setup_pes(&streams);
// Chain: full head (it's all TS data) + rest of reader
let chain: Box<dyn Read + Send> = Box::new(io::Cursor::new(head).chain(reader));
Ok(Self {
disc_title: DiscTitle {
duration_secs: 0.0, // unknown without seeking
streams,
..DiscTitle::empty()
},
mode: Mode::Read { reader: chain },
demuxer: if pids.is_empty() {
None
} else {
Some(ts::TsDemuxer::new(&pids))
},
parsers,
pending_frames: std::collections::VecDeque::new(),
pid_to_track,
pes_eof: false,
stored_codec_privates: Vec::new(),
muxer,
})
}
}
impl crate::pes::Stream for M2tsStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
if let Some(frame) = self.pending_frames.pop_front() {
return Ok(Some(frame));
}
if self.pes_eof {
return Ok(None);
}
loop {
let reader = match &mut self.mode {
Mode::Read { reader } => reader,
_ => return Err(crate::error::Error::StreamWriteOnly.into()),
};
let mut buf = vec![0u8; 192 * 1024];
let n = reader.read(&mut buf)?;
if n == 0 {
self.pes_eof = true;
// Flush demuxer — last PES packet may still be in the assembler
if let Some(ref mut demuxer) = self.demuxer {
for pes in &demuxer.flush() {
if let Some((_, track)) =
self.pid_to_track.iter().find(|(pid, _)| *pid == pes.pid)
{
if let Some((_, parser)) =
self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid)
{
for frame in parser.parse(pes) {
self.pending_frames.push_back(
crate::pes::PesFrame::from_codec_frame(*track, frame),
);
}
}
}
}
}
return Ok(self.pending_frames.pop_front());
}
if let Some(ref mut demuxer) = self.demuxer {
let packets = demuxer.feed(&buf[..n]);
for pes in &packets {
if let Some((_, track)) =
self.pid_to_track.iter().find(|(pid, _)| *pid == pes.pid)
{
if let Some((_, parser)) =
self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid)
{
for frame in parser.parse(pes) {
self.pending_frames.push_back(
crate::pes::PesFrame::from_codec_frame(*track, frame),
);
}
}
}
}
}
if let Some(frame) = self.pending_frames.pop_front() {
return Ok(Some(frame));
}
}
// Write-only sink. The m2ts:// read direction is served by
// `super::resolve::build_m2ts_pipeline` →
// `PipelinedPesStream`; routing through this type for reads
// was removed when the highway became the only ingress.
Err(crate::error::Error::StreamWriteOnly.into())
}
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
match &mut self.mode {
Mode::Write { muxer } => {
muxer.write_frame(frame.track, frame.pts, frame.keyframe, &frame.data)
}
Mode::Read { .. } => Err(crate::error::Error::StreamReadOnly.into()),
}
self.muxer
.write_frame(frame.track, frame.pts, frame.keyframe, &frame.data)
}
fn finish(&mut self) -> io::Result<()> {
match &mut self.mode {
Mode::Write { muxer } => muxer.finish(),
Mode::Read { .. } => Ok(()),
}
self.muxer.finish()
}
fn info(&self) -> &crate::disc::DiscTitle {
&self.disc_title
}
fn codec_private(&self, track: usize) -> Option<Vec<u8>> {
// First check stored codec_privates from FMKV header
if let Some(Some(cp)) = self.stored_codec_privates.get(track) {
return Some(cp.clone());
}
// Fall back to parser-extracted codec_private
let pid = self
.pid_to_track
.iter()
.find(|(_, idx)| *idx == track)
.map(|(pid, _)| *pid)?;
self.parsers
.iter()
.find(|(p, _)| *p == pid)
.and_then(|(_, parser)| parser.codec_private())
}
fn headers_ready(&self) -> bool {
for (idx, s) in self.disc_title.streams.iter().enumerate() {
if let crate::disc::Stream::Video(v) = s {
if !v.secondary && self.codec_private(idx).is_none() {
return false;
}
}
}
true
fn codec_private(&self, _track: usize) -> Option<Vec<u8>> {
// Write side doesn't have parsers; codec_private flows in
// via the title metadata at `create` time and gets baked
// into the FMKV header. Nothing to surface back here.
None
}
}
src/mux/mod.rs
+4 -2
View File
@@ -17,8 +17,9 @@
// Public modules — types here are intentionally part of the consumable API.
pub mod codec;
pub mod demux_thread;
pub mod disc;
pub mod iso;
pub mod pipelined_stream;
pub mod resolve;
// Internal modules — implementation details. Their *types* are re-exported
@@ -54,11 +55,12 @@ pub(crate) mod ts;
pub(crate) mod tsmux;
pub use disc::DiscStream;
pub use iso::IsoSectorReader;
pub use m2ts::M2tsStream;
pub use mkvstream::MkvStream;
pub use network::NetworkStream;
pub use null::NullStream;
pub use pipelined_stream::PipelinedPesStream;
pub use resolve::build_iso_pipeline;
pub use resolve::{InputOptions, StreamUrl, input, output, parse_url};
pub use stdio::StdioStream;
src/mux/pipelined_stream.rs
+225
View File
@@ -0,0 +1,225 @@
//! `PipelinedPesStream` — the read-side of the freemkv mux
//! highway.
//!
//! Given a [`crate::mux::demux_thread::DemuxThread`] (which has the
//! producer + demux workers already spawned), a set of codec
//! parsers, and the title metadata, this struct implements
//! [`crate::pes::Stream`] by running codec parse on the caller's
//! thread and emitting `PesFrame`s one at a time.
//!
//! The pipeline runs three threads in parallel:
//!
//! ```text
//! Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher)
//! Thread B: M2TS demux (DemuxThread)
//! Thread C: codec parse (this struct, on the caller's thread)
//! ```
//!
//! Communication between A→B and B→C is via bounded channels with
//! recycled buffer pools — no allocations or memcpys in the steady-
//! state hot loop.
//!
//! This is the *only* read-side `Stream` impl in tree. Both ISO file
//! mux ([`crate::mux::resolve`]) and BD-TS file mux ([`crate::mux::M2tsStream`])
//! return a `PipelinedPesStream`; the differences are in how the
//! producer thread (A) is configured — sector-aligned reads with
//! AACS decrypt for ISO, raw byte reads for M2TS.
use super::codec::CodecParser;
use super::demux_thread::{DemuxBatch, DemuxThread};
use super::ts::PesPacket;
use crate::disc::DiscTitle;
use crate::pes::{PesFrame, Stream};
use crossbeam_channel::Receiver;
use std::io;
/// Stream impl that consumes pre-demuxed `PesPacket` batches from a
/// [`DemuxThread`] and runs codec parse on the caller's thread.
pub struct PipelinedPesStream {
title: DiscTitle,
parsers: Vec<(u16, Box<dyn CodecParser>)>,
pid_to_track: Vec<(u16, usize)>,
demux_rx: Receiver<DemuxBatch>,
/// Kept alive so dropping this stream joins the demux + producer
/// workers deterministically. Never poked directly after spawn.
#[allow(dead_code)]
demux_thread: DemuxThread,
pending_frames: std::collections::VecDeque<PesFrame>,
eof: bool,
}
impl PipelinedPesStream {
/// Wire up the stream. Caller has already spawned the
/// `DemuxThread` (which in turn owns the producer); we take the
/// receiver end + the join handle bundle so cleanup is bounded
/// on drop.
pub fn new(
demux_thread: DemuxThread,
demux_rx: Receiver<DemuxBatch>,
title: DiscTitle,
parsers: Vec<(u16, Box<dyn CodecParser>)>,
pid_to_track: Vec<(u16, usize)>,
) -> Self {
Self {
title,
parsers,
pid_to_track,
demux_rx,
demux_thread,
pending_frames: std::collections::VecDeque::new(),
eof: false,
}
}
/// Pull one batch of `PesPacket`s from the demux thread, run
/// codec parse on each, enqueue resulting `PesFrame`s on
/// `pending_frames`. Returns Ok(true) on success, Ok(false) on
/// EOF (channel closed cleanly), Err on demuxer error.
fn pump_one_batch(&mut self) -> io::Result<bool> {
match self.demux_rx.recv() {
Ok(DemuxBatch::Ts(packets)) => {
self.consume_ts(packets);
Ok(true)
}
Ok(DemuxBatch::Ps(packets)) => {
self.consume_ps(packets);
Ok(true)
}
Ok(DemuxBatch::Err(e)) => Err(e),
Err(_) => Ok(false),
}
}
fn consume_ts(&mut self, packets: Vec<PesPacket>) {
let skip_parse = std::env::var_os("FREEMKV_SKIP_PARSE").is_some();
for pes in packets {
if let Some((_, track)) = self
.pid_to_track
.iter()
.find(|(pid, _)| *pid == pes.pid)
.copied()
{
if skip_parse {
// Profiling escape hatch — bypass codec parser.
self.pending_frames.push_back(PesFrame {
track,
pts: pes.pts.map(super::codec::pts_to_ns).unwrap_or(0),
keyframe: false,
data: pes.data,
});
} else if let Some((_, parser)) =
self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid)
{
for frame in parser.parse(&pes) {
self.pending_frames
.push_back(PesFrame::from_codec_frame(track, frame));
}
}
}
}
}
fn consume_ps(&mut self, packets: Vec<super::ps::PsPacket>) {
for ps in packets {
let track = match ps.stream_id {
0xE0..=0xEF => 0,
0xC0..=0xDF => 1,
0xBD => ps
.sub_stream_id
.map(|s| (s & 0x1F) as usize + 1)
.unwrap_or(1),
_ => continue,
};
if track >= self.title.streams.len() {
continue;
}
let pid = self
.pid_to_track
.iter()
.find(|(_, idx)| *idx == track)
.map(|(p, _)| *p)
.unwrap_or(0);
let pes = PesPacket {
pid,
pts: ps.pts.map(|p| p as i64),
dts: ps.dts.map(|d| d as i64),
data: ps.data,
};
if let Some((_, parser)) = self.parsers.iter_mut().find(|(p, _)| *p == pid) {
for frame in parser.parse(&pes) {
self.pending_frames
.push_back(PesFrame::from_codec_frame(track, frame));
}
}
}
}
}
impl Stream for PipelinedPesStream {
fn read(&mut self) -> io::Result<Option<PesFrame>> {
if let Some(frame) = self.pending_frames.pop_front() {
return Ok(Some(frame));
}
if self.eof {
return Ok(None);
}
loop {
match self.pump_one_batch()? {
true => {
if let Some(frame) = self.pending_frames.pop_front() {
return Ok(Some(frame));
}
// Batch contained no trackable packets — pull again.
}
false => {
self.eof = true;
return Ok(self.pending_frames.pop_front());
}
}
}
}
fn write(&mut self, _: &PesFrame) -> io::Result<()> {
Err(crate::error::Error::StreamReadOnly.into())
}
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
fn headers_ready(&self) -> bool {
// Match the previous DiscStream semantics: video tracks need
// codec_private before the consumer can write the container
// header. FREEMKV_SKIP_PARSE forces ready (no parser ever
// populates codec_private in that mode).
if std::env::var_os("FREEMKV_SKIP_PARSE").is_some() {
return true;
}
for (idx, s) in self.title.streams.iter().enumerate() {
if let crate::disc::Stream::Video(v) = s {
if !v.secondary && self.codec_private(idx).is_none() {
return false;
}
}
}
true
}
fn codec_private(&self, track: usize) -> Option<Vec<u8>> {
let pid = self
.pid_to_track
.iter()
.find(|(_, idx)| *idx == track)
.map(|(p, _)| *p)?;
self.parsers
.iter()
.find(|(p, _)| *p == pid)
.and_then(|(_, parser)| parser.codec_private())
}
}
src/mux/resolve.rs
+167 -10
View File
@@ -15,11 +15,13 @@
//! Bare paths without a scheme are rejected.
//! For disc→ISO (raw sector copy), use `Disc::copy()` instead.
use super::disc::DiscStream;
use super::network::NetworkStream;
use super::null::NullStream;
use super::pipelined_stream::PipelinedPesStream;
use super::stdio::StdioStream;
use super::{M2tsStream, MkvStream};
use crate::disc::{ContentFormat, DiscTitle};
use crate::sector::SectorSource;
use std::io;
use std::path::{Path, PathBuf};
@@ -185,7 +187,12 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
},
None => crate::disc::ScanOptions::default(),
};
let mut reader = super::iso::IsoSectorReader::open(&path.to_string_lossy())?;
// FileSectorSource is the sole file-backed sector source.
// It carries the platform-tuned SEQUENTIAL fadvise hint
// (so the kernel readahead window widens) and the periodic
// DONTNEED page-cache eviction that bounds memory pressure
// when the mux output is being written to the same disk.
let mut reader = crate::io::file_sector_source::FileSectorSource::open(path)?;
let capacity = reader.capacity_sectors();
let disc = crate::disc::Disc::scan_image(&mut reader, capacity, &scan_opts)
.map_err(|e| -> io::Error { e.into() })?;
@@ -203,21 +210,39 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
let title = disc.titles[idx].clone();
let keys = disc.decrypt_keys();
let format = disc.content_format;
// ISO file: use large batch size (16 MB) — sequential read from fast storage, no bad sectors.
// Physical drives need small batches for adaptive error handling and retry logic.
// ISO file: 16 MiB batch — sequential read from fast
// storage, no bad sectors. Measured optimum on the rip1
// testbed; bumping to 32 MiB regressed (more cache
// pressure, longer per-batch latency starves the consumer
// between iterations). Physical drives keep smaller
// batches for adaptive error handling.
const ISO_MUX_BATCH_SECTORS: u16 = 8192;
let mut stream =
DiscStream::new(Box::new(reader), title, keys, ISO_MUX_BATCH_SECTORS, format);
if opts.raw {
stream.set_raw();
}
// Pass `DecryptKeys::None` to the decrypt decorator when
// --raw is set — the read stack still flows through the
// same producer+demux+parse pipeline, just without the
// AACS / CSS step. Single highway for all ISO reads.
let effective_keys = if opts.raw {
crate::decrypt::DecryptKeys::None
} else {
keys
};
let stream = build_iso_pipeline(
reader,
title,
effective_keys,
ISO_MUX_BATCH_SECTORS,
format,
None,
);
Ok(Box::new(stream))
}
StreamUrl::M2ts { ref path } => {
validate_file_path(path, "m2ts")?;
let file = std::fs::File::open(path)?;
let reader = std::io::BufReader::with_capacity(IO_BUF_SIZE, file);
Ok(Box::new(M2tsStream::open(reader)?))
let stream = build_m2ts_pipeline(reader)?;
Ok(Box::new(stream))
}
StreamUrl::Mkv { ref path } => {
validate_file_path(path, "mkv")?;
@@ -281,3 +306,135 @@ pub fn output(
}
}
}
/// Demuxer-side state derived from a `DiscTitle`: the codec parser
/// table (keyed by PID), the PID-to-track index map, and an initial
/// `TsDemuxer` / `PsDemuxer` (whichever the content format calls
/// for).
type DemuxState = (
Vec<(u16, Box<dyn super::codec::CodecParser>)>,
Vec<(u16, usize)>,
Option<super::ts::TsDemuxer>,
Option<super::ps::PsDemuxer>,
);
/// Build the title's codec parser table + initial `TsDemuxer` /
/// `PsDemuxer`. Used by both the ISO and M2TS pipeline builders.
fn build_demux_state(title: &DiscTitle, format: ContentFormat) -> DemuxState {
let mut pids = Vec::new();
let mut parsers = Vec::new();
let mut pid_to_track = Vec::new();
for (idx, s) in title.streams.iter().enumerate() {
let (pid, codec) = match s {
crate::disc::Stream::Video(v) => (v.pid, v.codec),
crate::disc::Stream::Audio(a) => (a.pid, a.codec),
crate::disc::Stream::Subtitle(s) => (s.pid, s.codec),
};
pids.push(pid);
pid_to_track.push((pid, idx));
parsers.push((pid, super::codec::parser_for_codec(codec, None)));
}
let (ts, ps) = match format {
ContentFormat::MpegPs => (None, Some(super::ps::PsDemuxer::new())),
ContentFormat::BdTs => {
if pids.is_empty() {
(None, None)
} else {
(Some(super::ts::TsDemuxer::new(&pids)), None)
}
}
};
(parsers, pid_to_track, ts, ps)
}
/// Assemble the ISO mux pipeline (read+decrypt → demux → parse) for
/// a `FileSectorSource`-backed reader. Returns the resulting
/// `PipelinedPesStream`.
pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
reader: S,
title: DiscTitle,
keys: crate::decrypt::DecryptKeys,
batch_sectors: u16,
format: ContentFormat,
halt: Option<crate::halt::Halt>,
) -> PipelinedPesStream {
let extents = title.extents.clone();
let decrypting =
crate::sector::DecryptingSectorSource::new(Box::new(reader) as Box<dyn SectorSource>, keys);
let prefetched = crate::sector::PrefetchedSectorSource::new(
decrypting,
extents,
batch_sectors,
halt.clone(),
);
let (rx, recycle_tx, shell) = prefetched.into_channels();
let (parsers, pid_to_track, ts, ps) = build_demux_state(&title, format);
let (demux_thread, demux_rx) =
super::demux_thread::DemuxThread::spawn_zero_copy(rx, recycle_tx, shell, halt, ts, ps);
PipelinedPesStream::new(demux_thread, demux_rx, title, parsers, pid_to_track)
}
/// Assemble the M2TS file mux pipeline (read → demux → parse) for a
/// byte-stream reader. Scans the head for FMKV header or PMT/PAT,
/// rebuilds the title metadata, then wraps a chained reader (head +
/// remainder) in a `BytePrefetcher` feeding the demux + parse
/// threads.
fn build_m2ts_pipeline<R: std::io::Read + Send + 'static>(
mut reader: R,
) -> io::Result<PipelinedPesStream> {
use super::meta;
use std::io::Read;
const M2TS_SCAN_BYTES: usize = 1024 * 1024;
let mut head = vec![0u8; M2TS_SCAN_BYTES];
let head_len = {
let mut filled = 0;
while filled < head.len() {
match reader.read(&mut head[filled..])? {
0 => break,
n => filled += n,
}
}
filled
};
head.truncate(head_len);
// Try FMKV metadata header first; fall back to PMT scan.
let mut cursor = io::Cursor::new(&head);
let (title, head_consumed) = if let Ok(Some(m)) = meta::read_header(&mut cursor) {
(m.to_title(), cursor.position() as usize)
} else {
let streams = super::ts::scan_streams(&head)
.ok_or_else(|| -> io::Error { crate::error::Error::NoStreams.into() })?;
let t = DiscTitle {
duration_secs: 0.0,
streams,
..DiscTitle::empty()
};
(t, 0)
};
// Chain: any un-consumed head bytes + the remainder of the
// reader. The demuxer sees a contiguous M2TS byte stream.
let remaining_head = head[head_consumed..].to_vec();
let chained: Box<dyn Read + Send> = Box::new(io::Cursor::new(remaining_head).chain(reader));
let prefetcher = crate::io::byte_prefetcher::BytePrefetcher::new(
chained,
crate::io::byte_prefetcher::DEFAULT_CHUNK_BYTES,
None,
);
let (rx, recycle_tx, shell) = prefetcher.into_channels();
let (parsers, pid_to_track, ts, ps) = build_demux_state(&title, ContentFormat::BdTs);
let (demux_thread, demux_rx) =
super::demux_thread::DemuxThread::spawn_zero_copy(rx, recycle_tx, shell, None, ts, ps);
Ok(PipelinedPesStream::new(
demux_thread,
demux_rx,
title,
parsers,
pid_to_track,
))
}
src/mux/ts.rs
+108 -83
View File
@@ -37,11 +37,20 @@ struct PesAssembler {
active: bool,
}
/// Initial capacity for a fresh PES buffer. Sized to cover the
/// common BD-TS audio / subtitle PES outright (a few KB to ~16 KB).
/// Video PES (typically 150–300 KB on UHD) will grow this via the
/// standard Vec doubling, but the doublings hit the allocator's
/// slab caches instead of the 64-page first-touch faults that the
/// previous `Vec::with_capacity(256 * 1024)` triggered on every PES
/// boundary.
const PES_BUFFER_INIT_CAP: usize = 16 * 1024;
impl PesAssembler {
fn new(pid: u16) -> Self {
Self {
pid,
buffer: Vec::with_capacity(256 * 1024),
buffer: Vec::with_capacity(PES_BUFFER_INIT_CAP),
pts: None,
dts: None,
active: false,
@@ -55,7 +64,7 @@ impl PesAssembler {
pid: self.pid,
pts: self.pts,
dts: self.dts,
data: std::mem::replace(&mut self.buffer, Vec::with_capacity(256 * 1024)),
data: std::mem::replace(&mut self.buffer, Vec::with_capacity(PES_BUFFER_INIT_CAP)),
})
} else {
self.buffer.clear();
@@ -124,97 +133,54 @@ impl TsDemuxer {
}
}
/// Feed a chunk of BD transport stream data. Handles non-192-byte-aligned input
/// by buffering leftover bytes between calls. Returns completed PES packets.
/// Feed a chunk of BD transport stream data. Handles non-192-byte-
/// aligned input by buffering leftover bytes between calls. Returns
/// completed PES packets.
///
/// 16 MiB ISO batches never divide evenly into 192-byte BD-TS
/// packets, so every call after the first carries a ~64-byte
/// remainder. The pre-0.24 implementation handled this by building
/// a `combined` Vec containing remainder + the entire new input —
/// a 16 MiB+ memcpy on every call. Now we splice exactly one
/// boundary packet from a stack buffer, then process the rest of
/// `data` in place. Zero-copy on the bulk path; one 192-byte copy
/// on the boundary.
pub fn feed(&mut self, data: &[u8]) -> Vec<PesPacket> {
let mut completed = Vec::with_capacity(4);
// Prepend any remainder from previous call
let mut combined: Vec<u8> = Vec::new();
let work: &[u8] = if !self.remainder.is_empty() {
combined.reserve(self.remainder.len() + data.len());
combined.extend_from_slice(&self.remainder);
combined.extend_from_slice(data);
self.remainder.clear();
&combined
} else {
data
};
let mut offset = 0;
while offset + BD_TS_PACKET_SIZE <= work.len() {
let packet = &work[offset..offset + BD_TS_PACKET_SIZE];
offset += BD_TS_PACKET_SIZE;
// Skip 4-byte TP_extra_header, check sync byte
if packet[4] != SYNC_BYTE {
continue;
}
let ts = &packet[4..]; // 188-byte standard TS packet
// Parse TS header
let pid = (((ts[1] & 0x1F) as u16) << 8) | ts[2] as u16;
let pusi = ts[1] & 0x40 != 0; // Payload Unit Start Indicator
let adaptation = (ts[3] >> 4) & 0x03;
// Check if we're tracking this PID
let idx = if (pid as usize) < self.pid_index.len() {
self.pid_index[pid as usize]
} else {
-1
};
if idx < 0 {
continue;
}
let asm = &mut self.assemblers[idx as usize];
// Find payload start (skip adaptation field if present)
let payload_start = if adaptation == 0x03 || adaptation == 0x02 {
// Adaptation field present
let af_len = ts[4] as usize;
if af_len > 183 {
continue; // Malformed: AF length exceeds TS payload
}
5 + af_len
} else {
4
};
if payload_start >= TS_PACKET_SIZE {
continue;
}
// No payload
if adaptation == 0x02 {
continue;
}
let payload = &ts[payload_start..];
if pusi {
// New PES packet starts here — parse PES header
let (pts, dts, pes_data_start) = parse_pes_header(payload);
if let Some(prev) = asm.start(pts, dts) {
completed.push(prev);
}
if pes_data_start < payload.len() {
asm.push(&payload[pes_data_start..]);
}
} else {
// Continuation of current PES packet
asm.push(payload);
// Boundary packet: if a partial packet was left from the last
// call, complete it from the head of `data` without touching
// the rest of `data`.
if !self.remainder.is_empty() {
let need = BD_TS_PACKET_SIZE - self.remainder.len();
if data.len() < need {
// Still not a full packet — accumulate and wait.
self.remainder.extend_from_slice(data);
return completed;
}
let mut boundary = [0u8; BD_TS_PACKET_SIZE];
boundary[..self.remainder.len()].copy_from_slice(&self.remainder);
boundary[self.remainder.len()..].copy_from_slice(&data[..need]);
self.remainder.clear();
self.process_packet(&boundary, &mut completed);
offset = need;
}
// Save leftover bytes for next call (cap at one packet to prevent unbounded growth)
if offset < work.len() {
let leftover = &work[offset..];
// Aligned-packets fast path — reads directly out of `data`.
while offset + BD_TS_PACKET_SIZE <= data.len() {
let packet = &data[offset..offset + BD_TS_PACKET_SIZE];
offset += BD_TS_PACKET_SIZE;
self.process_packet(packet, &mut completed);
}
// Save leftover bytes for next call (cap at one packet to
// prevent unbounded growth on a desynchronised stream).
if offset < data.len() {
let leftover = &data[offset..];
if leftover.len() < BD_TS_PACKET_SIZE {
self.remainder.extend_from_slice(leftover);
} else {
// More than one full packet leftover — something is wrong, discard
self.remainder.clear();
}
}
@@ -222,6 +188,65 @@ impl TsDemuxer {
completed
}
/// Demux a single 192-byte BD-TS packet (4-byte TP_extra_header +
/// 188-byte TS). Routes payload bytes into the per-PID
/// `PesAssembler`; completed PES packets are pushed onto
/// `completed` so the caller's allocation amortises across the
/// batch.
fn process_packet(&mut self, packet: &[u8], completed: &mut Vec<PesPacket>) {
// Sync byte check skips malformed packets.
if packet[4] != SYNC_BYTE {
return;
}
let ts = &packet[4..]; // 188-byte standard TS packet
let pid = (((ts[1] & 0x1F) as u16) << 8) | ts[2] as u16;
let pusi = ts[1] & 0x40 != 0; // Payload Unit Start Indicator
let adaptation = (ts[3] >> 4) & 0x03;
let idx = if (pid as usize) < self.pid_index.len() {
self.pid_index[pid as usize]
} else {
-1
};
if idx < 0 {
return;
}
let asm = &mut self.assemblers[idx as usize];
let payload_start = if adaptation == 0x03 || adaptation == 0x02 {
let af_len = ts[4] as usize;
if af_len > 183 {
return; // Malformed: AF length exceeds TS payload
}
5 + af_len
} else {
4
};
if payload_start >= TS_PACKET_SIZE {
return;
}
// adaptation == 0x02 → AF only, no payload.
if adaptation == 0x02 {
return;
}
let payload = &ts[payload_start..];
if pusi {
let (pts, dts, pes_data_start) = parse_pes_header(payload);
if let Some(prev) = asm.start(pts, dts) {
completed.push(prev);
}
if pes_data_start < payload.len() {
asm.push(&payload[pes_data_start..]);
}
} else {
asm.push(payload);
}
}
/// Flush all assemblers, returning any remaining PES packets.
pub fn flush(&mut self) -> Vec<PesPacket> {
let mut completed = Vec::new();