mux: pipelined PES highway — read+decrypt → demux → parse on 3 threads

Introduces the freemkv mux throughput highway: a three-stage thread
pipeline that replaces the inline single-thread read path for any
file-backed source (ISO and m2ts file URLs both route through it).

  Thread A: read + decrypt  (PrefetchedSectorSource / BytePrefetcher)
  Thread B: M2TS demux      (DemuxThread)
  Thread C: codec parse     (PipelinedPesStream, on caller thread)

Each handoff uses a bounded crossbeam channel with a recycled buffer
pool — no allocations or memcpys in the steady-state hot loop.

Component map:

* io/byte_prefetcher.rs (new) — std::io::Read producer thread with
  recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector
  side) so demux_thread::spawn_zero_copy can wire either upstream.
* sector/prefetched.rs — recycled buffer pool added; into_channels()
  peels off the rx/recycle_tx/shell triple for zero-copy demux.
* mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs
  feed() on its thread, ships Vec<PesPacket> batches.
* mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls
  packets from the demux thread and runs codec parse on the caller.
* mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline
  (private) assemble the three stages; iso:// and m2ts:// both
  return PipelinedPesStream.
* mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted;
  the read direction lives on the highway now).
* mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find.
* mux/codec/hevc.rs — tightened frame_data initial capacity.
* mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB
  remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page
  first-touch fault tax that the previous 256 KiB pre-alloc paid on
  every PES boundary.
* mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as
  the legacy autorip ingress (drive + multipass paths still need
  on_event / skip_errors before they migrate to the highway).
* io/file_sector_source/* — per-OS prefetch() syscall hook
  (Linux readahead, macOS F_RDADVISE, Windows/other no-op).
* decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS;
  pool sized to all cores by default.

Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://):

  60 → 322 MB/s warm cache (old new_pipeline path)
  60 → 660 MB/s warm cache (highway path, this commit)
  60 → 126 MB/s sustained disk-bound

The IsoSectorReader baseline reader was deleted in favour of
FileSectorSource so the freemkv CLI and autorip exercise the same
read path.
This commit is contained in:
MattJackson
2026-05-19 13:35:32 -07:00
parent 2a31a47434
commit c51b3181f2
32 changed files with 2238 additions and 771 deletions
+6 -4
View File
@@ -176,14 +176,16 @@ impl<'a> Iterator for NalIterator<'a> {
}
/// Find the position of the next start code (00 00 01) at or after `from`.
///
/// Backed by `memchr::memmem::find` for SIMD-accelerated bytestring
/// search. On AVX2-capable x86_64 this runs ~510× the byte-by-byte
/// scan that preceded it; on a 200 KB UHD HEVC frame the saving is
/// in the hundreds of microseconds per call.
pub fn find_start_code(data: &[u8], from: usize) -> Option<usize> {
if data.len() < from + 3 {
return None;
}
// Range excludes last 2 bytes since we read 3 bytes at each position.
// data.len()-2 as exclusive upper bound means last checked index is data.len()-3,
// which accesses data[len-3], data[len-2], data[len-1] — all valid.
(from..data.len() - 2).find(|&i| data[i] == 0x00 && data[i + 1] == 0x00 && data[i + 2] == 0x01)
memchr::memmem::find(&data[from..], b"\x00\x00\x01").map(|rel| from + rel)
}
/// Skip past the start code at position `pos`, returning the first byte after it.
+4 -1
View File
@@ -52,7 +52,10 @@ impl CodecParser for HevcParser {
let pts_ns = pes.dts.or(pes.pts).map(pts_to_ns).unwrap_or(0);
let data = &pes.data;
let mut keyframe = false;
let mut frame_data = Vec::new();
// Pre-size: output is ~input bytes with a few 4-byte length
// prefixes added. UHD frames are 150-300 KB; the unsized Vec
// growth chain otherwise reallocs 5-7× per frame.
let mut frame_data = Vec::with_capacity(data.len() + 64);
// Single-pass NAL scan: extract params, detect keyframes, build length-prefixed output
let mut pos = 0;