MattJackson
c51b3181f2
Introduces the freemkv mux throughput highway: a three-stage thread pipeline that replaces the inline single-thread read path for any file-backed source (ISO and m2ts file URLs both route through it). Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher) Thread B: M2TS demux (DemuxThread) Thread C: codec parse (PipelinedPesStream, on caller thread) Each handoff uses a bounded crossbeam channel with a recycled buffer pool — no allocations or memcpys in the steady-state hot loop. Component map: * io/byte_prefetcher.rs (new) — std::io::Read producer thread with recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector side) so demux_thread::spawn_zero_copy can wire either upstream. * sector/prefetched.rs — recycled buffer pool added; into_channels() peels off the rx/recycle_tx/shell triple for zero-copy demux. * mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs feed() on its thread, ships Vec<PesPacket> batches. * mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls packets from the demux thread and runs codec parse on the caller. * mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline (private) assemble the three stages; iso:// and m2ts:// both return PipelinedPesStream. * mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted; the read direction lives on the highway now). * mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find. * mux/codec/hevc.rs — tightened frame_data initial capacity. * mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page first-touch fault tax that the previous 256 KiB pre-alloc paid on every PES boundary. * mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as the legacy autorip ingress (drive + multipass paths still need on_event / skip_errors before they migrate to the highway). * io/file_sector_source/* — per-OS prefetch() syscall hook (Linux readahead, macOS F_RDADVISE, Windows/other no-op). * decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS; pool sized to all cores by default. Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://): 60 → 322 MB/s warm cache (old new_pipeline path) 60 → 660 MB/s warm cache (highway path, this commit) 60 → 126 MB/s sustained disk-bound The IsoSectorReader baseline reader was deleted in favour of FileSectorSource so the freemkv CLI and autorip exercise the same read path.
66 lines
2.7 KiB
Rust
66 lines
2.7 KiB
Rust
//! Linux read-side platform hooks: sequential-access hint at open +
|
|
//! periodic page-cache eviction during streaming reads.
|
|
//!
|
|
//! ## Why both
|
|
//!
|
|
//! `POSIX_FADV_SEQUENTIAL` at open widens the kernel's readahead window
|
|
//! so each pread aggregates into fewer NFS round-trips. `DONTNEED` on
|
|
//! the consumed window (called periodically by the caller) drops the
|
|
//! already-read pages from the page cache so an 85 GB streaming ISO
|
|
//! read doesn't fill memory and starve concurrent writes (the MKV
|
|
//! output during mux). Together they mirror the write-side
|
|
//! WritebackPipeline's policy.
|
|
//!
|
|
//! ## History
|
|
//!
|
|
//! Pre-Phase-1 (0.20.7 baseline) had both. Phase 1's introduction of
|
|
//! `FileSectorSource` silently dropped the read-side DONTNEED, and
|
|
//! 0.21.2's revert of `SEQUENTIAL` (mistakenly attributing a regression
|
|
//! to it) removed the hint. Net effect: 85 GB of ISO reads pinned in
|
|
//! the page cache + no readahead widening → mux throughput collapse
|
|
//! from 18 MB/s historical to 2.7-8 MB/s on 0.21.x. Restored in 0.21.6.
|
|
|
|
use std::fs::File;
|
|
use std::os::unix::io::AsRawFd;
|
|
|
|
pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
|
|
// Best-effort: return value ignored. A fadvise failure has no
|
|
// user-observable consequence.
|
|
unsafe {
|
|
libc::posix_fadvise(file.as_raw_fd(), 0, 0, libc::POSIX_FADV_SEQUENTIAL);
|
|
}
|
|
}
|
|
|
|
/// Drop pages in the half-open byte range `[start, start+len)` from
|
|
/// the page cache. Called periodically by `read_sectors` to bound the
|
|
/// read-side page cache pressure.
|
|
pub(super) fn drop_window(file: &File, start: u64, len: u64) {
|
|
unsafe {
|
|
libc::posix_fadvise(
|
|
file.as_raw_fd(),
|
|
start as i64,
|
|
len as i64,
|
|
libc::POSIX_FADV_DONTNEED,
|
|
);
|
|
}
|
|
}
|
|
|
|
/// Async-prefetch `len` bytes at `offset` into the page cache. The
|
|
/// kernel `readahead(2)` syscall queues the I/O and returns
|
|
/// immediately — it does NOT wait for completion. Called right after
|
|
/// each consumed read so the next batch's I/O overlaps with the
|
|
/// caller's processing of the current batch (decrypt + demux + mux).
|
|
///
|
|
/// Without this hint, with a synchronous demux consumer running at
|
|
/// ~50 MB/s and a single-spindle disk capable of ~150 MB/s, the disk
|
|
/// sits idle ~70% of each iteration because kernel readahead alone
|
|
/// (capped at `/sys/block/<dev>/queue/read_ahead_kb`, default 128 KB)
|
|
/// can only pre-stage a tiny slice of the next batch. An explicit
|
|
/// `readahead()` of the same size as the current batch tells the
|
|
/// kernel to queue the full next-batch read now.
|
|
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
|
|
unsafe {
|
|
libc::readahead(file.as_raw_fd(), offset as i64, len as usize);
|
|
}
|
|
}
|