mux: pipelined PES highway — read+decrypt → demux → parse on 3 threads
Introduces the freemkv mux throughput highway: a three-stage thread pipeline that replaces the inline single-thread read path for any file-backed source (ISO and m2ts file URLs both route through it). Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher) Thread B: M2TS demux (DemuxThread) Thread C: codec parse (PipelinedPesStream, on caller thread) Each handoff uses a bounded crossbeam channel with a recycled buffer pool — no allocations or memcpys in the steady-state hot loop. Component map: * io/byte_prefetcher.rs (new) — std::io::Read producer thread with recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector side) so demux_thread::spawn_zero_copy can wire either upstream. * sector/prefetched.rs — recycled buffer pool added; into_channels() peels off the rx/recycle_tx/shell triple for zero-copy demux. * mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs feed() on its thread, ships Vec<PesPacket> batches. * mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls packets from the demux thread and runs codec parse on the caller. * mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline (private) assemble the three stages; iso:// and m2ts:// both return PipelinedPesStream. * mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted; the read direction lives on the highway now). * mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find. * mux/codec/hevc.rs — tightened frame_data initial capacity. * mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page first-touch fault tax that the previous 256 KiB pre-alloc paid on every PES boundary. * mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as the legacy autorip ingress (drive + multipass paths still need on_event / skip_errors before they migrate to the highway). * io/file_sector_source/* — per-OS prefetch() syscall hook (Linux readahead, macOS F_RDADVISE, Windows/other no-op). * decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS; pool sized to all cores by default. Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://): 60 → 322 MB/s warm cache (old new_pipeline path) 60 → 660 MB/s warm cache (highway path, this commit) 60 → 126 MB/s sustained disk-bound The IsoSectorReader baseline reader was deleted in favour of FileSectorSource so the freemkv CLI and autorip exercise the same read path.
This commit is contained in:
@@ -6,6 +6,7 @@
|
||||
|
||||
use std::ffi::CString;
|
||||
use std::os::unix::ffi::OsStrExt;
|
||||
use std::os::unix::io::RawFd;
|
||||
use std::path::Path;
|
||||
|
||||
use super::FsType;
|
||||
@@ -18,6 +19,27 @@ const XFS_SUPER_MAGIC: i64 = 0x5846_5342;
|
||||
const BTRFS_SUPER_MAGIC: i64 = 0x9123_683E;
|
||||
const TMPFS_MAGIC: i64 = 0x0102_1994;
|
||||
|
||||
/// Classify an `f_type` magic from `statfs`/`fstatfs`. Single source
|
||||
/// of truth for the magic comparisons used by both the path-based
|
||||
/// (`detect_impl`) and fd-based (`detect_fd_impl`) entry points.
|
||||
///
|
||||
/// Cast to `i64` because `statfs::f_type` is signed `__fsword_t` on
|
||||
/// glibc and unsigned `c_ulong` on musl; a portable comparison needs a
|
||||
/// common type. On glibc x86_64 both already are i64 — clippy flags
|
||||
/// the cast as unnecessary on that target only, but we need it for
|
||||
/// musl.
|
||||
#[allow(clippy::unnecessary_cast)]
|
||||
fn classify_f_type(f_type: i64) -> FsType {
|
||||
let nfs_magic = libc::NFS_SUPER_MAGIC as i64;
|
||||
if f_type == nfs_magic {
|
||||
return FsType::Nfs;
|
||||
}
|
||||
match f_type {
|
||||
EXT2_SUPER_MAGIC | XFS_SUPER_MAGIC | BTRFS_SUPER_MAGIC | TMPFS_MAGIC => FsType::Local,
|
||||
_ => FsType::Unknown,
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn detect_impl(path: &Path) -> FsType {
|
||||
let cpath = match CString::new(path.as_os_str().as_bytes()) {
|
||||
Ok(c) => c,
|
||||
@@ -30,20 +52,19 @@ pub(super) fn detect_impl(path: &Path) -> FsType {
|
||||
if rc != 0 {
|
||||
return FsType::Unknown;
|
||||
}
|
||||
// `f_type` is signed (`__fsword_t`) on glibc and unsigned
|
||||
// (`c_ulong`) on musl. Cast both sides to i64 for a portable
|
||||
// comparison. On glibc x86_64 both already are i64 — clippy flags
|
||||
// the cast as unnecessary on that target only, but we need it for
|
||||
// musl, so silence the lint.
|
||||
#[allow(clippy::unnecessary_cast)]
|
||||
let f_type = buf.f_type as i64;
|
||||
classify_f_type(buf.f_type as i64)
|
||||
}
|
||||
|
||||
/// fd-based variant of [`detect_impl`]. Same return semantics. Used
|
||||
/// by the writeback pipeline, which knows the open `File` but not its
|
||||
/// original path.
|
||||
pub(super) fn detect_fd_impl(fd: RawFd) -> FsType {
|
||||
let mut buf: libc::statfs = unsafe { std::mem::zeroed() };
|
||||
let rc = unsafe { libc::fstatfs(fd, &mut buf) };
|
||||
if rc != 0 {
|
||||
return FsType::Unknown;
|
||||
}
|
||||
#[allow(clippy::unnecessary_cast)]
|
||||
let nfs_magic = libc::NFS_SUPER_MAGIC as i64;
|
||||
if f_type == nfs_magic {
|
||||
return FsType::Nfs;
|
||||
}
|
||||
match f_type {
|
||||
EXT2_SUPER_MAGIC | XFS_SUPER_MAGIC | BTRFS_SUPER_MAGIC | TMPFS_MAGIC => FsType::Local,
|
||||
_ => FsType::Unknown,
|
||||
}
|
||||
classify_f_type(buf.f_type as i64)
|
||||
}
|
||||
|
||||
@@ -51,6 +51,9 @@ use other::detect_impl;
|
||||
#[cfg(target_os = "windows")]
|
||||
use windows::detect_impl;
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
use linux::detect_fd_impl;
|
||||
|
||||
/// Best-effort classification of the filesystem under `path`.
|
||||
///
|
||||
/// Falls back to [`FsType::Unknown`] on any syscall error or unrecognised
|
||||
@@ -60,6 +63,25 @@ pub fn detect(path: &Path) -> FsType {
|
||||
detect_impl(path)
|
||||
}
|
||||
|
||||
/// fd-based classification. Same return semantics as [`detect`], but
|
||||
/// takes a `RawFd` so callers that only have an open file (notably
|
||||
/// [`crate::io::writeback::WritebackPipeline`]) don't have to
|
||||
/// round-trip through the path.
|
||||
///
|
||||
/// Only implemented on Linux; other platforms return
|
||||
/// [`FsType::Unknown`] (none of them have a writeback policy that
|
||||
/// keys off this classification today).
|
||||
#[cfg(target_os = "linux")]
|
||||
pub fn detect_fd(fd: std::os::unix::io::RawFd) -> FsType {
|
||||
detect_fd_impl(fd)
|
||||
}
|
||||
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
#[allow(dead_code)] // API parity with the linux impl; callers cfg-gate.
|
||||
pub fn detect_fd(_fd: i32) -> FsType {
|
||||
FsType::Unknown
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
Reference in New Issue
Block a user