Introduces the freemkv mux throughput highway: a three-stage thread pipeline that replaces the inline single-thread read path for any file-backed source (ISO and m2ts file URLs both route through it). Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher) Thread B: M2TS demux (DemuxThread) Thread C: codec parse (PipelinedPesStream, on caller thread) Each handoff uses a bounded crossbeam channel with a recycled buffer pool — no allocations or memcpys in the steady-state hot loop. Component map: * io/byte_prefetcher.rs (new) — std::io::Read producer thread with recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector side) so demux_thread::spawn_zero_copy can wire either upstream. * sector/prefetched.rs — recycled buffer pool added; into_channels() peels off the rx/recycle_tx/shell triple for zero-copy demux. * mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs feed() on its thread, ships Vec<PesPacket> batches. * mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls packets from the demux thread and runs codec parse on the caller. * mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline (private) assemble the three stages; iso:// and m2ts:// both return PipelinedPesStream. * mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted; the read direction lives on the highway now). * mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find. * mux/codec/hevc.rs — tightened frame_data initial capacity. * mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page first-touch fault tax that the previous 256 KiB pre-alloc paid on every PES boundary. * mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as the legacy autorip ingress (drive + multipass paths still need on_event / skip_errors before they migrate to the highway). * io/file_sector_source/* — per-OS prefetch() syscall hook (Linux readahead, macOS F_RDADVISE, Windows/other no-op). * decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS; pool sized to all cores by default. Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://): 60 → 322 MB/s warm cache (old new_pipeline path) 60 → 660 MB/s warm cache (highway path, this commit) 60 → 126 MB/s sustained disk-bound The IsoSectorReader baseline reader was deleted in favour of FileSectorSource so the freemkv CLI and autorip exercise the same read path.
181 lines
5.9 KiB
Rust
181 lines
5.9 KiB
Rust
//! One-bit cooperative cancellation flag.
|
|
//!
|
|
//! `Halt` is a clonable token wrapping `Arc<AtomicBool>`. Pass clones into
|
|
//! every long-running loop; the loop polls `is_cancelled()` and bails out
|
|
//! cleanly. Calling `cancel()` from any clone flips the shared flag, and
|
|
//! every other clone observes it on its next poll.
|
|
//!
|
|
//! Why: `Ordering::Relaxed` is sufficient on both load and store because
|
|
//! this flag is purely advisory — no other memory operations piggyback on
|
|
//! it for happens-before ordering. Callers that need to publish data
|
|
//! across threads do so via channels or other synchronization, not via
|
|
//! this bit.
|
|
|
|
use std::sync::Arc;
|
|
use std::sync::atomic::{AtomicBool, Ordering};
|
|
|
|
/// Clonable, infallible cooperative-cancellation token.
|
|
///
|
|
/// Clones share the same underlying flag. `cancel()` is one-way; there is
|
|
/// no `reset()` by design — construct a fresh `Halt` for a fresh
|
|
/// operation.
|
|
///
|
|
/// Construct with [`Halt::new`] (or [`Halt::default`]). The `Default`
|
|
/// impl forwards to `new()` — both produce a fresh, uncancelled token.
|
|
/// The pair exists because clippy's `new_without_default` lint requires
|
|
/// `Default` whenever a public `new()` is present, even when the two
|
|
/// would do exactly the same thing.
|
|
#[derive(Clone, Debug)]
|
|
pub struct Halt(Arc<AtomicBool>);
|
|
|
|
impl Halt {
|
|
/// Construct a fresh, uncancelled token.
|
|
pub fn new() -> Self {
|
|
Self(Arc::new(AtomicBool::new(false)))
|
|
}
|
|
|
|
/// Wrap an existing `Arc<AtomicBool>` as a `Halt`. Useful as a
|
|
/// bridge during the 0.18 deprecation window: callers that already
|
|
/// hold an `Arc<AtomicBool>` (e.g. `Drive::halt_flag()`, the
|
|
/// deprecated `DiscStream::set_halt`) can adopt the new token API
|
|
/// without changing the underlying flag.
|
|
///
|
|
/// Cancelling either side flips the same bit — the wrapping `Halt`
|
|
/// and the original `Arc` are two views over one shared flag.
|
|
pub fn from_arc(flag: Arc<AtomicBool>) -> Self {
|
|
Self(flag)
|
|
}
|
|
|
|
/// Borrow the underlying `Arc<AtomicBool>`. Used at boundaries with
|
|
/// pre-`Halt` APIs that still take an `Arc<AtomicBool>` directly
|
|
/// (`CopyOptions::halt`, the deprecated `DiscStream::set_halt`).
|
|
/// Round 3 deletes those boundaries and this accessor with them.
|
|
pub fn as_arc(&self) -> &Arc<AtomicBool> {
|
|
&self.0
|
|
}
|
|
|
|
/// Flip the shared flag to cancelled. Idempotent.
|
|
pub fn cancel(&self) {
|
|
self.0.store(true, Ordering::Relaxed);
|
|
}
|
|
|
|
/// Read the shared flag.
|
|
pub fn is_cancelled(&self) -> bool {
|
|
self.0.load(Ordering::Relaxed)
|
|
}
|
|
}
|
|
|
|
impl Default for Halt {
|
|
fn default() -> Self {
|
|
Self::new()
|
|
}
|
|
}
|
|
|
|
/// Shared poll interval for halt-aware loops.
|
|
///
|
|
/// `bounded_syscall` checks the cancellation flag and the deadline
|
|
/// every [`POLL_INTERVAL`] while blocked on a worker; the same
|
|
/// cadence governs `Pipeline::send_with_halt`'s `try_send` retry.
|
|
/// 250 ms is the sweet spot between responsiveness (operator presses
|
|
/// Stop, sees it take effect within ~quarter-second) and waste
|
|
/// (atomic load + clock read is cheap but not free at thousands of
|
|
/// hertz).
|
|
///
|
|
/// Centralised here so the half-dozen halt-polling loops across `io`
|
|
/// can't drift apart silently.
|
|
pub const POLL_INTERVAL: std::time::Duration = std::time::Duration::from_millis(250);
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn fresh_is_not_cancelled() {
|
|
let h = Halt::new();
|
|
assert!(!h.is_cancelled());
|
|
}
|
|
|
|
#[test]
|
|
fn cancel_flips_state() {
|
|
let h = Halt::new();
|
|
assert!(!h.is_cancelled());
|
|
h.cancel();
|
|
assert!(h.is_cancelled());
|
|
}
|
|
|
|
#[test]
|
|
fn cancel_is_idempotent() {
|
|
let h = Halt::new();
|
|
h.cancel();
|
|
h.cancel();
|
|
assert!(h.is_cancelled());
|
|
}
|
|
|
|
#[test]
|
|
fn clone_shares_state() {
|
|
let original = Halt::new();
|
|
let cloned = original.clone();
|
|
assert!(!original.is_cancelled());
|
|
assert!(!cloned.is_cancelled());
|
|
|
|
// Cancel via the clone; the original observes it.
|
|
cloned.cancel();
|
|
assert!(original.is_cancelled());
|
|
assert!(cloned.is_cancelled());
|
|
}
|
|
|
|
#[test]
|
|
fn clone_shares_state_reverse_direction() {
|
|
let original = Halt::new();
|
|
let cloned = original.clone();
|
|
|
|
// Cancel via the original; the clone observes it.
|
|
original.cancel();
|
|
assert!(cloned.is_cancelled());
|
|
}
|
|
|
|
#[test]
|
|
fn clone_shares_state_across_threads() {
|
|
let h = Halt::new();
|
|
let h2 = h.clone();
|
|
let handle = std::thread::spawn(move || {
|
|
h2.cancel();
|
|
});
|
|
handle.join().unwrap();
|
|
assert!(h.is_cancelled());
|
|
}
|
|
|
|
#[test]
|
|
fn from_arc_shares_state() {
|
|
// The 0.18 deprecation-window bridge: a Halt built from an
|
|
// existing Arc<AtomicBool> must be a *view* over the same bit,
|
|
// not a fresh copy. Cancelling either side flips both.
|
|
let arc = Arc::new(AtomicBool::new(false));
|
|
let halt = Halt::from_arc(arc.clone());
|
|
assert!(!halt.is_cancelled());
|
|
assert!(!arc.load(Ordering::Relaxed));
|
|
|
|
// Cancel via the wrapping Halt; the original Arc observes it.
|
|
halt.cancel();
|
|
assert!(arc.load(Ordering::Relaxed));
|
|
|
|
// Conversely: flip the Arc directly; the Halt view observes it.
|
|
let arc2 = Arc::new(AtomicBool::new(false));
|
|
let halt2 = Halt::from_arc(arc2.clone());
|
|
arc2.store(true, Ordering::Relaxed);
|
|
assert!(halt2.is_cancelled());
|
|
}
|
|
|
|
#[test]
|
|
fn as_arc_returns_backing_flag() {
|
|
// `as_arc()` must hand back the *same* Arc, not a clone of a
|
|
// different bit. Verified by writing through the borrowed Arc
|
|
// and observing through the Halt.
|
|
let halt = Halt::new();
|
|
let arc = halt.as_arc().clone();
|
|
assert!(!halt.is_cancelled());
|
|
arc.store(true, Ordering::Relaxed);
|
|
assert!(halt.is_cancelled());
|
|
}
|
|
}
|