mirror of
https://github.com/freemkv/libfreemkv.git
synced 2026-10-07 18:13:10 +00:00
Matthew Jackson
6e14fc0c0c
Security - Remove the unsafe Vec::set_len over recycled sector buffers — UB reachable from safe code (GHSA-j8ww-f5fg-9pmh, low). Uses Vec::resize; pooling unchanged. CI / tooling - clippy on stable with the MSRV single-sourced from Cargo.toml; cargo-deny dependency-audit gate with first-party git sources allowed (#15); audience-based comment-guard (#17). Docs - README (keydb ownership, BD-J parser count), drive-access transport/reset notes, mux timeline module-doc links.
37 lines
1.7 KiB
Markdown
37 lines
1.7 KiB
Markdown
# `file_sector_source::linux` — read-side platform hints
|
|
|
|
## Why both `hint_sequential` and `drop_window`
|
|
|
|
`POSIX_FADV_SEQUENTIAL` at open widens the kernel's readahead window so
|
|
each pread aggregates into fewer NFS round-trips. `DONTNEED` on the
|
|
consumed window (called periodically by the caller) drops the
|
|
already-read pages from the page cache so an 85 GB streaming ISO read
|
|
doesn't fill memory and starve concurrent writes (the MKV output
|
|
during mux). Together they mirror the write-side WritebackPipeline's
|
|
policy.
|
|
|
|
## History
|
|
|
|
Pre-Phase-1 (0.20.7 baseline) had both. Phase 1's introduction of
|
|
`FileSectorSource` silently dropped the read-side DONTNEED, and
|
|
0.21.2's revert of `SEQUENTIAL` (mistakenly attributing a regression
|
|
to it) removed the hint. Net effect: 85 GB of ISO reads pinned in the
|
|
page cache + no readahead widening → mux throughput collapse from
|
|
18 MB/s historical to 2.7-8 MB/s on 0.21.x. Restored in 0.21.6.
|
|
|
|
## `prefetch`
|
|
|
|
Async-prefetch `len` bytes at `offset` into the page cache. The kernel
|
|
`readahead(2)` syscall queues the I/O and returns immediately — it
|
|
does NOT wait for completion. Called right after each consumed read
|
|
so the next batch's I/O overlaps with the caller's processing of the
|
|
current batch (decrypt + demux + mux).
|
|
|
|
Without this hint, with a synchronous demux consumer running at
|
|
~50 MB/s and a single-spindle disk capable of ~150 MB/s, the disk sits
|
|
idle ~70% of each iteration because kernel readahead alone (capped at
|
|
`/sys/block/<dev>/queue/read_ahead_kb`, default 128 KB) can only
|
|
pre-stage a tiny slice of the next batch. An explicit `readahead()` of
|
|
the same size as the current batch tells the kernel to queue the full
|
|
next-batch read now.
|