mirror of
https://github.com/freemkv/libfreemkv.git
synced 2026-10-07 18:13:10 +00:00
Matthew Jackson
6e14fc0c0c
Security - Remove the unsafe Vec::set_len over recycled sector buffers — UB reachable from safe code (GHSA-j8ww-f5fg-9pmh, low). Uses Vec::resize; pooling unchanged. CI / tooling - clippy on stable with the MSRV single-sourced from Cargo.toml; cargo-deny dependency-audit gate with first-party git sources allowed (#15); audience-based comment-guard (#17). Docs - README (keydb ownership, BD-J parser count), drive-access transport/reset notes, mux timeline module-doc links.
71 lines
3.4 KiB
Markdown
71 lines
3.4 KiB
Markdown
# `io::writeback::linux` — rationale
|
||
|
||
## Pathology this fixes
|
||
|
||
The kernel's default `vm.dirty_ratio` (~20 % of RAM) lets dirty pages
|
||
accumulate to hundreds of MB during a big sequential write, then bursts a
|
||
flush at 99 % disk utilisation. While the burst runs, app writes block on
|
||
the writeback queue — observed empirically as instantaneous speed dropping
|
||
from ~15 MB/s to ~1 MB/s every ~30 s during a Pass 1 sweep.
|
||
|
||
## Strategy
|
||
|
||
Every `chunk_bytes` of new sequential output, kick async writeback
|
||
(`SYNC_FILE_RANGE_WRITE`) on the just-completed chunk and finalise the
|
||
*previous* chunk via `WAIT_AFTER` + `posix_fadvise(DONTNEED)`. By the time
|
||
we finalise, that previous chunk has had a full chunk's worth of work to
|
||
flush — the wait is near-instant. Dirty cache stays bounded at ~2 ×
|
||
`chunk_bytes` and writes drain continuously instead of in bursts.
|
||
|
||
The chunk size is adaptive: we measure the elapsed time of the
|
||
`WAIT_AFTER` call over a rolling window of the last 16 chunks and resize
|
||
the chunk based on the p95. Slow storage (NFS, network shares, HDD) sees
|
||
larger chunks to amortise per-chunk overhead; fast storage (NVMe) sees
|
||
smaller chunks to keep cache pressure tight. Bounds: [4 MiB, 256 MiB].
|
||
|
||
## NFS escape hatch
|
||
|
||
`sync_file_range(WAIT_AFTER)` on an NFS-mounted file can block
|
||
indefinitely waiting for the server's commit ack. If the server never
|
||
acks (network partition, server-side hang, slow commit), the syscall
|
||
never returns and the consumer thread is stuck inside the kernel —
|
||
`/api/stop` can't reach it because halt is cooperative.
|
||
|
||
When `fstatfs` reports the file lives on an NFS mount
|
||
(`f_type == NFS_SUPER_MAGIC`), the pipeline skips the WAIT_AFTER +
|
||
`posix_fadvise(DONTNEED)` dance entirely. NFS clients have their own
|
||
buffering and commit semantics that handle dirty-page bounds without us
|
||
forcing the issue. The async `SYNC_FILE_RANGE_WRITE` kickoff still runs
|
||
(non-blocking by spec) so writeback still gets a nudge.
|
||
|
||
## Defence in depth: WAIT_AFTER timeout
|
||
|
||
Even on local storage, a degraded disk or odd filesystem driver could in
|
||
principle wedge inside WAIT_AFTER. Each WAIT_AFTER call runs on a worker
|
||
thread with a 30s recv_timeout on its result channel. On timeout we log a
|
||
loud error, set a `degraded` flag, and from then on skip WAIT_AFTER +
|
||
DONTNEED for the rest of the pipeline's life (same shape as the NFS
|
||
path). The worker thread is intentionally leaked — it unwinds whenever
|
||
the syscall eventually returns or the process exits. The mux continues;
|
||
the original dirty-burst pathology re-emerges but the rip can still
|
||
finish instead of freezing.
|
||
|
||
## `wait_after_with_timeout` fd lifetime / fd-reuse safety
|
||
|
||
`worker_file` is an *owned* `File` (produced by `File::try_clone` at
|
||
pipeline construction). It is moved into the worker closure so the file
|
||
description stays alive for exactly as long as the worker thread lives —
|
||
even if the original `WritebackFile` is closed and the OS reuses its fd
|
||
number before the worker's syscall returns.
|
||
|
||
`fallback_fd` is used only when `worker_file` is `None` (i.e. the
|
||
`try_clone` at construction failed). In that case the worker captures the
|
||
raw fd integer, which carries the original fd-reuse risk but is no worse
|
||
than the pre-fix behaviour.
|
||
|
||
This delegates to `crate::io::bounded::bounded_syscall`, the generic
|
||
worker-thread + `recv_timeout` primitive, and just adapts it to the
|
||
WAIT_AFTER call shape: it returns `elapsed_ms` instead of the syscall's
|
||
`()`, and treats `WorkerLost` as a benign no-op to match the original
|
||
semantics.
|