The pipeline's adaptive autotuner grows chunk_bytes only when the p95
WAIT_AFTER latency exceeds 200 ms. On NFS, sync_file_range(WAIT_AFTER)
translates to an NFS COMMIT RPC whose ack lands within ~10 ms — so
the autotuner never triggered and the pipeline stayed at the original
32 MiB initial value forever.
That capped sustained mux throughput by paying NFS COMMIT-RPC overhead
roughly once per second of writes. Bidirectional mountstats on rip1
2026-05-15: write side 29 MB/s (RTT 30 ms but exec_time 257 ms — 220 ms
queue/serial waiting) while concurrent dd on the same disk shows
~91 MB/s write + ~65 MB/s read available.
128 MiB initial drops COMMIT cadence 4x while keeping the bounded-
cache invariant intact (worst-case dirty pages ~2 x chunk = 256 MiB,
well under vm.dirty_ratio = 6.6 GB on the 32 GB rig). The adaptive
autotuner can still grow further (up to 256 MiB) or shrink if
WAIT_AFTER ever measures sub-20 ms p95 on faster media.
The WritebackPipeline's WAIT_AFTER + posix_fadvise(DONTNEED) dance
keeps dirty pages bounded at ~2 × chunk_bytes by waiting for each
chunk's writeback to commit before issuing the DONTNEED hint to drop
it from cache. The original 0.18-era design unconditionally skipped
this on NFS on the premise that "NFS clients have their own buffering
and commit semantics that handle dirty-page bounds without us forcing
the issue."
Empirically wrong. On unraid-1 NFS the kernel client buffers dirty
pages up to vm.dirty_ratio (default 20% of RAM = ~6.6 GB on the rip1
host) before the kernel forces writeback and throttles app writes.
Result on 0.21.11 mux measured 2026-05-15: mux throughput cycled
between ~45 MB/s (cache absorbing) and ~7 MB/s (cache draining under
throttle) on a ~100 s period — exactly the burst-flush pathology this
pipeline was built to fix, but disabled on the medium it actually
runs on. /proc/meminfo Dirty: column climbed lockstep with mux
write rate during the slow half of every cycle, confirming the cause.
The original safety concern — `sync_file_range(WAIT_AFTER)` hanging
indefinitely on a wedged NFS server — is already handled by
`wait_after_with_timeout`'s `bounded_syscall` wrapper (30 s deadline).
If a real WAIT_AFTER call exceeds the deadline the pipeline flips to
the `degraded` state and skips WAIT_AFTER + DONTNEED for the rest of
its life — same effect as the old NFS branch, but only triggered when
something is genuinely broken rather than as a blanket exception.
This change is medium-agnostic: every medium goes through the same
path now, every medium gets the same safety net, and the
ADAPTIVE_WINDOW chunk-size autotuner (lines 226-255 — measures p95 of
WAIT_AFTER and resizes between 4 MiB and 256 MiB) finally activates
on NFS where previously it was dead code. Slow medium auto-grows
chunks to amortise per-chunk overhead; fast medium auto-shrinks to
keep cache pressure tight; nothing in the code special-cases the
filesystem type.
`is_nfs` is still detected (for logging + observability) but no
longer keys `skip_wait`. Module doc + startup log line updated to
match.
Pre-coalescing the Phase-2.5 writer thread issued one `file.write_all`
syscall per `Cmd::Write` dequeued. The mux side calls
`WritebackFile::write_all(buf)` per PES frame, typically 30-200 KB.
On NFS that translates to one RPC per syscall, capping per-thread
throughput at `(wsize / rtt) × inflight` — well below what the same
disk delivers under a 1 MiB `dd oflag=direct` workload (empirical
2026-05-15: dd 71 MB/s vs mux ~25 MB/s sustained, with instantaneous
samples bursting 7→108 MB/s as the kernel page cache filled and
drained on its own cadence).
Coalesce instead: dequeue drains consecutive `Cmd::Write` items off
the ring up to a 1 MiB byte budget, returns them as
`DequeuedWork::Writes(Vec<Vec<u8>>)`, and the run loop concatenates
into one contiguous buffer and issues a single `file.write_all`.
Non-write commands (Seek, Flush, SyncAll, Finish) break the run and
are returned one at a time as `DequeuedWork::Other`, preserving their
ordering relative to the writes.
Single-buffer fast path avoids the concat allocation when only one
write is in the queue at dequeue time. A single oversize write (e.g.
the rare matroska cluster larger than 1 MiB) is admitted alone so it
still makes progress — the kernel splits internally.
This is generic across mediums: bigger app writes are at-least-as-
good on local SSD, HDD, or NFS. On fast storage the ring rarely fills
so coalescing is mostly a no-op; on slow storage with significant
per-RPC overhead it materially improves throughput.
Tests in `tests/` (write_then_drop_persists_bytes, sync_all_drains,
seek_then_patch_roundtrip, flush_is_observed_in_order) still pass —
ordering and durability semantics are unchanged.
The depth was conservative because pre-0.21.8 a full sync_file_range
stall on NFS could traverse this channel and pin the producer. With
0.21.8's restored Phase 2.5 writer thread + 128 MiB byte-bounded ring
inside WritebackFile, every blocking syscall happens downstream of
this channel — never on it. The original "smaller buffer reduces
backpressure risk" rationale no longer applies.
Empirical (2026-05-15 Civil War UHD remux on 0.21.8): 30.9 MB/s
sustained but instantaneous samples spanning 9-53 MB/s, stdev 9.3.
Distribution clusters 52% of samples in 25-35 MB/s but has a long
9-15 MB/s tail. The tail corresponds to brief stalls in the matroska
builder when the sink momentarily lags — exactly the case a deeper
inter-thread channel covers. 32 frames at PES-frame sizes is still
well under a megabyte of additional memory, so the cost is zero.
Reverts 523af46. That revert was made on the premise that Phase 2.5
caused a ~60% mux throughput regression on NFS bidirectional workloads.
The premise was wrong: at the time of measurement the producer was
capped at ~8 MB/s by a 50 ms thread::sleep poll in
Pipeline::send_with_halt (fixed in v0.21.7's io/pipeline change), so
the comparison was measuring the polling cap on both sides.
With the polling cap removed, direct passthrough exposes the kernel's
default dirty-page writeback pathology on NFS: writes accumulate, the
kernel periodically bursts a flush, app writes block for the burst.
Observed empirically on Civil War UHD remux 2026-05-14: 5-45 MB/s
spiking around a ~21 MB/s sustained mean, dominated by burst-flush
back-pressure cycles.
Phase 2.5 decouples the mux thread from the file syscall:
* mux writes complete instantly into a 128 MiB byte-bounded SPSC ring,
* a dedicated writer thread executes the real File writes, seeks,
and sync_file_range calls; can sit in a kernel burst without
blocking the mux pipeline,
* backpressure via Condvar notify/wait, no polling primitive,
* the ActiveClusterBuffer fast-path preserves the original MKV
cluster-backpatch optimisation so in-window seeks don't drain
the current writeback chunk.
Halt-safety is preserved: every blocking writeback syscall on the
writer thread still routes through bounded_syscall with a 60 s
deadline. A wedged NFS server cannot trap the writer indefinitely;
the muxer keeps queueing into the ring; the kernel page cache and
the ring together absorb the stall.
The halt-aware send loop polled try_send on a 50 ms sleep slice when
the channel was full. That capped producer throughput at 1 / 50 ms =
20 frames/sec ≈ 1 MB/s at typical PES frame sizes — way below NFS,
let alone local SSD/NVMe. Multi-day diagnostic 2026-05-13/14 surfaced
it as the root cause of the 18 → 3 MB/s mux throughput regression on
0.21.x.
Replaced std::sync::mpsc::sync_channel with crossbeam_channel::bounded
and switched send_with_halt to send_timeout. Producer now BLOCKS on
consumer drain (kernel-wakeup) instead of polling — the timeout slice
(250 ms) only fires when the producer is waiting for stop signal
observation, never on the happy path.
Result: no throughput cap from this primitive at any medium speed.
Mux is bounded by actual storage / network bandwidth, not by our
channel implementation.
Same fix needs to land in autorip's mux.rs producer-side
sync_channel (separate sibling commit).
Documented in (internal)/memory/
feedback_send_with_halt_poll_throttle.md.
Empirical: isolated NFS read 70 MB/s + write 93 MB/s on the rip1 setup
right now, but mux throughput pinned at 2.7 MB/s on 0.21.5. NOT
environmental — code regression.
Root cause: Phase 1 silently dropped the read-side
posix_fadvise(POSIX_FADV_DONTNEED) eviction that the pre-Phase-1 (0.20.7)
hot path had. Without it, an 85 GB streaming ISO read pins the entire
file in the kernel page cache, starving concurrent MKV writeback. 0.21.2
then also dropped the POSIX_FADV_SEQUENTIAL hint on the same theory,
compounding the regression.
Restored both, per-OS split:
- linux: posix_fadvise(SEQUENTIAL) at open + posix_fadvise(DONTNEED)
on consumed 32 MiB windows
- macos: F_RDADVISE hint at open (kept); drop_window no-op (macOS unified
buffer cache less prone to the pin pathology)
- windows / other: both no-op stubs
Target mux speed restored to 20+ MB/s (per concurrent-NFS math:
70/2 read × 0.73 MKV/ISO ratio ≈ 25 MB/s achievable).
The Phase 2.5 writer-thread + bounded ring + ActiveClusterBuffer
architecture introduced a ~60% mux throughput regression on NFS
bidirectional workloads (18 MB/s -> 7-8 MB/s in 0.21.x). Other
candidates (FileSectorSource readahead buffer, per-OS sink split)
ruled out empirically across 0.21.1-0.21.4.
Reverts WritebackFile's hot path to direct File passthrough matching
the 0.20.7 baseline:
* Write/write_all/flush call straight through to self.file.
* Seek calls through to self.file and notifies the pipeline.
* sync_all runs pipeline.finalize() then per-OS durable_sync.
Kept (intentional): the per-OS file split established in earlier work
(writeback_file/{linux,macos,windows,other}.rs). create_with_size_hint
still dispatches to platform::preallocate and sync_all still routes
through platform::durable_sync (bounded_syscall 60s deadline on
Linux/macOS), so halt-safety on a wedged NFS fsync is preserved.
Removed:
* Cmd / RingState / Shared / WriterState / writer_thread_main
* ActiveClusterBuffer (active-cluster window for in-window seeks)
* push_command / dequeue / publish_error / mark_writer_gone
* RING_CAPACITY_BYTES, ACTIVE_CLUSTER_WINDOW_BYTES,
MAX_WRITE_CHUNK_BYTES, WRITER_THREAD_NAME constants
* writer JoinHandle field + Drop join logic
* Tests that asserted internal writer-thread state
(backpressure_blocks_when_ring_full, in/out_of_window_seek_*,
active_cluster_buffer_*, writer_thread_panic_surfaces_on_drop)
Kept tests (correctness-of-output only):
* write_then_drop_persists_bytes
* sync_all_drains_and_flushes (renamed from sync_all_blocks_until_ring_drains)
* seek_then_patch_roundtrip (replaces in_window_seek_then_patch_roundtrip)
* flush_is_observed_in_order
mod.rs: 1026 -> 303 lines. linux/macos/windows/other.rs untouched.
The three tests (multi_sector_read_spanning_buffer_boundary,
backward_seek_rebuffers, partial_buffer_at_eof) were asserting
internal buf_start_lba / buf_len_sectors state. With 0.21.3's
read-path bypass, those fields are no longer mutated. The
byte-level contract assertions (read returns correct bytes for
every scenario the tests cover) remain intact.
The 32 MiB readahead window (0.21.0–0.21.1) regressed mux throughput
on NFS bidirectional workloads vs the pre-Phase-1 0.20.7 baseline
(18 -> 7-8 MB/s). The 0.21.2 4 MiB shrink made it worse (5-6 MB/s).
Both signs point at the application-level buffer itself, not the size.
This commit bypasses the buffer entirely on the read path — every
read_sectors call seeks and pread()s direct to the file. That matches
0.20.7's hot path. Kernel readahead handles the policy; on NFS that
interleaves naturally with concurrent writes on the same TCP
connection.
Buffer state fields and refill/buffer_covers are kept so the
structure is preserved for a future per-source-type policy (e.g. a
local-disk source where batched reads ARE beneficial), and so the
existing tests still exercise that machinery.
Empirical regression observed during 0.21.1 mux test on rip1/unraid-1:
historical 0.20.7 baseline averaged ~18 MB/s mux throughput; 0.21.1
dropped to ~7-8 MB/s flat. Same NFS source + destination, same disc.
Suspect: 32 MiB FileSectorSource readahead + posix_fadvise(SEQUENTIAL)
together saturate the TCP connection on read bursts, starving the
writer thread's concurrent NFS writes (mux reads the source ISO and
writes the MKV over the same connection).
- READAHEAD_BUF_BYTES: 32 MiB -> 4 MiB. Matches NFS rsize=1 MiB * 4
round-trips per refill, interleaves cleanly with writes.
- linux/hint_sequential: now no-op. Kernel's default ~128 KiB
readahead is what we want on NFS-backed ISOs (the dominant case).
Per-OS file stays so we can re-enable a hint cleanly later if a
different path benefits.
CI's lint workflow runs clippy on linux target where:
- platform/fs_type/linux.rs and io/writeback/linux.rs: the i64 cast
on buf.f_type / NFS_SUPER_MAGIC is unnecessary on glibc x86_64 (both
already i64) but required on musl (c_ulong); silence the lint via
inline allow with explanatory comment.
- mux/m2ts_mux/packet.rs: doc comment continuation across lines was
parsed as an unindented list item. Reworded to a single flowing
sentence.
- m2ts_mux: replace u64::is_multiple_of (stable in 1.87+) with %.
CI on the pinned 1.86 toolchain rejected the unstable feature use.
- mux/{fmp4,hevc,m2ts_mux}: rustfmt drift cleanup (test-code wrapping).
SocketSink + UdpSocketSink (`src/io/sink/socket.rs`) — sequential-only
TCP/UDP write destinations. SocketSink wraps BufWriter<TcpStream> with
1 MiB capacity, tunes SO_SNDBUF on construction, calls shutdown(Write)
on finish(). UdpSocketSink emits one datagram per write — caller
packetizes. Both impl Write+Send and thus satisfy SequentialSink via
the Phase 2 blanket; neither impls Seek, so RandomAccessSink is
correctly inaccessible (compile error to mux MKV onto a socket).
New sequential container muxers in src/mux/:
- hevc/ — raw HEVC Annex B elementary stream. Length-prefixed NALU
→ 00 00 00 01 NALU. hvcC parsing emits VPS/SPS/PPS once at stream
head. Fully ships.
- m2ts_mux/ — standard MPEG-TS (188-byte packets). Single program,
HEVC video on PID 0x100, optional AC3/TrueHD audio on PID 0x101.
PAT+PMT re-emitted every 250 packets; PCR stamped on video every
40 packets. Hand-rolled, no new deps. Distinct from the existing
BD-TS (192-byte) `mux::m2ts::M2tsStream` — that path stays as-is.
- fmp4/ — fragmented MP4. STUB: ftyp + minimal moov skeleton with
one HEVC video trak + mvex/trex. Media fragments (moof+mdat) are
TODO for v0.22.0 — write_video accumulates frames into a pending
buffer that finish() clears. Init segment is well-formed enough
that init_segment_starts_with_ftyp_then_moov asserts the box
chain.
17 new unit tests added (socket round-trip, HEVC Annex B conversion,
M2TS packet alignment + PAT/PMT cadence + per-PID CC, fMP4 box chain).
All 514 lib tests + 17 new = pass on Rust 1.86 (fmt + clippy + test
via (internal)/scripts/precommit.sh libfreemkv).
No new dependencies. No version bump. Don't-touch list clean.
WritebackFile now offloads all File I/O to a dedicated writer thread.
Muxer's Write/Seek/sync_all push into a bounded byte ring; the writer
drains the ring and runs the syscalls.
- Commands: Write(Vec<u8>), Seek(SeekFrom), SyncAll(oneshot),
Finish(oneshot). One ring carries all four; ordering preserved.
- ActiveClusterBuffer mirrors the last ACTIVE_CLUSTER_WINDOW_BYTES
written. Seek-back within window → in-memory patch + re-emit
(no forced drain). Seek-back outside window → real drain+seek.
Wins for MKV: cluster size patches almost always land inside the
active window; only the end-of-mux Cues + segment header backpatch
fall outside.
- Ring capacity RING_CAPACITY_BYTES = 128 MiB. Backpressure on full
blocks the muxer (correct semantics for archival workflows).
- All syscalls in writer thread wrapped in bounded_syscall(60s) so
a wedged NFS doesn't pin the thread forever.
- Drop drains via Finish + joins the writer thread; sync_all blocks
until ring drained AND underlying fsync completes.
Introduces the SequentialSink / RandomAccessSink trait pair under
io::sink and an open_for_mkv dispatch helper that picks WritebackFile
on Linux+NFS and LocalFileSink everywhere else. LocalFileSink wraps
BufWriter<File> with a 4 MiB buffer and exposes a per-OS preallocate
path (fallocate on Linux, F_PREALLOCATE on macOS, no-op fallback).
Adds platform::fs_type::detect with a per-OS split (statfs on Linux /
macOS, UNC heuristic on Windows, Unknown elsewhere) so construction-
site dispatch has a single primitive to call.
Blanket impls cover the common shapes: any Write+Send is a
SequentialSink, and any SequentialSink+Seek is a RandomAccessSink.
WritebackFile satisfies the random-access trait via the blanket impl
without needing an explicit per-type impl. No callers wired yet — the
mux::resolve construction sites stay on WritebackFile pending Phase 3.
Tests: 5 new sink/preallocate tests + 3 fs_type tests (1 ignored,
needs a real NFS mount). cargo +1.86 fmt + clippy + tests all green.
Three changes targeting 0.20.9's "muxer never read-stalls on NFS read
latency" invariant:
A. FileSectorSource gets a 32 MiB internal read-ahead buffer
(READAHEAD_BUF_BYTES). Splits out from src/sector/file.rs into
src/io/file_sector_source/ with per-OS open hints (Linux
posix_fadvise(SEQUENTIAL), macOS fcntl(F_RDADVISE) with 64 MiB
cap, Windows TODO stub, BSD/illumos no-op). Backward seeks
rebuffer; partial reads at EOF return only the bytes that exist;
oversize-request bypass for count > BUF_SECTORS.
B. WritebackFile inline #[cfg(target_os = "linux")] blocks split
into per-OS files under src/io/writeback_file/. Linux unchanged
(fallocate KEEP_SIZE, fsync via bounded_syscall). macOS gets a
real F_PREALLOCATE + F_FULLFSYNC impl (was a "skipped (non-linux)"
debug log before). Windows is a stub (FlushFileBuffers via
std sync_all; TODO for SetFileValidData). BSDs/illumos fall back
to std sync_all.
C. New byte_channel module — byte-bounded producer/consumer wrapping
std sync_channel with Mutex/Condvar byte accounting. Sender blocks
when used_bytes + item.byte_size() > capacity. HasByteSize impl
for PesFrame. Default cap BYTE_CHANNEL_DEFAULT_CAPACITY = 64 MiB,
sized to absorb worst-case NFS read p99 (~2 s × UHD peak compressed
~15 MB/s). The mux call site lives in autorip (out of scope here);
this lands the primitive in libfreemkv for autorip to adopt.
Test counts: byte_channel +6, file_sector_source +5, sector::file
round-trip suite (3) preserved. passn_handler_ab.rs A/B fixture
(8 profiles) still green.
precommit.sh libfreemkv: fmt + clippy + test all green on Rust 1.86.
No version bump; no Cargo.lock changes; no forbidden-file edits
(disc/patch.rs, disc/read_error.rs, io/pipeline.rs,
tests/passn_handler_ab.rs).
The 0.20.7 work lives downstream in autorip — process-level safety net
(hard watchdog 5-min mux escalation + std::process::exit, restart-loop
counter + auto-quarantine after 3 attempts, partial-state preservation,
failure_reason surfaced in /api/state).
Bumping libfreemkv to 0.20.7 keeps the 4 crates on the unified release
train per project docs.
Generalizes 0.20.5's hand-written wait_after_with_timeout into a
reusable primitive. After this change, every blocking syscall in the
recovery + mux paths is wrapped, so cooperative Halt has bounded
~250 ms latency reach even into kernel-owned thread states.
New module src/io/bounded.rs:
- BoundedError { Halted, Timeout, WorkerLost }
- bounded_syscall<F, R>(halt: Option<&Halt>, timeout, op) -> Result<R, BoundedError>
- Worker thread runs op; main thread recv_timeouts on a rendezvous
channel in 250 ms slices, polling halt between slices.
- Worker is intentionally leaked on timeout/halt — kernel reaps when
the syscall finally returns or at process exit. Calling thread is
NEVER trapped inside a kernel call.
- 6 unit tests cover the happy path + each error variant.
Refactored callsites:
- src/io/writeback/linux.rs::wait_after_with_timeout now delegates
to bounded_syscall. ~30 LOC of duplicated channel/thread plumbing
deleted. Same semantics, cleaner.
- src/io/writeback_file.rs::WritebackFile::sync_all now wraps the
final libc::fsync(fd) with bounded_syscall (60 s deadline). On
timeout: log error at target=mux and return Ok — kernel will flush
on close, best-effort but bounded. Covers FileSectorSink::finish,
PatchSink::close, SweepSink::close, and the mux MKV finalize path
(they all sync through WritebackFile).
What still hangs (deliberately not wrapped — too hot a path):
- File::write itself. Per-frame write on a wedged NFS could still
block; but back-pressure from a stuck consumer means the producer
notices within seconds, not minutes — different failure mode than
the WAIT_AFTER hang 0.20.5/0.20.6 fix.
Targets the recurring mux hang on NFS dest where the consumer thread
sits indefinitely inside libc::sync_file_range(SYNC_FILE_RANGE_WAIT_AFTER)
because the NFS server never returns a commit ack. The whole rip
wedges; halt is cooperative and can't reach inside a kernel syscall.
A. NFS detection at WritebackPipeline construction (fstatfs f_type ==
NFS_SUPER_MAGIC 0x6969). When NFS:
- Skip SYNC_FILE_RANGE_WAIT_AFTER entirely.
- Skip posix_fadvise(DONTNEED) — NFS client handles its own buffering.
- Still issue async SYNC_FILE_RANGE_WRITE (harmless hint).
Cannot hang on a syscall not made. fstatfs failure fails open (assume
local). Logged at info on construction so operators see which strategy
is active. The whole hang vector is removed for NFS deployments.
B. Hard timeout on WAIT_AFTER for non-NFS (defense in depth, since
even a degraded local disk could in principle hang the syscall).
Each WAIT_AFTER runs on a worker thread; main thread waits on a
sync_channel rendezvous with 30s deadline. On timeout: log error,
set per-pipeline 'degraded' Arc<AtomicBool>, downgrade to NFS-style
skip for the rest of the pipeline's life. Worker thread leaks
intentionally — it'll unwind when the syscall eventually returns or
the process exits. Converts indefinite freeze into 'log loud +
downgrade + keep ripping'.
C. Diagnostic logging for the 73%-of-this-movie reproduction:
- WritebackFile::seek logs every non-trivial seek (from, to, signed
delta) at target=mux so we can see if MkvMuxer seeks back before
a stall.
- WritebackPipeline::finalize logs the chunk being finalised before
any WAIT_AFTER call, so a hung chunk is identifiable by offset.
No new dependencies. macOS / Windows noop stubs unchanged. Net
+198 LOC libfreemkv (mostly writeback/linux.rs).
Four targeted changes to maximize mux throughput regardless of storage
backend (local SSD, local HDD, NFS, network share) and surface enough
log data to diagnose 'mux slow' reports without a re-rip:
1. POSIX_FADV_SEQUENTIAL on FileSectorSource::open (Linux only).
Widens the kernel readahead window for sequential ISO reads. One
syscall at open, free on every storage type.
2. POSIX_FADV_DONTNEED on the ISO read side after every 32 MiB chunk.
Mirrors the writeback DONTNEED that already runs on the write
side. Keeps the read-side page cache bounded during multi-GB ISO
reads — eliminates the OOM-pressure / eviction-storm risk on
long mux runs. Linux only; per-drop trace at target="mux".
3. WritebackFile::create_with_size_hint(path, size_bytes) calls
fallocate(FALLOC_FL_KEEP_SIZE) on Linux to pre-reserve extents
for the output. Reported file size stays 0 (writes grow it
naturally) but the on-disk extent allocation is contiguous —
reduces extent fragmentation for big sequential muxes. Wired
into mkv:// and m2ts:// output paths via DiscTitle::size_bytes.
No-op on macOS/Windows; old create() kept with #[allow(dead_code)]
for callers without a size hint.
4. Adaptive WRITEBACK_CHUNK_BYTES in the Linux writeback pipeline.
Tracks sync_file_range(WAIT_AFTER) elapsed_ms in a rolling
16-sample window. p95 > 200 ms → double chunk size (cap 256 MiB).
p95 < 20 ms → halve (floor 4 MiB). One algorithm, both
fast-storage (small chunks, responsive) and slow-storage (big
chunks, fewer commit round-trips) optimized. Per-chunk trace +
per-32-chunk debug snapshot + info-on-resize so an operator can
see where the autoscaler settled.
All four are universal — no storage-type detection, no env vars to
flip, no per-deploy tuning required. Total +201/-6 across four files.
WO-5 (partial): the patch backtrack inner loop ('while bt_pos <
backtrack_end' in disc/patch.rs) issues per-sector reads to fill the
gap created by a damage-window skip. A long backtrack span can run
minutes; without an inline halt poll, the outer halt only takes
effect when control returns to the per-range loop. Adds a halt poll
at the top of each iteration so cancellation propagates inside the
backtrack span.
Per-sector read failures inside the backtrack already drop through
to the main fail path; this only changes the cancellation latency
between an /api/stop call and the producer actually unwinding. Drops
worst-case unwind from 'whole backtrack span × per-sector recovery
timeout' (minutes) to 'one in-flight SCSI command' (seconds).
The broader Arc<AtomicBool> → Halt migration on CopyOptions /
SweepOptions / PatchOptions / Drive::halt and Pipeline::send halt-
awareness is deferred — separate cycle, larger API impact.
MattJackson