v0.17.12: mapfile time-batched persistence — unblock NFS staging
Pre-0.17.12 every Mapfile::record() persisted the full mapfile via tempfile-create + write + atomic-rename. On local LVM that's microseconds; on NFS each rename is multiple RPCs through the unraid user-share fuse layer, dragging a Black Mass UHD rip from ~11 MB/s on local to ~1.5 MB/s on NFS — the mapfile path alone burned multiple seconds of wall time per real-world second of work. Mapfile now batches the rename to once per second: - record() always updates in-memory state and stats; only fires write_to_disk when last_flushed.elapsed() >= FLUSH_INTERVAL (1 s). - New flush() API forces a persist; called by sweep_pipeline's consumer at end-of-sweep and by Disc::patch at end-of-patch, after the file's sync_all. - Drop impl best-effort flushes so an early-return / unwind doesn't silently lose pending state. Crash-safety changes from "lose at most one block" to "lose at most 1 s of recorded progress" — the ISO file's payload bytes are unaffected; only the mapfile's authority over which sectors are already-good is at risk, and a resume re-reads anything Pass 1 had already covered. Measured on the BU40N test bed against Black Mass UHD inner zone: - NFS staging: 1.5 MB/s → 16.48 MB/s (10.9× recovery) - Local LVM staging: 11.09 MB/s → 11.83 MB/s (+6.7 % bonus) Internal round_trip_load test now flushes before reading back from disk. External patch / copy tests are unaffected: patch and sweep_pipeline flush at completion before returning.
This commit is contained in:
@@ -199,7 +199,8 @@ fn consumer_loop(
|
||||
}
|
||||
}
|
||||
|
||||
// Final flush — drain the writeback pipeline + fsync the ISO.
|
||||
// Final flush — drain the writeback pipeline + fsync the ISO,
|
||||
// then persist any pending mapfile state.
|
||||
if first_error.is_none() {
|
||||
if let Err(e) = inputs.file.sync_all() {
|
||||
if inputs.is_regular {
|
||||
@@ -208,6 +209,9 @@ fn consumer_loop(
|
||||
// Non-regular outputs (/dev/null, pipes) always fail
|
||||
// sync_all; that's not a real error.
|
||||
}
|
||||
if let Err(e) = inputs.map.flush() {
|
||||
first_error = Some(Error::IoError { source: e });
|
||||
}
|
||||
}
|
||||
|
||||
ConsumerSummary {
|
||||
|
||||
Reference in New Issue
Block a user