v0.17.13: thread Writer through patch + mux for big-write consistency

The bounded-cache writeback wrapper (crate::io::Writer) was added in
0.17.10 and wired into Disc::sweep in 0.17.11, but the other two
paths in the crate that write large amounts of data sequentially —
Disc::patch and the MKV/M2TS mux — were still operating on raw
std::fs::File. That meant the dirty-page burst pathology the wrapper
exists to prevent could still bite on slow / network-attached staging
during recovery and mux phases.

This release plugs those gaps:

- Disc::patch (disc/mod.rs:1981) now wraps the reopened ISO in
  Writer before any seek / write. sync_all on Writer cleanly drains
  the in-flight chunk before the existing fsync.
- mux/resolve.rs MKV and M2TS branches wrap the output File in
  Writer underneath BufWriter. UHD MKV mux routinely produces 70+ GB
  of sequential output; the page cache no longer absorbs that as a
  single hot blast on slow targets.

Mapfile, log, settings, history, and stream-pipeline byte buffers
remain unchanged: those are either small one-shot writes (where
the wrapper has zero benefit and adds a stream_position syscall) or
already use bounded persistence (mapfile time-batched in 0.17.12).
The principle: any path that writes substantial sequential data to
a single file uses Writer; trivial writes don't.
This commit is contained in:
MattJackson
2026-05-09 06:32:08 -07:00
parent 3a6c1aa5a3
commit 6ec97af104
4 changed files with 49 additions and 5 deletions
+32
View File
@@ -1,5 +1,37 @@
# Changelog # Changelog
## 0.17.13 (2026-05-09)
### Use `crate::io::Writer` uniformly for all binary file output
0.17.10 introduced the bounded-cache writeback wrapper (`sync_file_range`
+ `posix_fadvise(DONTNEED)` per chunk) and 0.17.11 wired it into
`Disc::sweep`. The other two big-write paths in the crate were still
opening raw `std::fs::File` and would have hit the same dirty-page
burst pathology against slow / network-attached staging.
This release threads `crate::io::Writer` through the remaining sites:
- **`Disc::patch`** (`disc/mod.rs:1981`): the ISO file reopened for
Pass-N recovery now wraps in `Writer` before any seek / write.
Recovery writes are sparse, but the wrapper costs nothing when
there's no chunk crossing — and on heavily-damaged discs it
matters as patch accumulates GB of recovered data.
- **MKV mux output** (`mux/resolve.rs:243`): `BufWriter::with_capacity`
now wraps `Writer::new(file)` instead of a raw `File`. UHD MKVs
routinely exceed 70 GB of sequential writes; pre-0.17.13 those
bursts went straight to the kernel writeback queue.
- **M2TS mux output** (`mux/resolve.rs:251`): same change for
parity with the MKV path — anyone using `m2ts://` URLs gets it
too.
No new public surface. `Writer::sync_all()` is called from `Disc::patch`
and `sweep_pipeline`'s consumer at end of pass; mux exits via Drop on
the wrapping `BufWriter`, which propagates flush down to `Writer`'s
final `note_progress` (kernel finishes the in-flight chunk on file
close — there's no explicit `sync_all` for mux today, same behaviour
as before).
## 0.17.12 (2026-05-09) ## 0.17.12 (2026-05-09)
### Mapfile time-batched persistence — unblock NFS staging ### Mapfile time-batched persistence — unblock NFS staging
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "libfreemkv" name = "libfreemkv"
version = "0.17.12" version = "0.17.13"
edition = "2024" edition = "2024"
rust-version = "1.86" rust-version = "1.86"
license = "AGPL-3.0-only" license = "AGPL-3.0-only"
+5 -1
View File
@@ -1978,10 +1978,14 @@ impl Disc {
let is_regular = std::fs::metadata(path) let is_regular = std::fs::metadata(path)
.map(|m| m.file_type().is_file()) .map(|m| m.file_type().is_file())
.unwrap_or(false); .unwrap_or(false);
let mut file = std::fs::OpenOptions::new() let file = std::fs::OpenOptions::new()
.write(true) .write(true)
.open(path) .open(path)
.map_err(|e| Error::IoError { source: e })?; .map_err(|e| Error::IoError { source: e })?;
// Same bounded-cache writeback wrapper sweep uses, so patch's
// recovery writes (sparse but can be many across a damaged region)
// get the burst-flush protection on slow / NFS-backed staging.
let mut file = crate::io::Writer::new(file).map_err(|e| Error::IoError { source: e })?;
// Log ISO file size at patch start for write monitoring // Log ISO file size at patch start for write monitoring
if let Ok(metadata) = std::fs::metadata(path) { if let Ok(metadata) = std::fs::metadata(path) {
+11 -3
View File
@@ -242,15 +242,23 @@ pub fn output(
match parsed { match parsed {
StreamUrl::Mkv { ref path } => { StreamUrl::Mkv { ref path } => {
validate_file_path(path, "mkv")?; validate_file_path(path, "mkv")?;
// Wrap the raw `File` in `crate::io::Writer` (bounded-cache
// writeback) so a UHD-scale MKV mux to slow / network-attached
// staging doesn't hit the dirty-page burst pathology that
// sweep already side-steps. BufWriter sits on top to coalesce
// mux's many small EBML element writes.
let file = std::fs::File::create(path)?; let file = std::fs::File::create(path)?;
let writer: Box<dyn super::WriteSeek> = let writer: Box<dyn super::WriteSeek> = Box::new(std::io::BufWriter::with_capacity(
Box::new(std::io::BufWriter::with_capacity(IO_BUF_SIZE, file)); IO_BUF_SIZE,
crate::io::Writer::new(file)?,
));
Ok(Box::new(MkvStream::create(writer, title)?)) Ok(Box::new(MkvStream::create(writer, title)?))
} }
StreamUrl::M2ts { ref path } => { StreamUrl::M2ts { ref path } => {
validate_file_path(path, "m2ts")?; validate_file_path(path, "m2ts")?;
let file = std::fs::File::create(path)?; let file = std::fs::File::create(path)?;
let writer = std::io::BufWriter::with_capacity(IO_BUF_SIZE, file); let writer =
std::io::BufWriter::with_capacity(IO_BUF_SIZE, crate::io::Writer::new(file)?);
Ok(Box::new(M2tsStream::create(writer, title)?)) Ok(Box::new(M2tsStream::create(writer, title)?))
} }
StreamUrl::Network { ref addr } => { StreamUrl::Network { ref addr } => {