Bound the synthesized image, and stop it pinning the page cache

A DVD title set records where its VOBS begins as an offset inside its own
IFO, and the planner honours that offset because honouring it is what
makes a real backup readable. Nothing bounded it: a regenerated .BUP or a
hand-assembled folder naming an offset far past the content grew the
image to wherever it pointed — a u32 sector count reaches ~8.8 TB, and
writing that to an iso:// destination fills a disk with zeros before
anything notices. Capped at 128 GiB, which clears BD-100 with room.

Metadata is materialized up front and held for the life of the image, at
a 2 KiB File Entry per node, so the 100,000-entry cap alone permitted
~205 MB of it for content of no size at all — and the mux holds two
images at once while probing. The module claimed a budget of a few MiB;
that budget is now enforced rather than asserted.

Host reads had no page-cache eviction. The ISO source documents what that
costs, measured: an 85 GB read pins the whole file, starves the writer,
and collapses the mux to 2.7 MB/s against 70 MB/s isolated. A folder
source reads host files the same way, so it now uses the same eviction —
the hints move from private-to-that-module to crate-internal rather than
being reimplemented.
This commit is contained in:
Matthew Jackson
2026-08-05 17:10:19 -07:00
parent 3980aa8976
commit cbb3517afe
7 changed files with 95 additions and 21 deletions
+39 -1
View File
@@ -36,6 +36,14 @@ mod encode;
mod layout;
use crate::error::{Error, Result};
#[cfg(target_os = "linux")]
use crate::io::file_sector_source::linux::drop_window;
#[cfg(target_os = "macos")]
use crate::io::file_sector_source::macos::drop_window;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use crate::io::file_sector_source::other::drop_window;
#[cfg(target_os = "windows")]
use crate::io::file_sector_source::windows::drop_window;
use crate::sector::SectorSource;
use encode::{MetaSectors, SECTOR};
use std::fs::File;
@@ -78,6 +86,12 @@ struct FileRef {
/// Owns everything it reads through (`PathBuf`s and its own file handles), so
/// it is `Send + 'static` and can be moved into `build_iso_pipeline`, which
/// hands it to `PrefetchedSectorSource`'s producer thread.
/// Bytes read between page-cache eviction calls (32 MiB).
///
/// Large enough that the hint costs nothing measurable against a rip, small
/// enough that resident pages stay bounded well below any machine's RAM.
const DROP_CHUNK_BYTES: u64 = 32 * 1024 * 1024;
pub struct DirImage {
meta: MetaSectors,
/// Sorted by `start_lba`, non-overlapping.
@@ -87,6 +101,15 @@ pub struct DirImage {
total_sectors: u32,
volume_id: String,
data_bytes: u64,
/// Bytes read from host files since the last page-cache eviction.
///
/// A rip streams every byte of the folder exactly once. Without eviction
/// the kernel keeps all of it resident, which starves the concurrent writer
/// — `io::file_sector_source` records the measured cost of exactly this
/// omission on the ISO path (2.7 MB/s mux against 70 MB/s isolated reads).
/// A folder source reads host files the same way and needs the same
/// treatment.
bytes_since_drop: u64,
}
impl std::fmt::Debug for DirImage {
@@ -160,6 +183,7 @@ impl DirImage {
total_sectors: plan.total_sectors,
volume_id: plan.volume_id,
data_bytes,
bytes_since_drop: 0,
})
}
@@ -225,7 +249,21 @@ impl DirImage {
let file = r.file;
let h = self.handle(file)?;
h.seek(SeekFrom::Start(at)).map_err(Error::from)?;
match h.read_exact(&mut out[..want]) {
let res = h.read_exact(&mut out[..want]);
if res.is_ok() {
// Evict what we have consumed, per file handle. The window is the
// read just completed rather than a running offset, because reads
// here jump between files and a single monotonic cursor would name
// the wrong pages.
self.bytes_since_drop = self.bytes_since_drop.saturating_add(want as u64);
if self.bytes_since_drop >= DROP_CHUNK_BYTES {
if let Some((_, fh)) = self.open.iter().find(|(i, _)| *i == file) {
drop_window(fh, at, want as u64);
}
self.bytes_since_drop = 0;
}
}
match res {
Ok(()) => Ok(()),
// The file shrank while the handle was open. Same verdict as the
// size check in `handle`, reached the other way.