v0.11.17: adaptive batch sizer — no per-sector descent

Replace read_with_binary_search + 3×5s light recovery with an adaptive
sizer that shrinks on failure (halve, 3-aligned ≥6) and probes back up
after 100 MiB (51,200 sectors) of clean reads. Descent cost is paid
once per bad region, not once per bad sector.

Emit BatchSizeChanged { new_size, reason } on shrink and probe-up.
Remove BinarySearch event — no longer produced.

Side fix: scsi/macos.rs one-liner for manual_c_str_literals clippy
lint that surfaced on a newer toolchain.
This commit is contained in:
2026-04-23 20:24:40 -07:00
parent 63b68a01a5
commit 4a8913be22
5 changed files with 180 additions and 100 deletions
+16
View File
@@ -1,5 +1,21 @@
# Changelog # Changelog
## 0.11.17 (2026-04-23)
### Adaptive batch sizer in DiscStream — no more per-sector descent
Rip recovery rewritten. The old binary-search-per-bad-sector model paid the full descent (batch → half → quarter → … → single) for every bad sector in a region. On a damaged disc with 600 consecutive bad sectors this took 12+ hours. The new algorithm pays the descent once, remembers the working size, and ramps back up only after a sustained clean streak.
- **`BatchSizeChanged { new_size, reason }` event** — fires on shrink (read failed) and probe-up (clean streak threshold hit). Consumers use this to distinguish a "recovering" rip from a normal one.
- **Removed `BinarySearch` and `SectorRecovered` emissions from DiscStream** — no longer produced by the rip path. `SectorRecovered` still fires from `Drive::read`'s multi-phase recovery (unused by rips today, but kept for scan/other callers).
- **Removed `read_with_binary_search` and the 3×5s light-recovery loop** — no retry loops, no sleeps. One 5s attempt per read. On size-1 failure, skip (zero-fill) or error.
- **Probe-up threshold: 100 MiB (51,200 sectors) of clean reading at current size** before doubling toward preferred. Ramp 1 → preferred on good reading takes ~100 seconds for a typical BD — trivial vs. rip duration, conservative enough that a single lucky sector in a marginal zone can't trigger a premature probe.
- **Bad-region math**: ~600 consecutive bad sectors now complete in ~50 min (600 × 5s) instead of ~12h. The descent is O(log preferred) one time, not per sector.
### macOS
- Fix new clippy lint (`manual_c_str_literals`) in `scsi/macos.rs`.
## 0.11.16 (2026-04-21) ## 0.11.16 (2026-04-21)
### API cleanup — one method per action ### API cleanup — one method per action
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "libfreemkv" name = "libfreemkv"
version = "0.11.16" version = "0.11.17"
edition = "2021" edition = "2021"
rust-version = "1.86" rust-version = "1.86"
license = "AGPL-3.0-only" license = "AGPL-3.0-only"
+19 -3
View File
@@ -80,14 +80,21 @@ pub enum EventKind {
sector_count: u64, sector_count: u64,
}, },
/// Binary search isolated and recovered a marginal sector. /// Sector recovered after a retry (Drive::read multi-phase recovery).
SectorRecovered { sector: u64 }, SectorRecovered { sector: u64 },
/// Sector unreadable, zero-filled (skip mode). /// Sector unreadable, zero-filled (skip mode).
SectorSkipped { sector: u64 }, SectorSkipped { sector: u64 },
/// Binary search activated — batch failed, isolating bad sector. /// Adaptive batch sizer changed the read size.
BinarySearch { sector: u64, batch_size: u16 }, ///
/// Fires on shrink (read failed at larger size) and on probe-up
/// (enough clean reads to try larger again). Consumers use this to
/// display a "recovering" state distinct from "ripping normally".
BatchSizeChanged {
new_size: u16,
reason: BatchSizeReason,
},
/// Operation complete. /// Operation complete.
Complete { Complete {
@@ -98,5 +105,14 @@ pub enum EventKind {
}, },
} }
/// Why the adaptive batch sizer changed size.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BatchSizeReason {
/// Read failed; sizer halved the batch.
Shrunk,
/// Clean-read streak threshold hit; sizer doubled toward preferred.
Probed,
}
/// A no-op event handler. Ignores all events. /// A no-op event handler. Ignores all events.
pub fn ignore(_event: Event) {} pub fn ignore(_event: Event) {}
+135 -88
View File
@@ -6,10 +6,99 @@
//! Read-only. For disc→ISO (raw sector copy), use `Disc::copy()`. //! Read-only. For disc→ISO (raw sector copy), use `Disc::copy()`.
use crate::disc::{Disc, DiscTitle, Extent}; use crate::disc::{Disc, DiscTitle, Extent};
use crate::event::{Event, EventKind}; use crate::event::{BatchSizeReason, Event, EventKind};
use crate::sector::SectorReader; use crate::sector::SectorReader;
use std::io; use std::io;
/// Ramp back up to the preferred batch size after this many sectors
/// of clean reading at the current (reduced) size. 100 MiB = 51,200 sectors.
///
/// Chosen so that an isolated transient failure doesn't lock the rip at
/// size 1: once past the bad zone, we probe up after ~100 ms of good reads.
/// And so that noisy zones with occasional successes can't trigger a
/// premature probe — we need a sustained clean run.
const PROBE_THRESHOLD_SECTORS: u32 = 100 * 1024 * 1024 / 2048;
/// Halve a batch size, keeping 3-sector alignment when >= 6
/// (3-sector alignment = one AACS unit). At sizes < 6 we descend
/// through 3 → 1 without intermediate unaligned sizes.
fn halve_batch_size(size: u16) -> u16 {
let h = (size / 2).max(1);
if h >= 6 {
h - (h % 3)
} else {
h
}
}
/// Double a batch size toward a preferred max, keeping 3-sector alignment
/// when the result is >= 6.
fn double_batch_size(size: u16, preferred: u16) -> u16 {
let d = size.saturating_mul(2).min(preferred);
if d >= 6 {
d - (d % 3)
} else {
d
}
}
/// Adaptive batch sizer. Shrinks on read failure, grows after a sustained
/// clean streak. Amortizes the cost of entering a bad zone — descent happens
/// once, not once per bad sector.
#[derive(Debug)]
struct AdaptiveBatch {
preferred: u16,
current: u16,
streak_sectors: u32,
}
impl AdaptiveBatch {
fn new(preferred: u16) -> Self {
Self {
preferred,
current: preferred,
streak_sectors: 0,
}
}
fn current(&self) -> u16 {
self.current
}
/// Record a successful read of `sectors`. Returns an event if the
/// sizer probed up to a larger batch size.
fn on_success(&mut self, sectors: u16) -> Option<EventKind> {
self.streak_sectors = self.streak_sectors.saturating_add(sectors as u32);
if self.current < self.preferred && self.streak_sectors >= PROBE_THRESHOLD_SECTORS {
let new_size = double_batch_size(self.current, self.preferred);
if new_size != self.current {
self.current = new_size;
self.streak_sectors = 0;
return Some(EventKind::BatchSizeChanged {
new_size,
reason: BatchSizeReason::Probed,
});
}
}
None
}
/// Record a read failure. Returns an event if the sizer shrank.
/// Does nothing at size 1 (caller handles skip/error).
fn on_failure(&mut self) -> Option<EventKind> {
self.streak_sectors = 0;
if self.current <= 1 {
return None;
}
let new_size = halve_batch_size(self.current);
self.current = new_size;
Some(EventKind::BatchSizeChanged {
new_size,
reason: BatchSizeReason::Shrunk,
})
}
}
/// Disc stream. Reads sectors from any source → PES frames. /// Disc stream. Reads sectors from any source → PES frames.
/// ///
/// Sources: physical drive, ISO file, or any SectorReader. /// Sources: physical drive, ISO file, or any SectorReader.
@@ -31,8 +120,9 @@ pub struct DiscStream {
read_buf: Vec<u8>, read_buf: Vec<u8>,
buf_valid: usize, buf_valid: usize,
// Batch size for reads // Adaptive batch sizer — preferred comes from the caller
batch_sectors: u16, // (detect_max_batch_sectors), shrinks/grows based on read outcomes.
adaptive: AdaptiveBatch,
pub errors: u64, pub errors: u64,
pub skip_errors: bool, pub skip_errors: bool,
event_fn: Option<Box<dyn Fn(Event) + Send>>, event_fn: Option<Box<dyn Fn(Event) + Send>>,
@@ -99,7 +189,7 @@ impl DiscStream {
current_offset: 0, current_offset: 0,
read_buf: Vec::with_capacity(batch_sectors as usize * 2048), read_buf: Vec::with_capacity(batch_sectors as usize * 2048),
buf_valid: 0, buf_valid: 0,
batch_sectors, adaptive: AdaptiveBatch::new(batch_sectors),
errors: 0, errors: 0,
skip_errors: false, skip_errors: false,
event_fn: None, event_fn: None,
@@ -133,74 +223,6 @@ impl DiscStream {
self.disc.as_ref() self.disc.as_ref()
} }
/// Binary search to isolate failing sectors within a batch.
/// Good sub-batches read fast. Bad sectors get 3 retries × 5s — max 15s per sector.
/// No full Drive::read() recovery — that only runs on the initial batch attempt.
fn read_with_binary_search(&mut self, lba: u32, count: u16) -> io::Result<()> {
if count <= 1 {
// Single sector — light recovery: 3 attempts, 5s sleep between
let offset = self.buf_valid;
for attempt in 0..3u32 {
if attempt > 0 {
std::thread::sleep(std::time::Duration::from_secs(5));
}
if self
.reader
.read_sectors(lba, 1, &mut self.read_buf[offset..offset + 2048], false)
.is_ok()
{
self.emit(EventKind::SectorRecovered { sector: lba as u64 });
self.buf_valid += 2048;
return Ok(());
}
}
// 3 attempts failed
if self.skip_errors {
self.emit(EventKind::SectorSkipped { sector: lba as u64 });
self.read_buf[offset..offset + 2048].fill(0);
self.buf_valid += 2048;
self.errors += 1;
return Ok(());
} else {
return Err(crate::error::Error::DiscRead { sector: lba as u64 }.into());
}
}
// Try this sub-batch as a whole (fast read, no recovery)
let bytes = count as usize * 2048;
let offset = self.buf_valid;
if self
.reader
.read_sectors(
lba,
count,
&mut self.read_buf[offset..offset + bytes],
false,
)
.is_ok()
{
self.buf_valid += bytes;
return Ok(());
}
// Sub-batch failed — split in half and recurse
self.emit(EventKind::BinarySearch {
sector: lba as u64,
batch_size: count,
});
let half = count / 2;
let half = half - (half % 3).min(half);
let half = half.max(1);
let remainder = count - half;
self.read_with_binary_search(lba, half)?;
if remainder > 0 {
self.read_with_binary_search(lba + half as u32, remainder)?;
}
Ok(())
}
fn fill_extents(&mut self) -> io::Result<bool> { fn fill_extents(&mut self) -> io::Result<bool> {
if self.current_extent >= self.extents.len() { if self.current_extent >= self.extents.len() {
return Ok(false); return Ok(false);
@@ -214,32 +236,57 @@ impl DiscStream {
self.current_offset = 0; self.current_offset = 0;
return self.fill_extents(); return self.fill_extents();
} }
let mut sectors = remaining.min(self.batch_sectors as u32) as u16;
// Align to 3-sector AACS units when possible, but never drop let lba = ext_start + self.current_offset;
// trailing sectors at extent boundaries. decrypt_sectors() safely
// skips partial units (chunks shorter than ALIGNED_UNIT_LEN). // Adaptive sizer: start at current (preferred until a failure), shrink
// on failure, advance on success. One 5s read attempt per try — no
// retry loops, no sleeps. On size-1 failure, skip or error.
loop {
let mut sectors = remaining.min(self.adaptive.current() as u32) as u16;
// Align to 3-sector AACS units when possible. Partial units at
// extent boundaries are safely handled by decrypt_sectors().
if sectors >= 3 { if sectors >= 3 {
sectors -= sectors % 3; sectors -= sectors % 3;
} }
let lba = ext_start + self.current_offset;
let bytes = sectors as usize * 2048; let bytes = sectors as usize * 2048;
self.read_buf.resize(bytes, 0); self.read_buf.resize(bytes, 0);
if self let ok = self
.reader .reader
.read_sectors(lba, sectors, &mut self.read_buf[..bytes], false) .read_sectors(lba, sectors, &mut self.read_buf[..bytes], false)
.is_ok() .is_ok();
{
// Fast path: batch succeeded if ok {
self.buf_valid = bytes; if let Some(ev) = self.adaptive.on_success(sectors) {
} else { self.emit(ev);
// Batch failed fast — binary search to isolate bad sectors.
// Light recovery only (3x5s per sector, no full Drive::read).
self.buf_valid = 0;
self.read_with_binary_search(lba, sectors)?;
} }
self.buf_valid = bytes;
self.current_offset += sectors as u32; self.current_offset += sectors as u32;
break;
}
if sectors == 1 {
// Bottomed out. Skip this sector or bail.
if self.skip_errors {
self.read_buf.resize(2048, 0);
self.read_buf[..2048].fill(0);
self.buf_valid = 2048;
self.errors += 1;
self.emit(EventKind::SectorSkipped { sector: lba as u64 });
self.current_offset += 1;
break;
} else {
return Err(crate::error::Error::DiscRead { sector: lba as u64 }.into());
}
}
// Shrink and retry at the same LBA with a smaller batch.
if let Some(ev) = self.adaptive.on_failure() {
self.emit(ev);
}
}
if self.current_offset >= ext_sectors { if self.current_offset >= ext_sectors {
self.current_extent += 1; self.current_extent += 1;
self.current_offset = 0; self.current_offset = 0;
+3 -2
View File
@@ -455,8 +455,9 @@ fn walk_to_authoring_device(start: IOObject) -> Option<IOObject> {
// Walk up to 10 levels (more than enough) // Walk up to 10 levels (more than enough)
for _ in 0..10 { for _ in 0..10 {
let mut parent: IOObject = 0; let mut parent: IOObject = 0;
let kr = let kr = unsafe {
unsafe { IORegistryEntryGetParentEntry(current, b"IOService\0".as_ptr(), &mut parent) }; IORegistryEntryGetParentEntry(current, c"IOService".as_ptr() as *const u8, &mut parent)
};
if current != start { if current != start {
unsafe { IOObjectRelease(current) }; unsafe { IOObjectRelease(current) };