v0.11.17: adaptive batch sizer — no per-sector descent
Replace read_with_binary_search + 3×5s light recovery with an adaptive
sizer that shrinks on failure (halve, 3-aligned ≥6) and probes back up
after 100 MiB (51,200 sectors) of clean reads. Descent cost is paid
once per bad region, not once per bad sector.
Emit BatchSizeChanged { new_size, reason } on shrink and probe-up.
Remove BinarySearch event — no longer produced.
Side fix: scsi/macos.rs one-liner for manual_c_str_literals clippy
lint that surfaced on a newer toolchain.
This commit is contained in:
@@ -1,5 +1,21 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 0.11.17 (2026-04-23)
|
||||||
|
|
||||||
|
### Adaptive batch sizer in DiscStream — no more per-sector descent
|
||||||
|
|
||||||
|
Rip recovery rewritten. The old binary-search-per-bad-sector model paid the full descent (batch → half → quarter → … → single) for every bad sector in a region. On a damaged disc with 600 consecutive bad sectors this took 12+ hours. The new algorithm pays the descent once, remembers the working size, and ramps back up only after a sustained clean streak.
|
||||||
|
|
||||||
|
- **`BatchSizeChanged { new_size, reason }` event** — fires on shrink (read failed) and probe-up (clean streak threshold hit). Consumers use this to distinguish a "recovering" rip from a normal one.
|
||||||
|
- **Removed `BinarySearch` and `SectorRecovered` emissions from DiscStream** — no longer produced by the rip path. `SectorRecovered` still fires from `Drive::read`'s multi-phase recovery (unused by rips today, but kept for scan/other callers).
|
||||||
|
- **Removed `read_with_binary_search` and the 3×5s light-recovery loop** — no retry loops, no sleeps. One 5s attempt per read. On size-1 failure, skip (zero-fill) or error.
|
||||||
|
- **Probe-up threshold: 100 MiB (51,200 sectors) of clean reading at current size** before doubling toward preferred. Ramp 1 → preferred on good reading takes ~100 seconds for a typical BD — trivial vs. rip duration, conservative enough that a single lucky sector in a marginal zone can't trigger a premature probe.
|
||||||
|
- **Bad-region math**: ~600 consecutive bad sectors now complete in ~50 min (600 × 5s) instead of ~12h. The descent is O(log preferred) one time, not per sector.
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
|
||||||
|
- Fix new clippy lint (`manual_c_str_literals`) in `scsi/macos.rs`.
|
||||||
|
|
||||||
## 0.11.16 (2026-04-21)
|
## 0.11.16 (2026-04-21)
|
||||||
|
|
||||||
### API cleanup — one method per action
|
### API cleanup — one method per action
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "libfreemkv"
|
name = "libfreemkv"
|
||||||
version = "0.11.16"
|
version = "0.11.17"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
rust-version = "1.86"
|
rust-version = "1.86"
|
||||||
license = "AGPL-3.0-only"
|
license = "AGPL-3.0-only"
|
||||||
|
|||||||
+19
-3
@@ -80,14 +80,21 @@ pub enum EventKind {
|
|||||||
sector_count: u64,
|
sector_count: u64,
|
||||||
},
|
},
|
||||||
|
|
||||||
/// Binary search isolated and recovered a marginal sector.
|
/// Sector recovered after a retry (Drive::read multi-phase recovery).
|
||||||
SectorRecovered { sector: u64 },
|
SectorRecovered { sector: u64 },
|
||||||
|
|
||||||
/// Sector unreadable, zero-filled (skip mode).
|
/// Sector unreadable, zero-filled (skip mode).
|
||||||
SectorSkipped { sector: u64 },
|
SectorSkipped { sector: u64 },
|
||||||
|
|
||||||
/// Binary search activated — batch failed, isolating bad sector.
|
/// Adaptive batch sizer changed the read size.
|
||||||
BinarySearch { sector: u64, batch_size: u16 },
|
///
|
||||||
|
/// Fires on shrink (read failed at larger size) and on probe-up
|
||||||
|
/// (enough clean reads to try larger again). Consumers use this to
|
||||||
|
/// display a "recovering" state distinct from "ripping normally".
|
||||||
|
BatchSizeChanged {
|
||||||
|
new_size: u16,
|
||||||
|
reason: BatchSizeReason,
|
||||||
|
},
|
||||||
|
|
||||||
/// Operation complete.
|
/// Operation complete.
|
||||||
Complete {
|
Complete {
|
||||||
@@ -98,5 +105,14 @@ pub enum EventKind {
|
|||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Why the adaptive batch sizer changed size.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum BatchSizeReason {
|
||||||
|
/// Read failed; sizer halved the batch.
|
||||||
|
Shrunk,
|
||||||
|
/// Clean-read streak threshold hit; sizer doubled toward preferred.
|
||||||
|
Probed,
|
||||||
|
}
|
||||||
|
|
||||||
/// A no-op event handler. Ignores all events.
|
/// A no-op event handler. Ignores all events.
|
||||||
pub fn ignore(_event: Event) {}
|
pub fn ignore(_event: Event) {}
|
||||||
|
|||||||
+135
-88
@@ -6,10 +6,99 @@
|
|||||||
//! Read-only. For disc→ISO (raw sector copy), use `Disc::copy()`.
|
//! Read-only. For disc→ISO (raw sector copy), use `Disc::copy()`.
|
||||||
|
|
||||||
use crate::disc::{Disc, DiscTitle, Extent};
|
use crate::disc::{Disc, DiscTitle, Extent};
|
||||||
use crate::event::{Event, EventKind};
|
use crate::event::{BatchSizeReason, Event, EventKind};
|
||||||
use crate::sector::SectorReader;
|
use crate::sector::SectorReader;
|
||||||
use std::io;
|
use std::io;
|
||||||
|
|
||||||
|
/// Ramp back up to the preferred batch size after this many sectors
|
||||||
|
/// of clean reading at the current (reduced) size. 100 MiB = 51,200 sectors.
|
||||||
|
///
|
||||||
|
/// Chosen so that an isolated transient failure doesn't lock the rip at
|
||||||
|
/// size 1: once past the bad zone, we probe up after ~100 ms of good reads.
|
||||||
|
/// And so that noisy zones with occasional successes can't trigger a
|
||||||
|
/// premature probe — we need a sustained clean run.
|
||||||
|
const PROBE_THRESHOLD_SECTORS: u32 = 100 * 1024 * 1024 / 2048;
|
||||||
|
|
||||||
|
/// Halve a batch size, keeping 3-sector alignment when >= 6
|
||||||
|
/// (3-sector alignment = one AACS unit). At sizes < 6 we descend
|
||||||
|
/// through 3 → 1 without intermediate unaligned sizes.
|
||||||
|
fn halve_batch_size(size: u16) -> u16 {
|
||||||
|
let h = (size / 2).max(1);
|
||||||
|
if h >= 6 {
|
||||||
|
h - (h % 3)
|
||||||
|
} else {
|
||||||
|
h
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Double a batch size toward a preferred max, keeping 3-sector alignment
|
||||||
|
/// when the result is >= 6.
|
||||||
|
fn double_batch_size(size: u16, preferred: u16) -> u16 {
|
||||||
|
let d = size.saturating_mul(2).min(preferred);
|
||||||
|
if d >= 6 {
|
||||||
|
d - (d % 3)
|
||||||
|
} else {
|
||||||
|
d
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Adaptive batch sizer. Shrinks on read failure, grows after a sustained
|
||||||
|
/// clean streak. Amortizes the cost of entering a bad zone — descent happens
|
||||||
|
/// once, not once per bad sector.
|
||||||
|
#[derive(Debug)]
|
||||||
|
struct AdaptiveBatch {
|
||||||
|
preferred: u16,
|
||||||
|
current: u16,
|
||||||
|
streak_sectors: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl AdaptiveBatch {
|
||||||
|
fn new(preferred: u16) -> Self {
|
||||||
|
Self {
|
||||||
|
preferred,
|
||||||
|
current: preferred,
|
||||||
|
streak_sectors: 0,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn current(&self) -> u16 {
|
||||||
|
self.current
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Record a successful read of `sectors`. Returns an event if the
|
||||||
|
/// sizer probed up to a larger batch size.
|
||||||
|
fn on_success(&mut self, sectors: u16) -> Option<EventKind> {
|
||||||
|
self.streak_sectors = self.streak_sectors.saturating_add(sectors as u32);
|
||||||
|
if self.current < self.preferred && self.streak_sectors >= PROBE_THRESHOLD_SECTORS {
|
||||||
|
let new_size = double_batch_size(self.current, self.preferred);
|
||||||
|
if new_size != self.current {
|
||||||
|
self.current = new_size;
|
||||||
|
self.streak_sectors = 0;
|
||||||
|
return Some(EventKind::BatchSizeChanged {
|
||||||
|
new_size,
|
||||||
|
reason: BatchSizeReason::Probed,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Record a read failure. Returns an event if the sizer shrank.
|
||||||
|
/// Does nothing at size 1 (caller handles skip/error).
|
||||||
|
fn on_failure(&mut self) -> Option<EventKind> {
|
||||||
|
self.streak_sectors = 0;
|
||||||
|
if self.current <= 1 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let new_size = halve_batch_size(self.current);
|
||||||
|
self.current = new_size;
|
||||||
|
Some(EventKind::BatchSizeChanged {
|
||||||
|
new_size,
|
||||||
|
reason: BatchSizeReason::Shrunk,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Disc stream. Reads sectors from any source → PES frames.
|
/// Disc stream. Reads sectors from any source → PES frames.
|
||||||
///
|
///
|
||||||
/// Sources: physical drive, ISO file, or any SectorReader.
|
/// Sources: physical drive, ISO file, or any SectorReader.
|
||||||
@@ -31,8 +120,9 @@ pub struct DiscStream {
|
|||||||
read_buf: Vec<u8>,
|
read_buf: Vec<u8>,
|
||||||
buf_valid: usize,
|
buf_valid: usize,
|
||||||
|
|
||||||
// Batch size for reads
|
// Adaptive batch sizer — preferred comes from the caller
|
||||||
batch_sectors: u16,
|
// (detect_max_batch_sectors), shrinks/grows based on read outcomes.
|
||||||
|
adaptive: AdaptiveBatch,
|
||||||
pub errors: u64,
|
pub errors: u64,
|
||||||
pub skip_errors: bool,
|
pub skip_errors: bool,
|
||||||
event_fn: Option<Box<dyn Fn(Event) + Send>>,
|
event_fn: Option<Box<dyn Fn(Event) + Send>>,
|
||||||
@@ -99,7 +189,7 @@ impl DiscStream {
|
|||||||
current_offset: 0,
|
current_offset: 0,
|
||||||
read_buf: Vec::with_capacity(batch_sectors as usize * 2048),
|
read_buf: Vec::with_capacity(batch_sectors as usize * 2048),
|
||||||
buf_valid: 0,
|
buf_valid: 0,
|
||||||
batch_sectors,
|
adaptive: AdaptiveBatch::new(batch_sectors),
|
||||||
errors: 0,
|
errors: 0,
|
||||||
skip_errors: false,
|
skip_errors: false,
|
||||||
event_fn: None,
|
event_fn: None,
|
||||||
@@ -133,74 +223,6 @@ impl DiscStream {
|
|||||||
self.disc.as_ref()
|
self.disc.as_ref()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Binary search to isolate failing sectors within a batch.
|
|
||||||
/// Good sub-batches read fast. Bad sectors get 3 retries × 5s — max 15s per sector.
|
|
||||||
/// No full Drive::read() recovery — that only runs on the initial batch attempt.
|
|
||||||
fn read_with_binary_search(&mut self, lba: u32, count: u16) -> io::Result<()> {
|
|
||||||
if count <= 1 {
|
|
||||||
// Single sector — light recovery: 3 attempts, 5s sleep between
|
|
||||||
let offset = self.buf_valid;
|
|
||||||
for attempt in 0..3u32 {
|
|
||||||
if attempt > 0 {
|
|
||||||
std::thread::sleep(std::time::Duration::from_secs(5));
|
|
||||||
}
|
|
||||||
if self
|
|
||||||
.reader
|
|
||||||
.read_sectors(lba, 1, &mut self.read_buf[offset..offset + 2048], false)
|
|
||||||
.is_ok()
|
|
||||||
{
|
|
||||||
self.emit(EventKind::SectorRecovered { sector: lba as u64 });
|
|
||||||
self.buf_valid += 2048;
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// 3 attempts failed
|
|
||||||
if self.skip_errors {
|
|
||||||
self.emit(EventKind::SectorSkipped { sector: lba as u64 });
|
|
||||||
self.read_buf[offset..offset + 2048].fill(0);
|
|
||||||
self.buf_valid += 2048;
|
|
||||||
self.errors += 1;
|
|
||||||
return Ok(());
|
|
||||||
} else {
|
|
||||||
return Err(crate::error::Error::DiscRead { sector: lba as u64 }.into());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Try this sub-batch as a whole (fast read, no recovery)
|
|
||||||
let bytes = count as usize * 2048;
|
|
||||||
let offset = self.buf_valid;
|
|
||||||
if self
|
|
||||||
.reader
|
|
||||||
.read_sectors(
|
|
||||||
lba,
|
|
||||||
count,
|
|
||||||
&mut self.read_buf[offset..offset + bytes],
|
|
||||||
false,
|
|
||||||
)
|
|
||||||
.is_ok()
|
|
||||||
{
|
|
||||||
self.buf_valid += bytes;
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sub-batch failed — split in half and recurse
|
|
||||||
self.emit(EventKind::BinarySearch {
|
|
||||||
sector: lba as u64,
|
|
||||||
batch_size: count,
|
|
||||||
});
|
|
||||||
|
|
||||||
let half = count / 2;
|
|
||||||
let half = half - (half % 3).min(half);
|
|
||||||
let half = half.max(1);
|
|
||||||
let remainder = count - half;
|
|
||||||
|
|
||||||
self.read_with_binary_search(lba, half)?;
|
|
||||||
if remainder > 0 {
|
|
||||||
self.read_with_binary_search(lba + half as u32, remainder)?;
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn fill_extents(&mut self) -> io::Result<bool> {
|
fn fill_extents(&mut self) -> io::Result<bool> {
|
||||||
if self.current_extent >= self.extents.len() {
|
if self.current_extent >= self.extents.len() {
|
||||||
return Ok(false);
|
return Ok(false);
|
||||||
@@ -214,32 +236,57 @@ impl DiscStream {
|
|||||||
self.current_offset = 0;
|
self.current_offset = 0;
|
||||||
return self.fill_extents();
|
return self.fill_extents();
|
||||||
}
|
}
|
||||||
let mut sectors = remaining.min(self.batch_sectors as u32) as u16;
|
|
||||||
// Align to 3-sector AACS units when possible, but never drop
|
let lba = ext_start + self.current_offset;
|
||||||
// trailing sectors at extent boundaries. decrypt_sectors() safely
|
|
||||||
// skips partial units (chunks shorter than ALIGNED_UNIT_LEN).
|
// Adaptive sizer: start at current (preferred until a failure), shrink
|
||||||
|
// on failure, advance on success. One 5s read attempt per try — no
|
||||||
|
// retry loops, no sleeps. On size-1 failure, skip or error.
|
||||||
|
loop {
|
||||||
|
let mut sectors = remaining.min(self.adaptive.current() as u32) as u16;
|
||||||
|
// Align to 3-sector AACS units when possible. Partial units at
|
||||||
|
// extent boundaries are safely handled by decrypt_sectors().
|
||||||
if sectors >= 3 {
|
if sectors >= 3 {
|
||||||
sectors -= sectors % 3;
|
sectors -= sectors % 3;
|
||||||
}
|
}
|
||||||
|
|
||||||
let lba = ext_start + self.current_offset;
|
|
||||||
let bytes = sectors as usize * 2048;
|
let bytes = sectors as usize * 2048;
|
||||||
self.read_buf.resize(bytes, 0);
|
self.read_buf.resize(bytes, 0);
|
||||||
|
|
||||||
if self
|
let ok = self
|
||||||
.reader
|
.reader
|
||||||
.read_sectors(lba, sectors, &mut self.read_buf[..bytes], false)
|
.read_sectors(lba, sectors, &mut self.read_buf[..bytes], false)
|
||||||
.is_ok()
|
.is_ok();
|
||||||
{
|
|
||||||
// Fast path: batch succeeded
|
if ok {
|
||||||
self.buf_valid = bytes;
|
if let Some(ev) = self.adaptive.on_success(sectors) {
|
||||||
} else {
|
self.emit(ev);
|
||||||
// Batch failed fast — binary search to isolate bad sectors.
|
|
||||||
// Light recovery only (3x5s per sector, no full Drive::read).
|
|
||||||
self.buf_valid = 0;
|
|
||||||
self.read_with_binary_search(lba, sectors)?;
|
|
||||||
}
|
}
|
||||||
|
self.buf_valid = bytes;
|
||||||
self.current_offset += sectors as u32;
|
self.current_offset += sectors as u32;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
if sectors == 1 {
|
||||||
|
// Bottomed out. Skip this sector or bail.
|
||||||
|
if self.skip_errors {
|
||||||
|
self.read_buf.resize(2048, 0);
|
||||||
|
self.read_buf[..2048].fill(0);
|
||||||
|
self.buf_valid = 2048;
|
||||||
|
self.errors += 1;
|
||||||
|
self.emit(EventKind::SectorSkipped { sector: lba as u64 });
|
||||||
|
self.current_offset += 1;
|
||||||
|
break;
|
||||||
|
} else {
|
||||||
|
return Err(crate::error::Error::DiscRead { sector: lba as u64 }.into());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Shrink and retry at the same LBA with a smaller batch.
|
||||||
|
if let Some(ev) = self.adaptive.on_failure() {
|
||||||
|
self.emit(ev);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if self.current_offset >= ext_sectors {
|
if self.current_offset >= ext_sectors {
|
||||||
self.current_extent += 1;
|
self.current_extent += 1;
|
||||||
self.current_offset = 0;
|
self.current_offset = 0;
|
||||||
|
|||||||
+3
-2
@@ -455,8 +455,9 @@ fn walk_to_authoring_device(start: IOObject) -> Option<IOObject> {
|
|||||||
// Walk up to 10 levels (more than enough)
|
// Walk up to 10 levels (more than enough)
|
||||||
for _ in 0..10 {
|
for _ in 0..10 {
|
||||||
let mut parent: IOObject = 0;
|
let mut parent: IOObject = 0;
|
||||||
let kr =
|
let kr = unsafe {
|
||||||
unsafe { IORegistryEntryGetParentEntry(current, b"IOService\0".as_ptr(), &mut parent) };
|
IORegistryEntryGetParentEntry(current, c"IOService".as_ptr() as *const u8, &mut parent)
|
||||||
|
};
|
||||||
|
|
||||||
if current != start {
|
if current != start {
|
||||||
unsafe { IOObjectRelease(current) };
|
unsafe { IOObjectRelease(current) };
|
||||||
|
|||||||
Reference in New Issue
Block a user