mirror of
https://github.com/freemkv/libfreemkv.git
synced 2026-10-07 18:13:10 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 MattJackson
|
fe8933dad2 | ||
|
MattJackson
|
0a3c1b7b70 | ||
|
MattJackson
|
2bcdc97341 | ||
|
MattJackson
|
4863e9c545 | ||
|
MattJackson
|
03db038dd9 | ||
|
MattJackson
|
95001ba2c3 |
@@ -1,5 +1,45 @@
|
||||
# Changelog
|
||||
|
||||
## 0.16.1 (2026-04-30)
|
||||
|
||||
### Unified progress display for sweep and patch
|
||||
|
||||
- Patch passes now show `bytes_good / disc_total` as GB (grows when sectors are recovered), with pass progress `%` using `work_done / work_total` (always advances). `Xs unreadable` declines as data is recovered. Same UI for all pass types.
|
||||
|
||||
## 0.16.0 (2026-04-30)
|
||||
|
||||
### IOKit registry-based drive enumeration, BSD name matching, reverse patch default
|
||||
|
||||
- **`shim_open_exclusive`** now matches the correct `IOBDServices` for the requested BSD name. Walks all IOBDServices entries in the IOKit registry, matches child IOMedia `"BSD Name"` property. Falls back to IOMedia parent walk, then first-match. Fixes multi-drive systems.
|
||||
- **`shim_list_drives`** (new): registry-based drive enumeration. Reads IOBDServices `"Device Characteristics"` for vendor/model/firmware and child IOMedia for BSD name. Zero SCSI, zero exclusive access, zero unmounts. Fixes the enumeration blast that unmounted every disk on the system.
|
||||
- **`list_drives()`** and **`find_drives()`** rewritten to use `shim_list_drives`. No longer iterates `/dev/disk0..15` opening exclusive SCSI on each.
|
||||
- **`patch_internal`** defaults `reverse: true`. Sweep jumps forward with escalating gaps, so NonTrimmed ranges have good data at their tail. Reverse patch hits good data first, converges on actual bad block boundaries.
|
||||
|
||||
## 0.15.1 (2026-04-30)
|
||||
|
||||
### Fix damage-jump detection, fix dispatch covers_disc check
|
||||
|
||||
- Damage-jump tuning: `DAMAGE_WINDOW=16`, `DAMAGE_THRESHOLD_PCT=12%` (was 50/25%). Old params were too diluted by good reads between sparse failures; new params trigger on 2nd scattered failure.
|
||||
- `Disc::copy()` dispatch: `covers_disc` now checks `map.total_size() == disc_size` (byte-for-byte, not approximate). Fixes false sweep dispatch when mapfile exists but doesn't cover full disc.
|
||||
- Sweep resume: when dispatched from existing mapfile with NonTried, passes `resume: true`.
|
||||
|
||||
## 0.15.0 (2026-04-30)
|
||||
|
||||
### Multipass dispatch rewrite, speed control on damage zone entry/exit
|
||||
|
||||
- `CopyOptions { decrypt, multipass, progress, halt }`. `Disc::copy()` auto-detects sweep vs patch from mapfile state: no mapfile → sweep, NonTried → sweep with resume, only NonTrimmed/NonScraped/Unreadable → patch, clean → no-op.
|
||||
- `Disc::mapfile_for()`: `/dev/null` output → `/tmp/<disc_name>.mapfile`, otherwise `mapfile_path_for(path)`.
|
||||
- Speed control: damage zone entry → `set_speed(0x0000)` (minimum), 16 consecutive good reads → `set_speed(0xFFFF)` (maximum). Drive manages optimal speed in clean sections.
|
||||
- `SET CD SPEED` SCSI command via `SectorReader::set_speed()` (default no-op, Drive impl sends SCSI).
|
||||
|
||||
## 0.14.0 (2026-04-30)
|
||||
|
||||
### Damage-jump algorithm replaces probe, bridge degradation detection
|
||||
|
||||
- Damage-jump algorithm: when damage threshold exceeded in sliding window, jump ahead by `256×batch×multiplier` sectors. Doubles multiplier on each jump. Zero-fills gap as NonTrimmed.
|
||||
- `ecc_sectors()`: returns ECC block size per disc format (32 for UHD, 16 for BD, 16 for DVD).
|
||||
- Bridge degradation detection: NOT READY with sense key 2/ASC 0x04/ASCQ 0x3E triggers 10s cooldown, up to 5 times before treating as bad sector.
|
||||
|
||||
## 0.13.43 (2026-04-29)
|
||||
|
||||
### Pass 1 transport-failure recovery loop
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "libfreemkv"
|
||||
version = "0.16.0"
|
||||
version = "0.16.3"
|
||||
edition = "2024"
|
||||
rust-version = "1.86"
|
||||
license = "AGPL-3.0-only"
|
||||
|
||||
+11
-1
@@ -99,7 +99,6 @@ descriptors or calls ioctls outside of a `ScsiTransport` implementation.
|
||||
|----------|---------------|--------|
|
||||
| Linux | `SgIoTransport` — async `write`/`poll`/`read` on `/dev/sg*` | `/dev/sg*` |
|
||||
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
|
||||
| Windows | `WindowsScsiTransport` — SPTI | `\\.\CdRomN` |
|
||||
|
||||
The Linux backend uses the sg driver's asynchronous interface: `write()` submits
|
||||
the command, `poll()` waits with an enforceable wall-clock timeout, `read()`
|
||||
@@ -107,6 +106,17 @@ retrieves the result. If `poll()` times out, the fd is abandoned (closed in a
|
||||
background thread) and a fresh fd opened — the kernel's USB error recovery
|
||||
cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
|
||||
|
||||
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
|
||||
The shim handles:
|
||||
1. `shim_open_exclusive(bsd_name)` — unmounts the target device via `diskutil`,
|
||||
then walks the IOKit registry to find the `IOBDServices` matching the
|
||||
requested BSD name (IOBDServices → IOBDBlockStorageDriver → IOMedia → "BSD Name"),
|
||||
then creates MMCDeviceInterface → SCSITaskDeviceInterface → ObtainExclusiveAccess.
|
||||
2. `shim_list_drives()` — registry-based enumeration with zero SCSI, zero exclusive
|
||||
access, zero unmounts. Reads IOBDServices "Device Characteristics" for
|
||||
vendor/model/firmware and child IOMedia "BSD Name" for the device path.
|
||||
3. `shim_execute()` / `shim_close()` — raw CDB dispatch and cleanup.
|
||||
|
||||
On non-zero SCSI status, the transport parses sense key from the sense buffer
|
||||
and returns `Error::ScsiError`.
|
||||
|
||||
|
||||
+32
-33
@@ -62,52 +62,51 @@ Position and size are hex byte offsets into the ISO.
|
||||
|
||||
### `CopyOptions` and `PatchOptions`
|
||||
|
||||
Defaults preserve pre-`0.11.21` behavior — abort on the first unreadable
|
||||
sector. Opt in to the recovery-friendly path:
|
||||
`Disc::copy()` auto-detects the pass from mapfile state:
|
||||
|
||||
```rust
|
||||
CopyOptions {
|
||||
skip_on_error: true, // zero-fill bad blocks, continue
|
||||
skip_forward: true, // exponential skip-forward after a failure
|
||||
resume: true, // pick up from an existing ISO + mapfile
|
||||
decrypt: false, // keep the ISO a raw disc image
|
||||
..Default::default()
|
||||
decrypt: true, // decrypt AACS/CSS sectors
|
||||
multipass: true, // enable mapfile + skip-on-error + damage-jump
|
||||
progress: Some(&reporter), // progress callback
|
||||
halt: Some(flag), // halt flag for graceful stop
|
||||
}
|
||||
```
|
||||
|
||||
Dispatch logic:
|
||||
- No mapfile → sweep (fresh Pass 1)
|
||||
- Mapfile with NonTried (`?`) → sweep with resume
|
||||
- Mapfile covering full disc, only NonTrimmed/NonScraped/Unreadable → patch
|
||||
- Mapfile clean → no-op
|
||||
|
||||
## Algorithm
|
||||
|
||||
### Pass 1 — fast sweep (`Disc::copy`)
|
||||
### Pass 1 — fast sweep (`Disc::copy` → `sweep_internal`)
|
||||
|
||||
1. Read 64 KB (32 sectors, one BD ECC block) at the current LBA via
|
||||
`Drive::read(.., recovery=false)` — short 1.5 s timeout, single shot.
|
||||
2. On success: mark the range `+`, advance by one block.
|
||||
3. On failure (with `skip_on_error`): zero-fill the block in the ISO, mark
|
||||
it `*`, advance.
|
||||
4. If `skip_forward` is set: after a failure, jump ahead by an exponentially
|
||||
growing amount (256 KB initial, doubling on consecutive failures, capped at
|
||||
1% of disc). The skipped bytes are also marked `*` — `patch` will visit
|
||||
them later.
|
||||
5. Reset the skip size to 256 KB on the first success after a failure.
|
||||
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
|
||||
2. On success: write data to ISO, mark `+`, advance.
|
||||
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
|
||||
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
|
||||
→ **damage-jump**: skip ahead by `256×batch×multiplier` sectors (8 MB base for
|
||||
UHD). Double the multiplier on each jump (8→16→32→64 MB...). Zero-fill the gap as `*`.
|
||||
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
|
||||
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
|
||||
7. Only transport failures (USB bridge crash) abort the pass.
|
||||
|
||||
Pass 1 completes when every byte has terminal status (`+`, `-`, or the caller
|
||||
bails via the halt flag).
|
||||
Pass 1 completes when every byte has been visited (either `+` or `*`).
|
||||
|
||||
### Pass 2+ — patch (`Disc::patch`)
|
||||
### Pass 2+ — patch (`Disc::copy` → `patch_internal`)
|
||||
|
||||
`Disc::patch` reads the mapfile and iterates every non-`+` range. For each:
|
||||
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
|
||||
(walks ranges from highest LBA to lowest, within each range from end to start).
|
||||
|
||||
1. Issue a drive read via `Drive::read(.., recovery=true)` — long 30 s
|
||||
timeout, still single shot. Drive firmware does its own ECC and retries
|
||||
inside that window; userspace does not pile on additional retries here.
|
||||
2. On success: write the good bytes into the ISO at the exact byte offset,
|
||||
mark `+`.
|
||||
3. On failure: mark `-`.
|
||||
4. Update the mapfile after every block — crash-safe resume.
|
||||
|
||||
Idempotent. Call `patch` N times for N retry attempts; typically the caller
|
||||
stops early if a pass recovers zero bytes (structure-protected sectors will
|
||||
never yield).
|
||||
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
|
||||
does its own ECC recovery inside that window.
|
||||
2. On success: write the good bytes into the ISO, mark `+`.
|
||||
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
|
||||
4. On failure with marginal sense: mark `-`, continue.
|
||||
5. Update the mapfile after every block — crash-safe resume.
|
||||
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
|
||||
|
||||
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
|
||||
|
||||
|
||||
+1
-5
@@ -316,11 +316,7 @@ impl Mapfile {
|
||||
{
|
||||
let file = std::fs::File::create(&tmp)?;
|
||||
let mut w = std::io::BufWriter::new(file);
|
||||
writeln!(
|
||||
w,
|
||||
"# Rescue Logfile. Created by {}",
|
||||
self.version
|
||||
)?;
|
||||
writeln!(w, "# Rescue Logfile. Created by {}", self.version)?;
|
||||
writeln!(w, "# Current pos / status / pass / pass_time")?;
|
||||
writeln!(w, "0x000000000 ? 1 0")?;
|
||||
writeln!(w, "# pos size status")?;
|
||||
|
||||
+545
-97
@@ -1235,8 +1235,8 @@ impl Disc {
|
||||
if opts.multipass {
|
||||
let mf_path = self.mapfile_for(path);
|
||||
if mf_path.exists() {
|
||||
let map = mapfile::Mapfile::load(&mf_path)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
let map =
|
||||
mapfile::Mapfile::load(&mf_path).map_err(|e| Error::IoError { source: e })?;
|
||||
let stats = map.stats();
|
||||
let disc_size = self.capacity_bytes;
|
||||
let covers_disc = map.total_size() == disc_size;
|
||||
@@ -1317,8 +1317,7 @@ impl Disc {
|
||||
bytes_unreadable: pr.bytes_unreadable,
|
||||
bytes_pending: pr.bytes_pending,
|
||||
recovered_this_pass: pr.bytes_recovered_this_pass,
|
||||
complete: pr.bytes_pending == 0 && pr.bytes_unreadable > 0
|
||||
|| pr.bytes_pending == 0 && pr.bytes_unreadable == 0,
|
||||
complete: pr.bytes_pending == 0,
|
||||
halted: pr.halted,
|
||||
})
|
||||
}
|
||||
@@ -1343,13 +1342,19 @@ impl Disc {
|
||||
if !opts.resume {
|
||||
let _ = std::fs::remove_file(&mapfile_path);
|
||||
}
|
||||
let mut map =
|
||||
mapfile::Mapfile::open_or_create(&mapfile_path, total_bytes, concat!("libfreemkv v", env!("CARGO_PKG_VERSION")))
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
let mut map = mapfile::Mapfile::open_or_create(
|
||||
&mapfile_path,
|
||||
total_bytes,
|
||||
concat!("libfreemkv v", env!("CARGO_PKG_VERSION")),
|
||||
)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
|
||||
// ISO file: if resuming and mapfile has Finished ranges, open existing;
|
||||
// otherwise create fresh and pre-size to total_bytes (sparse holes for
|
||||
// non-tried regions).
|
||||
let is_regular = std::fs::metadata(path)
|
||||
.map(|m| m.file_type().is_file())
|
||||
.unwrap_or(false);
|
||||
let file = if opts.resume
|
||||
&& std::fs::metadata(path)
|
||||
.map(|m| m.len() > 0)
|
||||
@@ -1361,8 +1366,10 @@ impl Disc {
|
||||
.map_err(|e| Error::IoError { source: e })?
|
||||
} else {
|
||||
let f = std::fs::File::create(path).map_err(|e| Error::IoError { source: e })?;
|
||||
f.set_len(total_bytes)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
if is_regular {
|
||||
f.set_len(total_bytes)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
}
|
||||
f
|
||||
};
|
||||
|
||||
@@ -1386,10 +1393,11 @@ impl Disc {
|
||||
let mut bridge_degradation_count: u32 = 0;
|
||||
const BRIDGE_DEGRADATION_MAX: u32 = 5;
|
||||
const BRIDGE_DEGRADATION_COOLDOWN_SECS: u64 = 10;
|
||||
const DAMAGE_WINDOW: usize = 16;
|
||||
const DAMAGE_THRESHOLD_PCT: usize = 12;
|
||||
const JUMP_SECTORS_FACTOR: u64 = 256;
|
||||
let mut damage_window: Vec<bool> = Vec::with_capacity(DAMAGE_WINDOW);
|
||||
const PASS1_DAMAGE_WINDOW: usize = 16;
|
||||
const PASS1_DAMAGE_THRESHOLD_PCT: usize = 12;
|
||||
const PASS1_JUMP_SECTORS_FACTOR: u64 = 256;
|
||||
const PASS1_ESCALATION_RESET_GOOD: u64 = PASS1_DAMAGE_WINDOW as u64;
|
||||
let mut damage_window: Vec<bool> = Vec::with_capacity(PASS1_DAMAGE_WINDOW);
|
||||
let mut jump_multiplier: u64 = 1;
|
||||
let mut consecutive_good: u64 = 0;
|
||||
let mut in_damage_zone = false;
|
||||
@@ -1457,11 +1465,11 @@ impl Disc {
|
||||
if read_result.is_ok() {
|
||||
read_ok_count += 1;
|
||||
damage_window.push(true);
|
||||
if damage_window.len() > DAMAGE_WINDOW {
|
||||
if damage_window.len() > PASS1_DAMAGE_WINDOW {
|
||||
damage_window.remove(0);
|
||||
}
|
||||
consecutive_good += 1;
|
||||
if consecutive_good >= DAMAGE_WINDOW as u64 {
|
||||
if consecutive_good >= PASS1_ESCALATION_RESET_GOOD {
|
||||
jump_multiplier = 1;
|
||||
if in_damage_zone {
|
||||
in_damage_zone = false;
|
||||
@@ -1528,7 +1536,9 @@ impl Disc {
|
||||
error = %err,
|
||||
"bridge degradation; cooling down 10s"
|
||||
);
|
||||
std::thread::sleep(std::time::Duration::from_secs(BRIDGE_DEGRADATION_COOLDOWN_SECS));
|
||||
std::thread::sleep(std::time::Duration::from_secs(
|
||||
BRIDGE_DEGRADATION_COOLDOWN_SECS,
|
||||
));
|
||||
continue;
|
||||
}
|
||||
tracing::warn!(
|
||||
@@ -1544,7 +1554,9 @@ impl Disc {
|
||||
let asc = sense.map(|s| s.asc).unwrap_or(0);
|
||||
let ascq = sense.map(|s| s.ascq).unwrap_or(0);
|
||||
|
||||
if sense_key == crate::scsi::SENSE_KEY_NOT_READY && not_ready_retries < NOT_READY_MAX_RETRIES {
|
||||
if sense_key == crate::scsi::SENSE_KEY_NOT_READY
|
||||
&& not_ready_retries < NOT_READY_MAX_RETRIES
|
||||
{
|
||||
not_ready_retries += 1;
|
||||
tracing::warn!(
|
||||
target: "freemkv::disc",
|
||||
@@ -1582,15 +1594,16 @@ impl Disc {
|
||||
bytes_done = bytes_done.saturating_add(block_bytes);
|
||||
|
||||
damage_window.push(false);
|
||||
if damage_window.len() > DAMAGE_WINDOW {
|
||||
if damage_window.len() > PASS1_DAMAGE_WINDOW {
|
||||
damage_window.remove(0);
|
||||
}
|
||||
|
||||
let bad_count = damage_window.iter().filter(|&&b| !b).count();
|
||||
if damage_window.len() >= DAMAGE_WINDOW
|
||||
&& bad_count * 100 / damage_window.len() >= DAMAGE_THRESHOLD_PCT
|
||||
if damage_window.len() >= PASS1_DAMAGE_WINDOW
|
||||
&& bad_count * 100 / damage_window.len() >= PASS1_DAMAGE_THRESHOLD_PCT
|
||||
{
|
||||
let jump_sectors = JUMP_SECTORS_FACTOR * batch as u64 * jump_multiplier;
|
||||
let jump_sectors =
|
||||
PASS1_JUMP_SECTORS_FACTOR * batch as u64 * jump_multiplier;
|
||||
let jump_lba = ((pos / 2048) + jump_sectors) as u32;
|
||||
let region_end_lba = (region_end / 2048) as u32;
|
||||
if jump_lba < region_end_lba {
|
||||
@@ -1670,18 +1683,47 @@ impl Disc {
|
||||
work_done: pos,
|
||||
work_total: total_bytes,
|
||||
bytes_good_total: stats.bytes_good,
|
||||
bytes_bad_total: stats.bytes_unreadable + stats.bytes_retryable,
|
||||
bytes_unreadable_total: stats.bytes_unreadable,
|
||||
bytes_pending_total: stats.bytes_pending,
|
||||
bytes_total_disc: total_bytes,
|
||||
disc_duration_secs: self.titles.first().map(|t| t.duration_secs),
|
||||
bytes_bad_in_main_title: 0,
|
||||
main_title_duration_secs: None,
|
||||
main_title_size_bytes: None,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
file.sync_all().map_err(|e| Error::IoError { source: e })?;
|
||||
let stats = map.stats();
|
||||
tracing::trace!(
|
||||
tracing::debug!(
|
||||
target: "freemkv::disc",
|
||||
phase = "copy_done",
|
||||
phase = "sweep_sync",
|
||||
file_len = file.metadata().map(|m| m.len()).unwrap_or(0),
|
||||
"sweep: calling sync_all"
|
||||
);
|
||||
if let Err(e) = file.sync_all() {
|
||||
if is_regular {
|
||||
tracing::warn!(
|
||||
target: "freemkv::disc",
|
||||
phase = "sweep_sync_failed",
|
||||
error = %e,
|
||||
os_error = e.raw_os_error(),
|
||||
error_kind = ?e.kind(),
|
||||
"sweep: sync_all failed"
|
||||
);
|
||||
return Err(Error::IoError { source: e });
|
||||
}
|
||||
tracing::debug!(
|
||||
target: "freemkv::disc",
|
||||
phase = "sweep_sync_skipped",
|
||||
error = %e,
|
||||
"sweep: sync_all failed for non-regular file; ignoring"
|
||||
);
|
||||
}
|
||||
let stats = map.stats();
|
||||
tracing::debug!(
|
||||
target: "freemkv::disc",
|
||||
phase = "sweep_done",
|
||||
iter_count,
|
||||
read_ok_count,
|
||||
read_err_count,
|
||||
@@ -1689,7 +1731,7 @@ impl Disc {
|
||||
bytes_pending = stats.bytes_pending,
|
||||
halted = halt_requested,
|
||||
copy_elapsed_ms = copy_t0.elapsed().as_millis() as u64,
|
||||
"Disc::copy returning"
|
||||
"Disc::sweep returning"
|
||||
);
|
||||
Ok(CopyResult {
|
||||
bytes_total: total_bytes,
|
||||
@@ -1741,6 +1783,7 @@ pub(crate) struct PatchOpts<'a> {
|
||||
pub halt: Option<std::sync::Arc<std::sync::atomic::AtomicBool>>,
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub(crate) struct PatchOutcome {
|
||||
pub bytes_total: u64,
|
||||
pub bytes_good: u64,
|
||||
@@ -1763,9 +1806,18 @@ pub fn mapfile_path_for(iso_path: &std::path::Path) -> std::path::PathBuf {
|
||||
impl Disc {
|
||||
fn mapfile_for(&self, path: &std::path::Path) -> std::path::PathBuf {
|
||||
if path.as_os_str() == "/dev/null" {
|
||||
let name: String = self.meta_title.as_deref().unwrap_or(&self.volume_id)
|
||||
let name: String = self
|
||||
.meta_title
|
||||
.as_deref()
|
||||
.unwrap_or(&self.volume_id)
|
||||
.chars()
|
||||
.map(|c| if c.is_ascii_alphanumeric() || c == '-' || c == '_' { c } else { '_' })
|
||||
.map(|c| {
|
||||
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
|
||||
c
|
||||
} else {
|
||||
'_'
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
std::path::PathBuf::from(format!("/tmp/{name}.mapfile"))
|
||||
} else {
|
||||
@@ -1783,6 +1835,11 @@ impl Disc {
|
||||
) -> Result<PatchOutcome> {
|
||||
use std::io::{Seek, SeekFrom, Write};
|
||||
|
||||
const BRIDGE_DEGRADATION_PAUSE_SECS: u64 = 10;
|
||||
const POST_FAILURE_PAUSE_SECS: u64 = 1;
|
||||
const CONSECUTIVE_FAIL_LONG_PAUSE: u64 = 5;
|
||||
const CONSECUTIVE_FAIL_LONG_PAUSE_THRESHOLD: u64 = 10;
|
||||
|
||||
let mapfile_path = self.mapfile_for(path);
|
||||
let mut map =
|
||||
mapfile::Mapfile::load(&mapfile_path).map_err(|e| Error::IoError { source: e })?;
|
||||
@@ -1793,6 +1850,9 @@ impl Disc {
|
||||
crate::decrypt::DecryptKeys::None
|
||||
};
|
||||
|
||||
let is_regular = std::fs::metadata(path)
|
||||
.map(|m| m.file_type().is_file())
|
||||
.unwrap_or(false);
|
||||
let mut file = std::fs::OpenOptions::new()
|
||||
.write(true)
|
||||
.open(path)
|
||||
@@ -1811,26 +1871,30 @@ impl Disc {
|
||||
let mut unreadable_count: u64 = 0;
|
||||
let mut buf = vec![0u8; block_sectors as usize * 2048];
|
||||
|
||||
// Collect bad ranges up front. Iterating while mutating is fragile;
|
||||
// each recorded change is persisted, so resume works even if we crash
|
||||
// mid-loop.
|
||||
const PASSN_DAMAGE_WINDOW: usize = 8;
|
||||
const PASSN_DAMAGE_THRESHOLD_PCT: usize = 25;
|
||||
const PASSN_SKIP_SECTORS_BASE: u64 = 64;
|
||||
const PASSN_SKIP_SECTORS_CAP: u64 = 4096;
|
||||
const PASSN_ESCALATION_RESET_GOOD: u32 = 4;
|
||||
let mut damage_window: Vec<bool> = Vec::with_capacity(PASSN_DAMAGE_WINDOW);
|
||||
let mut consecutive_skips_without_recovery: u32;
|
||||
let mut consecutive_good_since_skip: u32;
|
||||
let mut last_skip_from: Option<u64> = None;
|
||||
|
||||
reader.set_speed(0x0000);
|
||||
|
||||
let mut bad_ranges = map.ranges_with(&[
|
||||
mapfile::SectorStatus::NonTried,
|
||||
mapfile::SectorStatus::NonTrimmed,
|
||||
mapfile::SectorStatus::NonScraped,
|
||||
mapfile::SectorStatus::Unreadable,
|
||||
]);
|
||||
// Reverse mode: walk ranges from highest LBA to lowest.
|
||||
if opts.reverse {
|
||||
bad_ranges.reverse();
|
||||
}
|
||||
// work_total = sum of all bad-range bytes. on_progress's third arg
|
||||
// is this value; the second arg (work_done) is incremented per block
|
||||
// attempted. UI consumers (autorip) compute pass_progress_pct =
|
||||
// work_done / work_total — true 0..100% per pass per RIP_DESIGN.md §16.
|
||||
let work_total: u64 = bad_ranges.iter().map(|(_, sz)| *sz).sum();
|
||||
let mut work_done: u64 = 0;
|
||||
tracing::trace!(
|
||||
tracing::info!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_start",
|
||||
block_sectors,
|
||||
@@ -1844,10 +1908,10 @@ impl Disc {
|
||||
|
||||
'outer: for (range_pos, range_size) in bad_ranges {
|
||||
let end = range_pos + range_size;
|
||||
// In reverse mode, walk this range from end - block_bytes back to range_pos.
|
||||
// Each iteration emits the block ending at `block_end` (so reads land on
|
||||
// increasing LBAs internally; we just choose blocks back-to-front).
|
||||
let mut block_end = if opts.reverse { end } else { range_pos };
|
||||
damage_window.clear();
|
||||
consecutive_skips_without_recovery = 0;
|
||||
consecutive_good_since_skip = 0;
|
||||
loop {
|
||||
if let Some(ref h) = opts.halt {
|
||||
if h.load(std::sync::atomic::Ordering::Relaxed) {
|
||||
@@ -1855,7 +1919,6 @@ impl Disc {
|
||||
break 'outer;
|
||||
}
|
||||
}
|
||||
// Compute block boundaries based on direction.
|
||||
let (pos, block_bytes) = if opts.reverse {
|
||||
if block_end <= range_pos {
|
||||
break;
|
||||
@@ -1873,66 +1936,203 @@ impl Disc {
|
||||
let count = (block_bytes / 2048) as u16;
|
||||
let bytes = count as usize * 2048;
|
||||
blocks_attempted += 1;
|
||||
|
||||
let read_result = reader.read_sectors(lba, count, &mut buf[..bytes], recovery);
|
||||
// 0.13.23: parity with Disc::copy — bail immediately on a
|
||||
// non-marginal SCSI sense (transport failure, HARDWARE
|
||||
// ERROR, DATA PROTECT, UNIT ATTENTION, NOT READY,
|
||||
// ILLEGAL REQUEST, kernel IoError). The wedged_threshold
|
||||
// counter is a defense-in-depth backstop for ~50
|
||||
// consecutive marginal failures, but a single
|
||||
// non-marginal error already proves the drive can't
|
||||
// produce data this pass — don't waste 50× the timeout
|
||||
// budget proving it again.
|
||||
if let Err(ref e) = read_result {
|
||||
if !e.is_marginal_read() {
|
||||
let err = read_result.err().unwrap();
|
||||
tracing::trace!(
|
||||
|
||||
match read_result {
|
||||
Ok(_) => {
|
||||
blocks_read_ok += 1;
|
||||
consecutive_failures = 0;
|
||||
consecutive_good_since_skip += 1;
|
||||
if consecutive_good_since_skip >= PASSN_ESCALATION_RESET_GOOD {
|
||||
consecutive_skips_without_recovery = 0;
|
||||
}
|
||||
damage_window.push(true);
|
||||
if damage_window.len() > PASSN_DAMAGE_WINDOW {
|
||||
damage_window.remove(0);
|
||||
}
|
||||
if opts.decrypt {
|
||||
crate::decrypt::decrypt_sectors(&mut buf[..bytes], &keys, 0)?;
|
||||
}
|
||||
file.seek(SeekFrom::Start(pos))
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
file.write_all(&buf[..bytes])
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
map.record(pos, block_bytes, mapfile::SectorStatus::Finished)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
|
||||
if let Some(skip_from) = last_skip_from.take() {
|
||||
let backtrack_start = block_end;
|
||||
let backtrack_end = skip_from;
|
||||
if opts.reverse && backtrack_start < backtrack_end {
|
||||
tracing::info!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_backtrack_start",
|
||||
from_lba = pos,
|
||||
to_lba = backtrack_end / 2048,
|
||||
"recovered after skip; backtracking into gap"
|
||||
);
|
||||
let mut bt_pos = backtrack_start;
|
||||
while bt_pos < backtrack_end {
|
||||
let span =
|
||||
(backtrack_end - bt_pos).min(block_sectors as u64 * 2048);
|
||||
let bt_lba = (bt_pos / 2048) as u32;
|
||||
let bt_count = (span / 2048) as u16;
|
||||
let bt_bytes = bt_count as usize * 2048;
|
||||
match reader.read_sectors(
|
||||
bt_lba,
|
||||
bt_count,
|
||||
&mut buf[..bt_bytes],
|
||||
recovery,
|
||||
) {
|
||||
Ok(_) => {
|
||||
blocks_read_ok += 1;
|
||||
if opts.decrypt {
|
||||
crate::decrypt::decrypt_sectors(
|
||||
&mut buf[..bt_bytes],
|
||||
&keys,
|
||||
0,
|
||||
)?;
|
||||
}
|
||||
file.seek(SeekFrom::Start(bt_pos))
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
file.write_all(&buf[..bt_bytes])
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
map.record(
|
||||
bt_pos,
|
||||
span,
|
||||
mapfile::SectorStatus::Finished,
|
||||
)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
}
|
||||
Err(err) => {
|
||||
if err.is_scsi_transport_failure() {
|
||||
return Err(err);
|
||||
}
|
||||
blocks_read_failed += 1;
|
||||
map.record(
|
||||
bt_pos,
|
||||
span,
|
||||
mapfile::SectorStatus::Unreadable,
|
||||
)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
tracing::info!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_backtrack_stop",
|
||||
lba = bt_lba,
|
||||
"backtrack hit damage; stopping"
|
||||
);
|
||||
break;
|
||||
}
|
||||
}
|
||||
work_done = work_done.saturating_add(span);
|
||||
bt_pos += span;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(err) => {
|
||||
if err.is_scsi_transport_failure() {
|
||||
tracing::warn!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_transport_failure",
|
||||
lba,
|
||||
error = %err,
|
||||
"transport failure (bridge crash); aborting pass"
|
||||
);
|
||||
return Err(err);
|
||||
}
|
||||
|
||||
blocks_read_failed += 1;
|
||||
consecutive_failures += 1;
|
||||
consecutive_good_since_skip = 0;
|
||||
unreadable_count += 1;
|
||||
map.record(pos, block_bytes, mapfile::SectorStatus::Unreadable)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
|
||||
damage_window.push(false);
|
||||
if damage_window.len() > PASSN_DAMAGE_WINDOW {
|
||||
damage_window.remove(0);
|
||||
}
|
||||
|
||||
let pause_secs = if err.is_bridge_degradation() {
|
||||
tracing::debug!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_bridge_degradation",
|
||||
lba,
|
||||
consecutive_failures,
|
||||
error = %err,
|
||||
"bridge degradation; cooling down"
|
||||
);
|
||||
BRIDGE_DEGRADATION_PAUSE_SECS
|
||||
} else if consecutive_failures >= CONSECUTIVE_FAIL_LONG_PAUSE_THRESHOLD {
|
||||
CONSECUTIVE_FAIL_LONG_PAUSE
|
||||
} else {
|
||||
POST_FAILURE_PAUSE_SECS
|
||||
};
|
||||
|
||||
tracing::debug!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_bail",
|
||||
phase = "patch_post_failure_pause",
|
||||
lba,
|
||||
error = %err,
|
||||
"patch read failed with non-marginal sense; bailing"
|
||||
consecutive_failures,
|
||||
pause_secs,
|
||||
"breathing room after failure"
|
||||
);
|
||||
return Err(err);
|
||||
std::thread::sleep(std::time::Duration::from_secs(pause_secs));
|
||||
}
|
||||
}
|
||||
|
||||
let bad_count = damage_window.iter().filter(|&&b| !b).count();
|
||||
let mut did_skip = false;
|
||||
if damage_window.len() >= PASSN_DAMAGE_WINDOW
|
||||
&& bad_count * 100 / damage_window.len() >= PASSN_DAMAGE_THRESHOLD_PCT
|
||||
{
|
||||
let skip_sectors = (PASSN_SKIP_SECTORS_BASE
|
||||
<< consecutive_skips_without_recovery)
|
||||
.min(PASSN_SKIP_SECTORS_CAP);
|
||||
let skip_bytes = skip_sectors * 2048;
|
||||
let new_block_end = if opts.reverse {
|
||||
block_end.saturating_sub(skip_bytes).max(range_pos)
|
||||
} else {
|
||||
(block_end + skip_bytes).min(end)
|
||||
};
|
||||
if new_block_end != block_end {
|
||||
tracing::info!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_damage_skip",
|
||||
from_lba = lba,
|
||||
skip_sectors,
|
||||
escalation = consecutive_skips_without_recovery,
|
||||
bad_pct = bad_count * 100 / damage_window.len(),
|
||||
"damage cluster detected; skipping within range"
|
||||
);
|
||||
let gap_bytes = if opts.reverse {
|
||||
block_end.saturating_sub(new_block_end)
|
||||
} else {
|
||||
new_block_end.saturating_sub(block_end)
|
||||
};
|
||||
work_done = work_done.saturating_add(gap_bytes);
|
||||
last_skip_from = Some(block_end);
|
||||
block_end = new_block_end;
|
||||
consecutive_skips_without_recovery += 1;
|
||||
did_skip = true;
|
||||
}
|
||||
}
|
||||
|
||||
if !did_skip {
|
||||
if opts.reverse {
|
||||
block_end = block_end.saturating_sub(block_bytes);
|
||||
} else {
|
||||
block_end += block_bytes;
|
||||
}
|
||||
}
|
||||
let read_ok = read_result.is_ok();
|
||||
if read_ok {
|
||||
blocks_read_ok += 1;
|
||||
consecutive_failures = 0;
|
||||
if opts.decrypt {
|
||||
crate::decrypt::decrypt_sectors(&mut buf[..bytes], &keys, 0)?;
|
||||
}
|
||||
file.seek(SeekFrom::Start(pos))
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
file.write_all(&buf[..bytes])
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
map.record(pos, block_bytes, mapfile::SectorStatus::Finished)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
} else {
|
||||
blocks_read_failed += 1;
|
||||
consecutive_failures += 1;
|
||||
unreadable_count += 1;
|
||||
map.record(pos, block_bytes, mapfile::SectorStatus::Unreadable)
|
||||
.map_err(|e| Error::IoError { source: e })?;
|
||||
}
|
||||
// Advance block_end in chosen direction.
|
||||
if opts.reverse {
|
||||
block_end = block_end.saturating_sub(block_bytes);
|
||||
} else {
|
||||
block_end += block_bytes;
|
||||
}
|
||||
|
||||
// Wedged-drive early-exit: many consecutive failures with zero
|
||||
// recovered bytes this pass means the drive is stuck and won't
|
||||
// produce data this pass. Save the wallclock budget for productive
|
||||
// grinding; future passes (with smaller block size, reverse, or
|
||||
// after settle) may still recover.
|
||||
if opts.wedged_threshold > 0
|
||||
&& consecutive_failures >= opts.wedged_threshold
|
||||
&& blocks_read_ok == 0
|
||||
{
|
||||
tracing::trace!(
|
||||
tracing::info!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_wedged_exit",
|
||||
consecutive_failures,
|
||||
@@ -1943,10 +2143,6 @@ impl Disc {
|
||||
break 'outer;
|
||||
}
|
||||
|
||||
// Track work done in this pass for the per-pass progress bar.
|
||||
// Each block iterated counts as work, regardless of read
|
||||
// outcome — a failed retry is still progress through the
|
||||
// bad-range walk.
|
||||
work_done = work_done.saturating_add(block_bytes);
|
||||
|
||||
if let Some(reporter) = opts.progress {
|
||||
@@ -1965,14 +2161,44 @@ impl Disc {
|
||||
work_done,
|
||||
work_total,
|
||||
bytes_good_total: s.bytes_good,
|
||||
bytes_bad_total: s.bytes_unreadable + s.bytes_retryable,
|
||||
bytes_unreadable_total: s.bytes_unreadable,
|
||||
bytes_pending_total: s.bytes_pending,
|
||||
bytes_total_disc: total_bytes,
|
||||
disc_duration_secs: self.titles.first().map(|t| t.duration_secs),
|
||||
bytes_bad_in_main_title: 0,
|
||||
main_title_duration_secs: None,
|
||||
main_title_size_bytes: None,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
file.sync_all().map_err(|e| Error::IoError { source: e })?;
|
||||
tracing::debug!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_sync",
|
||||
path = %path.display(),
|
||||
is_regular,
|
||||
"patch: calling sync_all"
|
||||
);
|
||||
if let Err(e) = file.sync_all() {
|
||||
if is_regular {
|
||||
tracing::warn!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_sync_failed",
|
||||
error = %e,
|
||||
os_error = e.raw_os_error(),
|
||||
error_kind = ?e.kind(),
|
||||
"patch: sync_all failed"
|
||||
);
|
||||
return Err(Error::IoError { source: e });
|
||||
}
|
||||
tracing::debug!(
|
||||
target: "freemkv::disc",
|
||||
phase = "patch_sync_skipped",
|
||||
error = %e,
|
||||
"patch: sync_all failed for non-regular file; ignoring"
|
||||
);
|
||||
}
|
||||
let stats = map.stats();
|
||||
tracing::info!(
|
||||
target: "freemkv::disc",
|
||||
@@ -2240,4 +2466,226 @@ mod tests {
|
||||
t.duration_secs = 24.0 * 3600.0;
|
||||
assert_eq!(t.duration_display(), "24h 00m");
|
||||
}
|
||||
|
||||
struct MockReader {
|
||||
total_sectors: u32,
|
||||
bad_sectors: std::collections::HashSet<u32>,
|
||||
}
|
||||
|
||||
impl crate::sector::SectorReader for MockReader {
|
||||
fn read_sectors(
|
||||
&mut self,
|
||||
lba: u32,
|
||||
count: u16,
|
||||
buf: &mut [u8],
|
||||
_recovery: bool,
|
||||
) -> crate::error::Result<usize> {
|
||||
let n = count as usize * 2048;
|
||||
for i in 0..count {
|
||||
if self.bad_sectors.contains(&(lba + i as u32)) {
|
||||
return Err(crate::error::Error::DiscRead {
|
||||
sector: (lba + i as u32) as u64,
|
||||
status: Some(0x02),
|
||||
sense: Some(crate::scsi::ScsiSense {
|
||||
sense_key: 0x02,
|
||||
asc: 0x04,
|
||||
ascq: 0x3E,
|
||||
}),
|
||||
});
|
||||
}
|
||||
}
|
||||
buf[..n].fill(0xAA);
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
fn capacity(&self) -> u32 {
|
||||
self.total_sectors
|
||||
}
|
||||
}
|
||||
|
||||
fn make_test_disc(sectors: u32, name: &str) -> Disc {
|
||||
Disc {
|
||||
volume_id: name.into(),
|
||||
meta_title: Some(name.into()),
|
||||
format: DiscFormat::Uhd,
|
||||
capacity_sectors: sectors,
|
||||
capacity_bytes: sectors as u64 * 2048,
|
||||
layers: 1,
|
||||
titles: Vec::new(),
|
||||
region: DiscRegion::Free,
|
||||
aacs: None,
|
||||
css: None,
|
||||
encrypted: false,
|
||||
content_format: ContentFormat::BdTs,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sweep_to_dev_null_no_enodev() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let iso_path = tmp.path().join("test.iso");
|
||||
let sectors: u32 = 1000;
|
||||
let bad: std::collections::HashSet<u32> = [500u32, 501, 502].into_iter().collect();
|
||||
let mut reader = MockReader {
|
||||
total_sectors: sectors,
|
||||
bad_sectors: bad,
|
||||
};
|
||||
let disc = make_test_disc(sectors, "T1");
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
progress: None,
|
||||
halt: None,
|
||||
};
|
||||
let result = disc.copy(&mut reader, &iso_path, &opts);
|
||||
assert!(
|
||||
result.is_ok(),
|
||||
"sweep to regular file should succeed: {:?}",
|
||||
result.err()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sweep_to_dev_null_real() {
|
||||
let _cleanup = CleanupGuard(std::path::PathBuf::from("/tmp/T2.mapfile"));
|
||||
let sectors: u32 = 1000;
|
||||
let bad: std::collections::HashSet<u32> = [500u32, 501, 502].into_iter().collect();
|
||||
let mut reader = MockReader {
|
||||
total_sectors: sectors,
|
||||
bad_sectors: bad,
|
||||
};
|
||||
let disc = make_test_disc(sectors, "T2");
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
progress: None,
|
||||
halt: None,
|
||||
};
|
||||
let result = disc.copy(&mut reader, std::path::Path::new("/dev/null"), &opts);
|
||||
assert!(
|
||||
result.is_ok(),
|
||||
"sweep to /dev/null should not fail with ENODEV: {:?}",
|
||||
result.err()
|
||||
);
|
||||
}
|
||||
|
||||
struct CleanupGuard(std::path::PathBuf);
|
||||
impl Drop for CleanupGuard {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_file(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sweep_dev_null_full_good() {
|
||||
let _cleanup = CleanupGuard(std::path::PathBuf::from("/tmp/T3.mapfile"));
|
||||
let sectors: u32 = 2000;
|
||||
let mut reader = MockReader {
|
||||
total_sectors: sectors,
|
||||
bad_sectors: std::collections::HashSet::new(),
|
||||
};
|
||||
let disc = make_test_disc(sectors, "T3");
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: false,
|
||||
progress: None,
|
||||
halt: None,
|
||||
};
|
||||
let result = disc.copy(&mut reader, std::path::Path::new("/dev/null"), &opts);
|
||||
assert!(
|
||||
result.is_ok(),
|
||||
"full-good sweep to /dev/null should succeed: {:?}",
|
||||
result.err()
|
||||
);
|
||||
let r = result.unwrap();
|
||||
assert!(r.complete, "should be complete");
|
||||
assert_eq!(r.bytes_good, sectors as u64 * 2048);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn patch_dev_null_after_sweep() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let iso_path = tmp.path().join("test.iso");
|
||||
let sectors: u32 = 500;
|
||||
let bad: std::collections::HashSet<u32> = [100u32, 200, 300].into_iter().collect();
|
||||
let mut reader = MockReader {
|
||||
total_sectors: sectors,
|
||||
bad_sectors: bad.clone(),
|
||||
};
|
||||
let disc = make_test_disc(sectors, "T4");
|
||||
|
||||
let sweep_opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
progress: None,
|
||||
halt: None,
|
||||
};
|
||||
let sweep_result = disc.copy(&mut reader, &iso_path, &sweep_opts);
|
||||
assert!(
|
||||
sweep_result.is_ok(),
|
||||
"sweep should succeed: {:?}",
|
||||
sweep_result.err()
|
||||
);
|
||||
|
||||
let mut reader2 = MockReader {
|
||||
total_sectors: sectors,
|
||||
bad_sectors: std::collections::HashSet::new(),
|
||||
};
|
||||
let patch_opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
progress: None,
|
||||
halt: None,
|
||||
};
|
||||
let patch_result = disc.copy(&mut reader2, &iso_path, &patch_opts);
|
||||
assert!(
|
||||
patch_result.is_ok(),
|
||||
"patch should succeed: {:?}",
|
||||
patch_result.err()
|
||||
);
|
||||
let pr = patch_result.unwrap();
|
||||
assert!(
|
||||
pr.complete,
|
||||
"patch should complete: bytes_pending={}",
|
||||
pr.bytes_pending
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn patch_dev_null_direct() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let iso_path = tmp.path().join("test.iso");
|
||||
let sectors: u32 = 500;
|
||||
let bad: std::collections::HashSet<u32> = [100u32, 200, 300].into_iter().collect();
|
||||
let mut reader = MockReader {
|
||||
total_sectors: sectors,
|
||||
bad_sectors: bad.clone(),
|
||||
};
|
||||
let disc = make_test_disc(sectors, "T5");
|
||||
|
||||
let sweep_opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
progress: None,
|
||||
halt: None,
|
||||
};
|
||||
let _sweep_result = disc.copy(&mut reader, &iso_path, &sweep_opts).unwrap();
|
||||
|
||||
let mut reader2 = MockReader {
|
||||
total_sectors: sectors,
|
||||
bad_sectors: std::collections::HashSet::new(),
|
||||
};
|
||||
let patch_opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
progress: None,
|
||||
halt: None,
|
||||
};
|
||||
let patch_result = disc.copy(&mut reader2, std::path::Path::new("/dev/null"), &patch_opts);
|
||||
assert!(
|
||||
patch_result.is_ok(),
|
||||
"patch to /dev/null should succeed: {:?}",
|
||||
patch_result.err()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -543,7 +543,7 @@ impl Error {
|
||||
/// classified as fatal; treat as recoverable
|
||||
///
|
||||
/// Returns `false` for transport failures (no sense data delivered),
|
||||
/// HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION, NOT READY, ILLEGAL
|
||||
/// HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION, ILLEGAL
|
||||
/// REQUEST, BLANK CHECK, kernel `IoError`, and any non-SCSI variant.
|
||||
/// Caller-agnostic predicate — describes a property of the *error*,
|
||||
/// not what one specific call site should do with it. Used by
|
||||
|
||||
+12
-9
@@ -39,19 +39,22 @@ pub enum PassKind {
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct PassProgress {
|
||||
pub kind: PassKind,
|
||||
/// Bytes processed in this pass so far. Monotonically non-decreasing.
|
||||
pub work_done: u64,
|
||||
/// Total bytes this pass will process. Constant for the duration of
|
||||
/// the pass.
|
||||
pub work_total: u64,
|
||||
/// Cumulative bytes confirmed clean (`Finished` mapfile state) across
|
||||
/// every pass run on this rip. Doesn't change across pass boundaries.
|
||||
pub bytes_good_total: u64,
|
||||
/// Cumulative bytes marked bad (`NonTrimmed` + `NonScraped` +
|
||||
/// `Unreadable`) across every pass run on this rip.
|
||||
pub bytes_bad_total: u64,
|
||||
/// Total disc capacity in bytes. Constant.
|
||||
pub bytes_unreadable_total: u64,
|
||||
pub bytes_pending_total: u64,
|
||||
pub bytes_total_disc: u64,
|
||||
pub disc_duration_secs: Option<f64>,
|
||||
/// How many bytes of the worst-case damage (unreadable + pending) fall
|
||||
/// within the main title's extents. Zero means none of the damage
|
||||
/// affects the main movie — it's all in extras/menus.
|
||||
pub bytes_bad_in_main_title: u64,
|
||||
/// Main title duration in seconds. Same as disc_duration_secs when the
|
||||
/// disc has one dominant title, but separate so consumers can show both.
|
||||
pub main_title_duration_secs: Option<f64>,
|
||||
/// Main title size in bytes (sum of extent sizes).
|
||||
pub main_title_size_bytes: Option<u64>,
|
||||
}
|
||||
|
||||
/// A consumer of pipeline progress events. Library code calls
|
||||
|
||||
+1
-2
@@ -158,8 +158,7 @@ pub(super) fn list_drives() -> Vec<super::DriveInfo> {
|
||||
let count = unsafe { shim_list_drives(buf.as_mut_ptr(), buf.len() as i32) };
|
||||
|
||||
let mut out = Vec::new();
|
||||
for i in 0..(count as usize).min(buf.len()) {
|
||||
let info = &buf[i];
|
||||
for info in buf.iter().take((count as usize).min(buf.len())) {
|
||||
let bsd_name = cstr_to_str(&info.bsd_name);
|
||||
if bsd_name.is_empty() {
|
||||
continue;
|
||||
|
||||
+5
-2
@@ -145,12 +145,14 @@ impl ScsiSense {
|
||||
/// (or a brief retry) sometimes succeeds:
|
||||
///
|
||||
/// - `MEDIUM ERROR` (3) — canonical bad-sector signal
|
||||
/// - `NOT READY` (2) — on many drives (notably BU40N), this is the
|
||||
/// dominant response for unreadable sectors (ASC 04/3E, 04/01, etc.)
|
||||
/// - `ABORTED COMMAND` (B) — transient; retry usually works
|
||||
/// - `RECOVERED ERROR` (1) / `NO SENSE` (0) — drive is healthy and
|
||||
/// either recovered the data or has no specific fault to report
|
||||
///
|
||||
/// `false` for HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION, NOT
|
||||
/// READY, ILLEGAL REQUEST, BLANK CHECK, and any unknown key. Used
|
||||
/// `false` for HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION,
|
||||
/// ILLEGAL REQUEST, BLANK CHECK, and any unknown key. Used
|
||||
/// by [`Error::is_marginal_read`] / `Disc::copy`'s hysteresis
|
||||
/// dispatch.
|
||||
pub fn is_marginal(&self) -> bool {
|
||||
@@ -158,6 +160,7 @@ impl ScsiSense {
|
||||
self.sense_key,
|
||||
SENSE_KEY_NO_SENSE
|
||||
| SENSE_KEY_RECOVERED_ERROR
|
||||
| SENSE_KEY_NOT_READY
|
||||
| SENSE_KEY_MEDIUM_ERROR
|
||||
| SENSE_KEY_ABORTED_COMMAND
|
||||
)
|
||||
|
||||
@@ -593,7 +593,7 @@ impl SectorReader for BlockSizeFailingReader {
|
||||
#[test]
|
||||
fn test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed() {
|
||||
let capacity_sectors: u32 = 256;
|
||||
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
||||
let _total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
||||
|
||||
let mut reader = BlockSizeFailingReader {
|
||||
capacity: capacity_sectors,
|
||||
@@ -634,3 +634,164 @@ fn test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed() {
|
||||
"complete=false when sectors remain NonTrimmed"
|
||||
);
|
||||
}
|
||||
|
||||
// ── 9. PassProgress carries separate unreadable vs pending byte counts ─────
|
||||
//
|
||||
// The video-damage-time display needs bytes_unreadable_total (confirmed dead)
|
||||
// separate from bytes_pending_total (might still recover). This test verifies
|
||||
// that a Pass 2 with some confirmed failures produces correct field values.
|
||||
|
||||
#[test]
|
||||
fn test_pass_progress_separates_unreadable_from_pending() {
|
||||
let capacity_sectors: u32 = 128;
|
||||
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
||||
|
||||
let mut reader = FailingSectorReader::new(capacity_sectors);
|
||||
let disc = synthetic_disc(capacity_sectors);
|
||||
|
||||
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
|
||||
let iso_path = tmp.path().to_path_buf();
|
||||
drop(tmp);
|
||||
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let pass1 = disc.copy(&mut reader, &iso_path, &opts).expect("pass1 ok");
|
||||
|
||||
assert_eq!(pass1.bytes_good, 0, "pass1: no good sectors");
|
||||
assert_eq!(pass1.bytes_unreadable, 0, "pass1: no confirmed unreadable");
|
||||
assert_eq!(
|
||||
pass1.bytes_pending, total_bytes,
|
||||
"pass1: all sectors NonTrimmed"
|
||||
);
|
||||
|
||||
let last_unreadable = Arc::new(AtomicU64::new(0));
|
||||
let last_pending = Arc::new(AtomicU64::new(0));
|
||||
let last_good = Arc::new(AtomicU64::new(0));
|
||||
let last_dur = Arc::new(AtomicU64::new(0));
|
||||
|
||||
struct SnapshotReporter {
|
||||
unreadable: Arc<AtomicU64>,
|
||||
pending: Arc<AtomicU64>,
|
||||
good: Arc<AtomicU64>,
|
||||
dur: Arc<AtomicU64>,
|
||||
}
|
||||
impl libfreemkv::progress::Progress for SnapshotReporter {
|
||||
fn report(&self, p: &libfreemkv::progress::PassProgress) {
|
||||
self.unreadable
|
||||
.store(p.bytes_unreadable_total, Ordering::Relaxed);
|
||||
self.pending.store(p.bytes_pending_total, Ordering::Relaxed);
|
||||
self.good.store(p.bytes_good_total, Ordering::Relaxed);
|
||||
if let Some(d) = p.disc_duration_secs {
|
||||
self.dur.store((d * 1000.0) as u64, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
}
|
||||
let reporter = SnapshotReporter {
|
||||
unreadable: last_unreadable.clone(),
|
||||
pending: last_pending.clone(),
|
||||
good: last_good.clone(),
|
||||
dur: last_dur.clone(),
|
||||
};
|
||||
|
||||
let pass2_opts = CopyOptions {
|
||||
decrypt: false,
|
||||
multipass: true,
|
||||
progress: Some(&reporter),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let pass2 = disc
|
||||
.copy(&mut reader, &iso_path, &pass2_opts)
|
||||
.expect("pass2 ok");
|
||||
|
||||
let _ = std::fs::remove_file(&iso_path);
|
||||
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
||||
|
||||
assert_eq!(
|
||||
pass2.bytes_good, 0,
|
||||
"pass2: still no good sectors (reader always fails)"
|
||||
);
|
||||
assert!(
|
||||
pass2.bytes_unreadable > 0,
|
||||
"pass2: some sectors confirmed unreadable"
|
||||
);
|
||||
assert!(
|
||||
pass2.bytes_pending < pass1.bytes_pending,
|
||||
"pass2: fewer pending sectors than pass1"
|
||||
);
|
||||
|
||||
let observed_unreadable = last_unreadable.load(Ordering::Relaxed);
|
||||
let observed_pending = last_pending.load(Ordering::Relaxed);
|
||||
assert!(
|
||||
observed_unreadable > 0,
|
||||
"progress should report confirmed unreadable bytes"
|
||||
);
|
||||
assert!(
|
||||
observed_pending == 0 || observed_pending < total_bytes,
|
||||
"pending should shrink as sectors are confirmed unreadable"
|
||||
);
|
||||
|
||||
// Video damage time: unreadable / total * duration
|
||||
// With no titles on synthetic disc, disc_duration_secs = None
|
||||
assert_eq!(
|
||||
last_dur.load(Ordering::Relaxed),
|
||||
0,
|
||||
"synthetic disc has no titles, duration should be None/0"
|
||||
);
|
||||
}
|
||||
|
||||
// ── 10. Damage time calculation (unit test) ────────────────────────────────
|
||||
//
|
||||
// Verifies the formula: damage_secs = bytes_unreadable / bytes_total * duration
|
||||
// This mirrors the CLI's print_disc_progress logic.
|
||||
|
||||
#[test]
|
||||
fn test_damage_time_calculation() {
|
||||
// 78.8 GB disc, 2h45m movie (9900s), 74 KB unreadable
|
||||
let disc_bytes: u64 = 78_800_000_000;
|
||||
let duration_secs: f64 = 9900.0;
|
||||
|
||||
let cases: Vec<(u64, &str)> = vec![
|
||||
(74 * 1024, "~10ms"), // 74 KB → ~9ms, negligible
|
||||
(10 * 1024 * 1024, "~1.3s"), // 10 MB → ~1.3s
|
||||
(100 * 1024 * 1024, "~13s"), // 100 MB → ~13s
|
||||
(1024 * 1024 * 1024, "~134s"), // 1 GB → ~134s
|
||||
];
|
||||
|
||||
for (bad_bytes, label) in cases {
|
||||
let damage_secs = bad_bytes as f64 / disc_bytes as f64 * duration_secs;
|
||||
match label {
|
||||
"~10ms" => assert!(damage_secs < 0.05, "{label}: {damage_secs:.3}s"),
|
||||
"~1.3s" => assert!(
|
||||
(damage_secs - 1.3).abs() < 0.2,
|
||||
"{label}: {damage_secs:.2}s"
|
||||
),
|
||||
"~13s" => assert!(
|
||||
(damage_secs - 13.0).abs() < 1.0,
|
||||
"{label}: {damage_secs:.1}s"
|
||||
),
|
||||
"~134s" => assert!(
|
||||
(damage_secs - 134.0).abs() < 2.0,
|
||||
"{label}: {damage_secs:.0}s"
|
||||
),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
// 0.25s threshold: how many bad bytes = 0.25s of damage?
|
||||
let threshold_bytes = (0.25 / duration_secs * disc_bytes as f64) as u64;
|
||||
assert!(
|
||||
threshold_bytes > 0,
|
||||
"0.25s damage threshold should be > 0 bytes"
|
||||
);
|
||||
// At 9900s / 78.8 GB ≈ 0.25s = ~2 MB
|
||||
let expected_mb = threshold_bytes as f64 / (1024.0 * 1024.0);
|
||||
assert!(
|
||||
(expected_mb - 2.0).abs() < 0.5,
|
||||
"0.25s ≈ {expected_mb:.2} MB (expected ~2 MB)"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -203,7 +203,10 @@ fn test_check_cond_not_ready_predicate() {
|
||||
|
||||
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
|
||||
assert!(sense.is_not_ready(), "sense_key 2 ⇒ is_not_ready");
|
||||
assert!(!err.is_marginal_read(), "NOT READY is not marginal");
|
||||
assert!(
|
||||
err.is_marginal_read(),
|
||||
"NOT READY is marginal (BU40N returns this for bad sectors)"
|
||||
);
|
||||
}
|
||||
|
||||
// ── 5. CHECK CONDITION + MEDIUM ERROR (canonical marginal-read) ───────────
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
//!
|
||||
//! These tests require a real /dev/sg* device and are therefore #[ignore].
|
||||
|
||||
use libfreemkv::scsi::{DataDirection, SCSI_STATUS_TRANSPORT_FAILURE};
|
||||
use std::path::Path;
|
||||
use std::time::Duration;
|
||||
|
||||
@@ -17,16 +16,17 @@ use std::time::Duration;
|
||||
fn test_sgio_transport_timeout_does_not_kill_transport() {
|
||||
let device = "/dev/sg2";
|
||||
let device = std::env::var("FREEMKV_TEST_SG_DEVICE").unwrap_or(device.to_string());
|
||||
let path = Path::new(&device);
|
||||
let _path = Path::new(&device);
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
use libfreemkv::scsi::ScsiTransport;
|
||||
use libfreemkv::scsi::linux::SgIoTransport;
|
||||
use libfreemkv::scsi::{DataDirection, SCSI_STATUS_TRANSPORT_FAILURE};
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicI32, Ordering};
|
||||
|
||||
let mut transport = SgIoTransport::open(path).expect("open device");
|
||||
let mut transport = SgIoTransport::open(_path).expect("open device");
|
||||
let fd_before = transport.fd;
|
||||
|
||||
// READ_10 with 1 ms timeout to force kernel timeout.
|
||||
@@ -90,11 +90,11 @@ fn test_sgio_transport_timeout_does_not_kill_transport() {
|
||||
fn test_drive_read_per_cdb_timeout_bounds_call() {
|
||||
let device = "/dev/sg2";
|
||||
let device = std::env::var("FREEMKV_TEST_SG_DEVICE").unwrap_or(device.to_string());
|
||||
let path = Path::new(&device);
|
||||
let _path = Path::new(&device);
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
let mut drive = libfreemkv::Drive::open(path).expect("open drive");
|
||||
let mut drive = libfreemkv::Drive::open(_path).expect("open drive");
|
||||
let timeout_ms: u32 = 5_000;
|
||||
|
||||
let start = std::time::Instant::now();
|
||||
|
||||
Reference in New Issue
Block a user