v0.13.37: Pass 1 is pure ECC-block sweep — read 32 sectors, fail → skip, no single-sector reads
This commit is contained in:
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "libfreemkv"
|
name = "libfreemkv"
|
||||||
version = "0.13.36"
|
version = "0.13.37"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
rust-version = "1.86"
|
rust-version = "1.86"
|
||||||
license = "AGPL-3.0-only"
|
license = "AGPL-3.0-only"
|
||||||
|
|||||||
@@ -0,0 +1,312 @@
|
|||||||
|
# Troubleshooting Guide
|
||||||
|
|
||||||
|
Common problems and solutions for optical drive ripping with freemkv.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. USB-SATA Bridge Issues
|
||||||
|
|
||||||
|
This is the single most common source of problems when ripping discs over USB.
|
||||||
|
|
||||||
|
### Symptoms
|
||||||
|
|
||||||
|
- The drive disappears mid-rip. The ripping tool reports the device is gone, and `ls /dev/sg*` no longer shows it.
|
||||||
|
- The device re-enumerates under a different name: `sg4` becomes `sg5`, then `sg7`, then `sg11` after each USB port reset.
|
||||||
|
- `dmesg` shows USB port resets: `usb X-Y: reset high-speed USB device`, `xhci_hcd 0000:00:14.0: Cannot enable. Maybe the USB cable is bad?`, or `usb-storage: device reset failed`.
|
||||||
|
- The SCSI layer reports `host_status=7` (Linux USB transport error) in sense data.
|
||||||
|
- The drive works fine for reading data discs or burning, but crashes when hitting damaged sectors during a rip.
|
||||||
|
- After the crash, the drive is completely invisible until physically unplugged and reconnected.
|
||||||
|
|
||||||
|
### Root Cause
|
||||||
|
|
||||||
|
USB-SATA bridges translate between the USB Mass Storage protocol (BOT or UAS) and the drive's native SATA interface. When the optical drive encounters an unreadable sector, it returns a SCSI CHECK CONDITION with sense key 0x03 (MEDIUM ERROR). Some bridge chipsets -- particularly the Initio INIC-36xx family -- have buggy firmware that mishandles this error response.
|
||||||
|
|
||||||
|
Specific failure modes:
|
||||||
|
|
||||||
|
- **Incorrect residue reporting.** The bridge claims a different number of bytes transferred than what actually occurred. The Linux USB storage driver sees this discrepancy as a protocol violation and resets the port to recover. The `US_FL_IGNORE_RESIDUE` quirk exists specifically for this class of bug (see `drivers/usb/storage/transport.c` in the Linux kernel).
|
||||||
|
- **Bridge firmware crash.** On some Initio bridges, a malformed SCSI error response from the drive causes the bridge MCU to hang entirely. The USB controller sees the device stop responding and initiates a port reset. The bridge recovers (it re-enumerates), but the rip is dead -- all state is lost.
|
||||||
|
- **Sense data corruption.** The bridge forwards garbled or truncated sense data to the host, which the SCSI midlayer cannot parse, leading to a transport reset.
|
||||||
|
|
||||||
|
This is a hardware + firmware problem, not a software bug. The same drive connected via direct SATA does not exhibit these symptoms.
|
||||||
|
|
||||||
|
### Known Affected Bridges
|
||||||
|
|
||||||
|
| Chipset | USB IDs | Notes |
|
||||||
|
|---------|---------|-------|
|
||||||
|
| Initio INIC-3609 | `13fd:3609` | Very common in cheap SATA-to-USB enclosures. Highly problematic. |
|
||||||
|
| Initio INIC-3619 | `13fd:3940` | Same firmware family as INIC-3609. |
|
||||||
|
| Initio INIC-3069 | `13fd:0840` | Older variant, same residue bug. |
|
||||||
|
| ASMedia ASM1051 | `174c:5106` | Early ASM SATA bridge. Residue issues on error paths. |
|
||||||
|
| JMicron JMB36x | `152d:0561` | Some firmware versions. Not all JMicroon chips are affected. |
|
||||||
|
|
||||||
|
If your drive came in a pre-built external enclosure (Vantec, Sabrent, OWC, etc.), it almost certainly uses one of these bridge chips internally.
|
||||||
|
|
||||||
|
### The Fix: USB Storage Quirk
|
||||||
|
|
||||||
|
Apply the `US_FL_IGNORE_RESIDUE` kernel quirk for your bridge. This tells the Linux USB storage driver to ignore the residue field in SCSI response frames, preventing the port reset on mismatched byte counts.
|
||||||
|
|
||||||
|
**Step 1: Identify your bridge's vendor:product ID.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsusb
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for your drive's entry. Example output:
|
||||||
|
|
||||||
|
```
|
||||||
|
Bus 002 Device 005: ID 13fd:0840 Initio Corporation INIC-3609
|
||||||
|
```
|
||||||
|
|
||||||
|
Here the vendor ID is `13fd` and the product ID is `0840`.
|
||||||
|
|
||||||
|
**Step 2: Apply the quirk at runtime.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo "13fd:0840:i" > /sys/module/usb_storage/parameters/quirks
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace `13fd:0840` with your device's actual IDs. The `:i` flag means `US_FL_IGNORE_RESIDUE`.
|
||||||
|
|
||||||
|
You can combine multiple flags. Common additions:
|
||||||
|
|
||||||
|
- `:i` -- ignore residue (`US_FL_IGNORE_RESIDUE`)
|
||||||
|
- `:u` -- force BOT mode instead of UAS, for bridges with UAS bugs
|
||||||
|
|
||||||
|
**Step 3: Reconnect the drive.** Unplug and replug the USB cable, or bind/unbind the device. The quirk is applied per-module-load, so existing sessions may need the drive reconnected.
|
||||||
|
|
||||||
|
### Making It Persistent
|
||||||
|
|
||||||
|
Add the quirk to your kernel boot parameters so it survives reboots.
|
||||||
|
|
||||||
|
Edit `/etc/default/grub` (GRUB) and add to `GRUB_CMDLINE_LINUX_DEFAULT`:
|
||||||
|
|
||||||
|
```
|
||||||
|
GRUB_CMDLINE_LINUX_DEFAULT="quiet usb_storage.quirks=13fd:0840:i"
|
||||||
|
```
|
||||||
|
|
||||||
|
Then rebuild the GRUB config:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo update-grub
|
||||||
|
```
|
||||||
|
|
||||||
|
For systemd-boot, add to your loader entry or `/etc/kernel/cmdline`:
|
||||||
|
|
||||||
|
```
|
||||||
|
usb_storage.quirks=13fd:0840:i
|
||||||
|
```
|
||||||
|
|
||||||
|
Multiple devices can be separated by commas:
|
||||||
|
|
||||||
|
```
|
||||||
|
usb_storage.quirks=13fd:0840:i,174c:5106:u
|
||||||
|
```
|
||||||
|
|
||||||
|
### Recommended Bridges
|
||||||
|
|
||||||
|
If you are buying a USB-SATA adapter or enclosure for optical drive use:
|
||||||
|
|
||||||
|
| Bridge | USB IDs | Notes |
|
||||||
|
|--------|---------|-------|
|
||||||
|
| ASMedia ASM1153 | `174c:1153` | Reliable. Widely available in SATA-USB 3.0 cables. |
|
||||||
|
| JMicron JMS578 | `152d:0578` | Good firmware. Supports UASP. |
|
||||||
|
| Icy Box IB-AC640-C3 | N/A | Uses a known-good bridge internally. Plug-and-play. |
|
||||||
|
|
||||||
|
Avoid any enclosure or adapter listing an Initio chipset.
|
||||||
|
|
||||||
|
### Best Solution: Direct SATA
|
||||||
|
|
||||||
|
Connect your optical drive directly to a motherboard SATA port. This eliminates the USB-SATA bridge entirely and is the most reliable configuration:
|
||||||
|
|
||||||
|
- No USB protocol overhead or translation errors.
|
||||||
|
- No bridge firmware bugs.
|
||||||
|
- No port resets or re-enumeration.
|
||||||
|
- Full SATA error recovery handled natively by the kernel's libata driver.
|
||||||
|
- Sustained read speeds are limited only by the drive, not the USB bus.
|
||||||
|
|
||||||
|
If your machine has a free SATA port, use it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Damaged Disc Handling
|
||||||
|
|
||||||
|
### Symptoms
|
||||||
|
|
||||||
|
- SCSI MEDIUM ERROR (sense key 0x03) at specific LBAs. `dmesg` shows `sr X:0:0:0: [srY] Unrecoverable read error` or similar.
|
||||||
|
- Read speed drops to near zero when approaching a damaged area.
|
||||||
|
- The drive makes audible retrying noises (laser repositioning, spindle speed changes).
|
||||||
|
- On USB-connected drives: the bridge crashes (see section 1 above) when the drive returns the error.
|
||||||
|
|
||||||
|
### How freemkv Handles This
|
||||||
|
|
||||||
|
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
|
||||||
|
|
||||||
|
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
|
||||||
|
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
|
||||||
|
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
|
||||||
|
|
||||||
|
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
|
||||||
|
|
||||||
|
### The Drive Taint Issue (LG BU40N)
|
||||||
|
|
||||||
|
Some drives, notably the LG BU40N, exhibit a "taint" behavior after encountering MEDIUM ERRORs:
|
||||||
|
|
||||||
|
1. The drive hits a damaged sector and returns a MEDIUM ERROR.
|
||||||
|
2. From that point forward, **all subsequent reads fail** -- even reads to sectors that were previously successful.
|
||||||
|
3. The only recovery is to physically unplug and reconnect the drive (or power-cycle it).
|
||||||
|
|
||||||
|
This is not a freemkv bug. It is a drive firmware behavior triggered by the interaction between the drive's internal error recovery and the USB-SATA bridge's handling of the error response. The drive firmware enters a degraded state that it does not recover from without a power cycle.
|
||||||
|
|
||||||
|
Workarounds:
|
||||||
|
|
||||||
|
- **Use a direct SATA connection.** This eliminates the bridge interaction that triggers the taint.
|
||||||
|
- **Use a different bridge.** The ASM1153 and JMS578 are less likely to trigger this behavior.
|
||||||
|
- **Accept the partial ISO.** freemkv's skip-forward recovery will zero-fill the unreadable blocks and continue. The resulting ISO may be playable with minor glitches in the affected areas.
|
||||||
|
- **Physical replug between retry passes.** If running multi-pass patch, replug the drive between passes to clear the taint state.
|
||||||
|
|
||||||
|
freemkv deliberately does not attempt inline SCSI resets or eject cycles to recover from this state, because those operations were found to make the problem worse on affected hardware (see the design rationale in [`docs/rip-recovery.md`](docs/rip-recovery.md)).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Drive Not Detected
|
||||||
|
|
||||||
|
### Check Hardware Visibility
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsusb
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify the drive appears in the USB device list. If it does not show up, the drive is not visible to the host at all -- check cables, power, and USB port.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ls /dev/sg*
|
||||||
|
```
|
||||||
|
|
||||||
|
On Linux, optical drives appear as `/dev/sg*` devices (the SCSI Generic interface). freemkv uses `/dev/sg*`, not `/dev/sr*`. If `lsusb` shows the device but no `/dev/sg*` entry exists, the `sg` kernel module may not be loaded:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo modprobe sg
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check Kernel Messages
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dmesg | grep -i usb | tail -30
|
||||||
|
dmesg | grep -i sg | tail -10
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for:
|
||||||
|
- USB enumeration errors or failed port resets.
|
||||||
|
- `sg_add` messages confirming the sg device was registered.
|
||||||
|
- Permission denied or access errors.
|
||||||
|
|
||||||
|
### Permission Issues
|
||||||
|
|
||||||
|
On most Linux distributions, `/dev/sg*` devices are owned by `root:disk` or `root:cdrom` with restricted permissions. Running freemkv as an unprivileged user will fail with permission errors.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
|
||||||
|
- Add your user to the appropriate group:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo usermod -aG disk $USER
|
||||||
|
```
|
||||||
|
|
||||||
|
Then log out and back in for the change to take effect. On some distributions the group is `cdrom` or `optical` instead of `disk`.
|
||||||
|
|
||||||
|
- Run with elevated privileges:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo freemkv ...
|
||||||
|
```
|
||||||
|
|
||||||
|
- Install a udev rule for persistent per-device permissions. Create `/etc/udev/rules.d/99-sg-optical.rules`:
|
||||||
|
|
||||||
|
```
|
||||||
|
SUBSYSTEM=="scsi_generic", ATTRS{type}=="5", MODE="0666"
|
||||||
|
```
|
||||||
|
|
||||||
|
Then reload udev rules:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo udevadm control --reload-rules && sudo udevadm trigger
|
||||||
|
```
|
||||||
|
|
||||||
|
### Spin-Up Delay
|
||||||
|
|
||||||
|
Optical drives take 30-60 seconds to spin up and become ready after hot-plug or disc insertion. During this window, SCSI commands may return NOT READY or timeout.
|
||||||
|
|
||||||
|
freemkv's `Drive::wait_ready()` handles this automatically by polling with TEST UNIT READY until the drive responds. If you are writing your own code using the library, always call `wait_ready()` before `init()`:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
|
||||||
|
drive.wait_ready()?; // blocks until disc is ready, up to 30s
|
||||||
|
drive.init()?;
|
||||||
|
```
|
||||||
|
|
||||||
|
If the drive was just plugged in, wait a full minute before concluding it is not detected.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. How to Identify Your USB-SATA Bridge
|
||||||
|
|
||||||
|
If you are experiencing the issues described in section 1, you need to know which bridge chipset your adapter or enclosure uses.
|
||||||
|
|
||||||
|
### Step 1: Find the Device
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsusb
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for entries matching your drive or enclosure. Bridges may appear under their own manufacturer name or as a generic SATA device. Common examples:
|
||||||
|
|
||||||
|
```
|
||||||
|
Bus 002 Device 005: ID 13fd:0840 Initio Corporation
|
||||||
|
Bus 002 Device 006: ID 174c:1153 ASMedia Technology Inc. ASM1153
|
||||||
|
Bus 002 Device 007: ID 152d:0578 JMicron Technology Corp. JMS578
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 2: Match the IDs
|
||||||
|
|
||||||
|
| Vendor | Product ID | Chipset | Status |
|
||||||
|
|--------|-----------|---------|--------|
|
||||||
|
| `13fd` | `3609` | Initio INIC-3609 | Affected. Apply quirk. |
|
||||||
|
| `13fd` | `3940` | Initio INIC-3619 | Affected. Apply quirk. |
|
||||||
|
| `13fd` | `0840` | Initio INIC-3069 | Affected. Apply quirk. |
|
||||||
|
| `174c` | `5106` | ASMedia ASM1051 | Affected (early firmware). Apply quirk. |
|
||||||
|
| `174c` | `1153` | ASMedia ASM1153 | Good. No quirk needed. |
|
||||||
|
| `152d` | `0561` | JMicron JMB36x | Affected (some firmware). Apply quirk if issues occur. |
|
||||||
|
| `152d` | `0578` | JMicron JMS578 | Good. No quirk needed. |
|
||||||
|
|
||||||
|
### Step 3: Check dmesg for the Bridge Name
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dmesg | grep -i "usb-storage\|uas\|initio\|asmedia\|jmicron"
|
||||||
|
```
|
||||||
|
|
||||||
|
This often reveals the bridge chipset even when `lsusb` shows a generic name.
|
||||||
|
|
||||||
|
### Step 4: If the Enclosure Is Sealed
|
||||||
|
|
||||||
|
Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not advertise the bridge chipset on the packaging. In this case:
|
||||||
|
|
||||||
|
1. Check `lsusb` while the enclosure is connected.
|
||||||
|
2. Search the vendor:product ID online -- there are community-maintained lists of which chipsets popular enclosures use.
|
||||||
|
3. If you cannot determine the chipset and are experiencing bridge crashes, assume it is an Initio and apply the quirk with its IDs.
|
||||||
|
4. The definitive test: connect the bare drive to a motherboard SATA port. If the problems disappear, the bridge was the cause.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. General Debugging Checklist
|
||||||
|
|
||||||
|
When something goes wrong during a rip, gather this information before filing an issue:
|
||||||
|
|
||||||
|
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
|
||||||
|
2. **Drive model:** from the drive label, or from `freemkv info`.
|
||||||
|
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
|
||||||
|
4. **Operating system and kernel:** `uname -a`.
|
||||||
|
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
|
||||||
|
6. **SCSI device:** which `/dev/sg*` the drive was on, and whether it changed after the failure.
|
||||||
|
7. **The disc:** title, format (BD/DVD/UHD), condition.
|
||||||
|
|
||||||
|
Include all of the above in bug reports. SCSI transport errors that resolve with the `US_FL_IGNORE_RESIDUE` quirk or by switching to direct SATA are bridge firmware bugs, not freemkv bugs.
|
||||||
+47
-110
@@ -1217,14 +1217,14 @@ impl Disc {
|
|||||||
/// `path + ".mapfile"` — flushed every block for crash-safe resume.
|
/// `path + ".mapfile"` — flushed every block for crash-safe resume.
|
||||||
///
|
///
|
||||||
/// # Options
|
/// # Options
|
||||||
/// - **default** (all false): behavior matches pre-v0.11.21 — uses full
|
/// - **default** (skip_on_error=false): uses full drive recovery (may take
|
||||||
/// drive recovery (may take minutes per bad sector), aborts on error.
|
/// minutes per bad sector), aborts on error. Mapfile is produced as a
|
||||||
/// Mapfile is produced as a side-effect.
|
/// side-effect.
|
||||||
/// - **skip_on_error**: zero-fill bad blocks in the ISO, mark them in the
|
/// - **skip_on_error**: zero-fill bad blocks in the ISO, mark them NonTrimmed
|
||||||
/// mapfile, and continue. Uses fast reads (no drive-level recovery loop).
|
/// in the mapfile, and continue. Reads in `batch`-sector chunks (32 sectors
|
||||||
/// - **skip_forward** (implies skip_on_error): on block failure, also skip
|
/// = 1 BD ECC block by default). Failed blocks are marked NonTrimmed for
|
||||||
/// forward by an exponentially-growing amount, marking the jumped region
|
/// recovery by `Disc::patch`. No single-sector reads in this pass —
|
||||||
/// as `non-trimmed` for later trimming/scraping by `Disc::patch`.
|
/// Pass 1 is pure ECC-block sweep.
|
||||||
/// - **resume**: if the mapfile exists, resume from its state — only
|
/// - **resume**: if the mapfile exists, resume from its state — only
|
||||||
/// `non-tried` ranges are read. Without `resume`, a fresh mapfile is
|
/// `non-tried` ranges are read. Without `resume`, a fresh mapfile is
|
||||||
/// written and the ISO recreated from scratch.
|
/// written and the ISO recreated from scratch.
|
||||||
@@ -1274,7 +1274,7 @@ impl Disc {
|
|||||||
let mut file = file;
|
let mut file = file;
|
||||||
let batch: u16 = match opts.batch_sectors {
|
let batch: u16 = match opts.batch_sectors {
|
||||||
Some(b) => b,
|
Some(b) => b,
|
||||||
None if opts.skip_forward => 32, // 64 KB = BD ECC block size
|
None if opts.skip_on_error => 32,
|
||||||
None => DEFAULT_BATCH_SECTORS,
|
None => DEFAULT_BATCH_SECTORS,
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -1286,23 +1286,15 @@ impl Disc {
|
|||||||
let mut read_ok_count: u64 = 0;
|
let mut read_ok_count: u64 = 0;
|
||||||
let mut read_err_count: u64 = 0;
|
let mut read_err_count: u64 = 0;
|
||||||
let mut last_log_iter: u64 = 0;
|
let mut last_log_iter: u64 = 0;
|
||||||
// Simple read strategy: start in block mode, drop to 1 sector on any
|
|
||||||
// failure, then after BPT1_EXIT_THRESHOLD consecutive good single-sector
|
|
||||||
// reads we try block mode again. This avoids hammering every sector
|
|
||||||
// in a big bad zone with a slow timeout at block size.
|
|
||||||
let mut use_single = false;
|
|
||||||
let mut consecutive_good: u64 = 0;
|
|
||||||
tracing::trace!(
|
tracing::trace!(
|
||||||
target: "freemkv::disc",
|
target: "freemkv::disc",
|
||||||
phase = "copy_start",
|
phase = "copy_start",
|
||||||
total_bytes,
|
total_bytes,
|
||||||
batch,
|
batch,
|
||||||
bpt1_exit_threshold = BPT1_EXIT_THRESHOLD,
|
skip_on_error = opts.skip_on_error,
|
||||||
"Disc::copy entered"
|
"Disc::copy entered"
|
||||||
);
|
);
|
||||||
|
|
||||||
// Iterate over not-yet-finished regions from the mapfile. We re-read the
|
|
||||||
// mapfile after each block because record() mutates the region list.
|
|
||||||
'outer: loop {
|
'outer: loop {
|
||||||
let regions_to_do = map.ranges_with(&[
|
let regions_to_do = map.ranges_with(&[
|
||||||
mapfile::SectorStatus::NonTried,
|
mapfile::SectorStatus::NonTried,
|
||||||
@@ -1318,9 +1310,6 @@ impl Disc {
|
|||||||
if regions_to_do.is_empty() {
|
if regions_to_do.is_empty() {
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
// Only process the first NonTried range per outer pass; skip_forward
|
|
||||||
// may turn others into NonTrimmed which we DO NOT re-enter here —
|
|
||||||
// Disc::patch handles those.
|
|
||||||
let Some((region_pos, region_size)) = map.next_with(0, mapfile::SectorStatus::NonTried)
|
let Some((region_pos, region_size)) = map.next_with(0, mapfile::SectorStatus::NonTried)
|
||||||
else {
|
else {
|
||||||
break;
|
break;
|
||||||
@@ -1337,7 +1326,6 @@ impl Disc {
|
|||||||
);
|
);
|
||||||
|
|
||||||
while pos < region_end {
|
while pos < region_end {
|
||||||
// Check halt
|
|
||||||
if let Some(ref h) = opts.halt {
|
if let Some(ref h) = opts.halt {
|
||||||
if h.load(std::sync::atomic::Ordering::Relaxed) {
|
if h.load(std::sync::atomic::Ordering::Relaxed) {
|
||||||
halt_requested = true;
|
halt_requested = true;
|
||||||
@@ -1345,12 +1333,7 @@ impl Disc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Decide read size: if use_single, read 1 sector; else use batch
|
let block_bytes = (region_end - pos).min(batch as u64 * 2048);
|
||||||
let block_bytes = if use_single {
|
|
||||||
(region_end - pos).min(2048) // 1 sector
|
|
||||||
} else {
|
|
||||||
(region_end - pos).min(batch as u64 * 2048)
|
|
||||||
};
|
|
||||||
let block_lba = (pos / 2048) as u32;
|
let block_lba = (pos / 2048) as u32;
|
||||||
let block_count = (block_bytes / 2048) as u16;
|
let block_count = (block_bytes / 2048) as u16;
|
||||||
let recovery = !opts.skip_on_error;
|
let recovery = !opts.skip_on_error;
|
||||||
@@ -1363,7 +1346,6 @@ impl Disc {
|
|||||||
);
|
);
|
||||||
|
|
||||||
if read_result.is_ok() {
|
if read_result.is_ok() {
|
||||||
// Good read — write to file
|
|
||||||
read_ok_count += 1;
|
read_ok_count += 1;
|
||||||
if opts.decrypt {
|
if opts.decrypt {
|
||||||
crate::decrypt::decrypt_sectors(
|
crate::decrypt::decrypt_sectors(
|
||||||
@@ -1379,17 +1361,7 @@ impl Disc {
|
|||||||
map.record(pos, block_bytes, mapfile::SectorStatus::Finished)
|
map.record(pos, block_bytes, mapfile::SectorStatus::Finished)
|
||||||
.map_err(|e| Error::IoError { source: e })?;
|
.map_err(|e| Error::IoError { source: e })?;
|
||||||
bytes_done = bytes_done.saturating_add(block_bytes);
|
bytes_done = bytes_done.saturating_add(block_bytes);
|
||||||
|
|
||||||
// If we're in single mode and hit threshold, try block mode again
|
|
||||||
if use_single {
|
|
||||||
consecutive_good = consecutive_good.saturating_add(1);
|
|
||||||
if consecutive_good >= BPT1_EXIT_THRESHOLD {
|
|
||||||
use_single = false;
|
|
||||||
consecutive_good = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else if !opts.skip_on_error {
|
} else if !opts.skip_on_error {
|
||||||
// Strict mode: abort
|
|
||||||
let (status, sense) = read_result
|
let (status, sense) = read_result
|
||||||
.as_ref()
|
.as_ref()
|
||||||
.err()
|
.err()
|
||||||
@@ -1400,71 +1372,49 @@ impl Disc {
|
|||||||
status: Some(status),
|
status: Some(status),
|
||||||
sense,
|
sense,
|
||||||
});
|
});
|
||||||
} else if use_single {
|
|
||||||
// Single sector mode — read failed, mark NonTrimmed
|
|
||||||
let err = read_result.err().unwrap();
|
|
||||||
read_err_count += 1;
|
|
||||||
if err.is_marginal_read()
|
|
||||||
|| err.scsi_sense().is_some_and(|s| s.is_medium_error())
|
|
||||||
{
|
|
||||||
// Disc-related error at 1 sector — bad sector, mark for pass 2
|
|
||||||
let is_medium = err.scsi_sense().is_some_and(|s| s.is_medium_error());
|
|
||||||
if is_medium {
|
|
||||||
tracing::warn!(
|
|
||||||
target: "freemkv::disc",
|
|
||||||
phase = "skip_bad_sector",
|
|
||||||
lba = block_lba,
|
|
||||||
error = %err,
|
|
||||||
"MEDIUM ERROR at 1 sector; marking NonTrimmed"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
// Zero fill, mark NonTrimmed
|
|
||||||
let zero = vec![0u8; block_bytes as usize];
|
|
||||||
file.seek(SeekFrom::Start(pos))
|
|
||||||
.map_err(|e| Error::IoError { source: e })?;
|
|
||||||
file.write_all(&zero)
|
|
||||||
.map_err(|e| Error::IoError { source: e })?;
|
|
||||||
map.record(pos, block_bytes, mapfile::SectorStatus::NonTrimmed)
|
|
||||||
.map_err(|e| Error::IoError { source: e })?;
|
|
||||||
bytes_done = bytes_done.saturating_add(block_bytes);
|
|
||||||
} else {
|
|
||||||
// Transport error at 1 sector — bail
|
|
||||||
return Err(err);
|
|
||||||
}
|
|
||||||
// Stay in single mode for next reads
|
|
||||||
use_single = true;
|
|
||||||
consecutive_good = 0;
|
|
||||||
} else {
|
} else {
|
||||||
// Batch mode failed — drop to 1 sector and retry
|
|
||||||
let err = read_result.err().unwrap();
|
let err = read_result.err().unwrap();
|
||||||
read_err_count += 1;
|
read_err_count += 1;
|
||||||
if err.is_marginal_read()
|
|
||||||
|| err.scsi_sense().is_some_and(|s| s.is_medium_error())
|
if err.is_scsi_transport_failure() {
|
||||||
{
|
tracing::warn!(
|
||||||
use_single = true;
|
target: "freemkv::disc",
|
||||||
consecutive_good = 0;
|
phase = "transport_failure",
|
||||||
if err.scsi_sense().is_some_and(|s| s.is_medium_error())
|
lba = block_lba,
|
||||||
&& opts
|
error = %err,
|
||||||
.halt
|
"transport failure (bridge crash); aborting copy"
|
||||||
.as_ref()
|
);
|
||||||
.is_none_or(|h| !h.load(std::sync::atomic::Ordering::Relaxed))
|
|
||||||
{
|
|
||||||
std::thread::sleep(std::time::Duration::from_secs(3));
|
|
||||||
}
|
|
||||||
continue;
|
|
||||||
} else {
|
|
||||||
// Non-recoverable transport error — bail
|
|
||||||
return Err(err);
|
return Err(err);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if !err.is_marginal_read()
|
||||||
|
&& err.scsi_sense().is_none_or(|s| !s.is_medium_error())
|
||||||
|
{
|
||||||
|
return Err(err);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ECC block failed — zero-fill, mark NonTrimmed, advance.
|
||||||
|
tracing::warn!(
|
||||||
|
target: "freemkv::disc",
|
||||||
|
phase = "skip_ecc_block",
|
||||||
|
lba = block_lba,
|
||||||
|
sectors = block_count,
|
||||||
|
error = %err,
|
||||||
|
"ECC block failed; marking NonTrimmed"
|
||||||
|
);
|
||||||
|
let zero = vec![0u8; block_bytes as usize];
|
||||||
|
file.seek(SeekFrom::Start(pos))
|
||||||
|
.map_err(|e| Error::IoError { source: e })?;
|
||||||
|
file.write_all(&zero)
|
||||||
|
.map_err(|e| Error::IoError { source: e })?;
|
||||||
|
map.record(pos, block_bytes, mapfile::SectorStatus::NonTrimmed)
|
||||||
|
.map_err(|e| Error::IoError { source: e })?;
|
||||||
|
bytes_done = bytes_done.saturating_add(block_bytes);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Advance position
|
|
||||||
pos += block_bytes;
|
pos += block_bytes;
|
||||||
|
|
||||||
// Progress callback throttling
|
|
||||||
iter_count += 1;
|
iter_count += 1;
|
||||||
|
|
||||||
// Throttled iter telemetry — every 100 inner iterations.
|
|
||||||
if iter_count - last_log_iter >= 100 {
|
if iter_count - last_log_iter >= 100 {
|
||||||
last_log_iter = iter_count;
|
last_log_iter = iter_count;
|
||||||
let stats = map.stats();
|
let stats = map.stats();
|
||||||
@@ -1532,15 +1482,11 @@ pub struct CopyOptions<'a> {
|
|||||||
/// Override the default block size in sectors. Callers should resolve
|
/// Override the default block size in sectors. Callers should resolve
|
||||||
/// this with `detect_max_batch_sectors(device_path)` for live drives.
|
/// this with `detect_max_batch_sectors(device_path)` for live drives.
|
||||||
/// When `None`, falls back to 32 sectors (64 KB BD ECC block) in
|
/// When `None`, falls back to 32 sectors (64 KB BD ECC block) in
|
||||||
/// `skip_forward` mode or `DEFAULT_BATCH_SECTORS=60` otherwise.
|
/// `skip_on_error` mode or `DEFAULT_BATCH_SECTORS=60` otherwise.
|
||||||
pub batch_sectors: Option<u16>,
|
pub batch_sectors: Option<u16>,
|
||||||
/// Zero-fill bad blocks in the ISO, mark them in the mapfile, continue.
|
/// Zero-fill bad blocks in the ISO, mark them NonTrimmed in the mapfile,
|
||||||
/// Uses fast reads (no drive-level recovery loop).
|
/// and continue. Failed ECC blocks are left for `Disc::patch` to recover.
|
||||||
pub skip_on_error: bool,
|
pub skip_on_error: bool,
|
||||||
/// ddrescue-style exponential skip-forward on block failure. Implies
|
|
||||||
/// `skip_on_error`. The skipped region is marked `non-trimmed` for later
|
|
||||||
/// trimming/scraping by `Disc::patch`.
|
|
||||||
pub skip_forward: bool,
|
|
||||||
/// Per-iteration progress reporter. v0.13.16 architecture: the library
|
/// Per-iteration progress reporter. v0.13.16 architecture: the library
|
||||||
/// emits a single `PassProgress` shape via the `Progress` trait;
|
/// emits a single `PassProgress` shape via the `Progress` trait;
|
||||||
/// consumers compute their own derived percentages / ETAs from it. No
|
/// consumers compute their own derived percentages / ETAs from it. No
|
||||||
@@ -1852,15 +1798,6 @@ const MAX_BATCH_SECTORS: u16 = 510;
|
|||||||
const DEFAULT_BATCH_SECTORS: u16 = 60;
|
const DEFAULT_BATCH_SECTORS: u16 = 60;
|
||||||
const MIN_BATCH_SECTORS: u16 = 3;
|
const MIN_BATCH_SECTORS: u16 = 3;
|
||||||
|
|
||||||
/// Number of consecutive good single-sector reads required to exit
|
|
||||||
/// `Single` mode (bpt=1) and return to `Block` mode (bpt=batch). 10 000
|
|
||||||
/// sectors ≈ 20 MB of clean data — long enough that we don't bounce in
|
|
||||||
/// and out of bpt=1 inside a sparse-bad cluster, short enough that we
|
|
||||||
/// don't waste much time reading clean territory at bpt=1 after the
|
|
||||||
/// damaged region ends. Tunable; calibrated from the 2026-04-26 BU40N
|
|
||||||
/// live test.
|
|
||||||
const BPT1_EXIT_THRESHOLD: u64 = 10_000;
|
|
||||||
|
|
||||||
/// Coarse damage tier for a finished or in-progress rip. Maps the
|
/// Coarse damage tier for a finished or in-progress rip. Maps the
|
||||||
/// observable signals (bad sector count + lost wallclock playback time)
|
/// observable signals (bad sector count + lost wallclock playback time)
|
||||||
/// onto a small discrete classification so UIs can render a colored badge
|
/// onto a small discrete classification so UIs can render a colored badge
|
||||||
|
|||||||
@@ -351,7 +351,7 @@ fn test_file_sector_reader_round_trip() {
|
|||||||
//
|
//
|
||||||
// Per RIP_DESIGN.md §2.1 + §3: Disc::copy must reach the end of the disc
|
// Per RIP_DESIGN.md §2.1 + §3: Disc::copy must reach the end of the disc
|
||||||
// regardless of how many reads fail. The only legitimate early exit is the
|
// regardless of how many reads fail. The only legitimate early exit is the
|
||||||
// halt flag. With `skip_on_error + skip_forward` and a reader that returns
|
// halt flag. With `skip_on_error` and a reader that returns
|
||||||
// Err for every read, Pass 1 must:
|
// Err for every read, Pass 1 must:
|
||||||
// - mark every sector NonTrimmed (so Pass 2 can retry them)
|
// - mark every sector NonTrimmed (so Pass 2 can retry them)
|
||||||
// - return cleanly (no panic, no hang)
|
// - return cleanly (no panic, no hang)
|
||||||
@@ -424,7 +424,7 @@ impl SectorReader for FailingSectorReader {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_disc_copy_completes_full_disc_with_failing_reader() {
|
fn test_disc_copy_completes_full_disc_with_failing_reader() {
|
||||||
// 1024 sectors = 2 MB. Reader fails every read. With skip_on_error +
|
// 1024 sectors = 2 MB. Reader fails every read. With skip_on_error +
|
||||||
// skip_forward, Pass 1 must mark every sector NonTrimmed and return
|
// skip_on_error, Pass 1 must mark every sector NonTrimmed and return
|
||||||
// cleanly — no bail, no hang.
|
// cleanly — no bail, no hang.
|
||||||
let capacity_sectors: u32 = 1024;
|
let capacity_sectors: u32 = 1024;
|
||||||
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
||||||
@@ -439,7 +439,7 @@ fn test_disc_copy_completes_full_disc_with_failing_reader() {
|
|||||||
let opts = CopyOptions {
|
let opts = CopyOptions {
|
||||||
decrypt: false,
|
decrypt: false,
|
||||||
skip_on_error: true,
|
skip_on_error: true,
|
||||||
skip_forward: true,
|
|
||||||
..Default::default()
|
..Default::default()
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -521,7 +521,7 @@ fn test_disc_copy_halts_promptly_on_failing_reader() {
|
|||||||
let opts = CopyOptions {
|
let opts = CopyOptions {
|
||||||
decrypt: false,
|
decrypt: false,
|
||||||
skip_on_error: true,
|
skip_on_error: true,
|
||||||
skip_forward: true,
|
|
||||||
halt: Some(halt),
|
halt: Some(halt),
|
||||||
..Default::default()
|
..Default::default()
|
||||||
};
|
};
|
||||||
@@ -553,16 +553,9 @@ fn test_disc_copy_halts_promptly_on_failing_reader() {
|
|||||||
|
|
||||||
// ── 8. Hysteresis recovers data the drive can read individually ──────────
|
// ── 8. Hysteresis recovers data the drive can read individually ──────────
|
||||||
//
|
//
|
||||||
// Empirically observed on the LG BU40N: in damaged regions the drive fails
|
// Pass 1 reads in batch (32 sectors = 1 ECC block). Failed blocks are marked
|
||||||
// multi-sector READ commands but reads each sector cleanly when asked one
|
// NonTrimmed for Pass 2 recovery. This test verifies that a reader where every
|
||||||
// at a time. Disc::copy's hysteresis state machine (0.13.22, replaces the
|
// multi-sector read fails produces all NonTrimmed output with zero bytes_good.
|
||||||
// 0.13.21 bisect-on-fail) drops to bpt=1 on the first multi-sector failure
|
|
||||||
// and stays there until BPT1_EXIT_THRESHOLD consecutive good single-sector
|
|
||||||
// reads, then returns to bpt=batch.
|
|
||||||
//
|
|
||||||
// Fixture: a reader that returns Err for any read with count > 1, and Ok
|
|
||||||
// for count == 1. The full disc must recover via the bpt=1 path with
|
|
||||||
// 100 % bytes_good outcome.
|
|
||||||
|
|
||||||
struct BlockSizeFailingReader {
|
struct BlockSizeFailingReader {
|
||||||
capacity: u32,
|
capacity: u32,
|
||||||
@@ -577,15 +570,11 @@ impl SectorReader for BlockSizeFailingReader {
|
|||||||
_recovery: bool,
|
_recovery: bool,
|
||||||
) -> Result<usize> {
|
) -> Result<usize> {
|
||||||
if count == 1 {
|
if count == 1 {
|
||||||
// Single-sector reads succeed — fill the sector with a marker.
|
|
||||||
for chunk in buf.chunks_mut(SECTOR_SIZE) {
|
for chunk in buf.chunks_mut(SECTOR_SIZE) {
|
||||||
chunk.fill((lba & 0xff) as u8);
|
chunk.fill((lba & 0xff) as u8);
|
||||||
}
|
}
|
||||||
Ok(buf.len())
|
Ok(buf.len())
|
||||||
} else {
|
} else {
|
||||||
// Multi-sector reads fail with the BU40N's signature: CHECK
|
|
||||||
// CONDITION + MEDIUM ERROR. The hysteresis must dispatch on
|
|
||||||
// this as marginal-read and drop to bpt=1.
|
|
||||||
Err(libfreemkv::error::Error::ScsiError {
|
Err(libfreemkv::error::Error::ScsiError {
|
||||||
opcode: libfreemkv::scsi::SCSI_READ_10,
|
opcode: libfreemkv::scsi::SCSI_READ_10,
|
||||||
status: libfreemkv::scsi::SCSI_STATUS_CHECK_CONDITION,
|
status: libfreemkv::scsi::SCSI_STATUS_CHECK_CONDITION,
|
||||||
@@ -604,13 +593,7 @@ impl SectorReader for BlockSizeFailingReader {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn test_disc_copy_hysteresis_recovers_via_single_sector_reads() {
|
fn test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed() {
|
||||||
// 256 sectors = 0.5 MB. Reader fails any multi-sector read but
|
|
||||||
// succeeds on bpt=1. The hysteresis path must drop to Single mode on
|
|
||||||
// the first multi-sector failure and recover every sector at bpt=1.
|
|
||||||
// Stays in Single mode until BPT1_EXIT_THRESHOLD reached (10 000
|
|
||||||
// sectors); since this disc is only 256 sectors we never re-enter
|
|
||||||
// Block mode, which is fine — every sector still recovers.
|
|
||||||
let capacity_sectors: u32 = 256;
|
let capacity_sectors: u32 = 256;
|
||||||
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
||||||
|
|
||||||
@@ -626,7 +609,6 @@ fn test_disc_copy_hysteresis_recovers_via_single_sector_reads() {
|
|||||||
let opts = CopyOptions {
|
let opts = CopyOptions {
|
||||||
decrypt: false,
|
decrypt: false,
|
||||||
skip_on_error: true,
|
skip_on_error: true,
|
||||||
skip_forward: true,
|
|
||||||
..Default::default()
|
..Default::default()
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -637,20 +619,17 @@ fn test_disc_copy_hysteresis_recovers_via_single_sector_reads() {
|
|||||||
let _ = std::fs::remove_file(&iso_path);
|
let _ = std::fs::remove_file(&iso_path);
|
||||||
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
||||||
|
|
||||||
// Bisect must recover every sector — the drive could read each one
|
|
||||||
// individually, and our algorithm must descend to that.
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
result.bytes_good, total_bytes,
|
result.bytes_good, 0,
|
||||||
"bisect-on-fail must recover every sector via single-sector reads. \
|
"Pass 1 should have 0 bytes_good when all batch reads fail. Got {} of {}",
|
||||||
Got bytes_good={} of total {}",
|
|
||||||
result.bytes_good, total_bytes
|
result.bytes_good, total_bytes
|
||||||
);
|
);
|
||||||
assert_eq!(
|
assert!(
|
||||||
result.bytes_pending, 0,
|
result.bytes_pending > 0,
|
||||||
"no sectors should be left NonTrimmed after a successful bisect"
|
"all sectors should be NonTrimmed pending Pass 2"
|
||||||
);
|
);
|
||||||
assert!(
|
assert!(
|
||||||
result.complete,
|
!result.complete,
|
||||||
"complete=true expected when every sector recovered"
|
"complete=false when sectors remain NonTrimmed"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user