pq/libfreemkv
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.13.23 — stop discarding the drive's SCSI sense data

Browse Source 
Through the entire 0.13.x line, every CHECK CONDITION reply from the
drive (the standard way SCSI tells you why a sector failed) was being
collapsed into a synthetic status=0xFF, sense_key=0 transport-wedge
sentinel and the actual sense data was thrown away. Confirmed live on
the BU40N reading Dune 2 on 2026-04-27: drive returned host_status=0,
driver_status=8, status=2, exec_elapsed_ms=1416 on every bad sector
— a clean CHECK CONDITION carrying full sense data — and Disc::copy
was bailing on it as if the bridge had wedged.

Root cause: scsi/linux.rs's wedge check was
  `host_status != 0 || driver_status != 0`
SG's DRIVER_SENSE bit (0x08) is set on every CHECK CONDITION reply
just to flag "sense buffer is populated" — it's not a transport
failure on its own. Pre-fix we conflated the two and silently lost
every drive-reported error reason. macOS and Windows backends had
the same shape: they extracted sense_key only, dropping ASC/ASCQ.

API restructure (clean separation):

  Error::ScsiError {
      opcode: u8,
      status: u8,                  // 0xFF = synthetic transport-failure
      sense: Option<ScsiSense>,    // None ⇔ no sense delivered
  }

  pub struct ScsiSense { sense_key: u8, asc: u8, ascq: u8 }
  impl ScsiSense {
      pub fn is_marginal(&self) -> bool       // keys 0/1/3/B
      pub fn is_medium_error(&self) -> bool
      pub fn is_hardware_error(&self) -> bool
      pub fn is_unit_attention(&self) -> bool
      pub fn is_data_protect(&self) -> bool
      pub fn is_not_ready(&self) -> bool
      pub fn is_illegal_request(&self) -> bool
      pub fn is_aborted_command(&self) -> bool
  }

  impl Error {
      pub fn scsi_sense(&self) -> Option<&ScsiSense>
      pub fn is_scsi_transport_failure(&self) -> bool
      pub fn is_marginal_read(&self) -> bool
  }

SCSI protocol constants (SCSI_STATUS_*, SENSE_KEY_*) moved from
error.rs to scsi/mod.rs where they belong alongside SCSI_INQUIRY,
SCSI_READ_10, etc. parse_sense replaces parse_sense_key (returns the
full triple, not just the key); inline tests now exercise ASC/ASCQ
extraction at the right offsets for both descriptor (0x72/0x73) and
fixed (0x70/0x71) sense formats.

Disc::copy + Disc::patch sense-aware dispatch:
  - marginal sense (MEDIUM ERROR / ABORTED COMMAND / RECOVERED ERROR
    / NO SENSE) → engage hysteresis (Block→Single, bpt=1)
  - non-marginal sense (HARDWARE / DATA PROTECT / UNIT ATTENTION /
    NOT READY / ILLEGAL REQUEST / transport failure / kernel
    IoError) → bail with full sense info preserved; caller (autorip)
    surfaces "physical replug" / "drive failing" / "media changed"

  Pre-fix: every CHECK CONDITION → 0xFF synthetic → Disc::copy bailed
  → bytes_good froze at the bad zone. The hysteresis from v0.13.22
  was correct but never got to run. This release unblocks it.

  Disc::patch's wedged_threshold (50 consecutive failures) stays as
  defense-in-depth for chains of marginal failures; a single
  non-marginal sense now short-circuits it.

New phase=bail trace event records the bail reason with the sense
triple. phase=transport_err remains for genuine bridge wedges /
kernel timeouts; phase=scsi_err carries the parsed sense_key, asc,
ascq for drive-reported errors.

All 350 tests pass. Clippy clean across all targets.
This commit is contained in:
matthew
2026-04-26 19:06:19 -07:00
parent 0ecf7c7c46
commit 0cb497b431
13 changed files with 761 additions and 219 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/scsi/linux.rs
+37 -23
View File
@@ -243,10 +243,19 @@ impl ScsiTransport for SgIoTransport {
/// Errors we surface to caller (any of these = command failed):
///
/// - ioctl returned -1 → `Error::IoError` (kernel-level failure)
/// - `hdr.host_status` != 0 → `Error::ScsiError` with status=0xFF
/// (transport-level: timeout, bridge wedge, etc.)
/// - `hdr.driver_status` != 0 → `Error::ScsiError` with status=0xFF
/// - `hdr.status` != 0 → `Error::ScsiError` with parsed sense key
/// - `hdr.host_status` != 0 OR `(hdr.driver_status & ~DRIVER_SENSE)` != 0
/// → `Error::ScsiError { status: 0xFF, sense_key: 0, asc: 0, ascq: 0 }`
/// (real transport-layer failure: kernel timeout, bridge wedge, bus error)
/// - `hdr.status` != 0 (typically `0x02` CHECK CONDITION) →
/// `Error::ScsiError { status, sense_key, asc, ascq }` carrying the
/// drive's full SPC-4 sense triple. Callers route on
/// `is_medium_error()`, `is_unit_attention()`, etc.
///
/// Note: SG's `DRIVER_SENSE` (0x08) bit indicates *sense data is
/// attached* — it's set on every CHECK CONDITION reply. It is **not**
/// a transport failure; pre-0.13.23 we conflated it with one and
/// silently lost every drive-reported error reason. The mask in the
/// transport-error check below is the fix.
///
/// Caller's `data` buffer is mutated only on success; partial
/// transfers are reported via `bytes_transferred = data.len() - resid`.
@@ -272,8 +281,8 @@ impl ScsiTransport for SgIoTransport {
if data.len() > u32::MAX as usize {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
@@ -323,7 +332,16 @@ impl ScsiTransport for SgIoTransport {
// bus error). `hdr.status` may still be zero — the SCSI device
// never got to send a status byte. Surface as 0xFF so callers
// (e.g. `drive_has_disc`) can detect the wedge signature.
if hdr.host_status != 0 || hdr.driver_status != 0 {
//
// 0.13.23: mask out `DRIVER_SENSE` (0x08) before treating
// `driver_status` as a transport failure. That bit is set on
// *every* CHECK CONDITION reply just to flag "sense data is
// attached in `sbp`" — it's not an error of its own. Pre-fix
// we collapsed every drive-reported error into a synthetic
// 0xFF wedge signature and discarded the sense data, which
// killed the rip's classification logic on damaged discs.
let driver_status_real = hdr.driver_status & !super::DRIVER_SENSE;
if hdr.host_status != 0 || driver_status_real != 0 {
tracing::trace!(
target: "freemkv::scsi",
phase = "transport_err",
@@ -336,28 +354,32 @@ impl ScsiTransport for SgIoTransport {
);
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
// SCSI-level failure: device responded, returned non-zero status.
// Parse sense key for the caller.
// SCSI-level failure: device responded, returned non-zero status
// (typically 0x02 CHECK CONDITION). Parse the full SPC-4 sense
// triple so callers can route on `ScsiSense::is_medium_error()`
// etc.
if hdr.status != 0 {
let sense_key = super::parse_sense_key(&sense, hdr.sb_len_wr);
let parsed = super::parse_sense(&sense, hdr.sb_len_wr);
tracing::trace!(
target: "freemkv::scsi",
phase = "scsi_err",
opcode = opcode,
status = hdr.status,
sense_key,
sense_key = parsed.sense_key,
asc = parsed.asc,
ascq = parsed.ascq,
exec_elapsed_ms,
"SCSI status non-zero"
);
return Err(Error::ScsiError {
opcode: cdb[0],
status: hdr.status,
sense_key,
sense: Some(parsed),
});
}
@@ -394,11 +416,6 @@ impl ScsiTransport for SgIoTransport {
/// Stored in `/sys/class/scsi_generic/sgN/device/type` as ASCII decimal.
const SCSI_TYPE_OPTICAL: &str = "5";
/// SCSI sense key 2 = "NOT READY". Sub-codes distinguish "medium not present"
/// (no disc) from other not-ready states (loading, etc.); for poll-loop
/// purposes any sense-key 2 means "no disc to act on".
const SENSE_KEY_NOT_READY: u8 = 2;
/// Maximum sg index probed in the fallback path when sysfs is unavailable.
/// Linux assigns `/dev/sgN` sequentially per host adapter; 16 covers any
/// realistic homelab (typical PERC + USB optical = ≤8 nodes).
@@ -547,10 +564,7 @@ pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
Err(Error::ScsiError {
sense_key: SENSE_KEY_NOT_READY,
..
}) => Ok(false),
Err(ref e) if e.scsi_sense().is_some_and(|s| s.is_not_ready()) => Ok(false),
Err(e) => Err(e),
}
}
src/scsi/macos.rs
+13 -10
View File
@@ -337,13 +337,11 @@ pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
Err(Error::ScsiError { sense_key, .. }) if sense_key == K_SENSE_KEY_NOT_READY => Ok(false),
Err(ref e) if e.scsi_sense().is_some_and(|s| s.is_not_ready()) => Ok(false),
Err(e) => Err(e),
}
}
const K_SENSE_KEY_NOT_READY: u8 = 2;
impl Drop for MacScsiTransport {
fn drop(&mut self) {
if self.device_iface.is_null() {
@@ -377,8 +375,8 @@ impl ScsiTransport for MacScsiTransport {
if task.is_null() {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
@@ -449,20 +447,25 @@ impl ScsiTransport for MacScsiTransport {
// the kernel mid-layer has already done what it can.
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
if task_status != K_SCSI_TASK_STATUS_GOOD as u32 {
// IOKit doesn't surface a "bytes written into sense buffer"
// count the way SG_IO does — pass the buffer's full length
// and let parse_sense_key inspect byte 0's response code.
let sense_key = super::parse_sense_key(&sense, sense.len() as u8);
// and let parse_sense inspect byte 0's response code to pick
// descriptor-vs-fixed format.
//
// 0.13.23: carry the full SPC-4 sense triple in
// `Error::ScsiError::sense` so callers can route on
// `ScsiSense::is_medium_error()` etc.
let parsed = super::parse_sense(&sense, sense.len() as u8);
return Err(Error::ScsiError {
opcode: cdb[0],
status: task_status as u8,
sense_key,
sense: Some(parsed),
});
}
src/scsi/mod.rs
+198 -24
View File
@@ -75,36 +75,207 @@ pub(crate) const READ_TIMEOUT_MS: u32 = 10_000;
/// (`DEF_TIMEOUT = 60000`).
pub(crate) const READ_RECOVERY_TIMEOUT_MS: u32 = 60_000;
// ── Sense-key parsing ───────────────────────────────────────────────────────
// ── SCSI status bytes (SPC-4 §4.5.5) ────────────────────────────────────────
/// Extract the SPC-4 sense key from a sense buffer.
/// Status byte 0x00 — `GOOD`. Command completed successfully.
pub const SCSI_STATUS_GOOD: u8 = 0x00;
/// Status byte 0x02 — `CHECK CONDITION`. Drive completed the command
/// reply and attached sense data describing the failure.
pub const SCSI_STATUS_CHECK_CONDITION: u8 = 0x02;
/// libfreemkv-synthesised sentinel: the transport never delivered a
/// SCSI status byte (kernel timeout, USB bridge wedge, IOKit service
/// failure). Distinct from any drive-returned value. Carriers
/// [`Error::ScsiError`] with `sense = None`.
pub const SCSI_STATUS_TRANSPORT_FAILURE: u8 = 0xFF;
// ── SPC-4 sense keys (§4.5.6 Table 28) ─────────────────────────────────────
//
// Broad failure category returned in a CHECK CONDITION reply's sense data.
// Names match the SCSI spec; predicate methods on [`ScsiSense`] (e.g.
// `is_medium_error`, `is_unit_attention`) read more fluently than raw
// constant comparisons at call sites.
pub const SENSE_KEY_NO_SENSE: u8 = 0x00;
pub const SENSE_KEY_RECOVERED_ERROR: u8 = 0x01;
pub const SENSE_KEY_NOT_READY: u8 = 0x02;
pub const SENSE_KEY_MEDIUM_ERROR: u8 = 0x03;
pub const SENSE_KEY_HARDWARE_ERROR: u8 = 0x04;
pub const SENSE_KEY_ILLEGAL_REQUEST: u8 = 0x05;
pub const SENSE_KEY_UNIT_ATTENTION: u8 = 0x06;
pub const SENSE_KEY_DATA_PROTECT: u8 = 0x07;
pub const SENSE_KEY_BLANK_CHECK: u8 = 0x08;
pub const SENSE_KEY_ABORTED_COMMAND: u8 = 0x0B;
// ── Sense parsing ───────────────────────────────────────────────────────────
/// Decoded SPC-4 sense triple — the precise reason a SCSI command failed.
///
/// Handles both response-code formats:
/// - Descriptor format (0x72 / 0x73): sense key in the low nibble of byte 1.
/// - Fixed format (0x70 / 0x71): sense key in the low nibble of byte 2.
/// Returned by [`parse_sense`] and embedded inside [`Error::ScsiError`]
/// (`sense: Option<ScsiSense>`). Predicate methods (`is_medium_error`,
/// `is_unit_attention`, `is_marginal`, …) read more fluently at call
/// sites than raw `sense_key` comparisons.
///
/// `sb_len_wr` is the number of bytes the transport actually wrote into
/// `sense`. When < 3 (or `sense.len() < 3`) we can't safely read either
/// the format byte or the key byte — return 0 (NO SENSE) per SPC-4 §4.5.3.
///
/// Pure function. Same parse runs on every platform backend so
/// callers don't have to special-case Linux SG_IO vs macOS IOKit vs
/// Windows SPTI sense layouts.
pub(crate) fn parse_sense_key(sense: &[u8], sb_len_wr: u8) -> u8 {
if (sb_len_wr as usize) < 3 || sense.len() < 3 {
return 0;
/// `Default::default()` and the [`ScsiSense::NONE`] constant both
/// produce the all-zero "no sense info" triple. Per SPC-4 §4.5.3, an
/// empty sense buffer is reported as NO SENSE (key 0); use the constant
/// for explicit intent at construction sites.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct ScsiSense {
/// Sense key — broad failure category (SPC-4 §4.5.6 Table 28).
/// See the `SENSE_KEY_*` constants for named values.
pub sense_key: u8,
/// Additional Sense Code — narrows the cause within a sense key
/// (SPC-4 §4.5.6 Table 29). E.g. `0x11` = UNRECOVERED READ ERROR.
pub asc: u8,
/// Additional Sense Code Qualifier — finest-grain disambiguation.
/// E.g. `0x05` (with `asc=0x11`) = L-EC UNCORRECTABLE.
pub ascq: u8,
}
impl ScsiSense {
/// Sense reply with all-zero fields — explicit "no sense info"
/// constructor for sites where `Default::default()` would be opaque.
pub const NONE: ScsiSense = ScsiSense {
sense_key: 0,
asc: 0,
ascq: 0,
};
/// `true` when the sense key indicates a *marginal-read* failure —
/// the kind of error where the same read at smaller granularity
/// (or a brief retry) sometimes succeeds:
///
/// - `MEDIUM ERROR` (3) — canonical bad-sector signal
/// - `ABORTED COMMAND` (B) — transient; retry usually works
/// - `RECOVERED ERROR` (1) / `NO SENSE` (0) — drive is healthy and
/// either recovered the data or has no specific fault to report
///
/// `false` for HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION, NOT
/// READY, ILLEGAL REQUEST, BLANK CHECK, and any unknown key. Used
/// by [`Error::is_marginal_read`] / `Disc::copy`'s hysteresis
/// dispatch.
pub fn is_marginal(&self) -> bool {
matches!(
self.sense_key,
SENSE_KEY_NO_SENSE
| SENSE_KEY_RECOVERED_ERROR
| SENSE_KEY_MEDIUM_ERROR
| SENSE_KEY_ABORTED_COMMAND
)
}
let response_code = sense[0] & 0x7F;
if response_code == 0x72 || response_code == 0x73 {
sense[1] & 0x0F
} else {
// Fixed format (0x70/0x71) and any unknown code fall through here;
// SPC-4 says implementations MUST tolerate unknown response codes
// and treat them as fixed — matches what reference projects do.
sense[2] & 0x0F
/// `true` if `sense_key == MEDIUM ERROR (3)` — canonical "bad sector"
/// signal from the drive.
pub fn is_medium_error(&self) -> bool {
self.sense_key == SENSE_KEY_MEDIUM_ERROR
}
/// `true` if `sense_key == HARDWARE ERROR (4)` — drive itself is
/// failing. Not recoverable by retry.
pub fn is_hardware_error(&self) -> bool {
self.sense_key == SENSE_KEY_HARDWARE_ERROR
}
/// `true` if `sense_key == NOT READY (2)` — medium not present /
/// drive becoming ready / etc.
pub fn is_not_ready(&self) -> bool {
self.sense_key == SENSE_KEY_NOT_READY
}
/// `true` if `sense_key == UNIT ATTENTION (6)` — disc/drive state
/// changed since the prior command (media inserted/removed,
/// power-on reset, parameters changed). Caller should rescan rather
/// than retry the read.
pub fn is_unit_attention(&self) -> bool {
self.sense_key == SENSE_KEY_UNIT_ATTENTION
}
/// `true` if `sense_key == DATA PROTECT (7)` — read blocked by
/// AACS / region / write-protect. Retry won't help.
pub fn is_data_protect(&self) -> bool {
self.sense_key == SENSE_KEY_DATA_PROTECT
}
/// `true` if `sense_key == ILLEGAL REQUEST (5)` — typically a bug
/// in the CDB we sent (LBA out of range, reserved bit, etc.). Don't
/// retry.
pub fn is_illegal_request(&self) -> bool {
self.sense_key == SENSE_KEY_ILLEGAL_REQUEST
}
/// `true` if `sense_key == ABORTED COMMAND (B)` — transient; one
/// retry is usually safe.
pub fn is_aborted_command(&self) -> bool {
self.sense_key == SENSE_KEY_ABORTED_COMMAND
}
}
/// Decode an SPC-4 sense buffer into the structured triple
/// `(sense_key, asc, ascq)`.
///
/// Handles both response-code formats SPC-4 mandates:
///
/// - **Descriptor format** (response code `0x72` / `0x73`):
/// - sense key = `sense[1] & 0x0F`
/// - asc = `sense[2]`
/// - ascq = `sense[3]`
/// - **Fixed format** (response code `0x70` / `0x71` and any unknown
/// code per SPC-4 §4.5.3):
/// - sense key = `sense[2] & 0x0F`
/// - asc = `sense[12]`
/// - ascq = `sense[13]`
///
/// `sb_len_wr` is the number of bytes the transport actually wrote into
/// `sense`. When the buffer is too short for the relevant fields we
/// return [`ScsiSense::NONE`] for the missing pieces rather than reading
/// uninitialised memory. The minimum useful sense reply per SPC-4 is 8
/// bytes (descriptor) or 14 bytes (fixed, to reach ASC/ASCQ at offsets
/// 12/13).
///
/// Pure function — same parse on every platform backend (Linux SG_IO,
/// macOS IOKit, Windows SPTI) so a regression here would silently
/// mis-route SCSI errors on all three OSes simultaneously.
pub(crate) fn parse_sense(sense: &[u8], sb_len_wr: u8) -> ScsiSense {
let n = (sb_len_wr as usize).min(sense.len());
if n < 3 {
return ScsiSense::NONE;
}
let response_code = sense[0] & 0x7F;
let descriptor = response_code == 0x72 || response_code == 0x73;
if descriptor {
// Descriptor format: key/asc/ascq are at fixed offsets 1/2/3.
let asc = if n >= 3 { sense[2] } else { 0 };
let ascq = if n >= 4 { sense[3] } else { 0 };
ScsiSense {
sense_key: sense[1] & 0x0F,
asc,
ascq,
}
} else {
// Fixed format: key at byte 2, ASC/ASCQ at bytes 12/13.
let asc = if n >= 13 { sense[12] } else { 0 };
let ascq = if n >= 14 { sense[13] } else { 0 };
ScsiSense {
sense_key: sense[2] & 0x0F,
asc,
ascq,
}
}
}
// ── SG_IO driver_status bits ────────────────────────────────────────────────
/// `DRIVER_SENSE` (0x08) — bit set in `driver_status` to indicate that
/// sense data was attached to a CHECK CONDITION reply. **Not** a transport
/// failure on its own. Mask this off before deciding whether `driver_status`
/// represents a real bus/host problem.
///
/// Used by Linux SG_IO (`sg_io_hdr.driver_status`); macOS IOKit and
/// Windows SPTI carry the equivalent signal in different fields and
/// don't need the same masking — the misclassification was Linux-only.
#[cfg(target_os = "linux")]
pub(crate) const DRIVER_SENSE: u16 = 0x08;
// ── Types ───────────────────────────────────────────────────────────────────
#[derive(Debug, Clone, Copy, PartialEq)]
@@ -396,7 +567,10 @@ mod parse_sense_tests {
//! same helper runs on every platform backend so a regression here
//! would silently miscategorize SCSI errors on Linux, macOS, and
//! Windows simultaneously.
use super::parse_sense_key;
use super::parse_sense;
fn parse_sense_key(sense: &[u8], sb_len_wr: u8) -> u8 {
parse_sense(sense, sb_len_wr).sense_key
}
/// Helper: build a 32-byte sense buffer whose first three bytes are
/// the given prefix; the rest are zeroes (sense data area).
src/scsi/windows.rs
+11 -7
View File
@@ -228,7 +228,7 @@ pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
Err(Error::ScsiError { sense_key: 2, .. }) => Ok(false),
Err(ref e) if e.scsi_sense().is_some_and(|s| s.is_not_ready()) => Ok(false),
Err(e) => Err(e),
}
}
@@ -308,21 +308,25 @@ impl ScsiTransport for SptiTransport {
// surfaces the failure to UX.
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
if sptwb.spt.ScsiStatus != 0 {
// SPTI doesn't surface a "bytes written into sense buffer"
// count separate from SenseInfoLength (input). Pass the full
// K_SENSE_SIZE; parse_sense_key keys off byte 0's response
// code to handle descriptor (0x72/0x73) vs fixed (0x70/0x71).
let sense_key = super::parse_sense_key(&sptwb.sense, K_SENSE_SIZE as u8);
// K_SENSE_SIZE; parse_sense keys off byte 0's response code
// to handle descriptor (0x72/0x73) vs fixed (0x70/0x71).
//
// 0.13.23: carry the full SPC-4 sense triple in
// `Error::ScsiError::sense` so callers can route on
// `ScsiSense::is_medium_error()` etc.
let parsed = super::parse_sense(&sptwb.sense, K_SENSE_SIZE as u8);
return Err(Error::ScsiError {
opcode: cdb[0],
status: sptwb.spt.ScsiStatus,
sense_key,
sense: Some(parsed),
});
}