Compare commits

...
155 Commits
Author SHA1 Message Date
MattJackson 385c9f094c v0.18.3: canonical_title_order — main feature first on branching UHDs 2026-05-09 20:08:56 -07:00
MattJackson 9f2a13739d Disc title order: main feature first on branching UHDs
Disc::titles previously sorted purely by duration_secs descending,
which puts a play-all virtual playlist at index 0 on UHDs that ship
one. Such playlists reference the same source clips multiple times
for seamless alternate-angle / alternate-ending playback and report
inflated duration AND inflated size_bytes that exceeds the disc's
physical capacity.

Concrete observed case (The Amateur 2025 4K UHD, 58.5 GB BD-100):
  Title 1 — 00020.mpls — 4h13m — 92.4 GB — 253 clips  ← impossible
  Title 2 — 00800.mpls — 2h02m — 57.2 GB — 1 clip      ← the movie

92.4 GB > 58.5 GB capacity is proof of clip double-counting. With
the duration-only sort, freemkv -t 1 / disc.titles.first() / autorip's
main-feature picker all selected the 4-hour composite instead of the
2-hour movie.

New canonical_title_order:
  1. Real titles (size_bytes <= capacity_bytes) before virtual
     composites — capacity gate is hard physical truth.
  2. Among real titles, fewer clips first (1-clip wins as the
     canonical main feature; multi-clip is either chapter-stitched
     or composite).
  3. Tiebreak on longer duration first.

Behaviour:
- Non-branching discs: unchanged. The longest 1-clip title is
  already the movie.
- Branching UHDs: virtual composite drops to the back, the real
  movie surfaces at index 0.

Comparator exposed as Disc::canonical_title_order for downstream
consumers that need the same logic on custom title sets.

Three regression tests (disc::tests::canonical_order_*):
- pushes_oversize_play_all_behind_real_main (The Amateur)
- preserves_natural_ranking_on_normal_disc
- fewer_clips_wins_tiebreak
2026-05-09 19:57:07 -07:00
MattJackson d9ce69bc9d v0.18.2: fix AACS nav-file scramble + sweep progress non-regression
decrypt::decrypt_sectors now restores chunks when decrypt_unit_full's
TS-sync verification fails, preventing 0.18.1's silent corruption of
MPLS/CLPI navigation files when DecryptingSectorSource decorates the
sweep reader. Fixes E6009 NoStreams on info iso:// for AACS-encrypted
UHDs ripped without --raw.

Disc::sweep progress takes max(snapshot.bytes_good, bytes_done) so
the user-visible counter never regresses below what the producer has
already sent.
2026-05-09 17:19:47 -07:00
MattJackson 7cd2c937ed 0.18.1 docs: refresh README, CHANGELOG, and docs/ for the trait split
The library's public-facing docs were sitting on the 0.17 trait
surface — Disc::copy, pes::Stream, SectorReader, etc. — even though
all in-tree callers migrated in 0.18 rounds 1-3. With 0.18.1 about
to ship, a user copy-pasting the README sample from crates.io would
have hit a compile error.

This commit is purely doc-side:

- README.md: Quick Start rewritten onto Disc::sweep + Disc::patch
  with caller-orchestrated multipass; Streams table footnote and
  Architecture row reference FrameSource / FrameSink.
- CHANGELOG.md: 0.18.1 entry describing the redesign — primitives,
  trait splits, deprecations (kept alive through 0.18.x, deletion
  target 0.18.2), throughput numbers.
- docs/{rip-recovery,api-design,architecture,disc-to-rip,
  drive-access,udf}.md: every Disc::copy / pes::Stream /
  SectorReader reference updated to the 0.18 trait surface.
- FEATURES.md: deleted (8+ versions stale; capabilities live in
  README.md and CHANGELOG.md now, matching the workspace-top
  FEATURES.md removal in 84acd65).
- examples/iso_dump.rs: verified compiles against 0.18.1.

No code changes.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 12:13:51 -07:00
MattJackson 107524da5e 0.18.1: bump version 2026-05-09 11:38:24 -07:00
MattJackson 055a3c5276 0.18 round 3: make Disc::sweep + Disc::patch pub (was pub(crate))
Round 3 step 1: lift the visibility on the two flat rip-phase verbs
so consumers (autorip + freemkv CLI) can call them directly instead
of going through Disc::copy's multipass dispatcher. Also lift their
option/outcome types and re-export at crate root.

- fn sweep -> pub fn sweep (with rustdoc explaining its role)
- fn patch -> pub fn patch (ditto)
- pub(crate) struct SweepOptions -> pub struct SweepOptions
- pub(crate) struct PatchOpts -> pub struct PatchOptions (renamed
  for consistency — both are 'Options')
- pub(crate) struct PatchOutcome -> pub struct PatchOutcome
- libfreemkv::{SweepOptions, PatchOptions, PatchOutcome} re-exports
  at crate root.

Disc::copy still exists and still calls Disc::sweep / Disc::patch
through the now-private sweep_internal / patch_internal wrappers.
Migration of the two autorip callers + the freemkv CLI's
disc_to_iso to direct sweep/patch is a follow-up; once those land
Disc::copy + CopyOptions + CopyResult delete in the same commit.

See (internal)/memory/0_18_redesign.md and
0_18_round3_migration_audit.md.

Single contributor: MattJackson.
2026-05-09 11:19:46 -07:00
MattJackson 50b04790ce 0.18 round 2: thread Halt through DiscStream construction
# Conflicts:
#	src/mux/disc.rs
2026-05-09 11:04:07 -07:00
MattJackson a00b2c884b 0.18 round 2: adopt DecryptingSectorSource decorator at sweep + patch + DiscStream 2026-05-09 11:03:43 -07:00
MattJackson 44e2c64d7e 0.18 round 2 (Halt threading): DiscStream accepts Halt at construction
Adds a Halt field to DiscStream, populated via the new
`with_halt(halt)` builder. The internal recovery / fill_extents
loops check `halt.is_cancelled()` directly. The existing
`set_halt(Arc<AtomicBool>)` method stays through the deprecation
window for callers (autorip mux) that haven't migrated; marked
#[deprecated] with a pointer to the constructor-time path.

Both signals are unified inside DiscStream: either Halt or the
legacy Arc<AtomicBool> triggers cancellation, so callers can mix
during the deprecation window without breaking stop behaviour.

See (internal)/memory/0_18_redesign.md and
0_18_round3_migration_audit.md.

Single contributor: MattJackson.
2026-05-09 10:58:54 -07:00
MattJackson d6535b8f57 0.18 round 2 (decrypt dedup): adopt DecryptingSectorSource at the two
existing call sites — sweep producer and DiscStream demux

Round 1 shipped the DecryptingSectorSource decorator
(libfreemkv/src/sector/decrypting.rs) but the existing decrypt
sites kept calling crate::decrypt::decrypt_sectors inline. This
commit migrates both:

- Disc::sweep (disc/mod.rs): producer wraps the input reader
  in DecryptingSectorSource::new(reader, keys) before the read loop.
  The inline decrypt_sectors call goes away — read_sectors yields
  plaintext directly.

- DiscStream (mux/disc.rs): constructor wraps the underlying
  Box<dyn SectorReader> in DecryptingSectorSource so the internal
  fill_extents / read path sees plaintext bytes. The DecryptKeys
  field stays on DiscStream for metadata-side use; it just no
  longer drives decryption.

Disc::patch carried the same inline decrypt step at three call
sites (main read, backtrack read, non-NOT_READY retry read). All
three migrated onto the same wrapping for a single audit surface.

Two small support changes carry the migration without touching
the round-1 decorator shape:
- sector/mod.rs gains specific SectorSource impls for
  &mut dyn SectorReader and Box<dyn SectorReader>, mirroring
  std's Read forwarding pattern. Generic blankets would conflict
  with the existing SectorReader → SectorSource blanket under the
  orphan rule (downstream could impl SectorReader for &mut U), so
  the impls are scoped to the dyn-trait shape we actually consume.
- sector/decrypting.rs gains DecryptingSectorSource::set_keys so
  DiscStream::set_raw() can flip the wrapped reader to a
  DecryptKeys::None pass-through without rebuilding the decorator
  (which would require moving the inner Box out from behind &mut self).

After this commit, grep `decrypt_sectors` in src/ shows the
function definition, its single use inside DecryptingSectorSource,
plus comments only. One audit surface for AACS / CSS / passthrough
correctness.

Behaviour-preserving: same plaintext bytes flow through; the only
difference is which type owns the decrypt step.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:49:53 -07:00
MattJackson 90ab00ed45 0.18 round 2: cargo fmt after sweep+patch+DiscStream merge
Auto-fmt nit on the multi-line map.flush() expression that landed
when the sweep + patch + DiscStream-FrameSource branches were merged
together. No semantic change.
2026-05-09 10:36:40 -07:00
MattJackson 6bfd7dfd13 0.18 round 2: confirm DiscStream as FrameSource via blanket impl 2026-05-09 10:32:56 -07:00
MattJackson b9a7f601d4 0.18 round 2: refactor Disc::patch onto Pipeline + PatchSink
# Conflicts:
#	src/io/mod.rs
#	src/io/pipeline.rs
2026-05-09 10:32:51 -07:00
MattJackson cf0a61f8b4 0.18 round 2: refactor Disc::sweep onto Pipeline + SweepSink (delete sweep_pipeline.rs) 2026-05-09 10:32:05 -07:00
MattJackson f98f07b2d3 0.18 round 2: refactor Disc::sweep onto Pipeline + SweepSink
Sweep was the original producer/consumer split that motivated the
generic Pipeline primitive (round 1, commit 198268b). Now that
Pipeline + Sink exist, sweep stops shipping its own bespoke
threading.

- New SweepSink: Sink<WorkItem> impl in src/disc/sweep.rs. Owns
  WritebackFile + Mapfile + ProgressSnapshot back-channel. apply()
  carries the file-write + mapfile.record per WorkItem; close()
  drains writeback, fsyncs, flushes mapfile.
- Disc::sweep: constructs SweepSink, calls Pipeline::spawn_named
  (so the consumer thread keeps showing up as
  freemkv-sweep-consumer), sends WorkItems, calls pipe.finish().
  The producer-side ReadCtx state machine, decrypt, set_speed,
  halt — all unchanged.
- Pipeline gains spawn_named(name, depth, sink) so callers can
  preserve identifiable thread names without the primitive baking
  one in. Also adds Pipeline::try_send for the throttled
  StatsRequest path that must not block the producer.
- Deleted src/disc/sweep_pipeline.rs entirely. WorkItem,
  ProgressSnapshot, ConsumerSummary moved into disc/sweep.rs as
  module-private types. WorkItem::Finish dropped — dropping the
  channel is the end-of-stream signal Pipeline already uses.

Behaviour-preserving: the sweep algorithm, mapfile invariants,
back-pressure via channel depth (DEFAULT_PIPELINE_DEPTH = 4) all
match the 0.17.13 implementation. New synthetic regression test
(sweep_pipeline_full_good_100_batches) exercises ~100 batches of
clean reads end-to-end through the new Pipeline path and verifies
bytes_good and ISO file size.

See (internal)/memory/0_18_redesign.md.
2026-05-09 10:31:05 -07:00
MattJackson b53454fa09 0.18 round 2: refactor Disc::patch onto Pipeline + PatchSink
Patch was strictly serial (per-sector recovery: read → seek+write
→ mapfile.record → next). Lifting the write+record onto a consumer
thread lets the drive issue the next per-sector retry while the
previous block's recovered bytes are being committed — small but
real win on damaged discs with many bad sectors, and uniform with
sweep's threading model.

- New PatchSink: Sink<PatchItem> impl in src/disc/patch.rs. Owns
  WritebackFile + Mapfile. apply() seeks+writes recovered bytes
  and records mapfile state per item; close() runs sync_all and
  mapfile.flush.
- Channel depth: WRITE_THROUGH_DEPTH (1). Patch wants minimum
  buffering — back-pressure should kick in immediately so the
  drive's per-sector retry budget isn't ahead of the writer.
- Disc::patch: keeps every existing recovery decision on the
  producer (reverse walk, damage-window skip, NOT_READY pauses,
  bridge-degradation handling, wedge exit, range watchdog).
  WritebackFile ownership moves to the sink.

Behaviour-preserving: per-sector single-shot read budget unchanged
(BU40N+Initio bridge wedge concern still respected); recovery
algorithm bit-identical.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:28:14 -07:00
MattJackson f28b6ee6d9 0.18 round 2: re-export Pipeline + Sink + Flow at the crate root
Round 2 #1 landed `Pipeline` / `Sink` / `Flow` / `DEFAULT_PIPELINE_DEPTH`
in `crate::io::pipeline` but only re-exported them through
`crate::io` (which is `pub(crate)`), so no out-of-tree consumer could
reach them. autorip's round 2 #2 (lifting the mux loop onto Pipeline +
MuxSink) is the first such consumer; surface the primitives at the
crate root for ergonomic access.

No behaviour change — the items themselves are unchanged from round
2 #1; this is just `pub use` plumbing.

Single contributor: MattJackson.
2026-05-09 10:22:44 -07:00
MattJackson d9d778fa64 0.18 round 2: confirm DiscStream as FrameSource
Mirror of the FrameSink concrete migrations slice (f52e4c5) on the
read side. DiscStream is the only meaningful source impl in tree;
all the mux/* impls are sinks.

The round-1 blanket impl<T: Stream + Send> FrameSource for T
already covers DiscStream if it's Send. This slice:

- Audits DiscStream's interior types for Send (its Box<dyn
  SectorReader> already requires Send via the trait's super-bound;
  verify nothing else interior breaks Send).
- Adds a synthetic-input test that constructs Box<dyn FrameSource>
  over a DiscStream, reads frames through the trait object, and
  exercises info() / headers_ready() / codec_private().
- (Conditional) Adds a direct FrameSource impl on DiscStream only
  if call-site ergonomics demand it; otherwise relies on the
  blanket.

No caller migrated. mux::resolve::input still returns
Box<dyn Stream>; autorip / CLI consumers still call Stream::read.
Per-caller migration is a later slice.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:13:14 -07:00
MattJackson bbfb887a35 0.18 round 1+2 integration fixes
Two clippy issues surfaced when round 1 polish + round 2 FrameSink
migrations both landed on libfreemkv main:

- src/halt.rs: clippy::new_without_default fires when a public new()
  exists without Default. The polish pass dropped the derive thinking
  it was redundant — clippy disagrees, so add a manual impl that
  forwards to new(). Doc-comment notes why both exist.

- src/disc/read_error.rs:372: pre-existing
  assert_eq!(.., true) trips clippy::bool_assert_comparison. Pre-0.18
  precommits passed because that lint sat outside the gate; the
  round-2 commits brought enough new clippy surface that it now
  shows up. Trivial cleanup: assert!(...) instead of assert_eq!.

Single contributor: MattJackson.
2026-05-09 10:00:06 -07:00
MattJackson 9884346c24 0.18 round 2: FrameSink concrete migrations for mux/* sinks 2026-05-09 09:53:12 -07:00
MattJackson 766b7c6636 0.18 round 1 polish: address libfreemkv code-review findings 2026-05-09 09:53:06 -07:00
MattJackson 925c30686b 0.18 round 1 polish: address libfreemkv code-review findings
Applies must-fix + in-scope should-fix items from the round-1 code
review:

- M1: FileSectorSource::open takes &Path (was &str — non-UTF-8 panic)
- M2: drop FileSectorSource's BufReader (defeated by absolute seeks)
- M3: WritebackFile Drop impl finalises the writeback pipeline
- M4: Pipeline::finish preserves panic payload in error message
- M5: pes::Stream is left without a : Send supertrait — concrete
  in-tree impls (MkvStream, M2tsStream) hold Box<dyn Read> /
  Box<dyn Write> trait objects that aren't Send, so the simple
  trait tightening would cascade into a wider Send audit. Per the
  review's escape clause the FrameSource blanket impl keeps its
  T: Send bound and the constraint is documented loudly there.
- S6: document Pipeline::send post-Flow::Stop semantics
- S9: truncate stale Stream docs (E9001/E9000 was runtime-only)
- S10: document WritebackPipeline.fd lifetime invariant
- S11: pub use pes::Stream as PesStream to disambiguate from
  disc::Stream codec enum at crate root
- S12: rename DEFAULT_DEPTH → DEFAULT_PIPELINE_DEPTH; add
  WRITE_THROUGH_DEPTH constant
- N14: drop Halt's Default derive (redundant with Halt::new)
- N17: Pipeline::spawn propagates thread-spawn error instead of expect
- N19: deprecation since = "0.18.0" (was "0.18.0-dev", non-conventional)
- N21: rename Apply enum to Flow

Deferred to follow-up commits: SectorReader/SectorSource competition
(migration commit), WritebackFile::create/open orphans (migration
commit), AACS round-trip test (design doc defers), various nits.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 09:52:25 -07:00
MattJackson f52e4c5d22 0.18 round 2: add FrameSink impls to concrete mux sinks
Per-impl migration of MkvStream / M2tsStream / NetworkStream /
NullStream / StdioStream from the deprecated pes::Stream trait
to the typed pes::FrameSink trait. Both impls coexist during
the 0.18 deprecation window — the existing Stream impls are
unchanged.

The FrameSink::finish signature differs (Box<Self> vs &mut self),
which is why this couldn't be a blanket impl. Each migration
re-borrows the box and delegates to the underlying Stream::finish
body.

FrameSink: Send forced two struct fields (M2tsStream's boxed
Write/Read, MkvStream's boxed WriteSeek/Read) to gain `+ Send`
bounds — minimum surface needed to make the Send-bounded trait
impl-able. mux::resolve::output's local Box<dyn WriteSeek>
construction picks up the same `+ Send`. tests/streams.rs's
shared `stream.write/.finish/.info/.read` calls were
disambiguated to `PesStream::*` to resolve the now-multiple
candidates from coexisting trait impls.

Caller migration (mux::resolve::output return type, autorip,
CLI) is a later slice. This commit only adds new impls; nothing
removed.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 09:49:18 -07:00
MattJackson 8d67790d2e 0.18: thread WritebackFile rename through FileSectorSink
The SectorSource/Sink agent and the WritebackFile-rename agent both
branched from main concurrently; the sector branch wrote against the
0.17 Writer name and only the rename branch knew about WritebackFile.
This integration commit reconciles the two: FileSectorSink::create /
::open / the inner-field type all use WritebackFile directly, and the
module-level + struct-level docs are corrected.
2026-05-09 09:17:39 -07:00
MattJackson 246a439990 0.18: FrameSource/FrameSink trait split (deprecate Stream) 2026-05-09 09:13:39 -07:00
MattJackson bd19041648 0.18: SectorSource/SectorSink trait split + DecryptingSectorSource decorator 2026-05-09 09:13:34 -07:00
MattJackson 198268b725 0.18: add crate::io::Pipeline + Sink trait
# Conflicts:
#	src/io/mod.rs
2026-05-09 09:13:30 -07:00
MattJackson 31424c8203 0.18: add crate::halt::Halt cancellation token 2026-05-09 09:13:07 -07:00
MattJackson c747ecc589 0.18: rename crate::io::Writer → WritebackFile 2026-05-09 09:12:59 -07:00
MattJackson 283a561c12 0.18 primitive: SectorSource/SectorSink trait split + DecryptingSectorSource
Splits the unidirectional read trait from a (planned) write trait at
the sector level, eliminating runtime "wrong direction" potential.
Keeps SectorReader alive as a pre-deprecation alias via blanket impl
so existing callers compile unchanged through the migration window.

Adds DecryptingSectorSource decorator: wrap any SectorSource in this
to get plaintext sectors out. Replaces the duplicate decrypt code
paths in sweep_pipeline and DiscStream (those migrations are
follow-up commits).

The formal #[deprecated] attribute on SectorReader is held back to a
follow-up commit because internal call sites in disc/, udf/, mux/,
and verify/ still go through the legacy trait, and the CI gauntlet
treats deprecation lints as errors. Behavioural intent — "this trait
is going away" — is documented on the trait itself.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 09:04:16 -07:00
MattJackson 2667d68675 0.18 primitive: FrameSource/FrameSink trait split (deprecate Stream)
Splits the bidirectional pes::Stream into one-direction traits so
calling read() on a write-only sink is a compile error instead of
runtime E9001. Keeps Stream alive as deprecated through 0.18 with a
blanket FrameSource impl so existing concrete types compile unchanged.

FrameSink can't be blanket-impl'd from Stream (different finish
signature), so concrete impls migrate per-type in a follow-up.

Concrete `impl pes::Stream for X` blocks in mux/* and the existing
tests gain a one-line `#[allow(deprecated)]` to keep `-D warnings`
clean during the deprecation window — no behavior changes.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 08:57:48 -07:00
MattJackson 57f2c22e30 0.18 primitive: crate::io::Pipeline + Sink trait
Generic bounded producer/consumer pipeline. The same shape applies to
sweep, patch, and mux today via three near-duplicate implementations
(or, in patch's and mux's case, no implementation at all). 0.18
collapses them onto one primitive. See
(internal)/memory/0_18_redesign.md for full context.

Single contributor: MattJackson.
2026-05-09 08:56:31 -07:00
MattJackson 5f3545d244 0.18 primitive: rename crate::io::Writer → WritebackFile
The type's job is the bounded-cache writeback pipeline (sync_file_range
+ posix_fadvise(DONTNEED)) — not generic writing. The 0.17 name was
ambiguous; reading `Writer::new(file)` gave no hint about what was
special. New name makes the role obvious at every call site.

Adds `WritebackFile::create(path)` and `WritebackFile::open(path)`
constructors so callers don't have to assemble a `File` first.

No alias kept; this is a clean 0.18 rename. See
(internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 08:53:17 -07:00
MattJackson 6e17ef0859 0.18 primitive: crate::halt::Halt cancellation token
One-bit cooperative cancellation flag. Replaces ad-hoc Arc<AtomicBool>
patterns scattered across libfreemkv (DiscStream::set_halt) and the
HALT_FLAGS global registry in autorip. See
(internal)/memory/0_18_redesign.md.

Single contributor: MattJackson (no attribution trailers).
2026-05-09 08:50:13 -07:00
MattJackson 6ec97af104 v0.17.13: thread Writer through patch + mux for big-write consistency
The bounded-cache writeback wrapper (crate::io::Writer) was added in
0.17.10 and wired into Disc::sweep in 0.17.11, but the other two
paths in the crate that write large amounts of data sequentially —
Disc::patch and the MKV/M2TS mux — were still operating on raw
std::fs::File. That meant the dirty-page burst pathology the wrapper
exists to prevent could still bite on slow / network-attached staging
during recovery and mux phases.

This release plugs those gaps:

- Disc::patch (disc/mod.rs:1981) now wraps the reopened ISO in
  Writer before any seek / write. sync_all on Writer cleanly drains
  the in-flight chunk before the existing fsync.
- mux/resolve.rs MKV and M2TS branches wrap the output File in
  Writer underneath BufWriter. UHD MKV mux routinely produces 70+ GB
  of sequential output; the page cache no longer absorbs that as a
  single hot blast on slow targets.

Mapfile, log, settings, history, and stream-pipeline byte buffers
remain unchanged: those are either small one-shot writes (where
the wrapper has zero benefit and adds a stream_position syscall) or
already use bounded persistence (mapfile time-batched in 0.17.12).
The principle: any path that writes substantial sequential data to
a single file uses Writer; trivial writes don't.
2026-05-09 06:32:08 -07:00
MattJackson 3a6c1aa5a3 v0.17.12: mapfile time-batched persistence — unblock NFS staging
Pre-0.17.12 every Mapfile::record() persisted the full mapfile via
tempfile-create + write + atomic-rename. On local LVM that's
microseconds; on NFS each rename is multiple RPCs through the
unraid user-share fuse layer, dragging a Black Mass UHD rip from
~11 MB/s on local to ~1.5 MB/s on NFS — the mapfile path alone burned
multiple seconds of wall time per real-world second of work.

Mapfile now batches the rename to once per second:

- record() always updates in-memory state and stats; only fires
  write_to_disk when last_flushed.elapsed() >= FLUSH_INTERVAL (1 s).
- New flush() API forces a persist; called by sweep_pipeline's
  consumer at end-of-sweep and by Disc::patch at end-of-patch,
  after the file's sync_all.
- Drop impl best-effort flushes so an early-return / unwind doesn't
  silently lose pending state.

Crash-safety changes from "lose at most one block" to "lose at most
1 s of recorded progress" — the ISO file's payload bytes are unaffected;
only the mapfile's authority over which sectors are already-good is at
risk, and a resume re-reads anything Pass 1 had already covered.

Measured on the BU40N test bed against Black Mass UHD inner zone:
- NFS staging: 1.5 MB/s → 16.48 MB/s (10.9× recovery)
- Local LVM staging: 11.09 MB/s → 11.83 MB/s (+6.7 % bonus)

Internal round_trip_load test now flushes before reading back from
disk. External patch / copy tests are unaffected: patch and
sweep_pipeline flush at completion before returning.
2026-05-08 23:00:52 -07:00
MattJackson 1ba3264747 v0.17.11: sweep producer/consumer split — overlap drive read with file write
Pre-0.17.11 sweep ran strictly serialised: SCSI read → decrypt → seek
+ write → mapfile.record → next read. Drive idled for the post-read
work; throughput capped at the sum of both costs. On a healthy disc
that's ~7-12 ms read + ~5-15 ms write/record per 64 KB batch, limiting
sustained throughput to ~10-12 MB/s on the test bed (BU40N + UHD inner
zone), well below the ~14-16 MB/s drive ceiling.

Decouples them: producer thread (caller's) owns SectorReader +
read_error state + decrypt + set_speed + halt; consumer thread (one
spawn) owns Writer + Mapfile, receives WorkItem messages, applies
file write + mapfile record. Bounded mpsc::sync_channel(4) gives
natural back-pressure. While the consumer writes batch N, the
producer is already reading batch N+1 — steady-state throughput is
now bound by the slower of the two pipelines (drive on healthy
discs), not their sum.

Side effects:
- Bisect path now decrypts. Pre-0.17.11 the bisect inner loop wrote
  raw cyphertext for single-sector recoveries on encrypted discs —
  quiet correctness bug exercised only by batch-fail-then-
  bisect-succeed on encrypted media. New producer-side decrypt
  covers main + bisect success paths uniformly.
- All read_ctx state stays single-threaded on producer (damage
  window, jump multiplier, etc.). No locking added.
- Mapfile remains single-writer on consumer. No locking.
- Halt latency: producer breaks loop, sends Finish, consumer drains
  ≤4 in-flight items + sync_all. ~1 batch (~12 ms) typical.
- BU40N + Initio bridge wedge concern unchanged: still single SCSI
  command in flight, error-path timing identical, no new retries.

New module: src/disc/sweep_pipeline.rs (WorkItem, ProgressSnapshot,
ConsumerInputs, spawn_consumer, consumer_loop, helpers). Public API
unchanged — Disc::copy / CopyOptions / CopyResult identical.

Patch (Pass N) is NOT changed; it's bound by drive recovery time, not
the read/write serialisation.
2026-05-08 21:32:11 -07:00
MattJackson ae2909fe8d v0.17.10: bounded-cache writeback pipeline for big sequential writes
Pass 1 sweep speed on a healthy disc previously dipped from ~15 MB/s
to ~1 MB/s every ~30 s on a host with default Linux dirty-page
settings. Empirical cause: the kernel's vm.dirty_ratio (~20% of RAM)
lets hundreds of MB of dirty pages accumulate, then bursts a flush at
99% disk utilisation that blocks app writes for ~1 s. Confirmed on
the BU40N test bed — dirty pages grew 112 → 563 MB between bursts;
lowering vm.dirty_bytes to 64 MB at the host sysctl level eliminated
the dips. Shipping the equivalent inside libfreemkv so users do not
need to tune the host kernel.

- New crate::io::Writer: drop-in File wrapper (impl Write + Seek).
  Wraps a per-platform WritebackPipeline that on Linux schedules
  sync_file_range(WRITE) + lagging sync_file_range(WAIT_AFTER) +
  posix_fadvise(DONTNEED) in 32 MB chunks, bounding dirty cache at
  ~64 MB. macOS and Windows ship a no-op stub.
- Disc::sweep wraps its output File in Writer. Loop body unchanged.
- Module is purpose-built so any large sequential output (patch,
  mux) can adopt the same wrapper as a one-line change later.
2026-05-08 19:54:25 -07:00
MattJackson 6a3d19a453 v0.17.7: sync release — no functional changes
Version bump to keep the four freemkv crates at unified versioning
after autorip's v0.17.6 + v0.17.7 work today. No libfreemkv code
changes; republished to crates.io so downstream consumers stay
aligned on the latest patch version.
2026-05-08 16:20:28 -07:00
MattJackson a35596d2d1 v0.17.5: Pass N kernel block-device fallback + per-range fixes
Direct-SATA BU40N + Dune Part Two UHD live testing exposed that the
v0.17.3 single-shot SCSI READ path matched 0/22 of the small bad-
sector LBAs that dd if=/dev/sr0 recovers on the same drive. This
release closes that gap and fixes adjacent bugs silently capping
recovery.

- /dev/sr0 pread fallback in Drive::read (Linux only): on SCSI READ
  Err, fall back to posix_fadvise(DONTNEED) + pread() against the
  corresponding block device. Kernel sr_mod runs ~5 internal retries
  with no per-attempt mid-layer escalation overhead — the mechanism
  behind dd's recovery advantage. End-to-end byte verification
  confirms the fallback path returns real disc data.

- Disc::patch per-range watchdog fix: MAX_RANGE_SECS was breaking
  'outer (one slow range killed the entire patch). Now skips to the
  next range. Pre-fix patch died after 4 sectors of range 1 of 47.

- Per-sector range budget: range_budget = sectors × 25 s, capped at
  1800 s. Replaces the flat 180 s/range that was unfair to medium
  ranges and pointlessly generous to single-sector ones.

- consecutive_failures resets per range. The wedge-exit detector is
  for stuck-on-one-range, not many-small-ranges-with-one-fail-each.

- Reverted inline 5× retry experiment (was hurting: each retry paid
  kernel SCSI escalation overhead). Restored READ_RECOVERY_TIMEOUT_MS
  to 60 s. The kernel-auto-retry pattern is now provided by sr0
  fallback.

Empirical: pass 1 recovered 94.6 MB / 11 s of main title (33 sr0
saves). Pass 2 added 0.6 MB. Remaining ~233 MB on the test disc
appears physically unrecoverable on this hardware.
2026-05-08 12:58:48 -07:00
MattJackson 318f654fed v0.17.3: fix scsi_recovery.rs Linux compile (missing Duration import)
tests/scsi_recovery.rs:
- Add `use std::time::Duration` inside both `#[cfg(target_os = "linux")]`
  blocks. Locally on macOS the linux blocks are cfg-out so the missing
  import was invisible to precommit on macOS.
- Bug pre-dated this branch but only surfaced when v0.17.2 release CI
  ran the test compile on Linux.

Cargo.toml: 0.17.2 -> 0.17.3.
2026-05-07 19:16:21 -07:00
MattJackson 5e82ef65de v0.17.2: bump version (skip 0.17.1 — stray tag exists)
Cargo.toml: 0.17.1 -> 0.17.2. Functionally identical to the prior
commit; 0.17.1 was never published to crates.io but a tag exists on
the remote pointing at an unrelated commit. Bumping past it.
2026-05-07 19:10:39 -07:00
MattJackson 8534607329 v0.17.1: cache priming, NonTrimmed marking, decrypt regression test
src/disc/mod.rs:
- Cache priming (3-sector lookback) before patch's single-sector reads.
  Drive read-ahead pulls in adjacent pages so the target may already be
  cached when we ask for it. Throwaway reads — failures here don't
  update mapfile state.
- When patch hits skip-limit on a range, leave remaining sectors
  NonTrimmed instead of marking Unreadable. We never tried to read those
  sectors, so don't give them terminal status — drive state evolves
  between passes (cache, mechanical settle), and a later pass may
  succeed.

tests/pass_n_patch_fix.rs:
- New regression test for the decrypt key inversion bug at
  src/disc/mod.rs:1938-1942. Asserts decrypt_sectors is invoked with
  the correct key when opts.decrypt=true.

tests/pass_n_size_aware_skip.rs:
- rustfmt-only changes.

Cargo.toml: 0.17.0 -> 0.17.1.
2026-05-07 19:09:17 -07:00
MattJackson 97e1a4cad3 unified read-error handler + pass N size-aware skip
New disc/read_error.rs as the single entry point all read failures flow
through. Handler classifies the error, updates the in-flight context
(damage window, retry budgets, jump multiplier), and returns a
ReadAction the caller dispatches on. Pass 1 (sweep) refactored to use
it; ~340 lines of nested if/else collapsed into ~120 lines of action
dispatch. Adding a new error class = one match arm. Logging is in one
place. Bisect inner failures don't poison the damage window. Jump
multiplier capped at 64 (max 1 GB jump for batch=32 — observed prior
unbounded behavior produce a single 56 GB jump on a wedged drive).

Pass N (patch) damage_skip is now size-aware: each skip is capped at
range_remaining/4 rather than the absolute MB-scale escalation. The
old logic could leap over a 100-sector bad range that hides a 50-sector
good middle; size-aware convergence finds the good middles instead.

Tests in tests/pass_n_size_aware_skip.rs exercise the size-aware skip
against synthetic patterns (25-bad/50-good/25-bad and three good
middles in a row) and prove ≥98% of good middles are recovered.
Existing test test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed
updated to reflect that MEDIUM_ERROR now triggers single-sector
bisect (which the BlockSizeFailingReader succeeds at).
2026-05-07 09:08:43 -07:00
MattJackson 3f6bf33de5 Format code 2026-05-04 20:06:10 -07:00
MattJackson 359301e8fc Pass 1 transport failure recovery without user intervention 2026-05-04 20:02:01 -07:00
MattJackson bfa527162a 0.17.0: unwrap safety fix, patch pass algorithm, clippy compliance
- Fix unwrap in disc/mod.rs sweep() hot path using pattern matching
- Patch pass excludes Unreadable sectors from work list
- Expose bytes_bad_in_title for accurate UI reporting
- All 256 tests pass, cargo clippy clean with -D warnings
2026-05-04 09:42:07 -07:00
MattJackson 45defd47f3 fix patch pass: exclude Unreadable from work list; expose bytes_bad_in_title; clippy 1.86 fixes
- patch(): only process NonTrimmed + NonScraped ranges (Unreadable=terminal, NonTried=not-yet-swept)
- bytes_bad_in_title: pub fn for autorip main-movie lost_ms computation
- Clippy 1.86: saturating_sub, unused assignments, unused variable
- fmt: rustfmt formatting
2026-05-03 16:35:06 -07:00
MattJackson fe8933dad2 v0.16.3: i18n damage display
- Add rip.damage_lost, rip.damage_lost_movie, rip.damage_lost_simple, rip.damage_none keys
- Update all 6 locale files (en, es, fr, de, it, pt, nl)
- CLI progress shows 'Xs lost (Y in movie)' instead of 'Xs unreadable'
- Perfect rips show '0s' instead of '0s unreadable'
2026-04-30 20:51:02 -07:00
MattJackson 0a3c1b7b70 fix: scsi_recovery test variable shadowing and missing imports for Linux CI 2026-04-30 19:57:16 -07:00
MattJackson 2bcdc97341 fix: add missing Duration import in scsi_recovery test 2026-04-30 19:52:53 -07:00
MattJackson 4863e9c545 v0.16.2: sticky escalation in patch, title-aware damage display, PASS1/PASSN constant naming 2026-04-30 19:50:08 -07:00
MattJackson 03db038dd9 docs: update project docs, CHANGELOG, drive-access.md, rip-recovery.md for v0.16.x 2026-04-30 15:43:58 -07:00
MattJackson 95001ba2c3 v0.16.1: bump version 2026-04-30 15:23:44 -07:00
MattJackson a1f4dff6f6 v0.16.0: IOKit registry-based drive enumeration, BSD name → IOBDServices matching, reverse patch default 2026-04-30 15:17:20 -07:00
MattJackson 7dd5001d45 v0.15.1: fix damage-jump detection (window=16, threshold=12%), fix dispatch covers_disc check, sweep resume from mapfile
- DAMAGE_WINDOW 50→16, DAMAGE_THRESHOLD_PCT 25→12: triggers on 2nd scattered failure
- Previous 50/25% was too diluted by good reads between sparse failures
- copy() dispatch checks mapfile total_size == disc capacity_bytes (covers_disc)
- NonTried regions → sweep with resume=true (preserves mapfile)
- Only NonTrimmed/NonScraped/Unreadable → patch
- sweep_internal takes resume param: true when dispatching from existing mapfile
- Verified: Pass 1 from 30-100% completes in ~20 min with correct jumps through 3 damage zones
2026-04-30 13:15:03 -07:00
MattJackson 67fe93c0b8 v0.15.0: multipass CopyOptions, auto-detect sweep vs patch, speed control on damage zone entry/exit
- CopyOptions: replace resume/skip_on_error/batch_sectors with single multipass bool
- Disc::copy() auto-detects pass from mapfile state: no mapfile or NonTried → sweep, only NonTrimmed/Unreadable → patch
- Fix bug where mapfile with NonTried regions incorrectly dispatched to patch mode
- SectorReader::set_speed() default method, Drive impl sends SET CD SPEED
- On damage zone entry: set_speed(0x0000) for better error recovery
- On damage zone exit (50 consecutive good): set_speed(0xFFFF) to restore max speed
- Disc::mapfile_for() returns /tmp/<name>.mapfile for null:// output
- patch_internal/sweep_internal as private helpers, CopyResult gains recovered_this_pass
2026-04-30 11:31:31 -07:00
MattJackson 5c4f575c46 v0.14.0: bump version 2026-04-30 08:45:32 -07:00
MattJackson e2cb7a78bb v0.13.46: damage-jump algorithm replaces probe, bridge degradation detection, ecc_sectors() 2026-04-29 22:08:52 -07:00
MattJackson a6f1bd19bc v0.13.45: multipass adaptive probe, NOT_READY retry, progress bytes_bad_total
- Adaptive probe algorithm in Disc::copy skip_on_error mode: after 4
  consecutive errors, probe 1 sector at 256x batch (8 MB) ahead. If
  good, zero-fill gap, mark NonTrimmed, jump. Clears bad zones in
  seconds instead of hours.
- NOT_READY sense key (0x02) now retries up to 3x with 3s pause before
  marking NonTrimmed. BU40N returns NOT READY for bad sectors, not
  MEDIUM ERROR.
- PassProgress struct gains bytes_bad_total field for consumer-side
  bad/retryable byte counts.
- Mapfile header version string fix: no longer duplicates 'libfreemkv v'
  prefix on each write.
- Structured sense_key/asc/ascq logging in copy error path.
2026-04-29 19:41:34 -07:00
MattJackson 9e3d0f1383 v0.13.44: macOS raw CDB transport via IOKit exclusive access
macOS SCSI transport rewritten from hybrid MMC+pread to single-path
raw CDB dispatch through SCSITaskDeviceInterface. All CDBs (INQUIRY,
READ, REPORT KEY, etc.) now go through ExecuteTaskSync — 1:1 with
the Linux SG_IO backend.

Key changes:
- New macos_shim.c: diskutil unmount → find IOBDServices →
  ObtainExclusiveAccess → raw CDB dispatch. Eliminates Rust-side
  IOKit COM vtable complexity.
- build.rs compiles macos_shim.c via cc into static lib
- macos.rs simplified to three FFI calls (open/close/execute)
- disc/mod.rs: graduated batch restore after errors, skip-ahead
  through bad zones, configurable error pause
2026-04-29 15:59:35 -07:00
MattJackson 1e6eb0698d v0.13.43: Pass 1 transport-failure recovery loop 2026-04-29 08:53:56 -07:00
MattJackson bd744171e4 v0.13.42: transport failure skips instead of aborting copy 2026-04-29 07:46:42 -07:00
MattJackson 7de9d4d42c v0.13.41: debug logging for sector-0 regression 2026-04-29 07:09:46 -07:00
MattJackson 9ac0c2fcdf v0.13.40 2026-04-28 21:50:32 -07:00
MattJackson 3aa27d5d1b fix CI: Rust 2024 match ergonomics - remove ref binding modifier 2026-04-28 21:47:19 -07:00
MattJackson 5083f70cff fix CI: import ScsiTransport trait in integration test 2026-04-28 21:45:34 -07:00
MattJackson 477cf7ee0e fix CI: pub scsi::linux module and SgIoTransport fields for integration tests 2026-04-28 21:43:14 -07:00
MattJackson 0b079e27ce fix CI: make scsi::linux pub(crate) for integration test access 2026-04-28 21:39:50 -07:00
MattJackson 5f69fd2a44 update CHANGELOG and README for v0.13.39 2026-04-28 21:36:37 -07:00
MattJackson befe6ef69b remove (internal) references from public code 2026-04-28 21:32:15 -07:00
MattJackson 29491d6eaf v0.13.39 2026-04-28 21:19:51 -07:00
MattJackson 2e602f7e37 fix scsi_recovery test: dereference status in pattern match 2026-04-28 21:17:11 -07:00
MattJackson a126f31b70 v0.13.38 2026-04-28 21:12:05 -07:00
MattJackson 6ff10df83a fix scsi_recovery test compilation (old Drive::read signature) 2026-04-28 21:11:37 -07:00
MattJackson 52b8522a75 v0.13.37: Pass 1 is pure ECC-block sweep — read 32 sectors, fail → skip, no single-sector reads 2026-04-28 21:08:58 -07:00
MattJackson 13eb4331b6 v0.13.36: remove unused reset() 2026-04-28 17:57:25 -07:00
MattJackson 1a3f7ce676 remove unused reset() 2026-04-28 17:56:42 -07:00
MattJackson e1a1f730be v0.13.35: pause 3s after first MEDIUM ERROR before retry to prevent USB bridge crash 2026-04-28 17:54:12 -07:00
MattJackson 646c22ae93 v0.13.34: open() just opens, drive_has_disc() is side-effect-free direct TUR, enumerate_sg_names skips unreadable type files, Disc::copy read_err_count fix 2026-04-28 16:00:00 -07:00
MattJackson 2f6092a686 scsi/linux: fix as_bytes() → as_encoded_bytes() for OsStrExt
- as_bytes() requires OsStrExt import which is platform-specific
- as_encoded_bytes() is the portable API available on all platforms
- Fixes CI failure on Linux (Ubuntu) in GitHub Actions
2026-04-28 14:59:55 -07:00
MattJackson e5a90a6567 disc: fix hysteresis, SgIoTransport recovery, wallclock budget, patch instrumentation
- Fix hysteresis: use_single now correctly forces block_count=1 (was computed before the check)
- Fix SgIoTransport: spawn close+reopen in background thread on transport error
- Fix autorip: rip_disc() spawns wallclock watcher thread, caps entire rip at max(disc_runtime, 1h)
- Fix Disc::patch: add unreadable_count counter + tracing instrumentation
- Simplify Disc::copy() error handling per RIP_DESIGN.md §2.1
2026-04-28 14:48:16 -07:00
MattJackson afce1031d5 disc: add eprintln debug 2026-04-27 23:46:34 -07:00
MattJackson 2c0bff1303 disc: add debug logging for error type 2026-04-27 22:39:03 -07:00
MattJackson 2067ebe485 disc: add warn trace for MEDIUM ERROR skip 2026-04-27 21:38:03 -07:00
MattJackson 2ce4c20221 disc: skip MEDIUM ERROR sectors instead of bailing
0.13.28 - when drive returns MEDIUM ERROR (bad sector), skip the sector
and continue instead of retrying or bailing. Write zero-fill, mark as
Unreadable for pass 2+ recovery.

Closes: (internal)#20260427
2026-04-27 20:44:59 -07:00
MattJackson da3004b120 disc: DiscRead carries SCSI status/sense
Fix extract_scsi_context() and Error::scsi_sense() to handle Error::DiscRead
in addition to Error::ScsiError, so is_marginal_read() works for DiscRead
errors and disc::copy() can properly route MEDIUM ERROR as a marginal
(bad sector) instead of bailing.

Closes: (internal)#20260427
2026-04-27 18:38:57 -07:00
MattJackson b5ed97801b libfreemkv: extend DiscRead with SCSI status/sense for 30% wedge diagnostics 2026-04-27 16:47:27 -07:00
MattJackson ae76aaf0fa v0.13.24 — MapStats: split bytes_pending into nontried / retryable
bytes_pending was an opaque aggregate of NonTried + NonTrimmed +
NonScraped. UIs that wanted a "will retry in Pass 2-N" bucket were
stuck showing the entire unread disc as Maybe at pct=0.

Adds two granular fields to MapStats:

  bytes_nontried   — Pass 1 hasn't read these yet
  bytes_retryable  — NonTrimmed + NonScraped, Pass 2-N will retry

bytes_pending stays for back-compat (= bytes_nontried + bytes_retryable).

Also picks up the cargo fmt --check lint that's been red on main CI
since v0.13.18 (rustfmt fold differences on a few long format-string
layouts; functional no-op).
2026-04-26 19:28:35 -07:00
MattJackson 2cd4fbead7 v0.13.23 — stop discarding the drive's SCSI sense data
Through the entire 0.13.x line, every CHECK CONDITION reply from the
drive (the standard way SCSI tells you why a sector failed) was being
collapsed into a synthetic status=0xFF, sense_key=0 transport-wedge
sentinel and the actual sense data was thrown away. Confirmed live on
the BU40N reading Dune 2 on 2026-04-27: drive returned host_status=0,
driver_status=8, status=2, exec_elapsed_ms=1416 on every bad sector
— a clean CHECK CONDITION carrying full sense data — and Disc::copy
was bailing on it as if the bridge had wedged.

Root cause: scsi/linux.rs's wedge check was
  `host_status != 0 || driver_status != 0`
SG's DRIVER_SENSE bit (0x08) is set on every CHECK CONDITION reply
just to flag "sense buffer is populated" — it's not a transport
failure on its own. Pre-fix we conflated the two and silently lost
every drive-reported error reason. macOS and Windows backends had
the same shape: they extracted sense_key only, dropping ASC/ASCQ.

API restructure (clean separation):

  Error::ScsiError {
      opcode: u8,
      status: u8,                  // 0xFF = synthetic transport-failure
      sense: Option<ScsiSense>,    // None ⇔ no sense delivered
  }

  pub struct ScsiSense { sense_key: u8, asc: u8, ascq: u8 }
  impl ScsiSense {
      pub fn is_marginal(&self) -> bool       // keys 0/1/3/B
      pub fn is_medium_error(&self) -> bool
      pub fn is_hardware_error(&self) -> bool
      pub fn is_unit_attention(&self) -> bool
      pub fn is_data_protect(&self) -> bool
      pub fn is_not_ready(&self) -> bool
      pub fn is_illegal_request(&self) -> bool
      pub fn is_aborted_command(&self) -> bool
  }

  impl Error {
      pub fn scsi_sense(&self) -> Option<&ScsiSense>
      pub fn is_scsi_transport_failure(&self) -> bool
      pub fn is_marginal_read(&self) -> bool
  }

SCSI protocol constants (SCSI_STATUS_*, SENSE_KEY_*) moved from
error.rs to scsi/mod.rs where they belong alongside SCSI_INQUIRY,
SCSI_READ_10, etc. parse_sense replaces parse_sense_key (returns the
full triple, not just the key); inline tests now exercise ASC/ASCQ
extraction at the right offsets for both descriptor (0x72/0x73) and
fixed (0x70/0x71) sense formats.

Disc::copy + Disc::patch sense-aware dispatch:
  - marginal sense (MEDIUM ERROR / ABORTED COMMAND / RECOVERED ERROR
    / NO SENSE) → engage hysteresis (Block→Single, bpt=1)
  - non-marginal sense (HARDWARE / DATA PROTECT / UNIT ATTENTION /
    NOT READY / ILLEGAL REQUEST / transport failure / kernel
    IoError) → bail with full sense info preserved; caller (autorip)
    surfaces "physical replug" / "drive failing" / "media changed"

  Pre-fix: every CHECK CONDITION → 0xFF synthetic → Disc::copy bailed
  → bytes_good froze at the bad zone. The hysteresis from v0.13.22
  was correct but never got to run. This release unblocks it.

  Disc::patch's wedged_threshold (50 consecutive failures) stays as
  defense-in-depth for chains of marginal failures; a single
  non-marginal sense now short-circuits it.

New phase=bail trace event records the bail reason with the sense
triple. phase=transport_err remains for genuine bridge wedges /
kernel timeouts; phase=scsi_err carries the parsed sense_key, asc,
ascq for drive-reported errors.

All 350 tests pass. Clippy clean across all targets.
2026-04-26 19:06:19 -07:00
MattJackson ebffc6eb88 v0.13.22 — replace bisect-on-fail with hysteresis Block↔Single
The v0.13.21 bisect-on-fail recovery was correct (100% of recoverable
sectors picked up) but slow on dense damage clusters. Live test on
Dune 2 v0.13.21 burned ~30 s per damaged 60-block — paying a ~5 s
kernel ABORT/timeout at every level of a log₂(60) ≈ 6 deep DFS, on
the failing branch each time.

Replaced with a two-state hysteresis machine in Disc::copy:

  Block(batch):
    read(batch) ok    → write, advance, stay Block
    read(batch) fail  → switch to Single, retry SAME range at bpt=1

  Single:
    read(1) ok    → write, consecutive_good++
                    if consecutive_good >= BPT1_EXIT_THRESHOLD:
                      switch to Block, reset counter
    read(1) fail  → mark NonTrimmed, consecutive_good = 0

BPT1_EXIT_THRESHOLD = 10_000 sectors (= 20 MB clean run). Calibrated
from the 2026-04-26 BU40N empirical probe data; tunable.

Per-block math on a damaged 60-block with 1 truly bad sector:

  Bisect      (v0.13.21): ~30 s  (5 s × 6 levels)
  Hysteresis  (v0.13.22): ~10 s  (5 s bpt=batch fail
                                  + 59 × 1 ms good
                                  + 1 × 5 s bad)

Inside a damaged cluster spanning many 60-blocks the win compounds:
hysteresis pays the bpt=batch fail cost ONCE on entry, then stays at
bpt=1 across the cluster; bisection re-paid it every 60 sectors. For
Dune 2's ~1248-sector boundary cluster that's ~21 fewer 5-sec
kernel timeouts ≈ 100 s saved per pass.

Telemetry: new phase=mode_change trace event with from, to, lba, and
consecutive_good. Replaces v0.13.21's phase=bisect. Worklist DFS is
gone — single iterative for s in 0..count on the failure path.

Test rename, same fixture and same 100% recovery expectation:
  test_disc_copy_bisect_recovers_via_single_sector_reads
  → test_disc_copy_hysteresis_recovers_via_single_sector_reads

Also adds DamageSeverity (Clean / Cosmetic / Moderate / Serious) +
classify_damage(bad_sectors, lost_ms), re-exported from libfreemkv,
so applications can render structured severity instead of formatting
their own from raw counters.
2026-04-26 17:27:57 -07:00
MattJackson 424d3cd4f2 v0.13.21 — bisect-on-fail in Disc::copy + 10s caller READ timeout
Fixes the BU40N wedge cycle that has been chasing us through
v0.13.18-20. Two changes, both backed by empirical live-hardware
probes recorded in (internal)/docs/TEST_PLAN.md:

1. scsi/mod.rs: READ_TIMEOUT_MS 1500 → 10000 ms.
   Cold-start seek on the BU40N takes ~1.5 s. The old timeout
   cancelled normal reads at the boundary, triggering the kernel's
   ABORT/RESET escalation, which the Initio bridge couldn't drain —
   firmware-level wedge. 10 s catches every legitimate slow read
   (max successful ECC recovery: 2.6 s; cold-start: 1.5 s) with
   margin and short-circuits truly bad sectors at ~10 s.

2. disc/mod.rs: Disc::copy bisect-on-fail (replaces skip-forward).
   Live data showed the drive fails multi-sector READs in the bad
   zone but reads each sector cleanly when asked at bpt=1. Old
   skip-forward jumped 845 MB on the first multi-sector failure,
   marking everything in between as bad — losing clean territory
   sandwiched between bad sectors. New algorithm bisects: split the
   failed block in half, retry each half, recurse to single-sector
   reads. Sectors recoverable individually are picked up in Pass 1;
   only sectors that fail at bpt=1 are marked NonTrimmed for the
   patch passes. Stack-based DFS, log2(batch) = 6 levels for the
   default 60-sector batch.

Multi-pass machinery is untouched. Pass 2..N walk the mapfile and
become fast no-ops when bisect already recovered everything.
Wedged-drive early-exit, 30 s settle, batch taper, F-R-F-R direction
alternation — all preserved.

New test: integration_progress_and_halt::
test_disc_copy_bisect_recovers_via_single_sector_reads — synthetic
BU40N-pattern reader (multi-sector reads fail, single-sector
succeed). Pre-patch: lost everything to skip-forward. Post-patch:
100 % bytes_good. Plus the 10 sense-key parser tests from the
0.13.20 test-coverage pass.

Empirical recovery on Dune 2 UHD on the BU40N (per TEST_PLAN.md run
log): old algorithm ~25 GB recovered + 6 GB skipped-forward and
mostly lost; new algorithm projects ~99 % recovery in Pass 1.

Audits + raw probe data:
- (internal)/docs/TEST_PLAN.md (run log)
- (internal)/docs/audits/2026-04-26-scsi-architecture-research.md
2026-04-26 15:57:44 -07:00
MattJackson d2905ba7bb v0.13.20 — sync blocking SG_IO + cross-platform parity strip
- scsi/linux.rs: full rewrite from async write/poll/read+1.5s timeout+
  close-on-timeout to one synchronous ioctl(fd, SG_IO, &hdr). Kernel
  honors hdr.timeout and runs its own ABORT/RESET escalation. Errors
  check host_status and driver_status (both 0xFF-synthesised) plus
  status. Sense-key parser handles descriptor (0x72/0x73) + fixed
  (0x70/0x71) formats. Deleted fd_recovery, bg close+open thread, fd
  swap dance. -331/+155 lines.

- scsi/macos.rs: try_recover() removed (userspace handle-recovery on
  task failure was the same anti-pattern stripped from Linux). bsd_name
  field deleted. Errors bubble up directly.

- scsi/windows.rs: try_recover() removed, wide_path field deleted,
  INVALID_HANDLE guard removed.

- scsi/mod.rs: parse_sense_key() helper extracted (used by all three
  platforms now — single canonical sense-key parse rather than three
  inlined copies). +10 unit tests covering descriptor format, fixed
  format, truncated buffers, unknown response codes.

- drive/mod.rs: Drive::reset() deleted (escalating eject + STOP/START +
  reinit recovery — per audit, kernel handles its own escalation;
  userspace shouldn't).
  pub fn find_drives() -> Vec<Drive> deleted (opened N drives just to
  throw most away). find_drive() now uses discover_drives() directly.
  wait_ready() simplified — drops the reset path on sense_key=5,
  just keeps polling TUR for 60 iterations.

- lib.rs: find_drives re-export removed.

- benches/sgio_read.rs: switched to find_drive() (no longer iterates a
  drive list).

Net: 9 files changed, 226 insertions(+), 473 deletions(-). 329 tests
pass, clippy -D warnings clean. No consumer breakage (CLI, autorip,
bdemu compile + test green).

Architecture decision documented in
(internal)/docs/audits/2026-04-26-scsi-architecture-research.md
(primary-source survey of MakeMKV, sg_dd, ddrescue, and the kernel
mid-layer's own scsi_eh.rst escalation ladder).
2026-04-26 09:51:46 -07:00
MattJackson b4de5d343d v0.13.18 — sync release (no functional changes; autorip two-bar UI fix) 2026-04-26 07:47:37 -07:00
MattJackson 8beeac7df9 v0.13.17 — sync release (no functional changes; actual fix is autorip hot-plug) 2026-04-26 07:27:07 -07:00
MattJackson b33f41e219 v0.13.16 — single Progress trait + PassProgress (RIP_DESIGN.md §16)
Pre-0.13.16 the rip API leaked internal mapfile concepts (pos,
bytes_good, work_done, bytes_pending, Finished/NonTrimmed) into per-pass
positional callbacks Fn(u64, u64, u64). Consumers reinvented the math
each time, and v0.13.15's UI bug surfaced exactly because of this —
autorip's web JS computed pct from bytes_good while the backend
computed from pos, silent drift, frozen UI bar.

This release replaces both Disc::copy::on_progress and
Disc::patch::on_progress callbacks with a single Progress trait +
PassProgress struct (new progress module).

  pub struct PassProgress {
      pub kind: PassKind,            // Sweep | Trim {reverse} | Scrape {reverse} | Mux
      pub work_done: u64,
      pub work_total: u64,
      pub bytes_good_total: u64,
      pub bytes_total_disc: u64,
  }

  pub trait Progress {
      fn report(&self, p: &PassProgress);
  }

  impl<F: Fn(&PassProgress)> Progress for F { ... }   // closures work directly

CopyOptions::on_progress and PatchOptions::on_progress are renamed to
progress: Option<&dyn Progress>. Closure callers update trivially via
the blanket impl; struct callers gain a clean named-field shape with no
positional-arg confusion.

PassKind carries the semantic (sweep vs trim vs scrape vs mux) so
consumers can label phases without reinventing detection logic.
Disc::patch reports Trim {reverse} for retry passes with block_sectors
>= 2 and Scrape {reverse} when block_sectors == 1. Direction comes
through reverse: bool. Mux variant is reserved for v0.13.17 when the
mux pipeline emits progress.

Tests + clippy clean across all 4 crates.
2026-04-26 07:15:26 -07:00
MattJackson e85e20f436 v0.13.15 — pos in on_progress, PatchOptions::reverse, wedged_threshold
Breaking: CopyOptions::on_progress + PatchOptions::on_progress now take
Fn(bytes_good, pos, total). Consumers display `pos` for "% swept" — the
true Pass 1 progress that advances through skip-forward bad zones, where
bytes_good (Finished sectors only) freezes. v0.13.14 live trace proved
the existing UI was lying for ~14 minutes about Dune 2 being "stuck at
30%" while Pass 1 was actually 83% through the disc via skip-forward.

PatchOptions::reverse: walk bad ranges from highest LBA to lowest. For
drives that wedge after a forward read of a bad sector, approaching the
post-bad-zone NonTrimmed range from end-of-disc reads good sectors before
the drive sees a bad one. Hypothesis informed by the BU40N + Initio
bridge live data — Pass 2 forward saw zero successful reads in 7 min
while Pass 1's pos walked all the way to end-of-disc.

PatchOptions::wedged_threshold: > 0 → exit early after that many
consecutive failures with zero successes in the same pass. Saves the
wallclock budget for productive grinding when the drive has wedged on
the bad zone for THIS pass; a different direction or block size in the
next pass may still recover. New PatchResult::wedged_exit reports it.

Trace: patch_start (block_sectors, recovery, reverse, wedged_threshold,
num_ranges) and patch_done (blocks_attempted, blocks_read_ok,
blocks_read_failed, wedged_exit, halted, bytes_recovered) at the
freemkv::disc target.
2026-04-25 20:06:03 -07:00
MattJackson 7bc54be8d3 v0.13.14 — sync release, no functional changes (autorip subscriber filter fix in autorip 0.13.14) 2026-04-25 18:38:17 -07:00
MattJackson 6d9083743b v0.13.13 — telemetry: tracing instrumentation in SgIoTransport + Disc::copy
v0.13.12 shipped the async fd_recovery design but a live test on Dune 2
showed Pass 1 sat for 14 minutes with bytes_good=0 — the inner loop iterates
(throttled on_progress log fires every 78s) but each iteration evidently
takes ~60s instead of the microseconds the design promises on fast-fail.
Without trace-level telemetry at the SCSI + Disc::copy boundaries we
can't tell where the time goes.

This release is instrumentation only — no behavior change.

- New dep: tracing 0.1. Per project docs, debug/trace logging is allowed in
  libfreemkv (the no-English rule applies to errors). Consumers wire a
  tracing subscriber.
- SgIoTransport::execute (Linux): trace at every state transition (entry,
  recovery_swap_ok, recovery_pending, write_ok / write_err, poll_done,
  timeout_spawn_recovery, scsi_err, read_err, ok). Each event includes
  opcode + elapsed timing. The bg recovery thread also traces close_ms +
  open_ms so we can see if the kernel really takes 60s to close+open on a
  wedged Initio bridge.
- Disc::copy: trace at copy_start, outer_loop, region_enter, every 100
  inner-loop iterations (iter_progress with pos/region_end/skip_size/
  bytes_good/read_ok_count/read_err_count/last_read_ms/copy_elapsed_ms),
  copy_done.
- All trace events use targets `freemkv::scsi` and `freemkv::disc` so
  consumers can filter by subsystem (e.g. autorip /api/debug?q=freemkv::scsi).

Next: run the live test on Dune 2 again, read the autorip JSONL log,
diagnose why each iter is slow, fix the actual bug.
2026-04-25 18:32:05 -07:00
MattJackson c49a68054f style: cargo fmt on integration test 2026-04-25 17:36:06 -07:00
MattJackson 4bd38787c6 test(integration): make halt-on-skip-forward test deterministic
The wallclock-based halt timing failed on fast CI runners where a 2 GB
synthetic-disc skip-forward sweep finishes in <100 ms — well under the
200 ms halt fire delay. Reader now signals halt on first read; the
inner-loop halt check on iteration 2 breaks 'outer. No wallclock race.
2026-04-25 17:33:52 -07:00
MattJackson b4951b1c5b v0.13.12 — Fix 1+2+4 + cross-platform SCSI parity (RIP_DESIGN.md §6, §7, §15.1)
Fix 1: delete stall guard from Disc::copy. Pass 1 must sweep end-to-end
per ddrescue model (RIP_DESIGN.md §2.1, §3, §9). The v0.13.9 guard at
disc/mod.rs broke Pass 1 at 30% on Dune 2 with 56 GB still NonTried.
Removed stall_secs field, narrative comment in scsi/linux.rs, and the
broken regression test. Replaced with test_disc_copy_completes_full_disc_
with_failing_reader and test_disc_copy_halts_promptly_on_failing_reader.

Fix 2: async SCSI transport recovery. Added Arc<AtomicI32> fd_recovery
on SgIoTransport. On poll timeout: spawn close + spawn open in
background, return Err immediately. Top of execute() swaps fd from
recovery atomic. Main thread never blocked beyond ~1.5s poll budget
(was up to ~60s per timeout because kernel serialized main-thread
open() against in-flight close()). Drop drains pending recovery fd.

§15.1 cross-platform parity: Windows + macOS now have the same
observable recovery contract. SptiTransport gets try_recover()
(synchronous CloseHandle + CreateFileW; Windows close is fast, no
in-flight CDB drain like Linux). MacScsiTransport gets try_recover()
(release IOKit interface + reacquire via new acquire_device_iface()
helper); stores bsd_name for re-resolution. Drop guards null'd-out
interfaces. Stripped English error strings ("try as root" / "run as
administrator") on Linux + Windows. Fixed Windows TimeOutValue
ms→s ceiling so 1500ms gets 2s (was 1s; broke Drive::read fast path).

Fix 4: instrument Disc::patch arms. PatchResult exposes
blocks_attempted, blocks_read_ok, blocks_read_failed so the v0.13.11
mystery (Dune 2 Pass 2 recovered 0 bytes in 100 min) is diagnosable
from the live device log without re-instrumenting from outside.

Cleanup: honor PatchOptions::full_recovery (was read into _ and
ignored; now routed to read_sectors recovery arg). Updated
CopyOptions::batch_sectors doc to describe the actual production
path (sysfs detect_max_batch_sectors, typically 60 sectors / ~120 KB
on BU40N) rather than the test-only 32-sector internal default.

All four crates clippy-clean and tests green on the host targets
(macOS native + cargo check on Linux). Cross-platform CI watches
Linux + Windows + macOS builds + tests.
2026-04-25 17:30:25 -07:00
MattJackson 92dc145864 v0.13.11: revert SgIoTransport timeout path — keep transport alive
v0.13.10's 'fd=-1 on first poll timeout' was too aggressive: a single
transient killed the entire transport, Pass 1 finished in 45ms with
0 GB good on Dune 2.

Revert to spawn-close + main-thread-reopen (the v0.13.5/8 pattern).
Per-timeout cost is up to ~60s while the kernel completes the
abandoned command, but the v0.13.9 Disc::copy stall guard caps
catastrophic stalls at 120s of bytes_good non-advance. Pass 1 bails
cleanly with NonTrimmed ranges; Pass 2 has a working Drive for
retries with recovery=true + 30s timeouts.
2026-04-25 08:47:10 -07:00
MattJackson 36b65f526a v0.13.10: version sync 2026-04-25 08:25:08 -07:00
MattJackson ca8ebf418f v0.13.9: Disc::copy stall guard + SgIoTransport no-reopen-on-timeout
Fixes the silent Pass 1 hang observed on Dune 2 with v0.13.8 (drive
grinding through bad sectors at 0 KB/s, errs=0, no error surfaced).

Root cause: SgIoTransport::execute's reopen-after-poll-timeout opened
a fresh /dev/sg* fd on the main thread, which serialized against the
spawned close() of the old fd via the kernel's per-device state lock.
The userspace 1.5s timeout still fired, but the abandon-and-reopen
recovery itself blocked the main thread for as long as close() took.
Net: reads returned slowly, skip-forward fired on every iteration,
bytes_good never advanced.

- SgIoTransport::execute: on poll timeout, spawn close, set fd=-1,
  return Err. No reopen on the main thread. The transport is now
  invalidated until the consumer creates a fresh Drive.
- Disc::copy: add stall guard. CopyOptions.stall_secs (default 120s).
  If bytes_good doesn't advance for the threshold, break 'outer
  cleanly with complete=false, bytes_pending > 0 so Pass 2 retries
  pick up the NonTrimmed ranges with recovery=true 30s timeouts.
- New regression test: test_disc_copy_stall_detection_triggers_
  skip_forward in tests/integration_progress_and_halt.rs.
2026-04-25 08:12:01 -07:00
MattJackson 34182d956a v0.13.8: version sync 2026-04-25 07:11:47 -07:00
MattJackson 0f967a2084 v0.13.7: version sync 2026-04-25 06:58:43 -07:00
MattJackson 0341995c0d fix: allow dead_code on Drive::emit (DiscStream owns BytesRead emission post-0.13.6 strip) 2026-04-24 21:43:30 -07:00
MattJackson fe0ec0c5cf fix: drop unused INQUIRY_* constants (clippy -D warnings) 2026-04-24 21:42:30 -07:00
MattJackson 022657d481 docs: complete 0.13.6 docs sweep (architecture, api-design, disc-to-rip, README)
Follow-up to the rip-recovery + drive-access updates: aligns the
remaining docs with the v0.13.6 single-shot read model and the
three-layer recovery architecture.

- architecture.md: module map says single-shot read; new paragraph on
  layered recovery with postmortem pointer.
- api-design.md: EventKind enum example expanded; emission notes
  document that BytesRead now fires from DiscStream::fill_extents and
  Retry/SectorRecovered are no longer emitted in 0.13.6+.
- disc-to-rip.md: Step 10 of the pipeline diagram + module table
  reflect single-shot read.
- docs/README.md: added rip-recovery.md to the TOC.
2026-04-24 21:36:49 -07:00
MattJackson fc8eca44e1 docs: rewrite rip-recovery + drive-access for 0.13.6 single-shot model
Updates docs/ to reflect the recovery-loop strip:
- rip-recovery.md: drops Phase 1/2/3 description, replaces with three-layer
  model (Disc::patch multi-pass / DiscStream batch halving / Drive::read
  single-shot). Notes that no SCSI resets fire from any retry path.
- drive-access.md: removes SG_SCSI_RESET + STOP/START UNIT escalation
  references; SgIoTransport::reset is now kernel SG_IO flush + ALLOW
  MEDIUM REMOVAL only.
- src/mux/disc.rs + tests/: cargo fmt cleanup.
2026-04-24 21:35:33 -07:00
MattJackson 43836865be v0.13.6: strip Drive::read inline recovery + reset escalation; emit BytesRead
Drive::read is now single-shot. Phase 1/2/3 retries + scsi::reset+reopen
removed (~80 lines). recovery=true bumps timeout to 30s; recovery=false
stays at 1.5s. On any failure returns Err(DiscRead) immediately — caller
(Disc::patch outer loop, DiscStream batch halver) handles retries.

Inline reset+reopen WAS the wedge primitive on the LG BU40N. Per prior
post-mortem, every USB/SCSI reset path tested fails to recover the
wedged Initio bridge — the inline retry was pure cost.

SgIoTransport::reset (Linux) trimmed to kernel SG_IO state flush +
ALLOW MEDIUM REMOVAL. SG_SCSI_RESET ioctl + STOP/START UNIT escalation
removed. macOS reset removed (no-op). scsi::reset() top-level family
removed (no callers).

EventKind::BytesRead { bytes, total } now actually emitted from
DiscStream::fill_extents after each successful sector read. Was
declared in 0.13.0, never fired. Drives autorip per-device progress
in direct mode.

EventKind::Retry / SectorRecovered no longer emitted (variants kept
for forward compat). SpeedChange still emitted via Drive::set_speed
public path.

Tests: new tests/integration_progress_and_halt.rs (5 tests). 233 unit
tests + 5 integration green.
2026-04-24 21:32:45 -07:00
MattJackson 5e22199441 v0.13.5: version sync (autorip-side fixes) 2026-04-24 20:13:07 -07:00
MattJackson 761b77bbd9 v0.13.4: roll back wedge recovery + add sysfs identity fallback
USB/SCSI recovery escalation in drive_has_disc (0.13.1-0.13.3) tested
on LG BU40N USB BD-RE: USBDEVFS_RESET, authorized toggle, driver
unbind/rebind, SCSI host rescan — all succeed at the USB transport
layer but the drive firmware below the bridge stays locked. Only
physical unplug-replug clears it. Rolled back so consumers can
surface the real failure to the user.

New: list_drives falls back to sysfs-cached vendor/model/rev from
/sys/class/scsi_generic/sgN/device/ when live INQUIRY returns empty,
so wedged drives still show their identity in UIs.

Removed: scsi::usb_reset, usb_reset_with_timeout, per-platform
usb_reset methods, recover_then_probe, is_wedge_signature. Breadcrumb
comment in scsi/linux.rs::drive_has_disc points at v0.13.3 tag for
the full implementation if future hardware needs it back.

Linux/macOS/Windows pass-through symmetric; 233 tests passing.
2026-04-24 19:53:44 -07:00
MattJackson cc05ef2a3a v0.13.3: broaden is_wedge_signature — fix dead-code wedge recovery
0.13.2's is_wedge_signature gated on opcode=SCSI_INQUIRY (0x12), but
drive_has_disc issues TEST UNIT READY (0x00). Production wedge errors
(E4000: 0x00/0xff/0x00) never matched → SCSI reset + USB reset
escalation never fired.

Drop the opcode gate. Status byte 0xFF is synthesised by our own
execute() path on poll() timeout — it's the ground-truth wedge
marker for any opcode.

Linux-only; macOS/Windows use sense-key-based wedge detection.
2026-04-24 19:26:11 -07:00
MattJackson e27b82ce5b v0.13.2: list_drives + drive_has_disc; SCSI primitives pub(crate)
Architectural cleanup. autorip + freemkv CLI were reimplementing drive
discovery (sysfs walking, type-5 filtering, sg-path construction) and
calling SCSI reset primitives directly. All of that hardware-aware code
moves into libfreemkv with two cheap public probes:

- DriveInfo + list_drives() — multi-OS enumeration (Linux/macOS/Windows)
  with peripheral-type-5 filtering and INQUIRY identity. Cheap.
- drive_has_disc(path) — single TUR with internal wedge recovery
  escalation (SCSI reset → USB reset → retry) hidden from callers.

USB-layer reset (USBDEVFS_RESET / IOUSBDeviceInterface::ResetDevice /
storport's combined reset) wired across all three platforms.

Visibility tightening — scsi::reset, scsi::usb_reset, and the timeout
constants are now pub(crate). Compile-time guarantee that no consumer
crate can issue SCSI commands directly.

233 lib tests pass; clippy clean.
2026-04-24 17:31:15 -07:00
MattJackson 8af47e4c19 v0.13.1: scsi::reset() bounded by wallclock timeout
Production incident: autorip's poll loop called scsi::reset() on a
wedged BU40N USB drive. The Linux SG_SCSI_RESET ioctl blocked
indefinitely (kernel SCSI subsystem waiting for a bus-wedged device to
ack a reset that will never come). Caller's poll loop hung for 60+
seconds before manual intervention.

scsi::reset() now spawns a detached worker for the platform-specific
reset and bounds the caller's wait via mpsc::recv_timeout. Default
30 s (DEFAULT_RESET_TIMEOUT_SECS); reset_with_timeout(device, dur)
exposes the bound for callers that want a different value. Returns
DeviceResetFailed on timeout. Worker thread keeps running until the
kernel eventually unblocks — leaks one OS thread per hard wedge, but
the daemon stays responsive instead of hanging forever.

Follow-up flagged for 0.13.2: USB-attached drives wedge at the USB
Mass Storage layer below SCSI; SG_SCSI_RESET doesn't help. A
scsi::usb_reset(path) using USBDEVFS_RESET is the proper escalation.
2026-04-24 16:58:13 -07:00
MattJackson d1f09439a5 v0.13.0: zero English in library + API hygiene + dead-code sweep
Audit pass against the project docs "no English text in library code" rule.
Found 9 call sites that violated the contract by stuffing English into
io::Error::new(kind, "…") or by abusing Error::DeviceNotFound { path }
as a free-form description field. Each is now a typed Error variant.

New variants and codes: ScsiInterfaceUnavailable (E1004), DeviceLocked
(E1005), IoKitPluginFailed (E1006), UnsupportedPlatform (E2003),
PlatformNotImplemented (E2004), MapfileInvalid (E6011), DiscUrlNotDirect
(E9009).

labels::apply() previously pushed Commentary/Descriptive/Score/IME and
" (Secondary)" English literals into AudioStream.label, leaking into
MKV titles + autorip UI. AudioStream now exposes structured `purpose:
LabelPurpose`, SubtitleStream `qualifier: LabelQualifier`. Callers
translate to localized text. label keeps codec-formatting only.

API hygiene: 11 mux/* modules dropped from `pub` to `pub(crate)` —
their *types* are still re-exported from lib.rs, but the modules were
leaking low-level EBML/TS/network primitives. Stream trait gets a real
rustdoc explaining read-vs-write split. lib.rs grouped re-exports into
documented sections. ScanOptions::with_keydb() removed (one-method-per-
action rule); use struct literal.

Dead-code sweep: removed lookahead.rs (orphan, never declared as mod),
tsreader.rs (TsDemuxReader unused), ebml::{write_int,read_vint,SEEK_*},
ts::{scan_first/last_pts,scan_duration,SCAN_HEAD/TAIL_SIZE,take/set_
remainder}, MkvMuxer codec_private_slots/filled fields and
fill_codec_private method (deferred-codecPrivate path never used since
the v0.10 PES rewrite). cargo clippy --all-targets -D warnings clean.

Tests: new error::tests for variant codes + Display "no English" guard +
io::ErrorKind mapping. 233 lib tests, all green (was 230).

Breaking: ScanOptions::with_keydb removed; mux/* modules pub(crate);
AudioStream and SubtitleStream gained required fields; UnsupportedDrive
{ product_revision: "Renesas not yet implemented" } no longer produced
(use PlatformNotImplemented).
2026-04-24 16:41:02 -07:00
MattJackson 37e721ee7e v0.12.2: hide halt behind checked_sleep / checked_exec primitives
Drive::read is now halt-check-free in its body. Previously, the halt
flag was checked in 4 places and the sleep logic was scattered across
4 "if halt_aware_sleep { return Halted }" call sites — correct, but
the ugliness invited drift: a new sleep added by someone unfamiliar
with the pattern would silently swallow Stop requests.

Two private primitives now own halt awareness:

  checked_sleep(Duration) -> Result<()>
  checked_exec(cdb, dir, buf, timeout_ms) -> Result<ScsiResult>

Both return Err(Halted) instead of a bool. The ? operator in read()
then propagates halts for free. The recovery path reads top-to-bottom
with no halt vocabulary.

sleep_until_halted lives as a free function so it's unit-testable
without a live Drive. 4 new tests: completes normally, bails on
pre-set flag within one slice, wakes mid-sleep, zero-duration no-op.

Public API unchanged — halt_flag/halt/clear_halt still exposed, the
refactor is entirely internal.
2026-04-24 13:12:24 -07:00
MattJackson 481b47d90d v0.12.1: halt-aware sleeps in Drive::read recovery
Stop was waiting up to 30 s to register when the drive hit L-EC
recovery mid-read — the recovery phase does 30 s sleeps between
retries and the halt flag was only checked at the start of each sleep.
UI feels broken ("stop isnt working") even though the halt was set.

halt_aware_sleep breaks each wait into 100 ms slices and returns
early on halt. Applied to all 4 sleeps in the recovery path (both
30 s retry delays, both 5 s reset-phase delays).
2026-04-24 12:59:36 -07:00
MattJackson dd7cbd70a9 v0.12.1: cut non-recovery read timeout 5s → 1500ms
Disc::copy fast pass (skip_on_error=true, recovery=false) was giving
the drive 5 s per 64 KB block. On structure-protected / marginal UHD
sectors the drive grinds L-EC for nearly the full budget per block,
pinning throughput at ~13 KB/s even though skip_forward would happily
skip past the region.

1500ms bounds the floor at ~43 KB/s/block. Recoverable sectors that
would have succeeded at 3-5 s get picked up on Disc::patch (pass 2+)
where recovery=true and the per-read budget is 30 s.
2026-04-24 12:57:53 -07:00
MattJackson 3dea679dac style: cargo fmt 2026-04-24 12:23:42 -07:00
MattJackson 64ecca0d12 fix(windows): Rust 2024 requires unsafe extern blocks
Missed in the prior 2024 edition sweep because local builds don't
exercise the cfg(target_os = "windows") path. check-windows CI caught it.
2026-04-24 12:21:43 -07:00
MattJackson 84f6baba38 v0.12.0: Rust 2024 edition migration
- edition = "2024" bump.
- FFI block in src/scsi/macos.rs wrapped in `unsafe extern "C" { }`.
- vtable_fn body gets an explicit unsafe block (unsafe_op_in_unsafe_fn).
- Match-ergonomics cleanup in mux/meta.rs, mkvstream.rs, network.rs,
  stdio.rs — removed redundant `ref` / `ref mut` bindings.

MSRV unchanged at 1.86. 226 tests pass. No behavior change.
2026-04-24 12:07:05 -07:00
MattJackson bd5b7795bd docs: multi-pass recovery — README snippet + new rip-recovery.md
- README quick-start gains a multi-pass example using the new
  Disc::copy + Disc::patch primitives.
- New docs/rip-recovery.md documents the two-stage rip model: mapfile
  format (ddrescue-compatible), CopyOptions/PatchOptions surface, the
  pass-1/pass-2 algorithm, and the design decisions (why no MODE
  SELECT, why ISO intermediate, why ddrescue mapfile).

No code change.
2026-04-24 11:53:28 -07:00
MattJackson a57506c1ce v0.11.22: version sync — no functional changes
autorip 0.11.22 ships the full multi-pass UI (live mapfile stats,
bad-range viz, Recovery settings). libfreemkv API is unchanged from
0.11.21. Part of the 0.11.22 ecosystem sync.
2026-04-24 11:43:32 -07:00
MattJackson c33f3e9557 v0.11.21: multi-pass rip — Disc::copy + Disc::patch + mapfile module
New primitives for two-stage rip workflows: fast forward pass with
zero-fill on failures, then targeted retries of bad ranges via a
ddrescue-compatible mapfile.

- Disc::copy now takes &CopyOptions (breaking change from positional
  args). Always writes a sidecar .mapfile. Opt-in skip_on_error +
  skip_forward give ddrescue-style fast sweep: 64 KB blocks,
  exponential skip-forward on failure, zero-fill bad blocks. Defaults
  preserve pre-0.11.21 behavior (recovery reads, abort on bad sector).

- Disc::patch is new and idempotent. Reads the mapfile, re-reads every
  non-finished range with full drive recovery, patches good bytes back
  into the ISO at exact offsets. Call N times for N retry attempts.

- disc::mapfile is a new module. ddrescue text format, crash-safe
  (flushed on every record()), greppable, human-editable, tool-compatible.
  Status chars match ddrescue: ? / * / / / - / +.

- Re-exports FileSectorReader from the crate root.

- freemkv CLI caller (pipe.rs) updated to the new Disc::copy signature
  in lockstep — shipped in the 0.11.21 freemkv CLI release.

Part of the 0.11.21 ecosystem sync (libfreemkv + freemkv + bdemu +
autorip all on 0.11.21).
2026-04-24 09:24:32 -07:00
MattJackson 166fc4bf8c v0.11.18: DiscStream halt flag — Stop works in dense bad-sector regions
DiscStream::fill_extents loops internally while the demuxer waits for
enough clean data to emit a PES frame. In a dense bad zone that loop
can run for minutes without returning to the outer read() call, so
the caller's Stop signal never gets serviced until a frame is finally
emitted — which may be very far away.

Add DiscStream::set_halt(Arc<AtomicBool>) — typically wired to
Drive::halt_flag() for unified Stop across drive recovery phases and
stream sector processing. fill_extents checks the flag at the top of
every retry iteration; raising it returns Err(Error::Halted) within
one SCSI round-trip.

No behavior change for callers that don't call set_halt. Unblocks the
architectural fix for the "Stop doesn't stop" bug observed on a
damaged UHD disc.
2026-04-24 07:41:13 -07:00
MattJackson 9aaddaa9b8 v0.11.17: adaptive batch sizer — no per-sector descent
Replace read_with_binary_search + 3×5s light recovery with an adaptive
sizer that shrinks on failure (halve, 3-aligned ≥6) and probes back up
after 100 MiB (51,200 sectors) of clean reads. Descent cost is paid
once per bad region, not once per bad sector.

Emit BatchSizeChanged { new_size, reason } on shrink and probe-up.
Remove BinarySearch event — no longer produced.

Side fix: scsi/macos.rs one-liner for manual_c_str_literals clippy
lint that surfaced on a newer toolchain.
2026-04-23 20:24:40 -07:00
Matt Jackson 40ec1e1eba v0.11.16: API cleanup — one method per action 2026-04-21 19:17:50 +00:00
Matt Jackson 8843833ea7 v0.11.15: lint cleanup — fmt + clippy clean 2026-04-21 18:52:55 +00:00
Matt Jackson afae37c8c3 fix: iso_dump example missing recovery arg 2026-04-21 18:41:41 +00:00
Matt Jackson d8089ec491 v0.11.14: audit fixes — trailing sectors, verify stop, SCSI sense, O_CLOEXEC
Fix trailing sectors dropped at extent boundaries when sector_count % 3 != 0.
Add verify_title stop support via progress callback returning bool.
Add O_CLOEXEC on all SCSI fd opens to prevent leak to child processes.
Fix SCSI sense descriptor format detection (0x72/0x73 vs 0x70/0x71).
2026-04-21 18:40:04 +00:00
Matt Jackson ce8132d181 Update docs: async sg transport, Drive::read recovery phases 2026-04-21 18:01:48 +00:00
Matt Jackson 71e1f57364 Async SG_IO: enforceable timeouts via write/poll/read
Replace blocking ioctl(SG_IO) with the sg driver's async interface.
Commands are submitted via write(), waited on via poll() with a hard
wall-clock timeout, and completed via read(). If poll() times out,
the fd is abandoned and a fresh one opened — the kernel can no longer
hold us hostage during USB error recovery.

- write() submits command, returns immediately
- poll() enforces exact timeout (EINTR-safe with deadline tracking)
- read() retrieves result + copies data to caller's buffer
- On timeout: old fd closed in background thread, new fd opened
- No SG_FLAG_DIRECT_IO — kernel buffers for safe timeout abandonment
- Store device_path for fd reopen after timeout
- Drop guards fd=-1 (abandoned fd)
2026-04-21 17:57:58 +00:00
Matt Jackson d8092ee093 Clean API: merge read() and read_fast() into read(recovery: bool) 2026-04-21 03:27:17 +00:00
Matt Jackson dd49d55b11 v0.11.13: all rip reads use fast timeout, no full recovery in read path 2026-04-21 02:16:49 +00:00
Matt Jackson 6ba958fe8a Fix: initial batch read uses fast read, not full recovery 2026-04-21 02:02:13 +00:00
Matt Jackson f30b958408 v0.11.12: halt, events, light recovery 2026-04-21 00:25:44 +00:00
Matt Jackson 6971f00be2 Drive halt flag, sector events, binary search light recovery (3x5s) 2026-04-21 00:18:46 +00:00
Matt Jackson ad86f92d85 v0.11.11: binary search error recovery in fill_extents 2026-04-20 21:51:46 +00:00
Matt Jackson cce1be29b2 v0.11.10: skip_errors, read_sectors_recover API 2026-04-20 15:27:56 +00:00
Matt Jackson 081ad4f619 DiscStream skip_errors, read_sectors_recover(recovery) API 2026-04-20 03:38:32 +00:00
Matt Jackson 43ecad2d43 Clean API: read_sectors_recover(recovery: bool) replaces read_sectors_fast 2026-04-20 01:12:59 +00:00
Matt Jackson 8870efa77b v0.11.9: fast verify reads — 5s timeout, no recovery loop 2026-04-20 01:01:44 +00:00
Matt Jackson a693ef965b v0.11.8: disc verify module 2026-04-20 00:03:26 +00:00
Matt Jackson 0596307c19 Add verify module: sector-by-sector disc health check 2026-04-20 00:01:02 +00:00
Matt Jackson 18c6365e08 Improve label generator: proper video labels, all resolutions, no false Atmos claims 2026-04-19 17:50:17 +00:00
Matt Jackson 981f30b1b0 Move label generation to labels system — fill_defaults() for all stream types 2026-04-19 17:23:50 +00:00
Matt Jackson 04086d73a0 Generate audio track labels when disc doesn't provide them 2026-04-19 16:34:35 +00:00
Matt Jackson 20e404e59a README: fix dep version 0.10 → 0.11, update share command 2026-04-19 04:57:55 +00:00
Matt Jackson 1bea8eb650 v0.11.7: TrueHD parser rewrite — 12-bit length, AC-3 skip, cross-PES buffering 2026-04-19 01:36:56 +00:00
Matt Jackson 0165f18fad v0.11.6: fix TrueHD BD-TS header corruption 2026-04-18 19:27:12 +00:00
Matt Jackson ca3914d537 Fix TrueHD: strip BD-TS access unit header before muxing to MKV 2026-04-18 19:26:14 +00:00
Matt Jackson 80fcffd190 v0.11.5: MKV container fixes — timestamps, frame rate, HDR, chapters, disposition 2026-04-18 16:29:21 +00:00
100 changed files with 14064 additions and 1795 deletions
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- run: cargo test
- run: cargo test --tests
check-macos:
runs-on: macos-latest
+1660
View File
File diff suppressed because it is too large Load Diff
+9 -2
View File
@@ -1,7 +1,7 @@
[package]
name = "libfreemkv"
version = "0.11.3"
edition = "2021"
version = "0.18.3"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
description = "Open source raw disc access library for optical drives"
@@ -24,6 +24,10 @@ rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# under project docs ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
@@ -31,6 +35,9 @@ libc = "0.2"
[target.'cfg(target_os = "macos")'.dependencies]
libc = "0.2"
[dev-dependencies]
tempfile = "3"
[[bench]]
name = "sgio_read"
harness = false
-34
View File
@@ -1,34 +0,0 @@
# libfreemkv — Feature List
## v0.10.10 (current)
### Done
- [x] Drive access: open, identify, unlock, firmware upload, speed calibration, eject
- [x] 206 bundled drive profiles (MediaTek MT1959 A + B variants)
- [x] SCSI transport: Linux SG_IO, macOS IOKit, Windows SPTI
- [x] UDF 2.50 filesystem parser (metadata partitions, Blu-ray profile)
- [x] MPLS playlist parser (play items, STN table, secondary streams)
- [x] CLPI clip info parser (EP map, sector extents)
- [x] AACS 1.0 decryption (4 VUK paths: KEYDB, media key, processing key, device key)
- [x] AACS 2.0 SCSI handshake (P-256/SHA-256 ECDH, bus decryption, read data key)
- [x] DVD IFO parser (VMG, VTS, PGC chains, cell addresses)
- [x] DVD CSS decryption (bus auth, disc key via player keys, title key, sector descramble)
- [x] CSS Stevenson plaintext attack for ISO key recovery
- [x] MPEG-2 PS demuxer (DVD Program Stream with PES extraction)
- [x] MPEG-2 video codec parser (sequence header, quantizer matrices, keyframe detection)
- [x] KEYDB.cfg download, verify, save (raw TCP, zero HTTP deps)
- [x] Content reading with adaptive batch sizing, error recovery, 12+ MB/s
- [x] Stream labels: 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
- [x] MKV muxer: 14 codec parsers (H.264, HEVC, MPEG-2, AC-3, DTS, TrueHD, PGS, DVD Sub, VC-1, LPCM, +4 more)
- [x] SectorReader trait: decouples disc scanning from SCSI
- [x] 7 stream types: Disc, ISO, MKV, M2TS, Network, Stdio, Null
- [x] PES pipeline with unified Stream trait (any source → any dest)
- [x] FMKV metadata header for M2TS and network streams
- [x] Numeric error codes only (no English text in library)
- [x] Event system for progress callbacks
### Planned
- [ ] Windows testing on real hardware
- [ ] Pioneer Renesas platform support (48 drives, need GET_CONFIG 010C)
- [ ] TranscodeStream (ffmpeg integration)
- [ ] ISO write with BD-compliant UDF structure
+20
View File
@@ -0,0 +1,20 @@
# libfreemkv — local dev helper.
# Mirrors the cross-crate scripts in (internal)/scripts/test-all.sh
# but scoped to this single crate.
.PHONY: test build check ci clean
test:
cargo test --tests
build:
cargo build --release
check:
cargo fmt --check
cargo clippy --all-targets -- -D warnings
ci: check build test
clean:
cargo clean
+54 -7
View File
@@ -18,7 +18,7 @@ Part of the [freemkv](https://github.com/freemkv) project.
```toml
[dependencies]
libfreemkv = "0.10"
libfreemkv = "0.18"
```
## Quick Start
@@ -51,6 +51,52 @@ while let Ok(Some(frame)) = input.read() {
output.finish()?;
```
### Multi-pass recovery rip
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) and the design notes in
`(internal)/memory/0_18_redesign.md`.
```rust
use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
## What It Does
- **Drive access** — open, identify, unlock, firmware upload, speed calibration, eject
@@ -59,7 +105,7 @@ output.finish()?;
- **Stream labels** — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
- **AACS decryption** — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
- **KEYDB updates** — download, verify, save from any HTTP URL (zero deps, raw TCP)
- **Content reading** — adaptive batch reads with automatic decryption and error recovery
- **Content reading** — adaptive batch reads with automatic decryption
- **Stream I/O** — unified stream pipeline for reading and writing any format
### Streams
@@ -67,21 +113,21 @@ output.finish()?;
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | Yes | Blu-ray ISO image file |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
Streams implement `IOStream` (byte-level) and `pes::Stream` (frame-level). `input()` / `output()` resolve URL strings to PES stream instances. `open_input()` / `open_output()` resolve to byte-level IOStream instances. All URLs use the `scheme://path` format — bare paths are rejected.
Streams implement `FrameSource` (read) and/or `FrameSink` (write); direction is type-checked. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
AACS decryption requires a KEYDB.cfg file. If available at `~/.config/aacs/KEYDB.cfg` or passed via `ScanOptions`, the library handles everything — handshake, key derivation, and per-sector decryption — without the application needing to know anything about encryption.
## Architecture
```text
Drive — open, identify, init, unlock, read (with recovery)
Drive — open, identify, init, unlock, single-shot read
├── ScsiTransport — SG_IO (Linux), IOKit (macOS), SPTI (Windows)
├── DriveProfile — per-drive unlock parameters (bundled)
└── PlatformDriver — MediaTek (supported), Renesas (planned)
@@ -97,7 +143,8 @@ Disc — scan titles, streams, AACS/CSS state
└── KEYDB — download + verify + save
Streams — unified PES pipeline
├── pes::Stream — read()/write() PES frames
├── FrameSource — read() PES frames (direction-typed)
├── FrameSink — write() PES frames (direction-typed)
├── DiscStream — sectors → decrypt → TS demux → PES
├── IsoStream — ISO file → decrypt → TS demux → PES
├── MkvStream — MKV mux/demux
@@ -134,7 +181,7 @@ All errors are structured with numeric codes. No user-facing English text — ap
## Contributing
Run `freemkv drive-info --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to contribute your drive's profile.
Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to contribute your drive's profile.
## License
+312
View File
@@ -0,0 +1,312 @@
# Troubleshooting Guide
Common problems and solutions for optical drive ripping with freemkv.
---
## 1. USB-SATA Bridge Issues
This is the single most common source of problems when ripping discs over USB.
### Symptoms
- The drive disappears mid-rip. The ripping tool reports the device is gone, and `ls /dev/sg*` no longer shows it.
- The device re-enumerates under a different name: `sg4` becomes `sg5`, then `sg7`, then `sg11` after each USB port reset.
- `dmesg` shows USB port resets: `usb X-Y: reset high-speed USB device`, `xhci_hcd 0000:00:14.0: Cannot enable. Maybe the USB cable is bad?`, or `usb-storage: device reset failed`.
- The SCSI layer reports `host_status=7` (Linux USB transport error) in sense data.
- The drive works fine for reading data discs or burning, but crashes when hitting damaged sectors during a rip.
- After the crash, the drive is completely invisible until physically unplugged and reconnected.
### Root Cause
USB-SATA bridges translate between the USB Mass Storage protocol (BOT or UAS) and the drive's native SATA interface. When the optical drive encounters an unreadable sector, it returns a SCSI CHECK CONDITION with sense key 0x03 (MEDIUM ERROR). Some bridge chipsets -- particularly the Initio INIC-36xx family -- have buggy firmware that mishandles this error response.
Specific failure modes:
- **Incorrect residue reporting.** The bridge claims a different number of bytes transferred than what actually occurred. The Linux USB storage driver sees this discrepancy as a protocol violation and resets the port to recover. The `US_FL_IGNORE_RESIDUE` quirk exists specifically for this class of bug (see `drivers/usb/storage/transport.c` in the Linux kernel).
- **Bridge firmware crash.** On some Initio bridges, a malformed SCSI error response from the drive causes the bridge MCU to hang entirely. The USB controller sees the device stop responding and initiates a port reset. The bridge recovers (it re-enumerates), but the rip is dead -- all state is lost.
- **Sense data corruption.** The bridge forwards garbled or truncated sense data to the host, which the SCSI midlayer cannot parse, leading to a transport reset.
This is a hardware + firmware problem, not a software bug. The same drive connected via direct SATA does not exhibit these symptoms.
### Known Affected Bridges
| Chipset | USB IDs | Notes |
|---------|---------|-------|
| Initio INIC-3609 | `13fd:3609` | Very common in cheap SATA-to-USB enclosures. Highly problematic. |
| Initio INIC-3619 | `13fd:3940` | Same firmware family as INIC-3609. |
| Initio INIC-3069 | `13fd:0840` | Older variant, same residue bug. |
| ASMedia ASM1051 | `174c:5106` | Early ASM SATA bridge. Residue issues on error paths. |
| JMicron JMB36x | `152d:0561` | Some firmware versions. Not all JMicroon chips are affected. |
If your drive came in a pre-built external enclosure (Vantec, Sabrent, OWC, etc.), it almost certainly uses one of these bridge chips internally.
### The Fix: USB Storage Quirk
Apply the `US_FL_IGNORE_RESIDUE` kernel quirk for your bridge. This tells the Linux USB storage driver to ignore the residue field in SCSI response frames, preventing the port reset on mismatched byte counts.
**Step 1: Identify your bridge's vendor:product ID.**
```bash
lsusb
```
Look for your drive's entry. Example output:
```
Bus 002 Device 005: ID 13fd:0840 Initio Corporation INIC-3609
```
Here the vendor ID is `13fd` and the product ID is `0840`.
**Step 2: Apply the quirk at runtime.**
```bash
echo "13fd:0840:i" > /sys/module/usb_storage/parameters/quirks
```
Replace `13fd:0840` with your device's actual IDs. The `:i` flag means `US_FL_IGNORE_RESIDUE`.
You can combine multiple flags. Common additions:
- `:i` -- ignore residue (`US_FL_IGNORE_RESIDUE`)
- `:u` -- force BOT mode instead of UAS, for bridges with UAS bugs
**Step 3: Reconnect the drive.** Unplug and replug the USB cable, or bind/unbind the device. The quirk is applied per-module-load, so existing sessions may need the drive reconnected.
### Making It Persistent
Add the quirk to your kernel boot parameters so it survives reboots.
Edit `/etc/default/grub` (GRUB) and add to `GRUB_CMDLINE_LINUX_DEFAULT`:
```
GRUB_CMDLINE_LINUX_DEFAULT="quiet usb_storage.quirks=13fd:0840:i"
```
Then rebuild the GRUB config:
```bash
sudo update-grub
```
For systemd-boot, add to your loader entry or `/etc/kernel/cmdline`:
```
usb_storage.quirks=13fd:0840:i
```
Multiple devices can be separated by commas:
```
usb_storage.quirks=13fd:0840:i,174c:5106:u
```
### Recommended Bridges
If you are buying a USB-SATA adapter or enclosure for optical drive use:
| Bridge | USB IDs | Notes |
|--------|---------|-------|
| ASMedia ASM1153 | `174c:1153` | Reliable. Widely available in SATA-USB 3.0 cables. |
| JMicron JMS578 | `152d:0578` | Good firmware. Supports UASP. |
| Icy Box IB-AC640-C3 | N/A | Uses a known-good bridge internally. Plug-and-play. |
Avoid any enclosure or adapter listing an Initio chipset.
### Best Solution: Direct SATA
Connect your optical drive directly to a motherboard SATA port. This eliminates the USB-SATA bridge entirely and is the most reliable configuration:
- No USB protocol overhead or translation errors.
- No bridge firmware bugs.
- No port resets or re-enumeration.
- Full SATA error recovery handled natively by the kernel's libata driver.
- Sustained read speeds are limited only by the drive, not the USB bus.
If your machine has a free SATA port, use it.
---
## 2. Damaged Disc Handling
### Symptoms
- SCSI MEDIUM ERROR (sense key 0x03) at specific LBAs. `dmesg` shows `sr X:0:0:0: [srY] Unrecoverable read error` or similar.
- Read speed drops to near zero when approaching a damaged area.
- The drive makes audible retrying noises (laser repositioning, spindle speed changes).
- On USB-connected drives: the bridge crashes (see section 1 above) when the drive returns the error.
### How freemkv Handles This
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
### The Drive Taint Issue (LG BU40N)
Some drives, notably the LG BU40N, exhibit a "taint" behavior after encountering MEDIUM ERRORs:
1. The drive hits a damaged sector and returns a MEDIUM ERROR.
2. From that point forward, **all subsequent reads fail** -- even reads to sectors that were previously successful.
3. The only recovery is to physically unplug and reconnect the drive (or power-cycle it).
This is not a freemkv bug. It is a drive firmware behavior triggered by the interaction between the drive's internal error recovery and the USB-SATA bridge's handling of the error response. The drive firmware enters a degraded state that it does not recover from without a power cycle.
Workarounds:
- **Use a direct SATA connection.** This eliminates the bridge interaction that triggers the taint.
- **Use a different bridge.** The ASM1153 and JMS578 are less likely to trigger this behavior.
- **Accept the partial ISO.** freemkv's skip-forward recovery will zero-fill the unreadable blocks and continue. The resulting ISO may be playable with minor glitches in the affected areas.
- **Physical replug between retry passes.** If running multi-pass patch, replug the drive between passes to clear the taint state.
freemkv deliberately does not attempt inline SCSI resets or eject cycles to recover from this state, because those operations were found to make the problem worse on affected hardware (see the design rationale in [`docs/rip-recovery.md`](docs/rip-recovery.md)).
---
## 3. Drive Not Detected
### Check Hardware Visibility
```bash
lsusb
```
Verify the drive appears in the USB device list. If it does not show up, the drive is not visible to the host at all -- check cables, power, and USB port.
```bash
ls /dev/sg*
```
On Linux, optical drives appear as `/dev/sg*` devices (the SCSI Generic interface). freemkv uses `/dev/sg*`, not `/dev/sr*`. If `lsusb` shows the device but no `/dev/sg*` entry exists, the `sg` kernel module may not be loaded:
```bash
sudo modprobe sg
```
### Check Kernel Messages
```bash
dmesg | grep -i usb | tail -30
dmesg | grep -i sg | tail -10
```
Look for:
- USB enumeration errors or failed port resets.
- `sg_add` messages confirming the sg device was registered.
- Permission denied or access errors.
### Permission Issues
On most Linux distributions, `/dev/sg*` devices are owned by `root:disk` or `root:cdrom` with restricted permissions. Running freemkv as an unprivileged user will fail with permission errors.
Options:
- Add your user to the appropriate group:
```bash
sudo usermod -aG disk $USER
```
Then log out and back in for the change to take effect. On some distributions the group is `cdrom` or `optical` instead of `disk`.
- Run with elevated privileges:
```bash
sudo freemkv ...
```
- Install a udev rule for persistent per-device permissions. Create `/etc/udev/rules.d/99-sg-optical.rules`:
```
SUBSYSTEM=="scsi_generic", ATTRS{type}=="5", MODE="0666"
```
Then reload udev rules:
```bash
sudo udevadm control --reload-rules && sudo udevadm trigger
```
### Spin-Up Delay
Optical drives take 30-60 seconds to spin up and become ready after hot-plug or disc insertion. During this window, SCSI commands may return NOT READY or timeout.
freemkv's `Drive::wait_ready()` handles this automatically by polling with TEST UNIT READY until the drive responds. If you are writing your own code using the library, always call `wait_ready()` before `init()`:
```rust
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?; // blocks until disc is ready, up to 30s
drive.init()?;
```
If the drive was just plugged in, wait a full minute before concluding it is not detected.
---
## 4. How to Identify Your USB-SATA Bridge
If you are experiencing the issues described in section 1, you need to know which bridge chipset your adapter or enclosure uses.
### Step 1: Find the Device
```bash
lsusb
```
Look for entries matching your drive or enclosure. Bridges may appear under their own manufacturer name or as a generic SATA device. Common examples:
```
Bus 002 Device 005: ID 13fd:0840 Initio Corporation
Bus 002 Device 006: ID 174c:1153 ASMedia Technology Inc. ASM1153
Bus 002 Device 007: ID 152d:0578 JMicron Technology Corp. JMS578
```
### Step 2: Match the IDs
| Vendor | Product ID | Chipset | Status |
|--------|-----------|---------|--------|
| `13fd` | `3609` | Initio INIC-3609 | Affected. Apply quirk. |
| `13fd` | `3940` | Initio INIC-3619 | Affected. Apply quirk. |
| `13fd` | `0840` | Initio INIC-3069 | Affected. Apply quirk. |
| `174c` | `5106` | ASMedia ASM1051 | Affected (early firmware). Apply quirk. |
| `174c` | `1153` | ASMedia ASM1153 | Good. No quirk needed. |
| `152d` | `0561` | JMicron JMB36x | Affected (some firmware). Apply quirk if issues occur. |
| `152d` | `0578` | JMicron JMS578 | Good. No quirk needed. |
### Step 3: Check dmesg for the Bridge Name
```bash
dmesg | grep -i "usb-storage\|uas\|initio\|asmedia\|jmicron"
```
This often reveals the bridge chipset even when `lsusb` shows a generic name.
### Step 4: If the Enclosure Is Sealed
Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not advertise the bridge chipset on the packaging. In this case:
1. Check `lsusb` while the enclosure is connected.
2. Search the vendor:product ID online -- there are community-maintained lists of which chipsets popular enclosures use.
3. If you cannot determine the chipset and are experiencing bridge crashes, assume it is an Initio and apply the quirk with its IDs.
4. The definitive test: connect the bare drive to a motherboard SATA port. If the problems disappear, the bridge was the cause.
---
## 5. General Debugging Checklist
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
6. **SCSI device:** which `/dev/sg*` the drive was on, and whether it changed after the failure.
7. **The disc:** title, format (BD/DVD/UHD), condition.
Include all of the above in bug reports. SCSI transport errors that resolve with the `US_FL_IGNORE_RESIDUE` quirk or by switching to direct SATA are bridge firmware bugs, not freemkv bugs.
+10 -5
View File
@@ -8,13 +8,12 @@ fn main() {
let device = std::env::args()
.skip(1)
.find(|a| !a.starts_with('-'))
.unwrap_or_else(|| {
let drives = libfreemkv::find_drives();
if drives.is_empty() {
.unwrap_or_else(|| match libfreemkv::find_drive() {
Some(d) => d.device_path().to_string(),
None => {
eprintln!("No drives found");
std::process::exit(1);
}
drives[0].device_path().to_string()
});
let mut drive = Drive::open(Path::new(&device)).unwrap_or_else(|e| {
@@ -44,9 +43,15 @@ fn main() {
let mut fail = 0u32;
let mut bytes: u64 = 0;
// Recovery flag: true matches pre-0.11.13 bench behavior — full SCSI
// ECC retry loop on errors (slower, what the rip path used before the
// adaptive batch sizer landed). Flip to `false` for the fast-fail path
// that current rips use; benches are configurable via this constant.
const READ_WITH_RECOVERY: bool = true;
for i in 0..1000u32 {
let lba = i * batch as u32;
match drive.read(lba, batch, &mut buf) {
match drive.read(lba, batch, &mut buf, READ_WITH_RECOVERY) {
Ok(_) => {
writer.write_all(&buf).unwrap();
ok += 1;
+29
View File
@@ -3,5 +3,34 @@ fn main() {
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
let lib = format!("{out_dir}/libmacos_scsi.a");
std::process::Command::new("cc")
.args([
"-c",
"src/scsi/macos_shim.c",
"-o",
&obj,
"-framework",
"IOKit",
"-framework",
"CoreFoundation",
"-Wall",
"-O2",
])
.status()
.expect("failed to compile macos_shim.c");
std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to create static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
println!("cargo:rerun-if-changed=src/scsi/macos_shim.c");
}
}
+1
View File
@@ -12,6 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
+72 -40
View File
@@ -44,35 +44,30 @@ while let Ok(Some(frame)) = input.read() {
output.finish()?;
```
The `pes::Stream` trait:
The `FrameSource` and `FrameSink` traits — direction is type-checked, so
calling `read()` on a write-only sink (or `write()` on a read-only source)
is a compile error rather than a runtime fault:
```rust
pub trait Stream {
fn read(&mut self) -> io::Result<Option<PesFrame>>;
fn write(&mut self, frame: &PesFrame) -> io::Result<()>;
fn finish(&mut self) -> io::Result<()>;
pub trait FrameSource: Send {
fn read(&mut self) -> Result<Option<PesFrame>, Error>;
fn info(&self) -> &DiscTitle;
fn codec_private(&self, track: usize) -> Option<Vec<u8>>;
fn headers_ready(&self) -> bool;
fn codec_private(&self, track: usize) -> Option<Vec<u8>> { None }
fn headers_ready(&self) -> bool { true }
}
```
## IOStream (byte-level API)
For raw byte copies (disc→ISO, resume, benchmarks). Lower level than PES.
```rust
let opts = InputOptions::default();
let mut input = open_input("iso://Disc.iso", &opts)?;
let mut output = open_output("mkv://Movie.mkv", input.info())?;
io::copy(&mut *input, &mut *output)?;
output.finish()?;
pub trait FrameSink: Send {
fn write(&mut self, frame: &PesFrame) -> Result<(), Error>;
fn finish(self: Box<Self>) -> Result<(), Error>;
fn info(&self) -> &DiscTitle;
}
```
## Streams
All streams implement `IOStream` (byte-level) and/or `pes::Stream` (frame-level).
URL-based resolvers open any stream by string.
All streams implement `FrameSource` (read) and/or `FrameSink` (write); the
directional split prevents runtime "wrong-direction" errors. URL-based
resolvers open any stream by string.
| Stream | Input | Output | URL | Transport |
|--------|-------|--------|-----|-----------|
@@ -87,21 +82,14 @@ URL-based resolvers open any stream by string.
All URLs require a `scheme://path` format. Bare paths are rejected.
```rust
// PES pipeline (frame-level)
// PES pipeline (frame-level) — input() returns Box<dyn FrameSource>,
// output() returns Box<dyn FrameSink>.
let input = libfreemkv::input("disc:///dev/sg4", &opts)?; // DiscStream
let input = libfreemkv::input("iso://Dune.iso", &opts)?; // IsoStream
let output = libfreemkv::output("mkv://Dune.mkv", &title)?; // MkvOutputStream
let output = libfreemkv::output("m2ts://Dune.m2ts", &title)?; // M2tsOutputStream
let output = libfreemkv::output("network://10.0.0.1:9000", &title)?; // NetworkOutputStream
let output = libfreemkv::output("null://", &title)?; // NullOutputStream
// IOStream (byte-level)
let input = open_input("disc://", &opts)?; // DiscStream
let input = open_input("iso://Dune.iso", &opts)?; // IsoStream
let output = open_output("iso://Copy.iso", &meta)?; // IsoStream (write)
let output = open_output("mkv://Dune.mkv", &meta)?; // MkvStream
let output = open_output("m2ts://Dune.m2ts", &meta)?; // M2tsStream
let output = open_output("null://", &meta)?; // NullStream
```
### FMKV Metadata Header
@@ -126,15 +114,45 @@ pub struct Event {
}
pub enum EventKind {
// Init / scan
DriveOpened { device: String },
DriveReady,
InitComplete { success: bool },
ProbeComplete { success: bool },
ScanComplete { titles: usize },
// Read pipeline
BytesRead { bytes: u64, total: u64 },
ReadError { sector: u64, error: Error },
Retry { attempt: u32 },
SpeedChange { speed_kbs: u16 },
ExtentStart { index: usize, start_sector: u64, sector_count: u64 },
SectorSkipped { sector: u64 },
BatchSizeChanged { new_size: u16, reason: BatchSizeReason },
Complete { bytes: u64, errors: u32 },
// Kept for forward-compat; not emitted in 0.13.6+
Retry { attempt: u32 },
SectorRecovered { sector: u64 },
}
```
Emission notes:
- `BytesRead { bytes, total }` is emitted from `DiscStream::fill_extents`
after each successful sector read. `bytes` is the cumulative running
total; `total` is the precomputed extent sum (0 if unknown).
- `SpeedChange` is emitted from the public `Drive::set_speed` API path.
It is no longer emitted from a recovery hot loop (recovery loop removed
in 0.13.6).
- `BatchSizeChanged` fires from the `DiscStream` adaptive sizer on shrink
(read failed at a larger size) and on probe-up (clean-read streak hit
the threshold). Consumers use it to display a "recovering" state
distinct from "ripping normally".
- `Retry` and `SectorRecovered` are NOT emitted in 0.13.6+. They were
tied to the inline `Drive::read` recovery phases that were removed; the
variants are kept for forward compatibility so consumers' match arms
don't need conditional compilation.
Events report what happened. App decides what to do. GUI shows a dialog. CLI
prints a line. Server logs to file.
@@ -145,20 +163,34 @@ libfreemkv/src/
├── lib.rs Public exports
├── error.rs Error codes (no English)
├── event.rs Event types for callbacks
├── drive/ Drive (open, init, read with recovery)
│ ├── mod.rs Drive struct, init, read, reset, eject
├── halt.rs Halt cancellation token (Arc<AtomicBool> wrapper)
├── io/ Pipeline + WritebackFile primitives
│ ├── mod.rs Re-exports WritebackFile, Pipeline, Sink, Flow
│ ├── pipeline.rs Generic Pipeline<I, R> + Sink trait
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
│ ├── capture.rs Drive profile capture for contribution
│ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup)
├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
│ ├── mod.rs Disc struct, scan, titles, formats
│ ├── sweep.rs Disc::sweep (Pass 1 forward sweep)
│ ├── patch.rs Disc::patch (Pass N retry over mapfile)
│ ├── mapfile.rs ddrescue-format mapfile
│ └── read_error.rs ReadCtx / ReadAction state machine
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── platform/ Drive unlock (MT1959 A/B)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
├── css/ DVD CSS cipher
├── decrypt.rs Unified decrypt dispatcher (AACS/CSS/None)
├── pes.rs PES frame types, Stream trait
├── sector.rs SectorReader trait
├── pes.rs PES frame types, FrameSource / FrameSink traits
├── sector/ Sector I/O (was sector.rs in 0.17)
│ ├── mod.rs SectorSource, SectorSink traits
│ ├── file.rs FileSectorSource, FileSectorSink (ISO-backed)
│ └── decrypting.rs DecryptingSectorSource decorator
├── udf.rs UDF 2.50 filesystem parser
├── mpls.rs MPLS playlist parser
├── clpi.rs CLPI clip info parser
@@ -169,15 +201,15 @@ libfreemkv/src/
├── profile.rs Bundled drive profiles
├── speed.rs DriveSpeed enum
├── mux/
│ ├── mod.rs IOStream trait, public exports
│ ├── resolve.rs URL parser + open_input/open_output + input/output
│ ├── mod.rs Public mux exports
│ ├── resolve.rs URL parser + input/output (Box<dyn FrameSource/Sink>)
│ ├── meta.rs FMKV header format
│ ├── disc.rs DiscStream (optical drive → PES)
│ ├── iso.rs IsoStream (ISO image read/write)
│ ├── iso.rs IsoStream (ISO image read)
│ ├── isowriter.rs ISO image writer (UDF, AVDP, multi-extent)
│ ├── mkvstream.rs MkvStream (bidirectional Matroska, IOStream)
│ ├── mkvstream.rs MkvStream (bidirectional Matroska)
│ ├── mkvout.rs MkvOutputStream (PES → MKV)
│ ├── m2ts.rs M2tsStream (BD-TS, IOStream)
│ ├── m2ts.rs M2tsStream (BD-TS)
│ ├── pesout.rs PES output streams (M2ts, Network, Stdio, Null)
│ ├── network.rs NetworkStream (TCP + FMKV header)
│ ├── stdio.rs StdioStream (stdin/stdout pipe)
+18 -7
View File
@@ -41,8 +41,8 @@ AACS keys are derived internally, and all SCSI communication is handled in-proce
libfreemkv (lib.rs)
│
├── Drive Access
│ ├── drive Drive — open, identify, init, unlock, read (with recovery)
│ ├── scsi ScsiTransport trait + platform backends (SG_IO, IOKit, SPTI)
│ ├── drive Drive — open, identify, init, unlock, single-shot read
│ ├── scsi ScsiTransport trait + platform backends (sg async, IOKit, SPTI)
│ ├── platform/ Platform trait — per-chipset command handlers
│ │ └── mt1959 MediaTek MT1959 driver (LG, ASUS, HP)
│ ├── profile DriveProfile loading, matching, bundled JSON
@@ -65,8 +65,12 @@ libfreemkv (lib.rs)
│
├── Streaming
│ ├── mux/ Stream implementations (Disc, ISO, MKV, M2TS, Network, Stdio, Null)
│ ├── pes PES frame types, Stream trait (read/write frames)
│ └── sector SectorReader trait — abstracts disc vs ISO vs file
│ ├── pes PES frame types; FrameSource / FrameSink direction-typed traits
│ └── sector/ SectorSource / SectorSink traits, FileSector{Source,Sink}, DecryptingSectorSource
│
├── I/O Primitives
│ ├── halt Halt cancellation token (one Arc<AtomicBool>, cloneable)
│ └── io/ Pipeline<I, R> + Sink trait + WritebackFile (bounded-cache writer)
│
├── Support
│ ├── keydb KEYDB.cfg download, parse, verify, save
@@ -83,7 +87,7 @@ libfreemkv (lib.rs)
```
Drive::open(Path::new("/dev/sg4"))
│
├─ scsi::open() Open /dev/sg4 via SG_IO
├─ scsi::open() Open /dev/sg4 (async write/poll/read)
├─ DriveId::from_drive() INQUIRY + GET_CONFIG 010C
├─ profile::find_by_drive_id() Match against bundled profiles
├─ Platform::new() Instantiate chipset driver (Mt1959)
@@ -93,10 +97,17 @@ Drive::open(Path::new("/dev/sg4"))
After open:
- `init()` -- unlock + firmware upload + speed calibration
- `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf)` -- single read method with built-in error recovery
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
- `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1
(`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
the stop-wedge postmortem (2026-04-25).
---
## Disc Scanning Flow
@@ -175,7 +186,7 @@ is baked into the library.
| Platform | Transport | Status |
|----------|-----------|--------|
| Linux | SG_IO ioctl on `/dev/sg*` | Supported |
| Linux | async sg write/poll/read on `/dev/sg*` | Supported |
| macOS | IOKit SCSITask | Supported |
| Windows | SPTI (`IOCTL_SCSI_PASS_THROUGH_DIRECT`) | Supported |
+12 -6
View File
@@ -77,7 +77,9 @@ Insert disc
10. Stream content (mux/disc.rs → DiscStream)
│ Read sectors → decrypt → TS demux → PES frames
│ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() handles error recovery (min speed → reset → retry)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — Disc::patch re-runs against the mapfile.
│
▼
PES frames → output stream (MKV, M2TS, network, etc.)
@@ -95,7 +97,9 @@ drive.probe_disc()?;
// Scan disc (UDF + playlists + AACS — all automatic)
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
// Stream pipeline — PES frames from any source to any output
// Stream pipeline — PES frames from any source to any output.
// 0.18: input() returns Box<dyn FrameSource>, output() returns Box<dyn FrameSink>;
// direction is type-checked, so calling .write() on an input is a compile error.
let opts = InputOptions::default();
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
let title = input.info().clone();
@@ -110,7 +114,7 @@ output.finish()?;
| Module | Doc | Purpose |
|--------|-----|---------|
| drive/ | [drive-access.md](drive-access.md) | Open, identify, init, unlock, read (with recovery) |
| drive/ | [drive-access.md](drive-access.md) | Open, identify, init, unlock, single-shot read |
| scsi/ | [drive-access.md](drive-access.md) | Platform SCSI transport (Linux, macOS, Windows) |
| udf.rs | [udf.md](udf.md) | UDF 2.50 filesystem |
| mpls.rs | [mpls.md](mpls.md) | MPLS playlists + STN streams |
@@ -119,11 +123,13 @@ output.finish()?;
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | -- | High-level scan + read API |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
| labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + Stream trait |
| sector.rs | -- | SectorReader trait |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
| sector/ | -- | SectorSource / SectorSink + DecryptingSectorSource decorator |
| io/ | -- | Pipeline<I, R> + Sink trait + WritebackFile |
| halt.rs | -- | Halt cancellation token |
| keydb.rs | -- | KEYDB download, parse, save |
| error.rs | -- | Error codes (E1xxx-E8xxx) |
| event.rs | -- | Drive event system |
+57 -18
View File
@@ -26,8 +26,8 @@ platform driver. The drive is ready for `wait_ready()` and `init()`.
| `wait_ready()` | Wait for disc insertion (30s timeout, TUR polling) |
| `init()` | Firmware upload + unlock + speed calibration |
| `probe_disc()` | Probe disc surface for optimal speeds |
| `read(lba, count, buf)` | Read sectors with built-in error recovery |
| `reset()` | Close/reopen device, TUR, escalate if needed |
| `read(lba, count, buf, recovery)` | Read sectors. Single-shot — no inline retries or reset. |
| `reset()` | Eject-cycle escape hatch. Caller-invoked only; not on the read path. |
| `lock_tray()` | Prevent tray ejection during rip |
| `unlock_tray()` | Allow tray ejection (also runs on Drop) |
| `eject()` | Eject disc tray |
@@ -44,16 +44,33 @@ platform driver. The drive is ready for `wait_ready()` and `init()`.
3. Speed calibration after unlock
4. Max 3 attempts before giving up
### read() with Recovery
### read() — single-shot
`Drive::read()` is the single read method. On error:
`Drive::read(lba, count, buf, recovery)` is the single read method. It issues
exactly one READ(10) CDB and returns the result. The `recovery` parameter only
selects the per-CDB timeout:
1. Set minimum speed immediately
2. Reset device (close/reopen/TUR)
3. Wait 2s for drive to settle
4. Retry at min speed, min batch (3 sectors)
5. If still failing: skip sectors, zero-fill, log
6. Stay at min speed for 500 MB after error (recovery window)
| `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------|
| `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 30 s | `Disc::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`:
- **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read
streak.
Inline recovery (5× gentle retry → close + reset + reopen → 5× more) was
removed in 0.13.6. See the stop-wedge postmortem (2026-04-25)
for rationale: the inline reset wedged drive firmware on the LG BU40N (Initio
USB-SATA bridge) without ever recovering a sector. See
[`rip-recovery.md`](rip-recovery.md) for the full three-layer model.
---
@@ -70,8 +87,6 @@ pub trait ScsiTransport: Send {
data: &mut [u8],
timeout_ms: u32,
) -> Result<ScsiResult>;
fn reset(&mut self, device: &str) -> Result<()>;
}
```
@@ -82,15 +97,39 @@ descriptors or calls ioctls outside of a `ScsiTransport` implementation.
| Platform | Implementation | Device |
|----------|---------------|--------|
| Linux | `SgIoTransport` — `ioctl(fd, SG_IO, &hdr)` | `/dev/sg*` |
| Linux | `SgIoTransport` — async `write`/`poll`/`read` on `/dev/sg*` | `/dev/sg*` |
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
| Windows | `WindowsScsiTransport` — SPTI | `\\.\CdRomN` |
The Linux backend opens with `O_RDWR | O_NONBLOCK`, constructs `sg_io_hdr`,
and returns `ScsiResult` with status, bytes transferred, and sense data.
The Linux backend uses the sg driver's asynchronous interface: `write()` submits
the command, `poll()` waits with an enforceable wall-clock timeout, `read()`
retrieves the result. If `poll()` times out, the fd is abandoned (closed in a
background thread) and a fresh fd opened — the kernel's USB error recovery
cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
On non-zero SCSI status, the transport parses sense key, ASC, and ASCQ from the
sense buffer and returns `Error::ScsiError`.
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
The shim handles:
1. `shim_open_exclusive(bsd_name)` — unmounts the target device via `diskutil`,
then walks the IOKit registry to find the `IOBDServices` matching the
requested BSD name (IOBDServices → IOBDBlockStorageDriver → IOMedia → "BSD Name"),
then creates MMCDeviceInterface → SCSITaskDeviceInterface → ObtainExclusiveAccess.
2. `shim_list_drives()` — registry-based enumeration with zero SCSI, zero exclusive
access, zero unmounts. Reads IOBDServices "Device Characteristics" for
vendor/model/firmware and child IOMedia "BSD Name" for the device path.
3. `shim_execute()` / `shim_close()` — raw CDB dispatch and cleanup.
On non-zero SCSI status, the transport parses sense key from the sense buffer
and returns `Error::ScsiError`.
`SgIoTransport::reset` (Linux) does pure userspace state cleanup: an open +
close pair to make the kernel cancel any SG_IO commands queued against a
previous fd, a 2 s sleep to let the kernel finish that cancellation, then a
fresh fd to send ALLOW MEDIUM REMOVAL to clear any stale tray lock. It does
NOT issue `SG_SCSI_RESET` or escalate via STOP+START UNIT. Both were tried
in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA bridge); both failed
to recover wedged drives and made the wedge worse. The macOS reset (which
had been a no-op) was removed entirely in 0.13.6, and the top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
removed at the same time (no callers).
### CDB Builders
+201
View File
@@ -0,0 +1,201 @@
# Rip recovery — three-layer architecture
`libfreemkv` supports a multi-stage rip model for damaged or protection-bearing
discs: a fast forward sweep that tolerates read failures, in-loop request-size
adaptation that survives transient drive trouble without bailing, and targeted
retry passes against a persistent bad-range map. The stream pipeline
(`DiscStream` + `input`/`output`) operates against the resulting ISO image, so
the mux stage never touches the drive.
Recovery is layered cleanly. Each layer has one responsibility and does not
reach into the others.
| Layer | Where it lives | What it does |
|-------|---------------|--------------|
| 1 — Bad-range retry | `Disc::patch` (one pass over the mapfile per call) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
The library exposes flat verbs; the caller drives the multipass loop. Autorip
runs `Disc::sweep` once, then loops `Disc::patch` until either the mapfile is
clean or the configured retry budget is exhausted, then hands the ISO off to
the mux pipeline. The `freemkv` CLI does the same shape with a
terminal-output progress sink. Layer 3 runs inside any consumer of
`DiscStream` (direct PES pipeline, ISO playback, etc.) without caller
involvement.
Three primitives compose the disc-side flow:
| Primitive | What it does |
|---------------------------|-----------------------------------------------------------------------|
| `Disc::sweep` | disc → ISO, one forward pass. Writes a sidecar `.mapfile`. Opt-in skip-on-error. |
| `Disc::patch` | Re-reads bad ranges from the drive. One pass per call; caller invokes N times. |
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
## Data model
### Mapfile
Format: [ddrescue](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)-compatible
plain text, greppable, tool-interoperable. Flushed to disk on every `record()`
so a crashed rip loses at most one block.
```
# Rescue Logfile. Created by libfreemkv v0.13.6
# Current pos / status / pass / pass_time
0x000000000 ? 1 0
# pos size status
0x000000000 0x12a35d000 +
0x12a35d000 0x000003000 -
0x12a360000 0x009c4a000 +
0x12d00a000 0x000064000 *
```
Status characters match ddrescue:
| Char | Meaning |
|------|----------------------------------------------------|
| `?` | Not yet attempted |
| `*` | Fast-pass failed; needs edge-trim |
| `/` | Trimmed; interior needs sector scrape |
| `-` | Unreadable this session |
| `+` | Finished (good) |
Position and size are hex byte offsets into the ISO.
### `SweepOptions` and `PatchOptions`
The library no longer dispatches between sweep and patch internally — the
caller picks the verb explicitly per pass. The two option structs are flat
and have no overlap:
```rust
SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true, // damage-jump + zero-fill on read failure
progress: Some(&reporter),
halt: Some(flag),
}
PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true, // walk bad ranges high → low LBA
wedged_threshold: 50,
progress: Some(&reporter),
halt: Some(flag),
}
```
Caller-orchestrated dispatch (the policy `Disc::copy` used to embed):
- No mapfile → `sweep` (fresh Pass 1).
- Mapfile with `?` ranges → `sweep` with `resume: true`.
- Mapfile covers full disc, only `*` / `/` / `-` ranges → `patch`.
- Mapfile clean → done; no further pass needed.
Each consumer (autorip, `freemkv` CLI) implements the loop in roughly five
lines of `Mapfile::stats()` checks.
## Algorithm
### Pass 1 — fast sweep (`Disc::sweep`)
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
2. On success: write data to ISO, mark `+`, advance.
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
→ **damage-jump**: skip ahead by `256×batch×multiplier` sectors (8 MB base for
UHD). Double the multiplier on each jump (8→16→32→64 MB...). Zero-fill the gap as `*`.
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
7. Only transport failures (USB bridge crash) abort the pass.
Pass 1 completes when every byte has been visited (either `+` or `*`).
### Pass 2+ — patch (`Disc::patch`)
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
(walks ranges from highest LBA to lowest, within each range from end to start).
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
does its own ECC recovery inside that window.
2. On success: write the good bytes into the ISO, mark `+`.
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
4. On failure with marginal sense: mark `-`, continue.
5. Update the mapfile after every block — crash-safe resume.
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
When a consumer reads a `DiscStream` directly (no ISO intermediate),
`fill_extents` runs an adaptive sizer in front of `Drive::read`:
1. Try the current preferred batch size (e.g. 32 sectors, one BD ECC block).
2. On failure: halve the batch and retry at the same LBA. Emit
`EventKind::BatchSizeChanged { reason: Shrunk }`.
3. On a clean-read streak: probe back up toward the preferred size. Emit
`EventKind::BatchSizeChanged { reason: Probed }`.
4. If a single-sector read fails: skip (zero-fill, emit
`EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return
`Err(DiscRead)`.
This is layer 3. It exists so a transient single-sector glitch in a 32-sector
batch can be isolated and read individually without the caller needing to
implement retry logic.
## Design choices
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset,
no eject cycle. The `recovery` flag controls only the per-CDB timeout
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
was removed in 0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale:
the inline reset on the LG BU40N (Initio USB-SATA bridge)
wedged drive firmware below the bridge without ever recovering a sector,
and the gentle-retry phase produced long stretches of 0 KB/s with no
recoveries to show for it. Recovery responsibility is now layered: layer 1
handles ranges, layer 3 handles request size, neither touches the
wedge-prone reset path.
**No `MODE SELECT` to disable drive retries.** Research showed neither ddrescue
nor MakeMKV does this. Drive firmware has access to raw analog signal, laser
power control, and drive-specific ECC tuning that userspace can't replicate —
disabling it throws away recovery headroom on marginal sectors. We fail fast
via short SG_IO timeouts in pass 1 and let the firmware work the long timeout
in pass 2 / patch.
**No SCSI reset from any retry path.** `SgIoTransport::reset` (Linux) is
trimmed to a kernel SG_IO state flush plus ALLOW MEDIUM REMOVAL — the
`SG_SCSI_RESET` ioctl and STOP/START UNIT escalation were removed in 0.13.6.
The macOS reset (which had been a no-op) was removed entirely. The top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
also removed (no callers). The remaining `Drive::reset()` is only invoked
explicitly by callers that need an eject-cycle escape hatch — it is never
reached from a read path.
**ISO intermediate, even for single-pass.** Pass 1 always writes an ISO. The
mux stage reads the ISO via `IsoSectorReader`. For single-pass (no retries),
this adds ~2-3 min (local disk mux) but gains resumability across crashes,
re-muxability without re-ripping, and a persistent forensic artifact. Callers
who need pure speed can bypass and use `DiscStream::new(Box::new(drive), …)`
directly — the lib doesn't forbid it, and layer 3 (adaptive batch halving)
still applies there.
**Mapfile in ddrescue format.** Plain text so users can `less` it, `diff` it,
or feed it to ddrescue's own tooling. Crash-safe (flush-per-record). Entries
coalesce on adjacent same-status ranges so files stay small.
**Patches target `-`, `*`, `/`, and `?` alike.** The status state machine is
ddrescue's but `patch` collapses the distinction — it just tries every
non-finished range with the long timeout. Future work can specialize (trim vs.
scrape vs. retry with direction reversal) if there's measured benefit.
## References
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/sweep.rs`](../src/disc/sweep.rs) (`Disc::sweep`), [`src/disc/patch.rs`](../src/disc/patch.rs) (`Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
+1 -1
View File
@@ -126,7 +126,7 @@ USB optical drives have ~500ms round-trip latency per SCSI command. Since `read_
`Disc::scan()` wraps the drive in a `BufferedSectorReader` before reading. On a single-sector read, the buffer prefetches a batch of sectors (sized from the kernel's `max_hw_sectors_kb` for the device) and caches them. Subsequent reads to nearby LBAs return from cache with zero SCSI overhead. After parsing the UDF directory structure, the entire metadata partition is pre-read into the cache, so all ICB lookups during title scanning and encryption resolution are instant.
The buffer is transparent -- `read_filesystem()`, `read_file()`, and all downstream code still call `read_sectors(lba, 1, buf)` as before. The batching happens inside the `SectorReader` implementation.
The buffer is transparent -- `read_filesystem()`, `read_file()`, and all downstream code still call `read_sectors(lba, 1, buf)` as before. The batching happens inside the `SectorSource` implementation.
### UDF Filename Encoding
+1 -1
View File
@@ -42,7 +42,7 @@ fn main() {
// Tiny yield between reads — test if pacing prevents firmware throttle
std::thread::yield_now();
let t0 = Instant::now();
let ok = drive.read(lba, count, &mut buf[..n]).is_ok();
let ok = drive.read(lba, count, &mut buf[..n], true).is_ok();
let read_ms = t0.elapsed().as_millis();
// Flag slow reads
+1 -1
View File
@@ -1,7 +1,7 @@
//! AACS content decryption — AES primitives, unit decryption, bus encryption.
use aes::cipher::{generic_array::GenericArray, BlockDecrypt, BlockEncrypt, KeyInit};
use aes::Aes128;
use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit, generic_array::GenericArray};
// ── AACS constants ──────────────────────────────────────────────────────────
+1 -1
View File
@@ -674,8 +674,8 @@ fn generate_host_key_pair() -> ([u8; 20], [u8; 20], [u8; 20]) {
/// AES-128-CMAC over 16 bytes of data.
fn aes_cmac_16(data: &[u8; 16], key: &[u8; 16]) -> [u8; 16] {
use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit};
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
let cipher = Aes128::new(GenericArray::from_slice(key));
+1 -5
View File
@@ -361,11 +361,7 @@ mod tests {
/// Get KEYDB path from KEYDB_PATH environment variable. Returns None if not set or not found.
fn keydb_path() -> Option<std::path::PathBuf> {
let path = std::path::PathBuf::from(std::env::var("KEYDB_PATH").ok()?);
if path.exists() {
Some(path)
} else {
None
}
if path.exists() { Some(path) } else { None }
}
#[test]
+3 -7
View File
@@ -661,18 +661,14 @@ pub fn resolve_keys(
#[cfg(test)]
mod tests {
use super::super::decrypt::{aes_ecb_encrypt, ALIGNED_UNIT_LEN};
use super::super::decrypt::{ALIGNED_UNIT_LEN, aes_ecb_encrypt};
use super::super::keydb::{DiscEntry, KeyDb};
use super::*;
/// Get KEYDB path from KEYDB_PATH environment variable. Returns None if not set or not found.
fn keydb_path() -> Option<std::path::PathBuf> {
let path = std::path::PathBuf::from(std::env::var("KEYDB_PATH").ok()?);
if path.exists() {
Some(path)
} else {
None
}
if path.exists() { Some(path) } else { None }
}
#[test]
@@ -849,7 +845,7 @@ mod tests {
data[23] = 1; // top_menu = CPS unit 1
data[24] = 0;
data[25] = 1; // num_titles = 1
// Title 0 entry: 2 bytes pad + CPS unit
// Title 0 entry: 2 bytes pad + CPS unit
data[28] = 0;
data[29] = 1; // CPS unit 1
+5 -5
View File
@@ -21,12 +21,12 @@ pub mod keys;
// Explicit re-exports — only items needed by external consumers and sibling crate modules.
// AES primitives (aes_ecb_encrypt, aes_ecb_decrypt, aes_cbc_decrypt) are pub(crate) in decrypt.rs.
pub use decrypt::{
decrypt_bus, decrypt_unit, decrypt_unit_full, decrypt_unit_try_keys, is_unit_encrypted,
ALIGNED_UNIT_LEN,
ALIGNED_UNIT_LEN, decrypt_bus, decrypt_unit, decrypt_unit_full, decrypt_unit_try_keys,
is_unit_encrypted,
};
pub use keydb::{DeviceKey, DiscEntry, HostCert, KeyDb};
pub use keys::{
decrypt_unit_key, derive_media_key_from_dk, derive_media_key_from_pk, derive_vuk, disc_hash,
disc_hash_hex, mkb_version, parse_content_cert, parse_unit_key_ro, read_mkb_from_drive,
resolve_keys, ContentCert, ResolvedKeys, UnitKeyFile,
ContentCert, ResolvedKeys, UnitKeyFile, decrypt_unit_key, derive_media_key_from_dk,
derive_media_key_from_pk, derive_vuk, disc_hash, disc_hash_hex, mkb_version,
parse_content_cert, parse_unit_key_ro, read_mkb_from_drive, resolve_keys,
};
+2 -1
View File
@@ -348,7 +348,7 @@ fn read_disc_key(drive: &mut Drive, agid: u8, bus_key: &[u8; 5]) -> Result<[u8;
let candidate = super::lfsr::decrypt_key(0x00, player_key, &enc);
// Check if any previous candidate matches (same disc key from different entry/pk)
for &(ref prev, _, _) in &candidates {
for (prev, _, _) in &candidates {
if *prev == candidate {
return Ok(candidate);
}
@@ -391,6 +391,7 @@ fn read_raw_title_key(drive: &mut Drive, agid: u8, lba: u32) -> Result<[u8; 5]>
Ok(key)
}
#[allow(dead_code)]
fn read_title_key(
drive: &mut Drive,
agid: u8,
+1 -1
View File
@@ -149,7 +149,7 @@ mod tests {
let key = [0x01, 0x02, 0x03, 0x04, 0x05];
let mut sector = vec![0xAA; 2048];
sector[0x14] = 0x30; // scramble flag set
// Set a sector seed
// Set a sector seed
sector[0x54..0x59].copy_from_slice(&[0x11, 0x22, 0x33, 0x44, 0x55]);
let original = sector.clone();
descramble_sector(&key, &mut sector);
+5 -1
View File
@@ -42,7 +42,11 @@ pub fn crack_key(reader: &mut dyn SectorReader, extents: &[Extent]) -> Option<Cs
let mut i = 0;
while i < ext.sector_count && tried < max_tries {
let mut buf = vec![0u8; 2048];
if reader.read_sectors(ext.start_lba + i, 1, &mut buf).is_ok() && is_scrambled(&buf) {
if reader
.read_sectors(ext.start_lba + i, 1, &mut buf, true)
.is_ok()
&& is_scrambled(&buf)
{
if let Some(key) = crack::crack_title_key(&buf) {
return Some(CssState { title_key: key });
}
+50 -1
View File
@@ -62,7 +62,23 @@ pub fn decrypt_sectors(
for chunk in buf.chunks_mut(unit_len) {
if chunk.len() == unit_len && aacs::is_unit_encrypted(chunk) {
aacs::decrypt_unit_full(chunk, &uk, rdk);
// `is_unit_encrypted` is a byte-0 heuristic: it fires on any
// unit whose first byte has the top 2 bits set, which is
// correct for m2ts source packets (where those bits are the
// copy-control marker) but false-positives on any other binary
// data with similarly-shaped first bytes — notably MPLS/CLPI
// navigation files that begin with ASCII magic ('M', 'H'…)
// and survive sweep mixed in with encrypted m2ts payloads.
// `decrypt_unit_full` self-checks via TS-sync verification and
// returns false on a misfire, but it has already mutated the
// chunk by then. Snapshot and restore on verification failure
// — same pattern `decrypt_unit_try_keys` uses for multi-key
// discs. Real m2ts units verify and stay decrypted; nav-file
// sectors get scrambled briefly and then put back as-was.
let original: Vec<u8> = chunk.to_vec();
if !aacs::decrypt_unit_full(chunk, &uk, rdk) {
chunk.copy_from_slice(&original);
}
}
}
}
@@ -74,3 +90,36 @@ pub fn decrypt_sectors(
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// Regression for the 0.18.1 nav-file scramble bug. A non-m2ts unit whose
/// first byte has the top 2 bits set (here: the ASCII letter 'M' that
/// MPLS files start with, 0x4D = 0b01001101) trips `is_unit_encrypted`,
/// gets AES-decrypted with the unit key, fails the TS-sync verification,
/// and must be restored to its original bytes — not left scrambled.
#[test]
fn nav_file_unit_survives_decrypt_attempt() {
let mut unit = vec![0u8; aacs::ALIGNED_UNIT_LEN];
unit[0] = b'M';
unit[1] = b'P';
unit[2] = b'L';
unit[3] = b'S';
for (i, b) in unit.iter_mut().enumerate().skip(4) {
*b = (i as u8).wrapping_mul(31);
}
let snapshot = unit.clone();
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0, [0xAB; 16])],
read_data_key: None,
};
decrypt_sectors(&mut unit, &keys, 0).unwrap();
assert_eq!(
unit, snapshot,
"non-m2ts unit must be restored after failed decrypt"
);
}
}
+5 -2
View File
@@ -132,6 +132,7 @@ impl Disc {
codec,
language: s.language.clone(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
}))
} else {
@@ -142,6 +143,7 @@ impl Disc {
language: s.language.clone(),
sample_rate: SampleRate::from_audio_rate(s.audio_rate),
secondary: s.stream_type == 5,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
}))
}
@@ -151,6 +153,7 @@ impl Disc {
codec,
language: s.language.clone(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
})),
// Stream type 4 = IG, unknown types -- skip
@@ -159,12 +162,12 @@ impl Disc {
})
.collect();
// Convert marks to chapters (filter mark_type == 1 = chapter entry)
// Convert marks to chapters (mark_type 0 or 1 = chapter entry, 2 = link)
let first_in_time = parsed.play_items.first().map(|pi| pi.in_time).unwrap_or(0);
let chapters: Vec<Chapter> = parsed
.marks
.iter()
.filter(|m| m.mark_type == 1)
.filter(|m| m.mark_type <= 1)
.enumerate()
.map(|(i, m)| {
let time_secs = (m.timestamp as f64 - first_in_time as f64) / 45000.0;
+13 -1
View File
@@ -48,6 +48,7 @@ impl Disc {
language: a.language.clone(),
sample_rate: SampleRate::from_hz(a.sample_rate),
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
})
})
@@ -92,6 +93,7 @@ impl Disc {
codec: Codec::DvdSub,
language: s.language.clone(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: codec_data.clone(),
})
})
@@ -101,6 +103,16 @@ impl Disc {
streams.extend(audio_streams.iter().cloned());
streams.extend(subtitle_streams);
let chapters: Vec<Chapter> = dvd_title
.chapter_times
.iter()
.enumerate()
.map(|(i, &t)| Chapter {
time_secs: t,
name: format!("Chapter {}", i + 1),
})
.collect();
titles.push(DiscTitle {
playlist: format!("VTS_{:02}_{}.VOB", ts.vts_number, title_number),
playlist_id: title_number,
@@ -108,7 +120,7 @@ impl Disc {
size_bytes,
clips: Vec::new(),
streams,
chapters: Vec::new(),
chapters,
extents,
content_format: ContentFormat::MpegPs,
codec_privates: Vec::new(),
+610
View File
@@ -0,0 +1,610 @@
//! ddrescue-compatible mapfile for tracking rip progress.
//!
//! Records which byte ranges of a disc image are good, unreadable,
//! or not-yet-attempted. Written as plain text so it's greppable,
//! human-editable, and interoperates with ddrescue's own tools.
//!
//! Format:
//! ```text
//! # Rescue Logfile. Created by libfreemkv v0.11.21
//! # Current pos / status / pass / pass_time (ddrescue state machine — we only populate pos)
//! 0x000000000 ? 1 0
//! # pos size status
//! 0x000000000 0x12345678 +
//! 0x012345678 0x00001000 -
//! 0x012346678 0x01234500 ?
//! ```
//!
//! Status chars: `?` non-tried · `*` non-trimmed · `/` non-scraped · `-` unreadable · `+` finished.
//!
//! The mapfile is flushed to disk at most once per `FLUSH_INTERVAL`
//! during `record()` calls, plus on explicit `flush()` and on `Drop`.
//! This bounds atomic-rename RPC rate on networked staging (e.g. NFS)
//! where per-record persists otherwise serialize the rip pipeline.
use std::io::{self, Write};
use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};
/// Minimum interval between mapfile persists. `record()` updates in-memory
/// state every call but only writes to disk when this interval has elapsed
/// since the last persist (or when `flush()` is called explicitly, or on
/// `Drop`). Bounds RPC rate on NFS staging where atomic-rename per record
/// otherwise dominates throughput. On crash the worst-case progress loss
/// is one interval's worth of records.
const FLUSH_INTERVAL: Duration = Duration::from_millis(1000);
/// Status of a byte range in the mapfile. ddrescue-compatible.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SectorStatus {
/// `?` — not yet attempted. Initial state for a fresh mapfile.
NonTried,
/// `*` — fast-pass read failed; edges need trimming.
NonTrimmed,
/// `/` — trimmed; interior needs sector scrape.
NonScraped,
/// `-` — drive couldn't read it this session.
Unreadable,
/// `+` — good.
Finished,
}
impl SectorStatus {
pub fn to_char(self) -> char {
match self {
Self::NonTried => '?',
Self::NonTrimmed => '*',
Self::NonScraped => '/',
Self::Unreadable => '-',
Self::Finished => '+',
}
}
pub fn from_char(c: char) -> Option<Self> {
Some(match c {
'?' => Self::NonTried,
'*' => Self::NonTrimmed,
'/' => Self::NonScraped,
'-' => Self::Unreadable,
'+' => Self::Finished,
_ => return None,
})
}
}
/// One contiguous range of bytes with a status.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MapEntry {
pub pos: u64,
pub size: u64,
pub status: SectorStatus,
}
/// Summary statistics over all entries.
///
/// `bytes_pending` aggregates `NonTried + NonTrimmed + NonScraped` for
/// back-compat. `bytes_nontried` and `bytes_retryable` (= NonTrimmed +
/// NonScraped) split that aggregate so UIs can distinguish *unread*
/// territory (still ahead of Pass 1's read head) from *needs-retry*
/// territory (Pass 1 already encountered, queued for Pass 2-N).
#[derive(Debug, Clone, Copy, Default)]
pub struct MapStats {
pub bytes_total: u64,
pub bytes_good: u64,
pub bytes_unreadable: u64,
pub bytes_pending: u64,
/// Sectors Pass 1 hasn't reached yet (`NonTried`). Subset of
/// `bytes_pending`.
pub bytes_nontried: u64,
/// Sectors flagged for Pass 2-N retry — `NonTrimmed` (multi-sector
/// read failed; needs split) + `NonScraped` (small-block read
/// partially recovered; remainder still pending). Subset of
/// `bytes_pending`. This is the right signal for a "MAYBE / will
/// retry" UI bucket; `bytes_pending` over-counts because it folds
/// in `bytes_nontried`.
pub bytes_retryable: u64,
/// Number of unreadable ranges (for UI display). Computed from
/// `ranges_with(&[Unreadable])`.
pub num_bad_ranges: u32,
/// Largest gap among unreadable ranges in milliseconds. Computed as
/// largest range size / bytes_per_sec * 1000. Set by caller (autorip)
/// since bytes_per_sec is application-specific.
pub main_lost_ms: f64,
}
/// Time-batched mapfile. `record()` keeps in-memory state up-to-date on
/// every call; persists to disk at most once per `FLUSH_INTERVAL`.
/// Explicit `flush()` and `Drop` guarantee state is on disk after a sweep
/// or patch finishes. On hard crash the worst-case loss is one flush
/// interval of records — the file's payload bytes are unaffected.
pub struct Mapfile {
path: PathBuf,
entries: Vec<MapEntry>,
total_size: u64,
version: String,
/// Incrementally maintained stats — updated on every `record()` call
/// so `stats()` is O(1) instead of O(n).
stats: MapStats,
/// True when in-memory state has changed but `write_to_disk` has not
/// yet captured it.
dirty: bool,
/// Wall-clock timestamp of the last successful `write_to_disk` (or
/// the moment the mapfile was constructed, whichever is later).
last_flushed: Instant,
}
impl Mapfile {
/// Create a new mapfile with one `NonTried` region covering the whole disc.
/// Writes to disk immediately so a resume can pick up even if the caller
/// never records anything.
pub fn create(path: &Path, total_size: u64, version: &str) -> io::Result<Self> {
let mut mf = Self {
path: path.to_path_buf(),
entries: vec![MapEntry {
pos: 0,
size: total_size,
status: SectorStatus::NonTried,
}],
total_size,
version: version.to_string(),
stats: MapStats {
bytes_total: total_size,
bytes_pending: total_size,
bytes_nontried: total_size,
..Default::default()
},
dirty: false,
last_flushed: Instant::now(),
};
// Eager initial persist so a resume can pick this up even if
// `record()` is never called.
mf.write_to_disk()?;
mf.last_flushed = Instant::now();
Ok(mf)
}
/// Load an existing mapfile from disk.
pub fn load(path: &Path) -> io::Result<Self> {
let text = std::fs::read_to_string(path)?;
let mut entries = Vec::new();
let mut saw_current_line = false;
let mut version = String::from("unknown");
for line in text.lines() {
let t = line.trim();
if t.is_empty() {
continue;
}
if let Some(rest) = t.strip_prefix('#') {
let rest = rest.trim();
if let Some(v) = rest.strip_prefix("Rescue Logfile. Created by ") {
version = v.to_string();
}
continue;
}
// First non-comment line is the "current" state line (pos status [pass] [pass_time]).
// We ignore its contents but skip over it.
if !saw_current_line {
saw_current_line = true;
// But if the line looks like an entry (has at least 3 fields starting 0x...),
// it's probably actually an entry for a mapfile we wrote without a current line.
// Heuristic: current line has status char as 2nd field; entry has size as 2nd field.
let fields: Vec<&str> = t.split_whitespace().collect();
if fields.len() >= 3 && fields[1].starts_with("0x") {
// It's an entry, not a current line — fall through to entry parse.
} else {
continue;
}
}
// Entry: `pos size statuschar`
let fields: Vec<&str> = t.split_whitespace().collect();
if fields.len() < 3 {
continue;
}
let pos = parse_hex(fields[0])?;
let size = parse_hex(fields[1])?;
let status = fields[2]
.chars()
.next()
.and_then(SectorStatus::from_char)
.ok_or_else(|| {
// No English text — the variant carries a stable
// language-neutral kind identifier (`status_char`).
let e: io::Error = crate::error::Error::MapfileInvalid {
kind: "status_char",
}
.into();
e
})?;
entries.push(MapEntry { pos, size, status });
}
entries.sort_by_key(|e| e.pos);
let total_size = entries.last().map(|e| e.pos + e.size).unwrap_or(0);
let stats = Self::compute_stats(&entries, total_size);
Ok(Self {
path: path.to_path_buf(),
entries,
total_size,
version,
stats,
dirty: false,
last_flushed: Instant::now(),
})
}
/// Load if the file exists, otherwise create a fresh mapfile.
pub fn open_or_create(path: &Path, total_size: u64, version: &str) -> io::Result<Self> {
match Self::load(path) {
Ok(mf) => Ok(mf),
Err(e) if e.kind() == io::ErrorKind::NotFound => {
Self::create(path, total_size, version)
}
Err(e) => Err(e),
}
}
/// Mark a byte range as having the given status. Splits any overlapping
/// existing entries, merges with adjacent same-status entries, and flushes
/// to disk.
pub fn record(&mut self, pos: u64, size: u64, status: SectorStatus) -> io::Result<()> {
if size == 0 {
return Ok(());
}
let end = pos.saturating_add(size);
let mut new_entries = Vec::with_capacity(self.entries.len() + 2);
for e in self.entries.drain(..) {
let e_end = e.pos + e.size;
if e_end <= pos || e.pos >= end {
// entirely before or after — keep
new_entries.push(e);
continue;
}
// Overlap — keep portions outside [pos, end)
if e.pos < pos {
new_entries.push(MapEntry {
pos: e.pos,
size: pos - e.pos,
status: e.status,
});
}
if e_end > end {
new_entries.push(MapEntry {
pos: end,
size: e_end - end,
status: e.status,
});
}
}
new_entries.push(MapEntry { pos, size, status });
new_entries.sort_by_key(|e| e.pos);
// Coalesce adjacent same-status entries.
let mut merged: Vec<MapEntry> = Vec::with_capacity(new_entries.len());
for e in new_entries {
if let Some(last) = merged.last_mut() {
if last.pos + last.size == e.pos && last.status == e.status {
last.size += e.size;
continue;
}
}
merged.push(e);
}
// Recompute stats from merged entries. record() is already O(n) due to
// drain-and-rebuild, so this is a constant-factor overhead. The critical
// win is that stats() is now O(1) — called millions of times in the hot
// path during sweep/patch, it just returns the cached value.
self.stats = Self::compute_stats(&merged, self.total_size);
self.entries = merged;
self.dirty = true;
if self.last_flushed.elapsed() >= FLUSH_INTERVAL {
self.write_to_disk()?;
self.dirty = false;
self.last_flushed = Instant::now();
}
Ok(())
}
/// Persist any pending in-memory changes to disk. No-op if clean.
/// Callers (sweep/patch finalisation) invoke this after their last
/// `record()` to guarantee state is durable before returning.
pub fn flush(&mut self) -> io::Result<()> {
if self.dirty {
self.write_to_disk()?;
self.dirty = false;
self.last_flushed = Instant::now();
}
Ok(())
}
pub fn entries(&self) -> &[MapEntry] {
&self.entries
}
pub fn total_size(&self) -> u64 {
self.total_size
}
/// First range with a given status starting at or after `from`.
pub fn next_with(&self, from: u64, status: SectorStatus) -> Option<(u64, u64)> {
for e in &self.entries {
if e.status != status {
continue;
}
let e_end = e.pos + e.size;
if e_end <= from {
continue;
}
let start = e.pos.max(from);
return Some((start, e_end - start));
}
None
}
/// All ranges matching one of the given statuses, in position order.
pub fn ranges_with(&self, statuses: &[SectorStatus]) -> Vec<(u64, u64)> {
self.entries
.iter()
.filter(|e| statuses.contains(&e.status))
.map(|e| (e.pos, e.size))
.collect()
}
pub fn stats(&self) -> MapStats {
self.stats
}
fn compute_stats(entries: &[MapEntry], total_size: u64) -> MapStats {
let mut s = MapStats {
bytes_total: total_size,
..Default::default()
};
for e in entries {
match e.status {
SectorStatus::Finished => s.bytes_good += e.size,
SectorStatus::Unreadable => s.bytes_unreadable += e.size,
SectorStatus::NonTried => {
s.bytes_pending += e.size;
s.bytes_nontried += e.size;
}
SectorStatus::NonTrimmed | SectorStatus::NonScraped => {
s.bytes_pending += e.size;
s.bytes_retryable += e.size;
}
}
}
s
}
fn write_to_disk(&self) -> io::Result<()> {
// Write to a tempfile then rename for atomicity. Appending ".tmp"
// rather than `with_extension` so we don't clobber the original
// extension (which may already be ".mapfile").
let tmp = {
let mut s = self.path.clone().into_os_string();
s.push(".tmp");
PathBuf::from(s)
};
{
let file = std::fs::File::create(&tmp)?;
let mut w = std::io::BufWriter::new(file);
writeln!(w, "# Rescue Logfile. Created by {}", self.version)?;
writeln!(w, "# Current pos / status / pass / pass_time")?;
writeln!(w, "0x000000000 ? 1 0")?;
writeln!(w, "# pos size status")?;
for e in &self.entries {
writeln!(
w,
"0x{:09x} 0x{:09x} {}",
e.pos,
e.size,
e.status.to_char()
)?;
}
w.flush()?;
}
std::fs::rename(&tmp, &self.path)?;
Ok(())
}
}
impl Drop for Mapfile {
/// Best-effort flush on drop so a sweep / patch that returns early
/// (or unwinds) doesn't lose its in-memory state. Errors here are
/// swallowed because Drop has no way to surface them; explicit
/// `flush()` on the success path gives callers proper error handling.
fn drop(&mut self) {
let _ = self.flush();
}
}
fn parse_hex(s: &str) -> io::Result<u64> {
let s = s.strip_prefix("0x").unwrap_or(s);
u64::from_str_radix(s, 16).map_err(|_| {
// Underlying ParseIntError dropped — its Display is OS-locale text.
// The typed variant carries `kind = "hex"` which is stable.
let e: io::Error = crate::error::Error::MapfileInvalid { kind: "hex" }.into();
e
})
}
#[cfg(test)]
mod tests {
use super::*;
fn tmpfile(tag: &str) -> PathBuf {
use std::sync::atomic::{AtomicU64, Ordering};
static CTR: AtomicU64 = AtomicU64::new(0);
let n = CTR.fetch_add(1, Ordering::Relaxed);
let name = format!(
"libfreemkv-mapfile-test-{}-{}-{}.mapfile",
std::process::id(),
tag,
n
);
std::env::temp_dir().join(name)
}
#[test]
fn create_has_one_nontried_region() {
let p = tmpfile("create_has_one_nontried_region");
let _ = std::fs::remove_file(&p);
let mf = Mapfile::create(&p, 1000, "test").unwrap();
assert_eq!(mf.entries().len(), 1);
assert_eq!(mf.entries()[0].pos, 0);
assert_eq!(mf.entries()[0].size, 1000);
assert_eq!(mf.entries()[0].status, SectorStatus::NonTried);
let _ = std::fs::remove_file(&p);
}
#[test]
fn record_splits_overlap() {
let p = tmpfile("record_splits_overlap");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
mf.record(200, 100, SectorStatus::Finished).unwrap();
let es = mf.entries();
assert_eq!(es.len(), 3);
assert_eq!(
(es[0].pos, es[0].size, es[0].status),
(0, 200, SectorStatus::NonTried)
);
assert_eq!(
(es[1].pos, es[1].size, es[1].status),
(200, 100, SectorStatus::Finished)
);
assert_eq!(
(es[2].pos, es[2].size, es[2].status),
(300, 700, SectorStatus::NonTried)
);
let _ = std::fs::remove_file(&p);
}
#[test]
fn record_coalesces_adjacent_same_status() {
let p = tmpfile("record_coalesces_adjacent_same_status");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
mf.record(100, 100, SectorStatus::Finished).unwrap();
mf.record(200, 100, SectorStatus::Finished).unwrap();
// Entries: [0..100 NonTried, 100..300 Finished (merged), 300..1000 NonTried]
let es = mf.entries();
assert_eq!(es.len(), 3);
assert_eq!(
(es[1].pos, es[1].size, es[1].status),
(100, 200, SectorStatus::Finished)
);
let _ = std::fs::remove_file(&p);
}
#[test]
fn record_replaces_existing_status() {
let p = tmpfile("record_replaces_existing_status");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
mf.record(200, 100, SectorStatus::Unreadable).unwrap();
mf.record(200, 100, SectorStatus::Finished).unwrap();
let es = mf.entries();
// The overwrite should result in all finished at 200..300, NonTried elsewhere — 3 entries.
assert_eq!(es.len(), 3);
assert_eq!(es[1].status, SectorStatus::Finished);
let _ = std::fs::remove_file(&p);
}
#[test]
fn round_trip_load() {
let p = tmpfile("round_trip_load");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
mf.record(100, 200, SectorStatus::Finished).unwrap();
mf.record(500, 100, SectorStatus::Unreadable).unwrap();
// record() batches; explicit flush before reading back from disk.
mf.flush().unwrap();
let loaded = Mapfile::load(&p).unwrap();
assert_eq!(loaded.entries(), mf.entries());
let _ = std::fs::remove_file(&p);
}
#[test]
fn stats_sum_correctly() {
let p = tmpfile("stats_sum_correctly");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
mf.record(0, 400, SectorStatus::Finished).unwrap();
mf.record(400, 100, SectorStatus::Unreadable).unwrap();
let s = mf.stats();
assert_eq!(s.bytes_good, 400);
assert_eq!(s.bytes_unreadable, 100);
assert_eq!(s.bytes_pending, 500);
assert_eq!(s.bytes_total, 1000);
let _ = std::fs::remove_file(&p);
}
#[test]
fn ranges_with_filters() {
let p = tmpfile("ranges_with_filters");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
mf.record(100, 50, SectorStatus::Unreadable).unwrap();
mf.record(300, 50, SectorStatus::Unreadable).unwrap();
let bad = mf.ranges_with(&[SectorStatus::Unreadable]);
assert_eq!(bad, vec![(100, 50), (300, 50)]);
let _ = std::fs::remove_file(&p);
}
#[test]
fn stats_consistent_after_overlapping_records() {
let p = tmpfile("stats_consistent_after_overlapping");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
// Record some finished, some unreadable, some nontrimmed
mf.record(0, 300, SectorStatus::Finished).unwrap();
mf.record(300, 200, SectorStatus::NonTrimmed).unwrap();
mf.record(500, 100, SectorStatus::Unreadable).unwrap();
mf.record(600, 400, SectorStatus::Finished).unwrap();
// Final entries: [0..300 Finished, 300..500 NonTrimmed, 500..600 Unreadable, 600..1000 Finished]
let s = mf.stats();
assert_eq!(s.bytes_good, 700); // 300 + 400
assert_eq!(s.bytes_unreadable, 100); // 100
assert_eq!(s.bytes_pending, 200); // NonTrimmed only (NonTried=0)
assert_eq!(s.bytes_nontried, 0);
assert_eq!(s.bytes_retryable, 200); // NonTrimmed
assert_eq!(s.bytes_total, 1000);
// Overwrite a NonTrimmed range with Finished
mf.record(300, 100, SectorStatus::Finished).unwrap();
// Entries: [0..400 Finished, 400..500 NonTrimmed, 500..600 Unreadable, 600..1000 Finished]
let s2 = mf.stats();
assert_eq!(s2.bytes_good, 800); // 400 + 400
assert_eq!(s2.bytes_unreadable, 100);
assert_eq!(s2.bytes_pending, 100); // NonTrimmed only
assert_eq!(s2.bytes_retryable, 100);
let _ = std::fs::remove_file(&p);
}
#[test]
fn stats_consistent_after_split_record() {
let p = tmpfile("stats_consistent_after_split");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
// Mark middle as NonTrimmed
mf.record(200, 400, SectorStatus::NonTrimmed).unwrap();
// Entries: [0..200 NonTried, 200..600 NonTrimmed, 600..1000 NonTried]
let s = mf.stats();
assert_eq!(s.bytes_pending, 1000); // NonTried(600) + NonTrimmed(400)
assert_eq!(s.bytes_retryable, 400); // NonTrimmed only
assert_eq!(s.bytes_nontried, 600); // 200 + 400
// Overwrite the NonTrimmed with Finished (splitting the remaining NonTried)
mf.record(200, 400, SectorStatus::Finished).unwrap();
// Entries: [0..200 NonTried, 200..600 Finished, 600..1000 NonTried]
let s2 = mf.stats();
assert_eq!(s2.bytes_good, 400);
assert_eq!(s2.bytes_pending, 600); // NonTried(200 + 400)
assert_eq!(s2.bytes_nontried, 600);
assert_eq!(s2.bytes_retryable, 0);
let _ = std::fs::remove_file(&p);
}
}
+2338 -88
View File
File diff suppressed because it is too large Load Diff
+243
View File
@@ -0,0 +1,243 @@
//! Producer / consumer split for `Disc::patch`.
//!
//! Background: pre-0.18 patch ran strictly serial — single-sector
//! recovery read → seek + write recovered bytes → mapfile.record →
//! next iteration. The drive sat idle while the previous block's
//! recovered bytes were committed. On a damaged disc with many bad
//! sectors that adds up: per-sector write + mapfile.record costs a
//! handful of milliseconds each, which the drive could be using to
//! issue the next per-sector retry.
//!
//! This module decouples them. A consumer thread owns the
//! [`crate::io::WritebackFile`] (the ISO file) and the
//! [`super::mapfile::Mapfile`]. The producer thread (`Disc::patch`)
//! keeps the [`crate::sector::SectorReader`], the wedge / damage-window
//! state, the per-range watchdog, decrypt — so what enters the channel
//! is already-clean cleartext bytes (or an "Unreadable" terminal mark).
//!
//! Producer and consumer run concurrently; the channel uses
//! [`crate::io::pipeline::WRITE_THROUGH_DEPTH`] (=1) so back-pressure
//! kicks in immediately. We want the drive's per-sector retry budget
//! to stay in lockstep with the writer — sweep's `DEFAULT_PIPELINE_DEPTH`
//! (4) would let several sectors of recovered bytes queue up between
//! the producer's retry decisions and the writer, and patch's recovery
//! loop reads stats (`bytes_good`, range progress) inline to drive its
//! skip / wedge decisions. WRITE_THROUGH_DEPTH gives "read N+1 while
//! writing N", no further pipelining — exactly the model the producer
//! logic was written against.
//!
//! Correctness invariants preserved:
//! - Mapfile is single-writer (consumer-only). No locking on it.
//! - All recovery state (damage window, consecutive_failures, skip
//! escalation, range watchdog) stays on the producer thread.
//! - `set_speed` calls happen on the producer thread (same thread that
//! owns the `SectorReader`). No new SCSI concurrency.
//! - Per-iteration ordering of file-write → mapfile-record is kept
//! intact in the consumer (write before record), so the on-disk
//! invariant "mapfile only marks Finished what the file has received"
//! survives a crash mid-pass.
//! - The BU40N+Initio bridge wedge concern is unchanged: only one
//! SCSI command in flight at a time, error-path timing identical,
//! no new retry logic. The threading primitive only overlaps the
//! *write* with the *next read*; the per-sector single-shot read
//! budget that the bridge wedge concern was originally about is
//! untouched.
//!
//! Per-range watchdog (`MAX_RANGE_SECS` / `RANGE_BUDGET_CAP_SECS`)
//! checks `bytes_good` for forward progress. With work in flight on
//! the consumer, the producer would otherwise see stale values; the
//! sink publishes a [`SharedPatchState`] snapshot after every record
//! so the producer's stall guards observe consumer side-effects with
//! at most one item of lag (which is fine — the watchdog uses minute-
//! scale budgets, not single-record latency).
use std::io::{Seek, SeekFrom, Write};
use std::sync::{Arc, Mutex};
use crate::error::{Error, Result};
use crate::io::pipeline::{Flow, Sink};
use super::mapfile::{MapStats, Mapfile, SectorStatus};
/// Item the producer hands to the patch consumer. One per per-sector
/// recovery decision.
pub(super) enum PatchItem {
/// Sector / small batch successfully recovered (and decrypted on the
/// producer side if `opts.decrypt` was set). Consumer seeks to
/// `pos`, writes `buf`, records the range as `Finished`.
Recovered { pos: u64, buf: Vec<u8> },
/// Producer exhausted retries on `[pos, pos+len)`. Consumer records
/// the range as `Unreadable`. No file write — the existing zero-fill
/// from sweep is preserved in place.
Unreadable { pos: u64, len: u64 },
/// Producer hit the per-range skip limit and is leaving the
/// remaining bytes as `NonTrimmed` for a future pass. CRITICAL:
/// this is not the same as `Unreadable` — sectors we never tried
/// stay hopeful. (See the comment at the skip-limit branch in
/// `Disc::patch`: ~36% of patch-marked Unreadable sectors are
/// actually readable on a later pass.) No file write.
NonTrimmed { pos: u64, len: u64 },
}
/// Mapfile snapshot the sink republishes after every record so the
/// producer can drive its stall / progress logic without holding the
/// mapfile lock for long. `bad_ranges` mirrors what
/// `Mapfile::ranges_with(&[NonTrimmed, Unreadable, NonScraped, NonTried])`
/// would return — same set the pre-split patch loop computed inline
/// for the progress callback.
pub(super) struct SharedPatchState {
pub stats: MapStats,
pub bad_ranges: Vec<(u64, u64)>,
}
impl SharedPatchState {
fn from_map(map: &Mapfile) -> Self {
Self {
stats: map.stats(),
bad_ranges: map.ranges_with(&[
SectorStatus::NonTrimmed,
SectorStatus::Unreadable,
SectorStatus::NonScraped,
SectorStatus::NonTried,
]),
}
}
}
/// Final summary returned by [`Sink::close`] when the consumer drains
/// cleanly. Mirrors what the pre-split patch loop computed at the end
/// of the function — final mapfile stats plus whether `sync_all`
/// failed on a regular file (the only kind of fsync error patch ever
/// surfaced; `/dev/null` and pipes always fail `sync_all`, that's not
/// a real error).
pub(super) struct PatchSummary {
pub stats: MapStats,
}
/// Consumer-side of the patch pipeline. Owns the ISO writeback file
/// and the mapfile; publishes a shared snapshot after every record so
/// the producer can read `bytes_good` for stall detection and
/// progress reporting.
pub(super) struct PatchSink {
file: crate::io::WritebackFile,
map: Mapfile,
/// Whether the output is a regular file (so a `sync_all` failure
/// is real). `/dev/null` etc. always fail `sync_all`; ignore those.
is_regular: bool,
/// Snapshot the producer reads. Updated after every successful
/// `record()` call. `Mutex` rather than separate atomics because
/// the producer wants stats + bad_ranges as a coherent pair.
shared: Arc<Mutex<SharedPatchState>>,
}
impl PatchSink {
/// Open `path` as a [`crate::io::WritebackFile`] and pair it with
/// `map` for the consumer. The producer holds onto the returned
/// `Arc<Mutex<SharedPatchState>>` so it can poll mapfile state
/// while the consumer is mutating it.
pub(super) fn new(
path: &std::path::Path,
map: Mapfile,
is_regular: bool,
) -> Result<(Self, Arc<Mutex<SharedPatchState>>)> {
let file =
crate::io::WritebackFile::open(path).map_err(|e| Error::IoError { source: e })?;
let shared = Arc::new(Mutex::new(SharedPatchState::from_map(&map)));
let shared_clone = shared.clone();
Ok((
Self {
file,
map,
is_regular,
shared,
},
shared_clone,
))
}
fn republish(&self) {
// Best-effort lock — only the producer reads, only the consumer
// writes; contention is single-acquire so the lock is never
// poisoned in practice. If it ever did get poisoned we'd want
// the underlying error surfaced rather than silently swallowed,
// so we propagate the poison panic. (Same posture as
// `sweep_pipeline.rs` — it never recovers from a poisoned
// mutex either.)
let mut guard = self
.shared
.lock()
.expect("PatchSink shared state mutex poisoned");
*guard = SharedPatchState::from_map(&self.map);
}
}
impl Sink<PatchItem> for PatchSink {
type Output = PatchSummary;
fn apply(&mut self, item: PatchItem) -> std::result::Result<Flow, Error> {
match item {
PatchItem::Recovered { pos, buf } => {
let len = buf.len() as u64;
self.file
.seek(SeekFrom::Start(pos))
.map_err(|e| Error::IoError { source: e })?;
self.file
.write_all(&buf)
.map_err(|e| Error::IoError { source: e })?;
self.map
.record(pos, len, SectorStatus::Finished)
.map_err(|e| Error::IoError { source: e })?;
}
PatchItem::Unreadable { pos, len } => {
self.map
.record(pos, len, SectorStatus::Unreadable)
.map_err(|e| Error::IoError { source: e })?;
}
PatchItem::NonTrimmed { pos, len } => {
self.map
.record(pos, len, SectorStatus::NonTrimmed)
.map_err(|e| Error::IoError { source: e })?;
}
}
self.republish();
Ok(Flow::Continue)
}
fn close(mut self) -> std::result::Result<Self::Output, Error> {
// Drain in-flight writeback then issue a full fsync. A failure
// here matters only on regular files — pipes / `/dev/null` etc.
// always fail `sync_all`.
if let Err(e) = self.file.sync_all() {
if self.is_regular {
tracing::warn!(
target: "freemkv::disc",
phase = "patch_sync_failed",
error = %e,
os_error = e.raw_os_error(),
error_kind = ?e.kind(),
"patch: sync_all failed"
);
return Err(Error::IoError { source: e });
}
tracing::debug!(
target: "freemkv::disc",
phase = "patch_sync_skipped",
error = %e,
"patch: sync_all failed for non-regular file; ignoring"
);
}
self.map.flush().map_err(|e| Error::IoError { source: e })?;
// Final republish so anyone reading the shared snapshot after
// `Pipeline::finish` sees the post-flush state. (The producer
// already has its own copy of the final `MapStats` in the
// returned `PatchSummary`, but the snapshot is part of the
// public-ish contract of the consumer: it stays current
// through close.)
self.republish();
Ok(PatchSummary {
stats: self.map.stats(),
})
}
}
+374
View File
@@ -0,0 +1,374 @@
//! Single source of truth for what to do when a sector read fails.
//!
//! Both Pass 1 (`Disc::sweep`) and Pass 2-N (`Disc::patch`) call into
//! `handle_read_error` after every failed `read_sectors`. The handler
//! classifies the error, updates the in-flight context (counters,
//! damage window, retry budgets), and returns a `ReadAction` the caller
//! dispatches on. Every read goes through the same gate — no path can
//! silently skip pause/skip/jump/abort logic.
//!
//! Adding a new error class = add one arm in `handle_read_error`.
//! Adding new logging on errors = one place.
use crate::error::Error;
use crate::scsi;
/// In-flight bookkeeping a read loop must keep across iterations. The
/// handler reads and mutates this. Caller owns the storage.
pub struct ReadCtx {
/// Number of sectors per read attempt. The handler uses this to
/// decide whether to bisect (only worthwhile when batch > 1).
pub batch: u16,
/// Successful reads since the last failure. Resets to 0 on failure.
/// Used by callers to drive damage-zone exit / speed restoration.
pub consecutive_good: u64,
/// Failed reads since the last success. Resets to 0 on success.
/// Drives long-pause escalation on persistent failure.
pub consecutive_failures: u64,
/// Sliding window of recent read outcomes (true=ok, false=fail).
/// Capped at `damage_window_max`. Drives damage-jump decisions.
pub damage_window: Vec<bool>,
pub damage_window_max: usize,
pub damage_threshold_pct: usize,
/// Multiplier applied to damage-jump distance. Doubles each jump,
/// resets to 1 after `damage_window_max` consecutive good reads.
pub jump_multiplier: u64,
/// NOT_READY retries used so far for the current LBA. Reset to 0
/// on any non-NOT_READY response.
pub not_ready_retries: u32,
/// Bridge-degradation cooldowns used so far.
pub bridge_degradation_count: u32,
/// Whether we're currently inside a damage-jump bisect attempt.
/// Caller sets this true when entering single-sector mode for a
/// failed batch, so the handler doesn't recursively request another
/// bisect on the inner-sector failures.
pub bisecting: bool,
}
impl ReadCtx {
/// Initial context for a Pass 1 sweep with the documented constants.
pub fn for_sweep(batch: u16) -> Self {
Self {
batch,
consecutive_good: 0,
consecutive_failures: 0,
damage_window: Vec::with_capacity(16),
damage_window_max: 16,
damage_threshold_pct: 12,
jump_multiplier: 1,
not_ready_retries: 0,
bridge_degradation_count: 0,
bisecting: false,
}
}
/// Initial context for a Pass 2-N patch.
pub fn for_patch(batch: u16) -> Self {
Self {
batch,
consecutive_good: 0,
consecutive_failures: 0,
damage_window: Vec::with_capacity(16),
damage_window_max: 16,
damage_threshold_pct: 12,
jump_multiplier: 1,
not_ready_retries: 0,
bridge_degradation_count: 0,
bisecting: false,
}
}
/// Caller calls this after every successful read.
pub fn on_success(&mut self) {
self.consecutive_good += 1;
self.consecutive_failures = 0;
self.not_ready_retries = 0;
self.damage_window.push(true);
if self.damage_window.len() > self.damage_window_max {
self.damage_window.remove(0);
}
}
}
/// What the caller should do after a read failure. The caller owns the
/// I/O side-effects (sleep, write zeros, advance pos) — the handler
/// only decides which side-effects.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ReadAction {
/// Pause `pause_secs` then retry the same LBA / batch. Used for
/// transient conditions (NOT_READY, bridge degradation) that the
/// drive may recover from on its own.
Retry { pause_secs: u64 },
/// Re-issue the failed batch as `batch` single-sector reads. Each
/// inner read is itself dispatched through `handle_read_error` with
/// `bisecting = true` so it cannot recurse.
Bisect,
/// Mark the failed range NonTrimmed (zero-fill, retry in Pass N+),
/// then pause `pause_secs` before resuming the next LBA.
SkipBlock { pause_secs: u64 },
/// Mark the failed range NonTrimmed AND advance position by
/// `sectors` (zero-filling the gap as NonTrimmed). Then pause
/// `pause_secs`. Used when the damage-window threshold is crossed.
JumpAhead { sectors: u64, pause_secs: u64 },
/// Unrecoverable at this layer. Caller propagates `Err` up to the
/// outer pass loop / autorip, which can attempt USB re-enumeration,
/// drop session, etc.
AbortPass,
}
// Pause budget constants. Tuned from 2026-05-07 BU40N traces showing
// bridge wedges 524 ms after a 5.4-second internal ECC retry. The
// post-failure pauses give the drive — and the bridge — time to settle.
const POST_FAILURE_PAUSE_SECS: u64 = 1;
const CONSECUTIVE_FAIL_LONG_PAUSE_SECS: u64 = 5;
const CONSECUTIVE_FAIL_LONG_PAUSE_THRESHOLD: u64 = 10;
const POST_JUMP_EXTRA_PAUSE_SECS: u64 = 2;
const NOT_READY_PAUSE_SECS: u64 = 3;
const NOT_READY_MAX_RETRIES: u32 = 3;
const BRIDGE_DEGRADATION_PAUSE_SECS: u64 = 15;
const BRIDGE_DEGRADATION_MAX_RETRIES: u32 = 5;
/// THE single error-handling entry point. Updates `ctx`, returns the
/// action the caller must apply.
///
/// New error class = add a new arm here. New logging on errors = add
/// it once at the top. New retry policy = adjust the constants. No
/// other read site needs to change.
pub fn handle_read_error(err: &Error, ctx: &mut ReadCtx) -> ReadAction {
ctx.consecutive_failures += 1;
ctx.consecutive_good = 0;
tracing::warn!(
target: "freemkv::disc",
phase = "read_error",
consecutive_failures = ctx.consecutive_failures,
batch = ctx.batch,
bisecting = ctx.bisecting,
sense_key = err.scsi_sense().map(|s| s.sense_key),
asc = err.scsi_sense().map(|s| s.asc),
ascq = err.scsi_sense().map(|s| s.ascq),
error = %err,
"read failed; classifying"
);
// 1. Transport failure: bridge crash / USB disconnect. The outer
// pass loop knows how to handle this (rediscover sg path,
// re-open drive). Inline single-sector retry here was tried in
// pre-v0.17.0 builds and observed to make wedges worse.
if err.is_scsi_transport_failure() {
return ReadAction::AbortPass;
}
// 2. Bridge degradation: NOT_READY with the well-known signature
// (sense_key=2, ASC=0x04, ASCQ=0x3E). Drive's bridge is in a
// semi-stuck state but typically recovers after a long cooldown.
// If we've exhausted our retry budget, fall through to the
// marginal/skip path below.
if err.is_bridge_degradation() && ctx.bridge_degradation_count < BRIDGE_DEGRADATION_MAX_RETRIES
{
ctx.bridge_degradation_count += 1;
return ReadAction::Retry {
pause_secs: BRIDGE_DEGRADATION_PAUSE_SECS,
};
}
let sense_key = err.scsi_sense().map(|s| s.sense_key).unwrap_or(0);
// 3. Generic NOT_READY (other ASC codes): drive's mechanical
// pickup may be moving. Pause and retry briefly.
if sense_key == scsi::SENSE_KEY_NOT_READY && ctx.not_ready_retries < NOT_READY_MAX_RETRIES {
ctx.not_ready_retries += 1;
return ReadAction::Retry {
pause_secs: NOT_READY_PAUSE_SECS,
};
}
if sense_key != scsi::SENSE_KEY_NOT_READY {
ctx.not_ready_retries = 0;
}
// 4. Hardware / illegal request: the drive said "no, won't do it".
// Retrying won't change the answer. Surface to the outer layer
// so it can eject + prompt user.
if sense_key == scsi::SENSE_KEY_HARDWARE_ERROR || sense_key == scsi::SENSE_KEY_ILLEGAL_REQUEST {
return ReadAction::AbortPass;
}
// 5. Marginal media (MEDIUM_ERROR / ABORTED_COMMAND) on a multi-
// sector batch: the drive can often read the same sectors
// individually. Bisect into single-sector reads (gentler on the
// bridge too — shorter SCSI transactions). Avoid recursive
// bisect.
let is_marginal = matches!(
sense_key,
scsi::SENSE_KEY_MEDIUM_ERROR | scsi::SENSE_KEY_ABORTED_COMMAND
);
if is_marginal && ctx.batch > 1 && !ctx.bisecting {
return ReadAction::Bisect;
}
// 6. Single-sector failure or unbisectable error — record in
// damage window, decide between skip-in-place vs damage-jump.
//
// SKIP damage-window updates while bisecting: the window
// represents per-batch outcomes, not per-sector. Updating it
// inside a bisect inner loop (potentially 32+ sector failures
// per batch) would over-weight the window and cause runaway
// JumpAhead distance via excessive multiplier doublings.
if !ctx.bisecting {
ctx.damage_window.push(false);
if ctx.damage_window.len() > ctx.damage_window_max {
ctx.damage_window.remove(0);
}
}
let bad_count = ctx.damage_window.iter().filter(|&&b| !b).count();
let bad_pct = if ctx.damage_window.is_empty() {
0
} else {
bad_count * 100 / ctx.damage_window.len()
};
let pause_secs = if ctx.consecutive_failures >= CONSECUTIVE_FAIL_LONG_PAUSE_THRESHOLD {
CONSECUTIVE_FAIL_LONG_PAUSE_SECS
} else {
POST_FAILURE_PAUSE_SECS
};
// 7. Damage-jump: too many failures in window → skip ahead by an
// escalating gap. Multiplier capped so we can't accidentally
// skip the entire rest of the disc (observed 2026-05-07: a
// saturated multiplier produced a 56 GB jump). Saturating
// arithmetic on the sector calc as defence in depth.
const MAX_JUMP_MULTIPLIER: u64 = 64; // 64 × 256 × batch sectors
if ctx.damage_window.len() >= ctx.damage_window_max && bad_pct >= ctx.damage_threshold_pct {
let mult = ctx.jump_multiplier.min(MAX_JUMP_MULTIPLIER);
let sectors = 256u64.saturating_mul(ctx.batch as u64).saturating_mul(mult);
ctx.jump_multiplier = (ctx.jump_multiplier.saturating_mul(2)).min(MAX_JUMP_MULTIPLIER);
return ReadAction::JumpAhead {
sectors,
pause_secs: pause_secs + POST_JUMP_EXTRA_PAUSE_SECS,
};
}
// 8. Default: zero-fill the failed batch as NonTrimmed and pause
// before the next read.
ReadAction::SkipBlock { pause_secs }
}
#[cfg(test)]
mod tests {
use super::*;
use crate::error::Error;
use crate::scsi::ScsiSense;
fn medium_err() -> Error {
Error::DiscRead {
sector: 100,
status: Some(2),
sense: Some(ScsiSense {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x05,
}),
}
}
fn hardware_err() -> Error {
Error::DiscRead {
sector: 100,
status: Some(2),
sense: Some(ScsiSense {
sense_key: scsi::SENSE_KEY_HARDWARE_ERROR,
asc: 0x44,
ascq: 0x00,
}),
}
}
#[test]
fn medium_error_with_batch_gt_1_bisects() {
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&medium_err(), &mut ctx);
assert_eq!(action, ReadAction::Bisect);
}
#[test]
fn medium_error_with_batch_1_skips() {
let mut ctx = ReadCtx::for_sweep(1);
let action = handle_read_error(&medium_err(), &mut ctx);
match action {
ReadAction::SkipBlock { pause_secs } => assert!(pause_secs >= 1),
other => panic!("expected SkipBlock, got {other:?}"),
}
}
#[test]
fn medium_error_while_bisecting_does_not_recurse() {
let mut ctx = ReadCtx::for_sweep(32);
ctx.bisecting = true;
let action = handle_read_error(&medium_err(), &mut ctx);
match action {
ReadAction::SkipBlock { .. } => {}
other => panic!("expected SkipBlock, got {other:?}"),
}
}
#[test]
fn hardware_error_aborts() {
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&hardware_err(), &mut ctx);
assert_eq!(action, ReadAction::AbortPass);
}
#[test]
fn long_failure_streak_extends_pause() {
let mut ctx = ReadCtx::for_sweep(1);
for _ in 0..15 {
handle_read_error(&medium_err(), &mut ctx);
}
// After many consecutive failures we should be in the long-pause regime
let final_action = handle_read_error(&medium_err(), &mut ctx);
match final_action {
ReadAction::SkipBlock { pause_secs } => {
assert!(pause_secs >= CONSECUTIVE_FAIL_LONG_PAUSE_SECS);
}
ReadAction::JumpAhead { pause_secs, .. } => {
assert!(pause_secs >= CONSECUTIVE_FAIL_LONG_PAUSE_SECS);
}
other => panic!("expected long-pause action, got {other:?}"),
}
}
#[test]
fn damage_window_fills_then_jumps() {
let mut ctx = ReadCtx::for_sweep(1);
ctx.damage_window_max = 4;
ctx.damage_threshold_pct = 50;
let mut saw_jump = false;
for _ in 0..6 {
let a = handle_read_error(&medium_err(), &mut ctx);
if matches!(a, ReadAction::JumpAhead { .. }) {
saw_jump = true;
break;
}
}
assert!(
saw_jump,
"expected at least one JumpAhead in 6 failures with 50% threshold"
);
}
#[test]
fn on_success_resets_failure_counters_and_pushes_window() {
let mut ctx = ReadCtx::for_sweep(32);
for _ in 0..3 {
handle_read_error(&medium_err(), &mut ctx);
}
assert!(ctx.consecutive_failures > 0);
ctx.bisecting = false;
ctx.on_success();
assert_eq!(ctx.consecutive_good, 1);
assert_eq!(ctx.consecutive_failures, 0);
assert!(*ctx.damage_window.last().unwrap());
}
}
+239
View File
@@ -0,0 +1,239 @@
//! `Disc::sweep`'s consumer-side `Sink<WorkItem>`.
//!
//! Background: the original sweep loop runs strictly serialised —
//! SCSI read → decrypt → seek + write → mapfile.record → next iter.
//! On a healthy disc the SCSI read costs ~5-12 ms per 64 KB batch and
//! the post-read work (decrypt 1-3 ms + file write + mapfile fsync
//! 5-15 ms) adds another batch's worth of latency. The drive idles
//! during the post-read work; throughput tops out at the *sum* of
//! both costs.
//!
//! 0.17.11 introduced a bespoke producer/consumer split (the now-
//! removed `disc/sweep_pipeline.rs`) to overlap the two stages. 0.18
//! collapses that split — together with the analogous splits patch
//! and mux need — onto the generic [`crate::io::Pipeline`] +
//! [`crate::io::Sink`] primitive. This module is the sweep-specific
//! `Sink` impl; the producer-side state machine (read_error context,
//! decrypt, set_speed, halt) stays in `Disc::sweep` in `disc/mod.rs`.
//!
//! Correctness invariants preserved (same as 0.17.11):
//! - Mapfile is single-writer (consumer-only). No locking.
//! - All `read_error::ReadCtx` state stays on the producer thread.
//! - `set_speed` calls happen on the producer thread (same thread that
//! owns the `SectorReader`). No new SCSI concurrency.
//! - Per-iteration ordering of file-write → mapfile-record is kept
//! intact in the consumer (write before record), so the on-disk
//! invariant "mapfile only marks Finished what the file has
//! received" survives a crash mid-pass.
//! - The BU40N+Initio bridge wedge concern is unchanged: only one
//! SCSI command in flight at a time, error-path timing identical,
//! no new retry logic.
use std::io::{Seek, SeekFrom, Write};
use std::sync::mpsc::{Receiver, SyncSender, sync_channel};
use crate::error::Error;
use crate::io::{Flow, Sink};
use super::mapfile::{MapStats, Mapfile, SectorStatus};
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. 64 KB
/// matches the existing zero_gap chunk size used by the pre-split
/// sweep loop.
const ZERO_CHUNK: usize = 65 * 1024;
/// Producer → Consumer messages. The consumer applies these in FIFO
/// order; ordering of file writes and mapfile records across items is
/// preserved.
pub(super) enum WorkItem {
/// Successful batch read. Producer has already decrypted `buf` if
/// `opts.decrypt` was set. Consumer writes `buf` at `pos` and
/// records the range as `Finished`.
Good { pos: u64, buf: Vec<u8> },
/// Bisect inner-loop good single sector (already decrypted by the
/// producer). 2048 bytes.
BisectGood { pos: u64, buf: Box<[u8; 2048]> },
/// Bisect inner-loop bad single sector. Consumer writes 2048
/// zeros at `pos` and records the sector as `NonTrimmed`.
BisectBad { pos: u64 },
/// Whole-batch zero-fill (failed batch on `SkipBlock`, or the
/// failed batch portion of `JumpAhead`). Consumer streams zeros
/// across `[pos, pos+len)` and records the range as `NonTrimmed`.
SkipFill { pos: u64, len: u64 },
/// Gap fill following a `JumpAhead`. Same effect as `SkipFill`;
/// distinguished only so future logging / instrumentation can
/// tell them apart without parsing a flag.
GapFill { pos: u64, len: u64 },
/// Producer wants the latest mapfile stats for the progress
/// callback. Consumer responds on `prog_tx` with a fresh
/// [`ProgressSnapshot`]. Best-effort: if the producer hasn't
/// drained the previous snapshot, the new one is silently
/// dropped — the producer's local cache stays current enough.
StatsRequest,
}
/// Snapshot the consumer sends back to the producer for the progress
/// callback.
pub(super) struct ProgressSnapshot {
pub stats: MapStats,
pub bad_ranges: Vec<(u64, u64)>,
}
/// Final summary returned by the consumer thread on shutdown — what
/// `SweepSink::close` produces, surfaced to the producer via
/// `Pipeline::finish`.
pub(super) struct ConsumerSummary {
pub stats: MapStats,
}
/// Drain any pending progress snapshots from the consumer. Returns
/// the most recent one, if any. The producer caches it and uses it
/// for subsequent progress callbacks until a fresh one arrives.
pub(super) fn try_recv_progress(rx: &Receiver<ProgressSnapshot>) -> Option<ProgressSnapshot> {
let mut latest = None;
while let Ok(snap) = rx.try_recv() {
latest = Some(snap);
}
latest
}
/// `Sink<WorkItem>` for sweep. Owns the writeback file + mapfile +
/// progress back-channel. `apply` carries the file-write +
/// mapfile.record per item; `close` drains the writeback pipeline,
/// fsyncs the ISO, and flushes the mapfile.
pub(super) struct SweepSink {
file: crate::io::WritebackFile,
map: Mapfile,
/// `sync_all`-on-failure-is-an-error iff the output is a regular
/// file. `/dev/null` and pipes always fail `sync_all`; that's not
/// a real error.
is_regular: bool,
/// Back-channel for `StatsRequest` responses. The producer caches
/// the latest snapshot and uses it for the progress callback;
/// dropped sends on a full channel are by design.
prog_tx: SyncSender<ProgressSnapshot>,
/// Reusable zero buffer for SkipFill / GapFill / BisectBad. Held
/// in the sink so each apply call doesn't reallocate.
zero: Box<[u8; ZERO_CHUNK]>,
}
impl SweepSink {
/// Construct a new `SweepSink` plus the matching progress
/// receiver. Channel depth on the back-channel is `1` — the
/// producer's cache is the source of truth between snapshots.
pub(super) fn new(
file: crate::io::WritebackFile,
map: Mapfile,
is_regular: bool,
) -> (Self, Receiver<ProgressSnapshot>) {
let (prog_tx, prog_rx) = sync_channel::<ProgressSnapshot>(1);
let sink = SweepSink {
file,
map,
is_regular,
prog_tx,
zero: Box::new([0u8; ZERO_CHUNK]),
};
(sink, prog_rx)
}
}
impl Sink<WorkItem> for SweepSink {
type Output = ConsumerSummary;
fn apply(&mut self, item: WorkItem) -> Result<Flow, Error> {
match item {
WorkItem::Good { pos, buf } => {
// Decrypt is on the producer; consumer assumes plaintext.
let len = buf.len() as u64;
self.file
.seek(SeekFrom::Start(pos))
.map_err(|e| Error::IoError { source: e })?;
self.file
.write_all(&buf)
.map_err(|e| Error::IoError { source: e })?;
self.map
.record(pos, len, SectorStatus::Finished)
.map_err(|e| Error::IoError { source: e })?;
}
WorkItem::BisectGood { pos, buf } => {
self.file
.seek(SeekFrom::Start(pos))
.map_err(|e| Error::IoError { source: e })?;
self.file
.write_all(&buf[..])
.map_err(|e| Error::IoError { source: e })?;
self.map
.record(pos, 2048, SectorStatus::Finished)
.map_err(|e| Error::IoError { source: e })?;
}
WorkItem::BisectBad { pos } => {
self.file
.seek(SeekFrom::Start(pos))
.map_err(|e| Error::IoError { source: e })?;
self.file
.write_all(&self.zero[..2048])
.map_err(|e| Error::IoError { source: e })?;
self.map
.record(pos, 2048, SectorStatus::NonTrimmed)
.map_err(|e| Error::IoError { source: e })?;
}
WorkItem::SkipFill { pos, len } | WorkItem::GapFill { pos, len } => {
self.file
.seek(SeekFrom::Start(pos))
.map_err(|e| Error::IoError { source: e })?;
// Subsequent writes are sequential; `WritebackFile`'s
// seek-elision keeps them on the writeback pipeline path.
let mut filled = 0u64;
while filled < len {
let chunk = (len - filled).min(self.zero.len() as u64) as usize;
self.file
.write_all(&self.zero[..chunk])
.map_err(|e| Error::IoError { source: e })?;
filled += chunk as u64;
}
self.map
.record(pos, len, SectorStatus::NonTrimmed)
.map_err(|e| Error::IoError { source: e })?;
}
WorkItem::StatsRequest => {
let stats = self.map.stats();
let bad_ranges = self.map.ranges_with(&[
SectorStatus::NonTrimmed,
SectorStatus::Unreadable,
SectorStatus::NonScraped,
SectorStatus::NonTried,
]);
// Best-effort: drop on backpressure; producer's cache
// stays current enough.
let _ = self
.prog_tx
.try_send(ProgressSnapshot { stats, bad_ranges });
}
}
Ok(Flow::Continue)
}
fn close(mut self) -> Result<Self::Output, Error> {
// Drain the writeback pipeline + fsync the ISO, then persist
// any pending mapfile state. Same finalisation order as the
// pre-Pipeline consumer loop.
if let Err(e) = self.file.sync_all() {
if self.is_regular {
return Err(Error::IoError { source: e });
}
// Non-regular outputs (/dev/null, pipes) always fail
// sync_all; that's not a real error.
}
self.map.flush().map_err(|e| Error::IoError { source: e })?;
Ok(ConsumerSummary {
stats: self.map.stats(),
})
}
}
+10 -11
View File
@@ -1,26 +1,25 @@
//! macOS drive discovery and device resolution.
//!
//! `find_drives` uses IOKit registry enumeration (via `scsi::list_drives`)
//! to discover optical drives without exclusive access or unmounts. Only
//! the returned paths are then opened for INQUIRY to build full `DriveId`.
use crate::error::{Error, Result};
use crate::identity::DriveId;
pub fn find_drives() -> Vec<(String, DriveId)> {
let mut drives = Vec::new();
for i in 0..16 {
let path = format!("/dev/disk{}", i);
if !std::path::Path::new(&path).exists() {
continue;
}
match crate::scsi::open(std::path::Path::new(&path)) {
let discovered = crate::scsi::list_drives();
for info in discovered {
let path = std::path::Path::new(&info.path);
match crate::scsi::open(path) {
Ok(mut transport) => {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty() && (id.raw_inquiry[0] & 0x1F) == 0x05 {
drives.push((path, id));
}
drives.push((info.path.clone(), id));
}
}
Err(_) => {
// Device exists but can't be opened (likely mounted).
// Use `diskutil unmountDisk /dev/diskN` to unmount before accessing.
continue;
}
}
}
+346 -203
View File
@@ -1,28 +1,38 @@
//! Drive session — open, identify, and read from optical drives.
//!
//! Three-step open:
//! 1. `open()` — open device, identify drive. Always OEM.
//! 2. `wait_ready()` — wait for disc to spin up. Call before reading.
//! 3. `init()` — activate custom firmware. Removes riplock.
//! 4. `probe_disc()` — probe disc surface. Drive learns optimal speeds.
pub(crate) fn extract_scsi_context(e: &Error) -> (u8, Option<crate::scsi::ScsiSense>) {
match e {
Error::ScsiError { status, sense, .. } => (*status, *sense),
Error::DiscRead { status, sense, .. } => (status.unwrap_or(0), *sense),
_ => (0, None),
}
}
pub mod capture;
// Per-platform discovery helpers (the `pub(crate)` `find_drives` /
// equivalents). Crate-public so `scsi/{linux,macos,windows}.rs` can
// reuse the existing enumeration logic when shaping `DriveInfo`.
#[cfg(target_os = "linux")]
mod linux;
pub(crate) mod linux;
#[cfg(target_os = "macos")]
mod macos;
pub(crate) mod macos;
#[cfg(windows)]
mod windows;
pub(crate) mod windows;
use crate::error::{Error, Result};
use crate::event::{Event, EventKind};
use crate::identity::DriveId;
use crate::platform::mt1959::Mt1959;
use crate::platform::PlatformDriver;
use crate::platform::mt1959::Mt1959;
use crate::profile::{self, DriveProfile};
use crate::scsi::ScsiTransport;
use crate::sector::SectorReader;
use std::path::Path;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
/// Physical state of the drive tray and disc.
#[derive(Debug, Clone, Copy, PartialEq)]
@@ -47,9 +57,6 @@ const SCSI_GET_EVENT_STATUS: u8 = 0x4A;
const SCSI_MODE_SENSE: u8 = 0x5A;
const SCSI_REPORT_KEY: u8 = 0xA4;
/// Recovery state after a read error — stay at min speed for N bytes.
const RECOVERY_WINDOW: u64 = 500 * 1024 * 1024; // 500 MB
/// Optical disc drive session -- open, identify, unlock, and read.
pub struct Drive {
scsi: Box<dyn ScsiTransport>,
@@ -58,9 +65,20 @@ pub struct Drive {
pub platform: Option<profile::Platform>,
pub drive_id: DriveId,
device_path: String,
/// Bytes remaining in the min-speed recovery window.
/// After a read error, we stay at min speed for RECOVERY_WINDOW bytes.
recovery_bytes_remaining: u64,
/// Halt flag — when set, Drive::read() bails at the next check point.
halt: Arc<AtomicBool>,
/// Event handler — fires for read errors and library-level state changes.
event_fn: Option<Box<dyn Fn(Event) + Send>>,
/// Linux only: raw fd for the corresponding block device (`/dev/sr*`)
/// used as a recovery fallback when SCSI READ via `/dev/sg*` returns
/// an error. The kernel `sr_mod` driver auto-retries failed reads
/// (~5× per command) — historically the reason `dd if=/dev/sr0`
/// recovers ~50% of bad sectors that single-shot `SG_IO` READ
/// misses on the same drive. `None` when the block device couldn't
/// be resolved or opened (no fallback in that case; SCSI read
/// errors propagate as before).
#[cfg(target_os = "linux")]
block_dev_fd: Option<std::os::unix::io::RawFd>,
}
impl Drive {
@@ -79,6 +97,9 @@ impl Drive {
None => (None, None, None),
};
#[cfg(target_os = "linux")]
let block_dev_fd = open_block_device_for_sg(device);
Ok(Drive {
scsi: transport,
driver,
@@ -86,10 +107,67 @@ impl Drive {
profile,
drive_id,
device_path: device.to_string_lossy().to_string(),
recovery_bytes_remaining: 0,
halt: Arc::new(AtomicBool::new(false)),
event_fn: None,
#[cfg(target_os = "linux")]
block_dev_fd,
})
}
/// Get a clone of the halt flag. Set to true to interrupt Drive::read().
pub fn halt_flag(&self) -> Arc<AtomicBool> {
self.halt.clone()
}
/// Halt the drive — Drive::read() will bail at the next check point.
pub fn halt(&self) {
self.halt.store(true, Ordering::Relaxed);
}
/// Clear the halt flag for the next operation.
pub fn clear_halt(&self) {
self.halt.store(false, Ordering::Relaxed);
}
/// Set an event handler for read recovery events.
pub fn on_event(&mut self, f: impl Fn(Event) + Send + 'static) {
self.event_fn = Some(Box::new(f));
}
#[allow(dead_code)] // public on_event registration kept; Drive currently
// has no internal emission sites after the 0.13.6 recovery strip.
// DiscStream is the BytesRead source. Plan to drop on_event in 0.14.
fn emit(&self, kind: EventKind) {
if let Some(ref f) = self.event_fn {
f(Event { kind });
}
}
fn is_halted(&self) -> bool {
self.halt.load(Ordering::Relaxed)
}
/// Halt-aware SCSI execute. Returns `Err(Halted)` if the flag is set
/// before the command dispatches or by the time it completes. The only
/// path to talk to the drive in the recovery hot loop; keeps Drive::read
/// free of explicit halt checks.
fn checked_exec(
&mut self,
cdb: &[u8],
dir: crate::scsi::DataDirection,
buf: &mut [u8],
timeout_ms: u32,
) -> Result<crate::scsi::ScsiResult> {
if self.is_halted() {
return Err(Error::Halted);
}
let r = self.scsi.as_mut().execute(cdb, dir, buf, timeout_ms)?;
if self.is_halted() {
return Err(Error::Halted);
}
Ok(r)
}
/// Close the drive cleanly. Unlocks tray, flushes SCSI state, closes fd.
/// Also runs automatically on Drop as a safety net.
pub fn close(self) {
@@ -101,11 +179,6 @@ impl Drive {
self.unlock_tray();
}
// NOTE: Debug aid — remove after fd issue is resolved
pub fn device_path_owned(&self) -> String {
self.device_path.clone()
}
/// Whether this drive has a known profile (unlock parameters available).
pub fn has_profile(&self) -> bool {
self.profile.is_some()
@@ -118,31 +191,16 @@ impl Drive {
pub fn wait_ready(&mut self) -> Result<()> {
let tur = [SCSI_TEST_UNIT_READY, 0x00, 0x00, 0x00, 0x00, 0x00];
let mut tried_reset = false;
for _ in 0..60 {
let mut buf = [0u8; 0];
match self.scsi.as_mut().execute(
&tur,
crate::scsi::DataDirection::None,
&mut buf,
5_000,
) {
Ok(_) => return Ok(()),
Err(Error::ScsiError { sense_key: 5, .. }) if !tried_reset => {
// Illegal Request on TUR — drive may be stuck from a previous session.
// Try reset() which attempts multiple recovery approaches.
tried_reset = true;
if self.reset().is_ok() {
return Ok(());
}
// If reset failed but disc is present, proceed anyway —
// the scan path will handle errors individually.
if self.drive_status() == DriveStatus::DiscPresent {
return Ok(());
}
}
Err(_) => {}
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
return Ok(());
}
std::thread::sleep(std::time::Duration::from_millis(500));
}
@@ -197,94 +255,18 @@ impl Drive {
5_000,
) {
Ok(_) => DriveStatus::DiscPresent,
Err(Error::ScsiError { sense_key: 2, .. }) => DriveStatus::NotReady,
Err(Error::ScsiError { sense_key: 6, .. }) => DriveStatus::NotReady, // UNIT ATTENTION
Err(ref e)
if e.scsi_sense()
.is_some_and(|s| s.is_not_ready() || s.is_unit_attention()) =>
{
DriveStatus::NotReady
}
_ => DriveStatus::Unknown,
}
}
}
}
/// Attempt to reset the drive to a clean state.
///
/// Escalates through increasingly aggressive recovery:
/// 1. Unlock tray + stop/start — handles normal stuck states
/// 2. Eject cycle — clears LibreDrive firmware stuck state (proven on BU40N)
/// 3. Re-init — firmware re-upload if profile available
///
/// Note: step 2 physically ejects the tray. On slimline drives the user
/// must push it back in manually. Returns Ok(()) if TUR succeeds after
/// any step, even if the drive reports "tray open" (that's a valid state).
pub fn reset(&mut self) -> Result<()> {
let mut buf = [0u8; 0];
let tur = [SCSI_TEST_UNIT_READY, 0x00, 0x00, 0x00, 0x00, 0x00];
// 1. Unlock + stop/start
self.unlock_tray();
let stop = [SCSI_START_STOP_UNIT, 0x00, 0x00, 0x00, 0x00, 0x00];
let _ =
self.scsi
.as_mut()
.execute(&stop, crate::scsi::DataDirection::None, &mut buf, 5_000);
std::thread::sleep(std::time::Duration::from_millis(500));
let start = [SCSI_START_STOP_UNIT, 0x00, 0x00, 0x00, 0x01, 0x00];
let _ =
self.scsi
.as_mut()
.execute(&start, crate::scsi::DataDirection::None, &mut buf, 5_000);
std::thread::sleep(std::time::Duration::from_millis(2000));
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
return Ok(());
}
// 2. Eject cycle — clears MT1959 LibreDrive stuck state.
// After eject, TUR returning "Not Ready — tray open" (sense key 2)
// counts as success: the drive is functional, just needs disc reinserted.
self.unlock_tray();
let eject = [SCSI_START_STOP_UNIT, 0x00, 0x00, 0x00, 0x02, 0x00];
let _ =
self.scsi
.as_mut()
.execute(&eject, crate::scsi::DataDirection::None, &mut buf, 30_000);
std::thread::sleep(std::time::Duration::from_millis(2000));
match self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
{
Ok(_) => return Ok(()),
Err(Error::ScsiError { sense_key: 2, .. }) => return Ok(()), // tray open = valid
_ => {}
}
// 3. If still stuck and we have a profile, try re-init
if self.driver.is_some() {
self.init()?;
std::thread::sleep(std::time::Duration::from_millis(1000));
match self.scsi.as_mut().execute(
&tur,
crate::scsi::DataDirection::None,
&mut buf,
5_000,
) {
Ok(_) => return Ok(()),
Err(Error::ScsiError { sense_key: 2, .. }) => return Ok(()),
_ => {}
}
}
Err(Error::DeviceResetFailed {
path: self.device_path.clone(),
})
}
pub fn platform_name(&self) -> &str {
match self.platform {
Some(ref p) => p.name(),
@@ -450,20 +432,41 @@ impl Drive {
}
}
/// Read sectors from the disc with automatic error recovery.
/// Read sectors from the disc. Single-shot — no inline retries, no
/// SCSI reset.
///
/// On failure: drops to min speed, waits with escalating patience
/// (5s, 10s, 15s, 30s, 60s), resets drive between attempts.
/// After recovery, stays at min speed for 500 MB before ramping up.
/// `recovery=true` uses [`crate::scsi::READ_RECOVERY_TIMEOUT_MS`] (60 s,
/// matches sg_dd) for the `Disc::patch` pass; `recovery=false` uses
/// [`crate::scsi::READ_TIMEOUT_MS`] (30 s, matches the kernel's
/// `/sys/block/sr*/device/timeout` default) for `Disc::copy`'s fast
/// skip-forward sweep. Both budgets are generous enough that the drive
/// can finish ECC recovery on a marginal sector — pre-0.13.21 this was
/// 1.5 s on the fast path which forced the kernel mid-layer to time
/// out and escalate while we waited anyway. On any failure returns
/// `Err(DiscRead)` immediately; orchestration (`Disc::patch` multi-pass,
/// `DiscStream` adaptive batch halving) handles retry policy.
///
/// Returns Err only after all attempts exhausted — user should clean
/// the disc and resume.
pub fn read(&mut self, lba: u32, count: u16, buf: &mut [u8]) -> Result<usize> {
let timeout_ms = if self.recovery_bytes_remaining > 0 {
30_000
/// Inline retry phases (5× gentle + reset+reopen + 5× more) were
/// removed in 0.13.6. Per
/// the stop-wedge postmortem (2026-04-25),
/// the inline reset on the LG BU40N (Initio bridge) wedged drive
/// firmware without ever recovering a sector. The remaining recovery
/// layers (Disc::patch multi-pass, DiscStream batch halving) do not
/// touch the wedge-prone reset path.
pub fn read(&mut self, lba: u32, count: u16, buf: &mut [u8], recovery: bool) -> Result<usize> {
let timeout_ms = if recovery {
crate::scsi::READ_RECOVERY_TIMEOUT_MS
} else {
10_000
crate::scsi::READ_TIMEOUT_MS
};
tracing::debug!(
target: "freemkv::drive",
lba,
count,
recovery,
timeout_ms,
"Drive::read enter"
);
let cdb = [
crate::scsi::SCSI_READ_10,
0x00,
@@ -477,69 +480,79 @@ impl Drive {
0x00,
];
// Normal read
if let Ok(result) = self.scsi.as_mut().execute(
match self.checked_exec(
&cdb,
crate::scsi::DataDirection::FromDevice,
buf,
timeout_ms,
) {
if self.recovery_bytes_remaining > 0 {
let bytes_read = count as u64 * 2048;
self.recovery_bytes_remaining =
self.recovery_bytes_remaining.saturating_sub(bytes_read);
if self.recovery_bytes_remaining == 0 {
self.set_speed(0xFFFF);
Ok(result) => Ok(result.bytes_transferred),
Err(Error::Halted) => Err(Error::Halted),
Err(e) => {
let (status, sense) = extract_scsi_context(&e);
tracing::warn!(
target: "freemkv::drive",
lba,
count,
inner_error = %e,
scsi_status = status,
"Drive::read checked_exec failed"
);
// /dev/sr0 pread fallback (Linux only). The kernel
// sr_mod driver auto-retries failed reads (~5× per
// command). Empirically (BU40N + Dune Part 2 UHD,
// 2026-05-08) dd via /dev/sr0 recovers ~50% of bad
// sectors that a single-shot SG_IO READ misses.
#[cfg(target_os = "linux")]
if recovery {
if let Some(fd) = self.block_dev_fd {
let len = count as usize * 2048;
if buf.len() >= len {
let offset = lba as i64 * 2048;
// Drop kernel cache for this region so we get
// a fresh device read, not stale page-cache
// data from a prior successful neighbour read.
let _ = unsafe {
libc::posix_fadvise(
fd,
offset,
len as i64,
libc::POSIX_FADV_DONTNEED,
)
};
let n = unsafe {
libc::pread(fd, buf.as_mut_ptr() as *mut libc::c_void, len, offset)
};
if n == len as isize {
tracing::info!(
target: "freemkv::drive",
lba,
count,
bytes = len,
"Drive::read recovered via /dev/sr0 pread fallback"
);
return Ok(len);
}
tracing::debug!(
target: "freemkv::drive",
lba,
count,
pread_ret = n as i64,
errno = std::io::Error::last_os_error().raw_os_error().unwrap_or(0),
"/dev/sr0 pread fallback also failed"
);
}
}
}
}
return Ok(result.bytes_transferred);
}
// Phase 1: gentle — sleep 30s, retry. 5 times.
self.set_speed(0);
for _ in 0..5 {
std::thread::sleep(std::time::Duration::from_secs(30));
if let Ok(result) = self.scsi.as_mut().execute(
&cdb,
crate::scsi::DataDirection::FromDevice,
buf,
30_000,
) {
self.recovery_bytes_remaining = RECOVERY_WINDOW;
return Ok(result.bytes_transferred);
Err(Error::DiscRead {
sector: lba as u64,
status: Some(status),
sense,
})
}
}
// Phase 2: fresh start — close, reset, open, init.
let device = std::path::PathBuf::from(&self.device_path);
std::thread::sleep(std::time::Duration::from_secs(5));
let _ = crate::scsi::reset(&device);
std::thread::sleep(std::time::Duration::from_secs(5));
self.scsi = crate::scsi::open(&device)?;
let _ = self.init();
let _ = self.wait_ready();
self.set_speed(0);
// Phase 3: gentle again on fresh connection — sleep 30s, retry. 5 times.
for _ in 0..5 {
std::thread::sleep(std::time::Duration::from_secs(30));
if let Ok(result) = self.scsi.as_mut().execute(
&cdb,
crate::scsi::DataDirection::FromDevice,
buf,
30_000,
) {
self.recovery_bytes_remaining = RECOVERY_WINDOW;
return Ok(result.bytes_transferred);
}
}
// Both phases failed.
self.recovery_bytes_remaining = RECOVERY_WINDOW;
Err(Error::DiscRead { sector: lba as u64 })
}
/// Read the disc capacity in sectors (2048 bytes each).
@@ -636,28 +649,110 @@ impl Drop for Drive {
fn drop(&mut self) {
self.cleanup();
// SgIoTransport::drop() runs next, calling libc::close(fd)
#[cfg(target_os = "linux")]
if let Some(fd) = self.block_dev_fd.take() {
unsafe { libc::close(fd) };
}
}
}
/// Resolve a `/dev/sg*` path to the corresponding `/dev/sr*` block
/// device by walking sysfs, then open it for read (no `O_DIRECT` —
/// `posix_fadvise(POSIX_FADV_DONTNEED)` flushes the cache before each
/// pread, which avoids buffer-alignment requirements while still
/// forcing fresh device reads).
///
/// Returns `None` on any error (sysfs not present, no matching block
/// device, open failed). Callers treat that as "no fallback available"
/// and propagate the original SCSI READ error.
#[cfg(target_os = "linux")]
fn open_block_device_for_sg(sg_path: &Path) -> Option<std::os::unix::io::RawFd> {
let basename = sg_path.file_name()?.to_str()?;
if !basename.starts_with("sg") {
return None;
}
let sysfs_dir = format!("/sys/class/scsi_generic/{}/device/block", basename);
let entries = std::fs::read_dir(&sysfs_dir).ok()?;
let block_name = entries
.flatten()
.find_map(|e| e.file_name().into_string().ok())?;
let block_path = format!("/dev/{}", block_name);
let mut bytes = block_path.as_bytes().to_vec();
bytes.push(0);
let fd = unsafe {
libc::open(
bytes.as_ptr() as *const libc::c_char,
libc::O_RDONLY | libc::O_CLOEXEC,
)
};
if fd < 0 {
tracing::debug!(
target: "freemkv::drive",
sg = basename,
block_path,
errno = std::io::Error::last_os_error().raw_os_error().unwrap_or(0),
"Failed to open block device for fallback; sr0 fallback disabled"
);
None
} else {
tracing::info!(
target: "freemkv::drive",
sg = basename,
block_path,
fd,
"Opened /dev/sr* as recovery fallback for failed SCSI reads"
);
Some(fd)
}
}
impl SectorReader for Drive {
fn read_sectors(&mut self, lba: u32, count: u16, buf: &mut [u8]) -> Result<usize> {
self.read(lba, count, buf)
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
self.read(lba, count, buf, recovery)
}
fn set_speed(&mut self, kbs: u16) {
Drive::set_speed(self, kbs);
}
}
/// Find all optical drives connected to this system.
/// Returns opened Drive objects ready for use.
pub fn find_drives() -> Vec<Drive> {
/// Find the first optical drive on this system and open it.
///
/// For just listing drives without opening (e.g. UI sidebar), use
/// `scsi::list_drives()` — that returns `DriveInfo` (path + identity)
/// without the cost of running every drive's profile + identity probe.
pub fn find_drive() -> Option<Drive> {
discover_drives()
.into_iter()
.filter_map(|(path, _)| Drive::open(std::path::Path::new(&path)).ok())
.collect()
.find_map(|(path, _)| Drive::open(std::path::Path::new(&path)).ok())
}
/// Find the first optical drive.
/// Returns an opened Drive ready for use.
pub fn find_drive() -> Option<Drive> {
find_drives().into_iter().next()
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping
/// to true. Kept for the unit tests that cover the slicing behaviour;
/// production code paths no longer sleep on the recovery hot path
/// (recovery loop removed in 0.13.6).
#[cfg(test)]
fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<()> {
const SLICE: std::time::Duration = std::time::Duration::from_millis(100);
let deadline = std::time::Instant::now() + total;
loop {
if halt.load(Ordering::Relaxed) {
return Err(Error::Halted);
}
let now = std::time::Instant::now();
if now >= deadline {
return Ok(());
}
let remaining = deadline - now;
std::thread::sleep(remaining.min(SLICE));
}
}
/// Internal: discover drive paths + IDs without opening full Drive objects.
@@ -700,10 +795,58 @@ fn create_driver(
match platform {
profile::Platform::Mt1959A => Ok(Box::new(Mt1959::new(profile.clone(), false))),
profile::Platform::Mt1959B => Ok(Box::new(Mt1959::new(profile.clone(), true))),
profile::Platform::Renesas => Err(Error::UnsupportedDrive {
vendor_id: profile.identity.vendor_id.trim().to_string(),
product_id: String::new(),
product_revision: "Renesas not yet implemented".to_string(),
profile::Platform::Renesas => Err(Error::PlatformNotImplemented {
platform: "renesas".to_string(),
}),
}
}
#[cfg(test)]
mod halt_tests {
use super::*;
use std::time::{Duration, Instant};
#[test]
fn sleep_until_halted_completes_when_not_halted() {
let flag = AtomicBool::new(false);
let t0 = Instant::now();
let r = sleep_until_halted(&flag, Duration::from_millis(150));
assert!(r.is_ok());
assert!(t0.elapsed() >= Duration::from_millis(140));
}
#[test]
fn sleep_until_halted_returns_immediately_if_preflagged() {
let flag = AtomicBool::new(true);
let t0 = Instant::now();
let r = sleep_until_halted(&flag, Duration::from_secs(10));
assert!(matches!(r, Err(Error::Halted)));
// Must wake within one slice (100 ms) — the whole point of the
// primitive is that a 30 s sleep doesn't block Stop.
assert!(t0.elapsed() < Duration::from_millis(200));
}
#[test]
fn sleep_until_halted_wakes_mid_sleep() {
let flag = Arc::new(AtomicBool::new(false));
let f2 = flag.clone();
let t0 = Instant::now();
std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(150));
f2.store(true, Ordering::Relaxed);
});
let r = sleep_until_halted(&flag, Duration::from_secs(10));
assert!(matches!(r, Err(Error::Halted)));
let waited = t0.elapsed();
// Flag flipped at ~150 ms; we wake within one 100 ms slice → <300 ms.
assert!(waited < Duration::from_millis(350), "waited {waited:?}");
assert!(waited >= Duration::from_millis(140), "waited {waited:?}");
}
#[test]
fn sleep_until_halted_zero_duration_is_noop_when_not_halted() {
let flag = AtomicBool::new(false);
let r = sleep_until_halted(&flag, Duration::ZERO);
assert!(r.is_ok());
}
}
+341 -10
View File
@@ -24,10 +24,15 @@ pub const E_DEVICE_NOT_FOUND: u16 = 1000;
pub const E_DEVICE_PERMISSION: u16 = 1001;
pub const E_DEVICE_NOT_READY: u16 = 1002;
pub const E_DEVICE_RESET_FAILED: u16 = 1003;
pub const E_SCSI_INTERFACE_UNAVAILABLE: u16 = 1004;
pub const E_DEVICE_LOCKED: u16 = 1005;
pub const E_IOKIT_PLUGIN_FAILED: u16 = 1006;
// Profile (2xxx)
pub const E_UNSUPPORTED_DRIVE: u16 = 2000;
pub const E_PROFILE_PARSE: u16 = 2002;
pub const E_UNSUPPORTED_PLATFORM: u16 = 2003;
pub const E_PLATFORM_NOT_IMPLEMENTED: u16 = 2004;
// Unlock (3xxx)
pub const E_UNLOCK_FAILED: u16 = 3000;
@@ -41,6 +46,7 @@ pub const E_IO_ERROR: u16 = 5000;
// Disc format (6xxx)
pub const E_DISC_READ: u16 = 6000;
pub const E_HALTED: u16 = 6010;
pub const E_MPLS_PARSE: u16 = 6001;
pub const E_CLPI_PARSE: u16 = 6002;
pub const E_UDF_NOT_FOUND: u16 = 6003;
@@ -48,6 +54,7 @@ pub const E_DISC_TITLE_RANGE: u16 = 6005;
pub const E_IFO_PARSE: u16 = 6007;
pub const E_MKV_INVALID: u16 = 6008;
pub const E_NO_STREAMS: u16 = 6009;
pub const E_MAPFILE_INVALID: u16 = 6011;
// AACS (7xxx)
pub const E_AACS_NO_KEYS: u16 = 7000;
@@ -83,6 +90,7 @@ pub const E_PES_FRAME_TOO_LARGE: u16 = 9005;
pub const E_PES_INVALID_MAGIC: u16 = 9006;
pub const E_ISO_TOO_LARGE: u16 = 9007;
pub const E_NO_METADATA: u16 = 9008;
pub const E_DISC_URL_NOT_DIRECT: u16 = 9009;
// ── Error enum ──────────────────────────────────────────────────────────────
@@ -102,6 +110,24 @@ pub enum Error {
DeviceResetFailed {
path: String,
},
/// Platform-specific SCSI interface couldn't be obtained from the OS
/// (macOS: `SCSITaskDeviceInterface` unavailable). The `path` field
/// carries the device path; no English commentary on the failure mode.
ScsiInterfaceUnavailable {
path: String,
},
/// Device is held by another process / kernel state. `kr` is the
/// platform return code (macOS IOReturn, Linux errno-equivalent).
DeviceLocked {
path: String,
kr: u32,
},
/// macOS IOKit plugin couldn't be created for this device. `kr` is
/// the IOReturn code from `IOCreatePlugInInterfaceForService`.
IoKitPluginFailed {
path: String,
kr: u32,
},
// Profile (2xxx)
UnsupportedDrive {
@@ -110,6 +136,16 @@ pub enum Error {
product_revision: String,
},
ProfileParse,
/// SCSI transport was requested on an OS without a backend
/// implementation. `target` is the `std::env::consts::OS` value.
UnsupportedPlatform {
target: String,
},
/// Drive matched a known platform that we haven't implemented yet
/// (e.g. Renesas firmware). `platform` is a stable identifier.
PlatformNotImplemented {
platform: String,
},
// Unlock (3xxx)
UnlockFailed,
@@ -119,10 +155,24 @@ pub enum Error {
},
// SCSI (4xxx)
/// SCSI command failed.
///
/// `opcode` is the failing CDB byte 0. `status` is the raw SCSI
/// status byte: `0x02` = CHECK CONDITION (drive replied with sense
/// data), `0xFF` = libfreemkv-synthesised sentinel meaning "no SCSI
/// status delivered" (kernel timeout, USB bridge wedge, IOKit
/// service failure). `sense` carries the drive's SPC-4 sense triple
/// when the drive replied; `None` for transport-layer failures.
///
/// Recommended dispatch (callers shouldn't pattern-match raw
/// fields):
/// - [`Error::is_scsi_transport_failure`] — bail; bridge/transport wedge
/// - [`Error::is_marginal_read`] — drive said this read was marginal; smaller block may recover
/// - [`Error::scsi_sense`] — borrow the sense triple for finer routing ([`ScsiSense::is_medium_error`] etc.)
ScsiError {
opcode: u8,
status: u8,
sense_key: u8,
sense: Option<crate::scsi::ScsiSense>,
},
// I/O (5xxx)
@@ -133,7 +183,11 @@ pub enum Error {
// Disc format (6xxx)
DiscRead {
sector: u64,
status: Option<u8>,
sense: Option<crate::scsi::ScsiSense>,
},
/// Drive was halted by caller.
Halted,
MplsParse,
ClpiParse,
UdfNotFound {
@@ -146,6 +200,12 @@ pub enum Error {
IfoParse,
MkvInvalid,
NoStreams,
/// ddrescue mapfile parse failed. `kind` is a stable, language-neutral
/// identifier (e.g. `"status_char"`, `"hex"`); not a translatable
/// English message.
MapfileInvalid {
kind: &'static str,
},
// AACS (7xxx)
AacsNoKeys,
@@ -199,6 +259,10 @@ pub enum Error {
path: String,
},
NoMetadata,
/// `disc://` URLs aren't openable through `input()` — callers must use
/// `Drive::open() + Disc::scan() + DiscStream::new()` directly. This
/// is a structural API constraint, not a parse failure.
DiscUrlNotDirect,
}
impl Error {
@@ -208,13 +272,19 @@ impl Error {
Error::DevicePermission { .. } => E_DEVICE_PERMISSION,
Error::DeviceNotReady { .. } => E_DEVICE_NOT_READY,
Error::DeviceResetFailed { .. } => E_DEVICE_RESET_FAILED,
Error::ScsiInterfaceUnavailable { .. } => E_SCSI_INTERFACE_UNAVAILABLE,
Error::DeviceLocked { .. } => E_DEVICE_LOCKED,
Error::IoKitPluginFailed { .. } => E_IOKIT_PLUGIN_FAILED,
Error::UnsupportedDrive { .. } => E_UNSUPPORTED_DRIVE,
Error::ProfileParse => E_PROFILE_PARSE,
Error::UnsupportedPlatform { .. } => E_UNSUPPORTED_PLATFORM,
Error::PlatformNotImplemented { .. } => E_PLATFORM_NOT_IMPLEMENTED,
Error::UnlockFailed => E_UNLOCK_FAILED,
Error::SignatureMismatch { .. } => E_SIGNATURE_MISMATCH,
Error::ScsiError { .. } => E_SCSI_ERROR,
Error::IoError { .. } => E_IO_ERROR,
Error::DiscRead { .. } => E_DISC_READ,
Error::Halted => E_HALTED,
Error::MplsParse => E_MPLS_PARSE,
Error::ClpiParse => E_CLPI_PARSE,
Error::UdfNotFound { .. } => E_UDF_NOT_FOUND,
@@ -222,6 +292,7 @@ impl Error {
Error::IfoParse => E_IFO_PARSE,
Error::MkvInvalid => E_MKV_INVALID,
Error::NoStreams => E_NO_STREAMS,
Error::MapfileInvalid { .. } => E_MAPFILE_INVALID,
Error::AacsNoKeys => E_AACS_NO_KEYS,
Error::AacsCertShort => E_AACS_CERT_SHORT,
Error::AacsAgidAlloc => E_AACS_AGID_ALLOC,
@@ -251,6 +322,7 @@ impl Error {
Error::PesInvalidMagic => E_PES_INVALID_MAGIC,
Error::IsoTooLarge { .. } => E_ISO_TOO_LARGE,
Error::NoMetadata => E_NO_METADATA,
Error::DiscUrlNotDirect => E_DISC_URL_NOT_DIRECT,
}
}
}
@@ -263,6 +335,22 @@ impl std::fmt::Display for Error {
Error::DevicePermission { path } => write!(f, "E{}: {}", self.code(), path),
Error::DeviceNotReady { path } => write!(f, "E{}: {}", self.code(), path),
Error::DeviceResetFailed { path } => write!(f, "E{}: {}", self.code(), path),
Error::ScsiInterfaceUnavailable { path } => write!(f, "E{}: {}", self.code(), path),
Error::DeviceLocked { path, kr } => {
write!(f, "E{}: {} 0x{:08x}", self.code(), path, kr)
}
Error::IoKitPluginFailed { path, kr } => {
write!(f, "E{}: {} 0x{:08x}", self.code(), path, kr)
}
Error::UnsupportedPlatform { target } => {
write!(f, "E{}: {}", self.code(), target)
}
Error::PlatformNotImplemented { platform } => {
write!(f, "E{}: {}", self.code(), platform)
}
Error::MapfileInvalid { kind } => {
write!(f, "E{}: {}", self.code(), kind)
}
Error::UnsupportedDrive {
vendor_id,
product_id,
@@ -291,19 +379,47 @@ impl std::fmt::Display for Error {
Error::ScsiError {
opcode,
status,
sense_key,
} => {
write!(
sense,
} => match sense {
Some(s) => write!(
f,
"E{}: 0x{:02x}/0x{:02x}/0x{:02x}",
"E{}: 0x{:02x}/0x{:02x}/0x{:02x}/0x{:02x}/0x{:02x}",
self.code(),
opcode,
status,
sense_key
)
}
s.sense_key,
s.asc,
s.ascq,
),
None => write!(f, "E{}: 0x{:02x}/0x{:02x}", self.code(), opcode, status,),
},
Error::IoError { source } => write!(f, "E{}: {}", self.code(), source),
Error::DiscRead { sector } => write!(f, "E{}: {}", self.code(), sector),
Error::DiscRead {
sector,
status,
sense,
} => match (status, sense) {
(Some(st), Some(s)) => write!(
f,
"E{}: {} 0x{:02x}/0x{:02x}/0x{:02x}",
self.code(),
sector,
st,
s.sense_key,
s.asc,
),
(Some(st), None) => write!(f, "E{}: {} 0x{:02x}", self.code(), sector, st,),
(None, Some(s)) => write!(
f,
"E{}: {} 0x{:02x}/0x{:02x}",
self.code(),
sector,
s.sense_key,
s.asc,
),
(None, None) => write!(f, "E{}: {}", self.code(), sector),
},
Error::Halted => write!(f, "E{}", self.code()),
Error::UdfNotFound { path } => write!(f, "E{}: {}", self.code(), path),
Error::DiscTitleRange { index, count } => {
write!(f, "E{}: {}/{}", self.code(), index, count)
@@ -352,7 +468,10 @@ impl From<Error> for std::io::Error {
7000..=7999 => std::io::ErrorKind::PermissionDenied,
8000..=8999 => std::io::ErrorKind::Other,
9000..=9001 => std::io::ErrorKind::Unsupported,
9002..=9009 => std::io::ErrorKind::InvalidInput,
9002..=9008 => std::io::ErrorKind::InvalidInput,
// 9009 DiscUrlNotDirect: structurally unsupported entry point,
// not a parse failure — caller used the wrong API.
9009 => std::io::ErrorKind::Unsupported,
_ => std::io::ErrorKind::Other,
};
std::io::Error::new(kind, msg)
@@ -361,3 +480,215 @@ impl From<Error> for std::io::Error {
/// Convenience alias for `Result<T, Error>`.
pub type Result<T> = std::result::Result<T, Error>;
impl Error {
/// Borrow the drive-returned SPC-4 sense triple if this error is a
/// [`Error::ScsiError`] carrying sense data. `None` for any other
/// variant **and** for `ScsiError`s that represent a transport-layer
/// failure (where the device never delivered a SCSI status reply, so
/// no sense data exists).
pub fn scsi_sense(&self) -> Option<&crate::scsi::ScsiSense> {
match self {
Error::ScsiError { sense: Some(s), .. } => Some(s),
Error::DiscRead { sense: Some(s), .. } => Some(s),
_ => None,
}
}
/// True if this is a [`Error::ScsiError`] representing a transport-layer
/// failure — kernel timeout, USB bridge wedge, IOKit service error.
/// The device never delivered a SCSI status reply, so there is no
/// sense data to inspect; retrying typically requires physical
/// intervention (replug).
pub fn is_scsi_transport_failure(&self) -> bool {
matches!(
self,
Error::ScsiError {
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
..
}
) || matches!(
self,
Error::DiscRead {
status: Some(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE),
..
}
)
}
/// True if this error indicates bridge degradation — the SCSI status
/// is neither GOOD (0x00), CHECK CONDITION (0x02), nor transport failure
/// (0xFF). Observed on the Initio INIC-1618L USB bridge preceding a full
/// crash: the bridge firmware returns non-standard status bytes (e.g.
/// 0x04, 0x05) with empty sense data. The caller should cool down
/// (10 s pause) and retry rather than hammering the bridge.
pub fn is_bridge_degradation(&self) -> bool {
let status = match self {
Error::ScsiError { status, .. } => *status,
Error::DiscRead { status, .. } => status.unwrap_or(0),
_ => return false,
};
status != crate::scsi::SCSI_STATUS_GOOD
&& status != crate::scsi::SCSI_STATUS_CHECK_CONDITION
&& status != crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE
}
/// True if the underlying SCSI failure is a *marginal read* — the
/// drive returned an error category in which smaller-granularity
/// retries can sometimes recover the data:
///
/// - MEDIUM ERROR (sense key 3) — canonical bad-sector signal
/// - ABORTED COMMAND (sense key B) — transient; retry usually works
/// - RECOVERED ERROR (sense key 1) / NO SENSE (sense key 0) — not
/// classified as fatal; treat as recoverable
///
/// Returns `false` for transport failures (no sense data delivered),
/// HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION, ILLEGAL
/// REQUEST, BLANK CHECK, kernel `IoError`, and any non-SCSI variant.
/// Caller-agnostic predicate — describes a property of the *error*,
/// not what one specific call site should do with it. Used by
/// `Disc::copy`'s hysteresis dispatch.
pub fn is_marginal_read(&self) -> bool {
self.scsi_sense()
.map(crate::scsi::ScsiSense::is_marginal)
.unwrap_or(false)
}
}
#[cfg(test)]
mod tests {
//! Smoke tests for the error code → variant mapping. Each new variant
//! added in 0.13.0 (English-elimination work) gets a code() check + a
//! Display sanity-check (no English words) + an io::ErrorKind mapping
//! check. Without these, future drift between the const codes and the
//! match arms in `code()` / the From impl could silently miscategorize.
use super::*;
#[test]
fn new_variants_have_distinct_codes() {
let codes = [
Error::ScsiInterfaceUnavailable { path: "p".into() }.code(),
Error::DeviceLocked {
path: "p".into(),
kr: 0,
}
.code(),
Error::IoKitPluginFailed {
path: "p".into(),
kr: 0,
}
.code(),
Error::UnsupportedPlatform { target: "x".into() }.code(),
Error::PlatformNotImplemented {
platform: "renesas".into(),
}
.code(),
Error::MapfileInvalid { kind: "hex" }.code(),
Error::DiscUrlNotDirect.code(),
];
let mut sorted = codes.to_vec();
sorted.sort();
sorted.dedup();
assert_eq!(
sorted.len(),
codes.len(),
"two new variants share a code — check error.rs constants"
);
}
#[test]
fn display_emits_no_english_words() {
// Every variant's Display must be `E{code}: {data}` — no English.
// Sample a few of the new variants and a few existing ones to
// catch accidental string-stuffing in future edits.
let cases: &[(Error, u16)] = &[
(
Error::ScsiInterfaceUnavailable {
path: "/dev/sg4".into(),
},
E_SCSI_INTERFACE_UNAVAILABLE,
),
(
Error::DeviceLocked {
path: "/dev/sg4".into(),
kr: 0xE00002C5,
},
E_DEVICE_LOCKED,
),
(
Error::UnsupportedPlatform {
target: "freebsd".into(),
},
E_UNSUPPORTED_PLATFORM,
),
(
Error::PlatformNotImplemented {
platform: "renesas".into(),
},
E_PLATFORM_NOT_IMPLEMENTED,
),
(Error::MapfileInvalid { kind: "hex" }, E_MAPFILE_INVALID),
(Error::DiscUrlNotDirect, E_DISC_URL_NOT_DIRECT),
];
for (e, want_code) in cases {
let s = e.to_string();
assert!(
s.starts_with(&format!("E{}", want_code)),
"{:?} display does not lead with code: {}",
e,
s
);
// Crude English filter — `Display` should never emit ASCII words
// longer than 4 chars (codes/paths/identifiers like `/dev/sg4`,
// `renesas`, `freebsd` all pass; "exclusive access denied" would
// not).
for word in s.split(|c: char| !c.is_ascii_alphabetic()) {
assert!(
word.len() <= 8
|| word.eq_ignore_ascii_case("renesas")
|| word.eq_ignore_ascii_case("freebsd"),
"Display contains suspicious English-looking word `{word}` in `{s}`"
);
}
}
}
#[test]
fn iokind_mapping_for_new_variants() {
use std::io::ErrorKind;
let mapped = |e: Error| -> ErrorKind {
let io: std::io::Error = e.into();
io.kind()
};
// 1xxx range → NotFound
assert_eq!(
mapped(Error::ScsiInterfaceUnavailable { path: "p".into() }),
ErrorKind::NotFound
);
assert_eq!(
mapped(Error::DeviceLocked {
path: "p".into(),
kr: 0
}),
ErrorKind::NotFound
);
// 2xxx range → Unsupported
assert_eq!(
mapped(Error::UnsupportedPlatform { target: "x".into() }),
ErrorKind::Unsupported
);
assert_eq!(
mapped(Error::PlatformNotImplemented {
platform: "x".into()
}),
ErrorKind::Unsupported
);
// 6xxx range → InvalidData
assert_eq!(
mapped(Error::MapfileInvalid { kind: "hex" }),
ErrorKind::InvalidData
);
// 9009 special-cased to Unsupported
assert_eq!(mapped(Error::DiscUrlNotDirect), ErrorKind::Unsupported);
}
}
+25
View File
@@ -80,6 +80,22 @@ pub enum EventKind {
sector_count: u64,
},
/// Sector recovered after a retry (Drive::read multi-phase recovery).
SectorRecovered { sector: u64 },
/// Sector unreadable, zero-filled (skip mode).
SectorSkipped { sector: u64 },
/// Adaptive batch sizer changed the read size.
///
/// Fires on shrink (read failed at larger size) and on probe-up
/// (enough clean reads to try larger again). Consumers use this to
/// display a "recovering" state distinct from "ripping normally".
BatchSizeChanged {
new_size: u16,
reason: BatchSizeReason,
},
/// Operation complete.
Complete {
/// Total bytes written.
@@ -89,5 +105,14 @@ pub enum EventKind {
},
}
/// Why the adaptive batch sizer changed size.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BatchSizeReason {
/// Read failed; sizer halved the batch.
Shrunk,
/// Clean-read streak threshold hit; sizer doubled toward preferred.
Probed,
}
/// A no-op event handler. Ignores all events.
pub fn ignore(_event: Event) {}
+166
View File
@@ -0,0 +1,166 @@
//! One-bit cooperative cancellation flag.
//!
//! `Halt` is a clonable token wrapping `Arc<AtomicBool>`. Pass clones into
//! every long-running loop; the loop polls `is_cancelled()` and bails out
//! cleanly. Calling `cancel()` from any clone flips the shared flag, and
//! every other clone observes it on its next poll.
//!
//! Why: `Ordering::Relaxed` is sufficient on both load and store because
//! this flag is purely advisory — no other memory operations piggyback on
//! it for happens-before ordering. Callers that need to publish data
//! across threads do so via channels or other synchronization, not via
//! this bit.
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
/// Clonable, infallible cooperative-cancellation token.
///
/// Clones share the same underlying flag. `cancel()` is one-way; there is
/// no `reset()` by design — construct a fresh `Halt` for a fresh
/// operation.
///
/// Construct with [`Halt::new`] (or [`Halt::default`]). The `Default`
/// impl forwards to `new()` — both produce a fresh, uncancelled token.
/// The pair exists because clippy's `new_without_default` lint requires
/// `Default` whenever a public `new()` is present, even when the two
/// would do exactly the same thing.
#[derive(Clone, Debug)]
pub struct Halt(Arc<AtomicBool>);
impl Halt {
/// Construct a fresh, uncancelled token.
pub fn new() -> Self {
Self(Arc::new(AtomicBool::new(false)))
}
/// Wrap an existing `Arc<AtomicBool>` as a `Halt`. Useful as a
/// bridge during the 0.18 deprecation window: callers that already
/// hold an `Arc<AtomicBool>` (e.g. `Drive::halt_flag()`, the
/// deprecated `DiscStream::set_halt`) can adopt the new token API
/// without changing the underlying flag.
///
/// Cancelling either side flips the same bit — the wrapping `Halt`
/// and the original `Arc` are two views over one shared flag.
pub fn from_arc(flag: Arc<AtomicBool>) -> Self {
Self(flag)
}
/// Borrow the underlying `Arc<AtomicBool>`. Used at boundaries with
/// pre-`Halt` APIs that still take an `Arc<AtomicBool>` directly
/// (`CopyOptions::halt`, the deprecated `DiscStream::set_halt`).
/// Round 3 deletes those boundaries and this accessor with them.
pub fn as_arc(&self) -> &Arc<AtomicBool> {
&self.0
}
/// Flip the shared flag to cancelled. Idempotent.
pub fn cancel(&self) {
self.0.store(true, Ordering::Relaxed);
}
/// Read the shared flag.
pub fn is_cancelled(&self) -> bool {
self.0.load(Ordering::Relaxed)
}
}
impl Default for Halt {
fn default() -> Self {
Self::new()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fresh_is_not_cancelled() {
let h = Halt::new();
assert!(!h.is_cancelled());
}
#[test]
fn cancel_flips_state() {
let h = Halt::new();
assert!(!h.is_cancelled());
h.cancel();
assert!(h.is_cancelled());
}
#[test]
fn cancel_is_idempotent() {
let h = Halt::new();
h.cancel();
h.cancel();
assert!(h.is_cancelled());
}
#[test]
fn clone_shares_state() {
let original = Halt::new();
let cloned = original.clone();
assert!(!original.is_cancelled());
assert!(!cloned.is_cancelled());
// Cancel via the clone; the original observes it.
cloned.cancel();
assert!(original.is_cancelled());
assert!(cloned.is_cancelled());
}
#[test]
fn clone_shares_state_reverse_direction() {
let original = Halt::new();
let cloned = original.clone();
// Cancel via the original; the clone observes it.
original.cancel();
assert!(cloned.is_cancelled());
}
#[test]
fn clone_shares_state_across_threads() {
let h = Halt::new();
let h2 = h.clone();
let handle = std::thread::spawn(move || {
h2.cancel();
});
handle.join().unwrap();
assert!(h.is_cancelled());
}
#[test]
fn from_arc_shares_state() {
// The 0.18 deprecation-window bridge: a Halt built from an
// existing Arc<AtomicBool> must be a *view* over the same bit,
// not a fresh copy. Cancelling either side flips both.
let arc = Arc::new(AtomicBool::new(false));
let halt = Halt::from_arc(arc.clone());
assert!(!halt.is_cancelled());
assert!(!arc.load(Ordering::Relaxed));
// Cancel via the wrapping Halt; the original Arc observes it.
halt.cancel();
assert!(arc.load(Ordering::Relaxed));
// Conversely: flip the Arc directly; the Halt view observes it.
let arc2 = Arc::new(AtomicBool::new(false));
let halt2 = Halt::from_arc(arc2.clone());
arc2.store(true, Ordering::Relaxed);
assert!(halt2.is_cancelled());
}
#[test]
fn as_arc_returns_backing_flag() {
// `as_arc()` must hand back the *same* Arc, not a clone of a
// different bit. Verified by writing through the borrowed Arc
// and observing through the Halt.
let halt = Halt::new();
let arc = halt.as_arc().clone();
assert!(!halt.is_cancelled());
arc.store(true, Ordering::Relaxed);
assert!(halt.is_cancelled());
}
}
+40 -3
View File
@@ -47,6 +47,8 @@ pub struct DvdTitle {
pub duration_secs: f64,
/// Cell sector ranges
pub cells: Vec<DvdCell>,
/// Chapter start times in seconds (derived from program map + cell times)
pub chapter_times: Vec<f64>,
/// Subtitle palette from PGC: 16 entries of [padding, Y, Cb, Cr].
pub palette: Option<Vec<[u8; 4]>>,
}
@@ -541,6 +543,39 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
duration_secs
};
// Extract chapter times from program map + cell durations
// PGC program map offset at 0xE6, maps program_number → first cell_number
let chapter_times = {
let pgm_map_offset = be_u16(data, pgc_offset + 0xE6).unwrap_or(0) as usize;
let nr_of_programs = byte_at(data, pgc_offset + 0x02).unwrap_or(0) as usize;
let mut times = Vec::new();
if pgm_map_offset > 0 && nr_of_programs > 0 && cell_playback_offset > 0 {
let pgm_base = pgc_offset + pgm_map_offset;
// Collect cell durations
let mut cell_durations = Vec::with_capacity(num_cells);
let cell_base = pgc_offset + cell_playback_offset;
for i in 0..num_cells {
let co = cell_base + i * 24;
if co + 8 <= data.len() {
cell_durations.push(bcd_to_secs(&data[co + 4..co + 8]));
} else {
cell_durations.push(0.0);
}
}
// Program map: each byte is the first cell number (1-based) for that program
for p in 0..nr_of_programs {
if pgm_base + p >= data.len() {
break;
}
let first_cell = data[pgm_base + p] as usize;
// Chapter time = sum of cell durations before this program's first cell
let time: f64 = cell_durations[..first_cell.saturating_sub(1)].iter().sum();
times.push(time);
}
}
times
};
// Extract subtitle palette at PGC offset 0xA4: 16 colors × 4 bytes [padding, Y, Cb, Cr]
let palette = if pgc_offset + 0xA4 + 64 <= data.len() {
let mut colors = Vec::with_capacity(16);
@@ -562,6 +597,7 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
chapters,
duration_secs,
cells,
chapter_times,
palette,
})
}
@@ -651,6 +687,7 @@ mod tests {
chapters: 5,
duration_secs: 3600.0,
cells: vec![cell.clone()],
chapter_times: Vec::new(),
palette: None,
};
assert_eq!(title.chapters, 5);
@@ -697,12 +734,12 @@ mod tests {
let mut pgc = vec![0u8; 0xEA];
pgc[0x02] = 1; // 1 program
pgc[0x03] = 2; // 2 cells
// 1h 59m 30s at 29.97fps, 0 frames
// 1h 59m 30s at 29.97fps, 0 frames
pgc[0x04] = 0x01; // hours BCD
pgc[0x05] = 0x59; // minutes BCD
pgc[0x06] = 0x30; // seconds BCD
pgc[0x07] = 0b11_000000; // 29.97fps, 0 frames
// Cell playback info offset at PGC+0xE8
// Cell playback info offset at PGC+0xE8
let cell_offset: u16 = 0xEA; // right after minimum header
pgc[0xE8] = (cell_offset >> 8) as u8;
pgc[0xE9] = cell_offset as u8;
@@ -718,7 +755,7 @@ mod tests {
pgc[co + 21] = 0;
pgc[co + 22] = 0;
pgc[co + 23] = 200; // last sector
// Cell 1: sectors 300-400
// Cell 1: sectors 300-400
let co = 0xEA + 24;
pgc[co + 8] = 0;
pgc[co + 9] = 0;
+31
View File
@@ -0,0 +1,31 @@
//! File I/O helpers that bound kernel cache pressure on big writes.
//!
//! `WritebackFile` is a drop-in wrapper around `std::fs::File` for any
//! call site that performs large sequential writes (sweep, patch, mux,
//! etc.). It implements `Write` and `Seek` so existing code paths can
//! swap `File` for `WritebackFile` with no body changes. Internally it
//! drives a `WritebackPipeline` that, on Linux, drains dirty pages
//! continuously at 32 MB granularity to avoid the kernel's
//! accumulate-then-burst flush behaviour. macOS and Windows use a
//! no-op pipeline — their default cache policies have not been shown
//! to exhibit the same pathology for this access pattern.
//!
//! `Pipeline` + `Sink` (0.18) is the generic producer/consumer primitive
//! used by sweep, patch, and mux to overlap reads with writes via a
//! bounded channel + dedicated consumer thread.
mod writeback;
mod writeback_file;
pub mod pipeline;
pub(crate) use writeback_file::WritebackFile;
// Re-exports for the 0.18 redesign. Sweep + patch are both wired up
// (disc/sweep.rs, disc/patch.rs); mux migrates separately in autorip.
// `WRITE_THROUGH_DEPTH` is patch-specific and has no other in-tree
// caller — the targeted `#[allow]` keeps the re-export visible without
// dragging the rest of the module under `dead_code`.
#[allow(unused_imports)]
pub use pipeline::WRITE_THROUGH_DEPTH;
pub use pipeline::{DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink};
+496
View File
@@ -0,0 +1,496 @@
//! Generic bounded producer/consumer pipeline.
//!
//! `Pipeline<I, R>` spawns a single consumer thread, hands it items
//! through a bounded `mpsc::sync_channel`, and joins it on `finish()`.
//! The consumer's behaviour is supplied by a [`Sink`] implementation:
//! `apply` is called once per item, `close` is called once at the end.
//!
//! Three call sites in libfreemkv want a producer/consumer split —
//! sweep (migrated to `disc/sweep.rs::SweepSink`), patch, and mux.
//! 0.18 collapses all three onto this primitive; sweep is in,
//! patch and mux migrate in later 0.18 slices.
//! See `(internal)/memory/0_18_redesign.md` for the full picture.
//!
//! ## Cancellation and error semantics
//!
//! - Producer dropping the channel (via `Pipeline::finish` dropping
//! `tx`) signals end-of-stream; consumer flushes via `close()` and
//! returns its `Output`.
//! - Consumer returning [`Flow::Stop`] also calls `close()` and
//! returns its `Output`. `send()` from the producer will then either
//! succeed (if the item already fit in the channel buffer) or fail
//! with `Err(item)` once the consumer has dropped its receiver.
//! - Consumer returning `Err` from `apply` skips `close()` entirely;
//! the consumer keeps draining the channel so the producer never
//! blocks on a dead receiver, and the first error is propagated as
//! the `JoinHandle` result.
//! - Consumer panic is converted into
//! `Error::IoError { source: io::Error::other(...) }`.
//!
//! ## Dead-code suppression
//!
//! Both `Disc::sweep` (`DEFAULT_PIPELINE_DEPTH`, `spawn_named`,
//! `try_send`) and `Disc::patch` (`WRITE_THROUGH_DEPTH`, `Flow::Stop`)
//! are now in-tree callers (0.18 round 2). The targeted `#[allow]`
//! markers below cover the corners of the API not exercised by either
//! call site (e.g. `Pipeline::spawn` without a name; `Flow::Stop`
//! return path from sweep's sink — sweep always consumes the producer's
//! full work-list). Drop the allows once mux migrates.
use std::io;
use std::sync::mpsc::{SyncSender, sync_channel};
use std::thread::{self, JoinHandle};
use crate::error::Error;
/// Default channel depth for callers without a specific reason to
/// pick another value.
///
/// Empirically tuned for sweep and mux — both want enough slack that
/// short consumer stalls don't immediately back up onto the producer,
/// but not so much that a producer outpacing the consumer accumulates
/// arbitrary buffered work. `4` matches the depth sweep has used
/// since 0.17.11 (originally in `disc/sweep_pipeline.rs`, now in
/// `disc/sweep.rs::SweepSink`). Patch should usually use
/// [`WRITE_THROUGH_DEPTH`] (`1`) instead — write-through gives clean
/// back-pressure between every read attempt and the matching write,
/// which matters when the consumer is updating the mapfile in lockstep.
pub const DEFAULT_PIPELINE_DEPTH: usize = 4;
/// Channel depth for write-through pipelines. Each `send` fully
/// drains before the next can enqueue. Use this when the producer
/// must observe consumer side-effects (e.g. mapfile state) before
/// emitting the next item.
#[allow(dead_code)]
pub const WRITE_THROUGH_DEPTH: usize = 1;
/// Outcome of [`Sink::apply`]: either keep feeding items
/// ([`Flow::Continue`]), or stop the pipeline early and run `close()`
/// ([`Flow::Stop`]).
///
/// `Stop` has no in-tree caller in this slice — sweep never returns
/// it (it always processes the producer's full work-list before the
/// channel is dropped). Patch and mux are the intended consumers and
/// migrate in later 0.18 slices. The variant ships now so the contract
/// is fixed; the targeted `#[allow]` is removed when patch lands.
pub enum Flow {
Continue,
#[allow(dead_code)]
Stop,
}
/// Consumer-side of a [`Pipeline`]. The pipeline owns one of these on
/// its consumer thread and calls `apply` once per received item, then
/// `close` once at end-of-stream.
pub trait Sink<I>: Send + 'static {
/// Type returned from `close()` and surfaced via
/// [`Pipeline::finish`].
type Output: Send + 'static;
/// Apply one item. Returning [`Flow::Continue`] keeps the
/// pipeline running; [`Flow::Stop`] ends it cleanly (still calls
/// `close()`). An error short-circuits: `close()` is *not* called
/// and the error is what `finish()` will return, but the consumer
/// keeps draining the channel so the producer never blocks on a
/// dead receiver.
fn apply(&mut self, item: I) -> Result<Flow, Error>;
/// Called once at end-of-stream — either because the producer
/// dropped `tx` or because `apply` returned [`Flow::Stop`]. Use
/// this to flush, fsync, finalise. Skipped if any prior `apply`
/// returned `Err`.
fn close(self) -> Result<Self::Output, Error>;
}
/// Bounded producer/consumer pipeline. Holds the producer-side
/// channel and the consumer thread's join handle.
pub struct Pipeline<I: Send + 'static, R: Send + 'static> {
tx: SyncSender<I>,
handle: JoinHandle<Result<R, Error>>,
}
impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Spawn the consumer thread with the given channel depth and
/// [`Sink`].
///
/// The thread is named `freemkv-pipeline-consumer` so it shows up
/// distinctly in stack traces and `top -H`. Callers that want a
/// more specific name (e.g. `freemkv-sweep-consumer`) should use
/// [`Pipeline::spawn_named`] instead. Returns an `Error::IoError`
/// if the OS refuses the thread spawn (resource exhaustion);
/// callers already operate in fallible context, so this is
/// propagated rather than panicked.
///
/// Sweep uses [`Pipeline::spawn_named`] directly so the consumer
/// thread shows up as `freemkv-sweep-consumer`; this function has
/// no in-tree caller yet. Patch and mux migrate in later 0.18
/// slices. The targeted `#[allow]` is removed when one of them
/// lands on the default name.
#[allow(dead_code)]
pub fn spawn<S: Sink<I, Output = R>>(depth: usize, sink: S) -> Result<Self, Error> {
Self::spawn_named("freemkv-pipeline-consumer", depth, sink)
}
/// Like [`Pipeline::spawn`] but lets the caller supply the
/// consumer thread's name. Useful when several pipelines run in
/// the same process and stack traces / `top -H` need to tell them
/// apart (e.g. `freemkv-sweep-consumer`, `freemkv-mux-consumer`).
pub fn spawn_named<S: Sink<I, Output = R>>(
name: &str,
depth: usize,
sink: S,
) -> Result<Self, Error> {
let (tx, rx) = sync_channel::<I>(depth);
let handle = thread::Builder::new()
.name(name.into())
.spawn(move || -> Result<R, Error> {
let mut sink = sink;
let mut first_err: Option<Error> = None;
let mut stopped = false;
while let Ok(item) = rx.recv() {
if first_err.is_some() || stopped {
// Drain remaining items so the producer never
// blocks on a dead receiver. `apply` is not
// called once we've decided to stop.
continue;
}
match sink.apply(item) {
Ok(Flow::Continue) => {}
Ok(Flow::Stop) => {
stopped = true;
}
Err(e) => {
first_err = Some(e);
}
}
}
match first_err {
Some(e) => Err(e),
None => sink.close(),
}
})
.map_err(|e| Error::IoError { source: e })?;
Ok(Pipeline { tx, handle })
}
/// Push one item. Blocks if the channel is full — that's the
/// back-pressure the whole primitive exists to provide. Returns
/// the item back if the consumer thread is gone (panicked or
/// already returned).
///
/// After the consumer returns [`Flow::Stop`], `send` will silently
/// buffer items into the channel until the channel fills, then
/// return `Err(item)` once the consumer has dropped its receiver.
/// Producers that need to stop pushing on `Stop` should track an
/// independent signal (e.g. `Halt`) — `send` alone is not the
/// notification edge.
pub fn send(&self, item: I) -> Result<(), I> {
self.tx.send(item).map_err(|e| e.0)
}
/// Non-blocking variant of [`Pipeline::send`]. If the channel is
/// full or the consumer has hung up, the item is returned in
/// `Err`. Useful for best-effort signalling (e.g. sweep's
/// throttled `StatsRequest`) where dropping the message is
/// preferable to blocking the producer.
pub fn try_send(&self, item: I) -> Result<(), std::sync::mpsc::TrySendError<I>> {
self.tx.try_send(item)
}
/// Drop the producer-side channel and wait for the consumer
/// thread to finish. Returns whatever the consumer's `close()`
/// produced, or the first `apply` error, or — on consumer panic —
/// an `Error::IoError` whose source is `io::Error::other(...)`
/// with a "pipeline consumer panicked: <payload>" message
/// (callers can match on the constant prefix).
pub fn finish(self) -> Result<R, Error> {
let Pipeline { tx, handle } = self;
// Explicit drop, although the destructure already drops `tx`
// at end-of-scope. Being explicit keeps the intent obvious.
drop(tx);
match handle.join() {
Ok(result) => result,
Err(payload) => {
// Preserve the original panic message when the
// consumer's panic payload was a `&str` or `String`
// (the two stdlib formats that `panic!` produces).
// Anything else falls back to "(no message)".
let msg = payload
.downcast_ref::<&'static str>()
.copied()
.or_else(|| payload.downcast_ref::<String>().map(|s| s.as_str()))
.unwrap_or("(no message)");
Err(Error::IoError {
source: io::Error::other(format!("pipeline consumer panicked: {msg}")),
})
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, Instant};
/// Sums u64s; returns the total from `close`.
struct SumSink {
total: u64,
}
impl Sink<u64> for SumSink {
type Output = u64;
fn apply(&mut self, item: u64) -> Result<Flow, Error> {
self.total += item;
Ok(Flow::Continue)
}
fn close(self) -> Result<u64, Error> {
Ok(self.total)
}
}
#[test]
fn happy_path_sums_items() {
let pipe = Pipeline::spawn(DEFAULT_PIPELINE_DEPTH, SumSink { total: 0 })
.expect("spawn should succeed");
let mut expected = 0u64;
for i in 0..100u64 {
expected += i;
pipe.send(i).expect("send should succeed");
}
let total = pipe.finish().expect("finish should succeed");
assert_eq!(total, expected);
assert_eq!(total, (0..100u64).sum::<u64>());
}
/// Sleeps `delay` per apply; counts how many it received.
struct SlowSink {
delay: Duration,
count: Arc<AtomicUsize>,
}
impl Sink<()> for SlowSink {
type Output = usize;
fn apply(&mut self, _item: ()) -> Result<Flow, Error> {
std::thread::sleep(self.delay);
self.count.fetch_add(1, Ordering::SeqCst);
Ok(Flow::Continue)
}
fn close(self) -> Result<usize, Error> {
Ok(self.count.load(Ordering::SeqCst))
}
}
#[test]
fn back_pressure_blocks_sender() {
// depth=2 + 5 sends + 50ms/apply: with the consumer pinned at
// 50 ms per item, the producer can buffer 2 (channel cap) +
// 1 (consumer in flight) = 3 items before sends 4 and 5 must
// block on consumer progress. Wall-clock floor across all 5
// sends is therefore ~2 * 50ms = 100ms (sends 4 and 5 each
// wait roughly one apply-cycle). Use 80 ms as the assertion
// floor to stay above the 50ms-per-item progress floor while
// tolerating CI jitter — it still proves blocking is real.
let count = Arc::new(AtomicUsize::new(0));
let sink = SlowSink {
delay: Duration::from_millis(50),
count: count.clone(),
};
let pipe = Pipeline::spawn(2, sink).expect("spawn should succeed");
let start = Instant::now();
for _ in 0..5 {
pipe.send(()).expect("send should succeed");
}
let elapsed_send = start.elapsed();
let total = pipe.finish().expect("finish should succeed");
assert_eq!(total, 5);
assert!(
elapsed_send >= Duration::from_millis(80),
"back-pressure not observed: 5 sends with depth=2 and 50ms/apply \
took {elapsed_send:?}, expected ≥ ~100ms (one or more sends \
should have blocked behind the consumer)"
);
}
/// Returns `Err` on the Nth apply (1-indexed). Tracks all calls.
struct FailOnNthSink {
n: usize,
seen: Arc<AtomicUsize>,
close_called: Arc<AtomicUsize>,
}
impl Sink<u64> for FailOnNthSink {
type Output = ();
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
let i = self.seen.fetch_add(1, Ordering::SeqCst) + 1;
if i == self.n {
Err(Error::DecryptFailed)
} else {
Ok(Flow::Continue)
}
}
fn close(self) -> Result<(), Error> {
self.close_called.fetch_add(1, Ordering::SeqCst);
Ok(())
}
}
#[test]
fn apply_error_drains_then_propagates() {
let seen = Arc::new(AtomicUsize::new(0));
let close_called = Arc::new(AtomicUsize::new(0));
let pipe = Pipeline::spawn(
DEFAULT_PIPELINE_DEPTH,
FailOnNthSink {
n: 3,
seen: seen.clone(),
close_called: close_called.clone(),
},
)
.expect("spawn should succeed");
// Send 10 items. Subsequent sends after the 3rd error must
// still succeed (the consumer is draining).
for i in 0..10u64 {
pipe.send(i).expect("send should succeed even after error");
}
let res = pipe.finish();
assert!(matches!(res, Err(Error::DecryptFailed)));
assert_eq!(
close_called.load(Ordering::SeqCst),
0,
"close() must not be called when apply returned Err"
);
// The consumer kept calling `recv` to drain after the error;
// it just stopped invoking `apply`. So `seen` is exactly 3
// (apply was called for items 1, 2, 3).
assert_eq!(seen.load(Ordering::SeqCst), 3);
}
/// Returns `Flow::Stop` on the Nth apply.
struct StopOnNthSink {
n: usize,
seen: Arc<AtomicUsize>,
close_called: Arc<AtomicUsize>,
}
impl Sink<u64> for StopOnNthSink {
type Output = usize;
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
let i = self.seen.fetch_add(1, Ordering::SeqCst) + 1;
if i >= self.n {
Ok(Flow::Stop)
} else {
Ok(Flow::Continue)
}
}
fn close(self) -> Result<usize, Error> {
self.close_called.fetch_add(1, Ordering::SeqCst);
Ok(self.seen.load(Ordering::SeqCst))
}
}
#[test]
fn apply_stop_calls_close_and_returns_output() {
let seen = Arc::new(AtomicUsize::new(0));
let close_called = Arc::new(AtomicUsize::new(0));
let pipe = Pipeline::spawn(
DEFAULT_PIPELINE_DEPTH,
StopOnNthSink {
n: 3,
seen: seen.clone(),
close_called: close_called.clone(),
},
)
.expect("spawn should succeed");
// Send 10 items. After Stop, subsequent sends may either
// succeed (already buffered) or fail with Err(I) (channel
// closed). Both are valid — we don't assert on the send
// results.
for i in 0..10u64 {
let _ = pipe.send(i);
}
let out = pipe.finish().expect("finish should succeed after Stop");
assert_eq!(close_called.load(Ordering::SeqCst), 1);
// At least 3 items processed (the one that returned Stop).
assert!(
out >= 3,
"expected ≥ 3 applies before Stop took effect, got {out}"
);
}
/// Panics on the first apply call.
struct PanickingSink;
impl Sink<u64> for PanickingSink {
type Output = ();
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
panic!("synthetic test panic");
}
fn close(self) -> Result<(), Error> {
Ok(())
}
}
#[test]
fn consumer_panic_becomes_io_error() {
// Silence the panic message that would otherwise pollute the
// test output — we expect this panic.
let prev = std::panic::take_hook();
std::panic::set_hook(Box::new(|_| {}));
let pipe =
Pipeline::spawn(DEFAULT_PIPELINE_DEPTH, PanickingSink).expect("spawn should succeed");
// First send may succeed (item buffered before panic) or fail
// (channel closed after panic) — either is fine.
let _ = pipe.send(1);
// Drain a few more sends; once the channel is closed they'll
// return Err(I), which we just discard.
for i in 0..5u64 {
let _ = pipe.send(i);
}
let res = pipe.finish();
std::panic::set_hook(prev);
match res {
Err(Error::IoError { source }) => {
let msg = source.to_string();
// Constant prefix lets callers match without parsing
// the variable payload tail.
assert!(
msg.contains("pipeline consumer panicked"),
"expected constant panic prefix, got: {msg}"
);
// The original `panic!` payload (a `&'static str`) must
// be preserved — without the downcast the message
// would just be the prefix.
assert!(
msg.contains("synthetic test panic"),
"expected original panic payload, got: {msg}"
);
}
other => panic!("expected Err(IoError), got {other:?}"),
}
}
}
+17
View File
@@ -0,0 +1,17 @@
//! Per-platform writeback pipeline. On Linux, drains dirty pages
//! continuously at chunk granularity to keep the kernel's writeback
//! queue bounded. On macOS and Windows, a no-op stub.
//!
//! The platform decision lives entirely in this file (the cfg-gated
//! `pub use` below). Callers — and `DiskWriter` itself — are
//! platform-independent.
#[cfg(target_os = "linux")]
mod linux;
#[cfg(not(target_os = "linux"))]
mod noop;
#[cfg(target_os = "linux")]
pub(super) use linux::WritebackPipeline;
#[cfg(not(target_os = "linux"))]
pub(super) use noop::WritebackPipeline;
+104
View File
@@ -0,0 +1,104 @@
//! Linux writeback pipeline using `sync_file_range` + `posix_fadvise`.
//!
//! Pathology this fixes: the kernel's default `vm.dirty_ratio` (~20 %
//! of RAM) lets dirty pages accumulate to hundreds of MB during a
//! big sequential write, then bursts a flush at 99 % disk utilisation.
//! While the burst runs, app writes block on the writeback queue —
//! observed empirically as instantaneous speed dropping from ~15 MB/s
//! to ~1 MB/s every ~30 s during a Pass 1 sweep.
//!
//! Strategy: every `chunk_bytes` of new sequential output, kick async
//! writeback (`SYNC_FILE_RANGE_WRITE`) on the just-completed chunk and
//! finalise the *previous* chunk via `WAIT_AFTER` + `posix_fadvise
//! (DONTNEED)`. By the time we finalise, that previous chunk has had
//! a full chunk's worth of work to flush — the wait is near-instant.
//! Dirty cache stays bounded at ~2 × `chunk_bytes` and writes drain
//! continuously instead of in bursts.
use std::fs::File;
use std::os::unix::io::{AsRawFd, RawFd};
pub(crate) struct WritebackPipeline {
/// Aliases the wrapping `WritebackFile::file`. Only valid for the
/// lifetime of that struct — moving the `File` independently
/// would silently UAF this fd. The pipeline is a private field of
/// `WritebackFile` and never exposed outside that wrapper, which
/// is what keeps the alias sound.
fd: RawFd,
chunk_bytes: u64,
last_flush_pos: u64,
pending: Option<(u64, u64)>,
}
impl WritebackPipeline {
/// Construct a pipeline aliasing `file`'s file descriptor. The
/// returned `WritebackPipeline` MUST be dropped before `file`
/// itself, or kept inside the same struct that owns `file` — the
/// alias is unchecked.
pub(crate) fn new(file: &File, start_pos: u64, chunk_bytes: u64) -> Self {
Self {
fd: file.as_raw_fd(),
chunk_bytes,
last_flush_pos: start_pos,
pending: None,
}
}
/// Caller advanced the file position to `pos`. If a chunk boundary
/// was crossed, kick async writeback for the just-completed chunk
/// and finalise the previous one.
pub(crate) fn note_progress(&mut self, pos: u64) {
if pos < self.last_flush_pos.saturating_add(self.chunk_bytes) {
return;
}
unsafe {
let chunk_off = self.last_flush_pos as i64;
let chunk_len = (pos - self.last_flush_pos) as i64;
libc::sync_file_range(self.fd, chunk_off, chunk_len, libc::SYNC_FILE_RANGE_WRITE);
if let Some((prev_off, prev_len)) = self.pending.take() {
libc::sync_file_range(
self.fd,
prev_off as i64,
prev_len as i64,
libc::SYNC_FILE_RANGE_WAIT_AFTER,
);
libc::posix_fadvise(
self.fd,
prev_off as i64,
prev_len as i64,
libc::POSIX_FADV_DONTNEED,
);
}
self.pending = Some((chunk_off as u64, chunk_len as u64));
}
self.last_flush_pos = pos;
}
/// Caller is about to seek away from the current write region.
/// Drain any in-flight chunk and reset tracking.
pub(crate) fn handle_seek(&mut self, new_pos: u64) {
self.finalize();
self.last_flush_pos = new_pos;
}
/// Drain any in-flight chunk. Idempotent. Call before `sync_all()`
/// or when discarding the pipeline.
pub(crate) fn finalize(&mut self) {
if let Some((prev_off, prev_len)) = self.pending.take() {
unsafe {
libc::sync_file_range(
self.fd,
prev_off as i64,
prev_len as i64,
libc::SYNC_FILE_RANGE_WAIT_AFTER,
);
libc::posix_fadvise(
self.fd,
prev_off as i64,
prev_len as i64,
libc::POSIX_FADV_DONTNEED,
);
}
}
}
}
+18
View File
@@ -0,0 +1,18 @@
//! No-op writeback pipeline for non-Linux targets. macOS and Windows
//! page cache policies have not been shown to exhibit the Linux
//! accumulate-then-burst flush pathology for our access pattern.
//! If that changes, replace this stub with a real implementation
//! (e.g. `F_NOCACHE` on macOS, `FILE_FLAG_WRITE_THROUGH` on Windows).
use std::fs::File;
pub(crate) struct WritebackPipeline;
impl WritebackPipeline {
pub(crate) fn new(_file: &File, _start_pos: u64, _chunk_bytes: u64) -> Self {
Self
}
pub(crate) fn note_progress(&mut self, _pos: u64) {}
pub(crate) fn handle_seek(&mut self, _new_pos: u64) {}
pub(crate) fn finalize(&mut self) {}
}
+129
View File
@@ -0,0 +1,129 @@
//! `WritebackFile` — a `File` wrapper whose reason for existing is the
//! bounded-cache writeback pipeline.
//!
//! Why: large sequential writes (sweep, patch, mux on UHD-scale output)
//! left to the kernel's default writeback policy accumulate hundreds of
//! megabytes of dirty pages and then burst-flush, stalling subsequent
//! writes for seconds at a time. `WritebackFile` drives a continuous
//! [`super::writeback::WritebackPipeline`] that on Linux issues
//! incremental `sync_file_range` + `posix_fadvise(DONTNEED)` calls at
//! 32 MB granularity so dirty pages drain at the same rate they're
//! produced. macOS and Windows fall through to a no-op pipeline — their
//! default cache policies have not been shown to exhibit the same
//! pathology for this access pattern.
//!
//! It implements `Write` and `Seek` so any call site that wrote to a
//! plain `File` through those traits (sweep, patch, mux) can swap in
//! `WritebackFile` without touching the body of the loop. The wrapper
//! also tracks the current file position to feed the pipeline with
//! progress + seek boundaries.
//!
//! See `super::writeback::linux` for the underlying pathology and the
//! strategy.
use std::fs::{File, OpenOptions};
use std::io::{self, Seek, SeekFrom, Write};
use std::path::Path;
use super::writeback::WritebackPipeline;
const CHUNK_BYTES: u64 = 32 * 1024 * 1024;
pub(crate) struct WritebackFile {
file: File,
pipeline: WritebackPipeline,
pos: u64,
}
impl WritebackFile {
/// Wrap an open `File`. The current OS file position is queried
/// once so the pipeline starts tracking from wherever the file
/// already is (typically 0 for fresh files; non-zero for resumed
/// or appended files).
pub(crate) fn new(mut file: File) -> io::Result<Self> {
let pos = file.stream_position()?;
let pipeline = WritebackPipeline::new(&file, pos, CHUNK_BYTES);
Ok(Self {
file,
pipeline,
pos,
})
}
/// Create a new file at `path` (truncating any existing contents)
/// and wrap it. Convenience for the common
/// `File::create(path)` + `WritebackFile::new(file)` pair so callers
/// don't have to assemble a `File` first.
pub(crate) fn create(path: &Path) -> io::Result<Self> {
let file = File::create(path)?;
Self::new(file)
}
/// Open an existing file at `path` for writing (no truncation) and
/// wrap it. Mirrors `File::open` semantics for the writable case
/// — used by patch / resume paths that mutate an existing ISO in
/// place.
pub(crate) fn open(path: &Path) -> io::Result<Self> {
let file = OpenOptions::new().write(true).open(path)?;
Self::new(file)
}
/// Drain in-flight writeback then issue a full fsync. Use this in
/// place of `File::sync_all`.
pub(crate) fn sync_all(&mut self) -> io::Result<()> {
self.pipeline.finalize();
self.file.sync_all()
}
}
impl Write for WritebackFile {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
let n = self.file.write(buf)?;
self.pos += n as u64;
self.pipeline.note_progress(self.pos);
Ok(n)
}
fn write_all(&mut self, buf: &[u8]) -> io::Result<()> {
self.file.write_all(buf)?;
self.pos += buf.len() as u64;
self.pipeline.note_progress(self.pos);
Ok(())
}
fn flush(&mut self) -> io::Result<()> {
self.file.flush()
}
}
impl Seek for WritebackFile {
fn seek(&mut self, from: SeekFrom) -> io::Result<u64> {
let p = self.file.seek(from)?;
// Only treat seeks that actually move the position as
// boundaries — sweep does a redundant `seek(Current(pos))`
// before every write, and we don't want that to drain the
// pipeline on every iteration.
if p != self.pos {
self.pipeline.handle_seek(p);
self.pos = p;
}
Ok(p)
}
}
impl Drop for WritebackFile {
fn drop(&mut self) {
// Run the pipeline's tail finalize so the last in-flight chunk
// gets its `WAIT_AFTER` + `posix_fadvise(DONTNEED)`. Without
// this, callers that drop a `WritebackFile` without calling
// `sync_all` (panic, early-return, idiomatic `let _ = w;`)
// leave the trailing chunk in cache; the kernel still flushes
// on close, but the bounded-cache invariant fails at the tail.
// We deliberately do *not* call `self.file.sync_all()` here —
// close already triggers a flush, and an `fsync` from `Drop`
// would silently swallow its `io::Error` anyway. `finalize` is
// idempotent so an explicit `sync_all` followed by drop is
// still safe.
self.pipeline.finalize();
}
}
+1 -1
View File
@@ -4,7 +4,7 @@
//! When both exist, language_streams.txt provides structured types while
//! menu_base.prop provides stream number → button name mapping.
use super::{vocab, LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType};
use super::{LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType, vocab};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
use std::collections::HashMap;
+140 -11
View File
@@ -17,6 +17,10 @@ use crate::disc::{DiscTitle, Stream};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
// Re-exported via crate::disc — the public API surfaces these next to
// AudioStream/SubtitleStream so callers can map purpose/qualifier to display
// text in their own locale.
/// A stream label extracted from disc config files.
#[derive(Debug, Clone)]
#[allow(dead_code)]
@@ -46,7 +50,6 @@ pub enum StreamLabelType {
}
#[derive(Debug, Clone, Copy, PartialEq)]
#[allow(dead_code)]
pub enum LabelPurpose {
Normal,
Commentary,
@@ -99,16 +102,12 @@ pub fn apply(reader: &mut dyn SectorReader, udf: &UdfFs, titles: &mut [DiscTitle
if let Some(label) = labels.iter().find(|l| {
l.stream_type == StreamLabelType::Audio && l.stream_number == audio_idx
}) {
// Structured fields — callers translate purpose to UI text.
a.purpose = label.purpose;
// a.label only carries codec/variant info. NEVER any
// English purpose text — the CLI handles that via i18n.
let mut parts = Vec::new();
match label.purpose {
LabelPurpose::Commentary => parts.push("Commentary".to_string()),
LabelPurpose::Descriptive => {
parts.push("Descriptive Audio".to_string())
}
LabelPurpose::Score => parts.push("Score".to_string()),
LabelPurpose::Ime => parts.push("IME".to_string()),
LabelPurpose::Normal => {}
}
if !label.variant.is_empty() {
parts.push(format!("({})", label.variant));
}
@@ -117,7 +116,10 @@ pub fn apply(reader: &mut dyn SectorReader, udf: &UdfFs, titles: &mut [DiscTitle
}
if !parts.is_empty() {
a.label = parts.join(" ");
} else if !label.name.is_empty() {
} else if !label.name.is_empty() && label.purpose == LabelPurpose::Normal {
// Only fall back to the parser-supplied display
// name when there's no purpose to flag — the CLI
// handles purpose rendering itself.
a.label = label.name.clone();
}
}
@@ -127,6 +129,7 @@ pub fn apply(reader: &mut dyn SectorReader, udf: &UdfFs, titles: &mut [DiscTitle
if let Some(label) = labels.iter().find(|l| {
l.stream_type == StreamLabelType::Subtitle && l.stream_number == sub_idx
}) {
s.qualifier = label.qualifier;
if label.qualifier == LabelQualifier::Forced {
s.forced = true;
}
@@ -138,6 +141,132 @@ pub fn apply(reader: &mut dyn SectorReader, udf: &UdfFs, titles: &mut [DiscTitle
}
}
/// Fill in default labels for any streams that don't have one.
/// Runs after BD-J label extraction — fills gaps with codec + channel descriptions.
/// This is the central place for all fallback label generation.
pub fn fill_defaults(titles: &mut [crate::disc::DiscTitle]) {
use crate::disc::Stream;
for title in titles.iter_mut() {
for stream in &mut title.streams {
match stream {
Stream::Audio(a) if a.label.is_empty() => {
a.label = generate_audio_label(&a.codec, &a.channels, a.secondary);
}
Stream::Video(v) if v.label.is_empty() => {
v.label =
generate_video_label(&v.codec, v.resolution.pixels(), &v.hdr, v.secondary);
}
Stream::Subtitle(s) if s.forced => {
// Ensure forced subs are labeled even if BD-J didn't set a name
// (subtitle labels are generally not set — this just marks forced)
}
_ => {}
}
}
}
}
fn generate_video_label(
codec: &crate::disc::Codec,
pixels: (u32, u32),
hdr: &crate::disc::HdrFormat,
secondary: bool,
) -> String {
use crate::disc::HdrFormat;
if secondary {
// "Dolby Vision EL" is a brand identifier, not English prose, so the
// library may emit it. Other "secondary video" wording is a CLI
// concern — the library just leaves the label empty.
return match hdr {
HdrFormat::DolbyVision => "Dolby Vision EL".to_string(),
_ => String::new(),
};
}
let mut parts = Vec::new();
// Codec
parts.push(codec.name().to_string());
// Resolution
let (w, h) = pixels;
let res = if w >= 7680 {
"8K"
} else if w >= 3840 {
"4K"
} else if w >= 1920 {
"1080p"
} else if w >= 1280 {
"720p"
} else if h >= 576 {
"576p"
} else if h >= 480 {
"480p"
} else {
""
};
if !res.is_empty() {
parts.push(res.into());
}
// HDR
match hdr {
HdrFormat::Sdr => {}
_ => parts.push(hdr.name().to_string()),
}
parts.join(" ")
}
fn generate_audio_label(
codec: &crate::disc::Codec,
channels: &crate::disc::AudioChannels,
_secondary: bool,
) -> String {
use crate::disc::{AudioChannels, Codec};
// Full marketing names for disc audio codecs.
// These are codec brand identifiers, not user-facing English prose.
let codec_name = match codec {
Codec::TrueHd => "Dolby TrueHD",
Codec::Ac3 => "Dolby Digital",
Codec::Ac3Plus => "Dolby Digital Plus",
Codec::DtsHdMa => "DTS-HD Master Audio",
Codec::DtsHdHr => "DTS-HD High Resolution",
Codec::Dts => "DTS",
Codec::Lpcm => "LPCM",
Codec::Aac => "AAC",
Codec::Mp2 => "MPEG Audio",
Codec::Mp3 => "MP3",
Codec::Flac => "FLAC",
Codec::Opus => "Opus",
_ => return String::new(),
};
// Channel layout
let channel_str = match channels {
AudioChannels::Mono => "1.0",
AudioChannels::Stereo => "2.0",
AudioChannels::Stereo21 => "2.1",
AudioChannels::Quad => "4.0",
AudioChannels::Surround50 => "5.0",
AudioChannels::Surround51 => "5.1",
AudioChannels::Surround61 => "6.1",
AudioChannels::Surround71 => "7.1",
AudioChannels::Unknown => "",
};
// The "(Secondary)" suffix is a CLI/UI concern — callers display it from
// the AudioStream::secondary bool, not the library.
if channel_str.is_empty() {
codec_name.to_string()
} else {
format!("{} {}", codec_name, channel_str)
}
}
fn extract(reader: &mut dyn SectorReader, udf: &UdfFs) -> Vec<StreamLabel> {
for (_name, detect, parse) in PARSERS {
if detect(udf) {
+1 -1
View File
@@ -5,7 +5,7 @@
//!
//! Token format: `{lang}_{codec?}_{purpose?}_{region?}_`
use super::{vocab, LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType};
use super::{LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType, vocab};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
+113 -11
View File
@@ -79,8 +79,10 @@ pub mod disc;
pub mod drive;
pub mod error;
pub mod event;
pub mod halt;
pub(crate) mod identity;
pub(crate) mod ifo;
pub(crate) mod io;
pub mod keydb;
pub(crate) mod labels;
pub(crate) mod mpls;
@@ -88,34 +90,134 @@ pub mod mux;
pub mod pes;
pub(crate) mod platform;
pub mod profile;
pub mod progress;
pub mod scsi;
pub mod sector;
pub(crate) mod speed;
pub(crate) mod udf;
pub mod verify;
// Re-export verify types at the crate root for ergonomic imports.
pub use verify::{SectorRange, SectorStatus, VerifyResult, verify_title};
// ─── Drive lifecycle ────────────────────────────────────────────────────────
//
// `Drive::open(path)` → `wait_ready()` → `init()` → `Disc::scan()`. `Drive`
// owns the SCSI session; `DriveCapture` etc. let advanced callers introspect
// drive identity / profile data for sharing.
pub use drive::capture::{
capture_drive_data, mask_bytes, mask_string, CapturedFeature, DriveCapture,
CapturedFeature, DriveCapture, capture_drive_data, mask_bytes, mask_string,
};
pub use drive::{find_drive, find_drives, Drive, DriveStatus};
pub use drive::{Drive, DriveStatus, find_drive};
// ─── Errors ─────────────────────────────────────────────────────────────────
//
// All fallible APIs return `Result<T, Error>`. `Error` is a typed enum with a
// numeric `code()`; **no English text in the library** — applications map
// codes to localized messages. See `error.rs` for the full taxonomy.
pub use error::{Error, Result};
// ─── 0.18 primitives ────────────────────────────────────────────────────────
//
// One-bit cooperative cancellation token, shared by every long-running loop
// in libfreemkv (sweep, patch, mux). Replaces the ad-hoc `Arc<AtomicBool>`
// flags scattered across 0.17 (`DiscStream::set_halt`, autorip's
// `HALT_FLAGS` registry). Clone it cheaply; pass it by value into each
// component; poll `is_cancelled()` inside the loop body.
pub use halt::Halt;
// Generic bounded producer/consumer primitive used by sweep, patch, and
// mux to overlap reads with writes via a dedicated consumer thread.
// `Pipeline::spawn(depth, sink)` spawns the consumer; `pipe.send(item)`
// pushes one item with back-pressure; `pipe.finish()` joins the
// consumer and surfaces its `close()` output. Callers implement `Sink`
// to define per-item behaviour and end-of-stream finalisation.
//
// `DEFAULT_PIPELINE_DEPTH` (=4) is the depth sweep + mux use; patch
// uses `WRITE_THROUGH_DEPTH` (=1) so each read fully drains before the
// next can enqueue. Returning `Flow::Stop` from `apply` ends the
// consumer cleanly (still calls `close()`).
pub use io::pipeline::{DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, Sink, WRITE_THROUGH_DEPTH};
// ─── Drive events (low-level callbacks) ─────────────────────────────────────
pub use event::{Event, EventKind};
pub use identity::DriveId;
pub use profile::DriveProfile;
// Platform trait is pub(crate) -- callers use Drive, not Platform directly
pub use decrypt::{decrypt_sectors, DecryptKeys};
// Platform trait is pub(crate) — callers use Drive, not Platform directly.
// ─── Decryption (AACS / CSS) ────────────────────────────────────────────────
//
// `Disc::scan()` resolves keys and stores them on `Disc`; in most flows you
// don't touch `DecryptKeys` directly — `DiscStream::new(reader, title, keys, …)`
// accepts whatever `Disc::decrypt_keys()` returned. `decrypt_sectors()` is
// for callers that operate on raw sector buffers (e.g. ISO patching).
pub use decrypt::{DecryptKeys, decrypt_sectors};
// ─── Disc structure ─────────────────────────────────────────────────────────
//
// `Disc::scan()` produces a fully-populated `Disc` (titles, streams, AACS
// state). `Disc::identify()` is the fast path — UDF only, no playlist parse,
// for displaying disc name + format quickly while a full scan runs in the
// background. The codec / channel / resolution enums are the canonical
// structured representation; never compare against display strings.
// Note: `disc::Stream` here is the codec enum (audio / video / sub kind)
// — not the `pes::Stream` trait re-exported below as `PesStream`. Two
// different concepts, the same short name; both stay because both are
// load-bearing in their respective domains.
pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
DiscFormat, DiscId, DiscTitle, Extent, FrameRate, HdrFormat, KeySource, Resolution, SampleRate,
ScanOptions, Stream, SubtitleStream, VideoStream,
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, DamageSeverity,
Disc, DiscFormat, DiscId, DiscTitle, Extent, FrameRate, HdrFormat, KeySource, LabelPurpose,
LabelQualifier, PatchOptions, PatchOutcome, Resolution, SampleRate, ScanOptions, Stream,
SubtitleStream, SweepOptions, VideoStream, classify_damage,
};
// ─── Streams ────────────────────────────────────────────────────────────────
//
// All stream types implement `pes::Stream` — read PES frames from a source,
// write PES frames to a sink. Pick the right type at construction:
//
// - `DiscStream` — physical drive or ISO (any `SectorReader`). Always read.
// - `MkvStream` — Matroska container. Read on `open()`, write on `create()`.
// - `M2tsStream` — Blu-ray Transport Stream. Read on `open()`, write on `create()`.
// - `NetworkStream` — TCP. Read on `listen()`, write on `connect()`.
// - `NullStream` — write-only black-hole sink. Useful for benchmarks.
// - `StdioStream` — pipe to/from stdin/stdout. Read or write.
//
// Most consumers use the URL resolvers (`input()` / `output()`) which pick
// the right type from a scheme:// URL. Direct construction is for callers
// that need to wire custom readers (e.g. autorip's drive-session reuse).
// 0.18 trait split: `FrameSource` (read-only) and `FrameSink` (write-only)
// supersede the unified `pes::Stream`. The old `Stream` re-export below
// stays available for the deprecation window — re-exported as
// `PesStream` to disambiguate from `disc::Stream` (the codec-kind enum
// re-exported above), which would otherwise collide at the crate root.
#[allow(deprecated)]
pub use pes::Stream as PesStream;
pub use pes::{FrameSink, FrameSource, PesFrame};
pub use mux::DiscStream;
pub use mux::M2tsStream;
pub use mux::MkvStream;
pub use mux::NetworkStream;
pub use mux::NullStream;
pub use mux::StdioStream;
pub use mux::{input, output, parse_url, InputOptions, StreamUrl};
pub use scsi::ScsiTransport;
pub use sector::SectorReader;
pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
// ─── Lower-level surfaces ───────────────────────────────────────────────────
//
// `ScsiTransport` is the platform-abstraction trait Drive uses; expose for
// out-of-tree platform backends. `SectorSource` / `SectorSink` are the 0.18
// direction-typed read/write traits; `FileSectorSource` and `FileSectorSink`
// are the ISO-on-disk implementations. [`DecryptingSectorSource`] is the
// single decrypt-on-read decorator (AACS / CSS / none) — wrap any
// `SectorSource` to get plaintext sectors out. The legacy `SectorReader` /
// `FileSectorReader` names stay re-exported through the 0.18 migration
// window so existing call sites compile unchanged; a blanket impl makes
// every `SectorReader` automatically usable as a `SectorSource`.
pub use scsi::{DriveInfo, ScsiSense, ScsiTransport, drive_has_disc, list_drives};
pub use sector::{
DecryptingSectorSource, FileSectorReader, FileSectorSink, FileSectorSource, SectorReader,
SectorSink, SectorSource,
};
pub use speed::DriveSpeed;
pub use udf::{read_filesystem, UdfFs};
pub use udf::{UdfFs, read_filesystem};
+1 -1
View File
@@ -256,7 +256,7 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
// Parse PlayListMark section
let mut marks = Vec::new();
if mark_start > 0 && mark_start + 4 < data.len() {
if mark_start > 0 && mark_start + 4 <= data.len() {
let ms = &data[mark_start..];
if ms.len() >= 6 {
let num_marks = u16::from_be_bytes([ms[4], ms[5]]) as usize;
+1 -1
View File
@@ -4,7 +4,7 @@
//! Buffers across PES boundaries so frames that span two PES packets
//! are emitted complete, not truncated.
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
pub struct Ac3Parser {
/// Leftover bytes from previous PES (incomplete frame at end).
+1 -1
View File
@@ -5,7 +5,7 @@
//! Buffers across PES boundaries so frames spanning two PES packets
//! are emitted complete.
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
const DTS_CORE_SYNC: [u8; 4] = [0x7F, 0xFE, 0x80, 0x01];
const DTS_HD_EXT_SYNC: [u8; 4] = [0x64, 0x58, 0x20, 0x25];
+8 -19
View File
@@ -7,26 +7,15 @@
//! For MKV: codec ID "S_VOBSUB".
//! All frames are keyframes (each is a complete bitmap).
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
pub struct DvdSubParser {
/// Pre-formatted VobSub .idx palette header for codec_private.
codec_data: Option<Vec<u8>>,
}
impl Default for DvdSubParser {
fn default() -> Self {
Self::new()
}
}
impl DvdSubParser {
pub fn new() -> Self {
Self { codec_data: None }
}
/// Create a parser with pre-formatted codec private data (palette header).
pub fn with_codec_data(codec_data: Option<Vec<u8>>) -> Self {
pub fn new(codec_data: Option<Vec<u8>>) -> Self {
Self { codec_data }
}
}
@@ -109,7 +98,7 @@ mod tests {
#[test]
fn passthrough_data() {
let mut parser = DvdSubParser::new();
let mut parser = DvdSubParser::new(None);
let sub_data = vec![0x00, 0x0A, 0x00, 0x08, 0x01, 0xFF, 0x02, 0x03, 0x04, 0x05];
let pes = make_pes(sub_data.clone(), Some(90000));
let frames = parser.parse(&pes);
@@ -124,7 +113,7 @@ mod tests {
#[test]
fn always_keyframe() {
let mut parser = DvdSubParser::new();
let mut parser = DvdSubParser::new(None);
for i in 0..3u8 {
let data = vec![0x00, i, 0x00, i + 1];
let pes = make_pes(data, Some(90000 * i as i64));
@@ -139,21 +128,21 @@ mod tests {
#[test]
fn empty_pes_returns_no_frames() {
let mut parser = DvdSubParser::new();
let mut parser = DvdSubParser::new(None);
let pes = make_pes(Vec::new(), Some(0));
assert!(parser.parse(&pes).is_empty());
}
#[test]
fn codec_private_none_by_default() {
let parser = DvdSubParser::new();
let parser = DvdSubParser::new(None);
assert!(parser.codec_private().is_none());
}
#[test]
fn codec_private_returns_palette_when_set() {
let palette_data = b"palette: 000000, ffffff\n".to_vec();
let parser = DvdSubParser::with_codec_data(Some(palette_data.clone()));
let parser = DvdSubParser::new(Some(palette_data.clone()));
let cp = parser.codec_private();
assert!(cp.is_some());
assert_eq!(cp.unwrap(), palette_data);
@@ -161,7 +150,7 @@ mod tests {
#[test]
fn no_pts_defaults_to_zero() {
let mut parser = DvdSubParser::new();
let mut parser = DvdSubParser::new(None);
let pes = make_pes(vec![0x01, 0x02], None);
let frames = parser.parse(&pes);
assert_eq!(frames.len(), 1);
+2 -2
View File
@@ -4,7 +4,7 @@
//! Detects keyframes (IDR slices).
//! Each PES packet = one access unit = one frame.
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// H.264 NAL unit types we care about.
const NAL_SLICE_IDR: u8 = 5;
@@ -263,7 +263,7 @@ mod tests {
data.extend_from_slice(&[0x00, 0x00, 0x01]);
data.push(0x67); // SPS
data.extend_from_slice(&[0x42, 0x00, 0x1E, 0xAB, 0xCD]); // profile=0x42, compat=0x00, level=0x1E
// PPS: 00 00 01 [68 <payload>]
// PPS: 00 00 01 [68 <payload>]
data.extend_from_slice(&[0x00, 0x00, 0x01]);
data.push(0x68); // PPS
data.extend_from_slice(&[0xCE, 0x01]);
+4 -4
View File
@@ -5,7 +5,7 @@
//! Each PES packet = one access unit = one frame.
use super::h264::{find_start_code, skip_start_code};
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
// HEVC NAL unit types
const NAL_VPS: u8 = 32;
@@ -122,7 +122,7 @@ impl CodecParser for HevcParser {
// Minimal HEVCDecoderConfigurationRecord header
record.push(1); // configurationVersion
// General profile space, tier flag, profile IDC from SPS
// General profile space, tier flag, profile IDC from SPS
if sps.len() > 3 {
record.push(sps[1]); // general_profile_space + general_tier_flag + general_profile_idc
} else {
@@ -154,7 +154,7 @@ impl CodecParser for HevcParser {
record.push(0xFC);
// chromaFormat (6 + 2 bits)
record.push(0xFC | 1); // 4:2:0
// bitDepthLumaMinus8 (5 + 3 bits)
// bitDepthLumaMinus8 (5 + 3 bits)
record.push(0xF8);
// bitDepthChromaMinus8 (5 + 3 bits)
record.push(0xF8);
@@ -162,7 +162,7 @@ impl CodecParser for HevcParser {
record.extend_from_slice(&[0, 0]);
// constantFrameRate + numTemporalLayers + temporalIdNested + lengthSizeMinusOne
record.push(0x03); // lengthSizeMinusOne = 3 (4 bytes)
// numOfArrays
// numOfArrays
record.push(3); // VPS, SPS, PPS
// VPS array
+1 -1
View File
@@ -13,7 +13,7 @@
//! For MKV: codec ID "A_PCM/INT/BIG" (BD) or "A_PCM/INT/LIT" (DVD).
//! All frames are keyframes (uncompressed audio).
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// BD LPCM header size in bytes.
const BD_LPCM_HEADER_SIZE: usize = 4;
+2 -10
View File
@@ -79,18 +79,10 @@ impl CodecParser for PassthroughParser {
}
}
/// Create the appropriate parser for a codec.
pub fn parser_for_codec(codec: Codec) -> Box<dyn CodecParser> {
parser_for_codec_with_data(codec, None)
}
/// Create the appropriate parser for a codec, with optional codec private data.
///
/// For DvdSub, `codec_data` should be the pre-formatted VobSub .idx palette header.
pub fn parser_for_codec_with_data(
codec: Codec,
codec_data: Option<Vec<u8>>,
) -> Box<dyn CodecParser> {
pub fn parser_for_codec(codec: Codec, codec_data: Option<Vec<u8>>) -> Box<dyn CodecParser> {
match codec {
Codec::H264 => Box::new(h264::H264Parser::new()),
Codec::Hevc => Box::new(hevc::HevcParser::new()),
@@ -101,7 +93,7 @@ pub fn parser_for_codec_with_data(
Codec::TrueHd => Box::new(truehd::TrueHdParser::new()),
Codec::Pgs => Box::new(pgs::PgsParser::new()),
Codec::Lpcm => Box::new(lpcm::LpcmParser::new()),
Codec::DvdSub => Box::new(dvdsub::DvdSubParser::with_codec_data(codec_data)),
Codec::DvdSub => Box::new(dvdsub::DvdSubParser::new(codec_data)),
_ => Box::new(PassthroughParser::new(true)),
}
}
+3 -3
View File
@@ -9,7 +9,7 @@
//! - Sequence extension: 00 00 01 B5
//! - Picture header: 00 00 01 00
use super::{pts_to_ns, CodecParser, Frame};
use super::{CodecParser, Frame, pts_to_ns};
use crate::mux::ts::PesPacket;
/// Sequence header start code suffix.
@@ -153,7 +153,7 @@ impl CodecParser for Mpeg2Parser {
bits += 64 * 8;
}
}
let total_bytes = 4 + ((bits + 7) / 8) as usize;
let total_bytes = 4 + bits.div_ceil(8) as usize;
(sc + total_bytes).min(data.len())
}
}
@@ -415,7 +415,7 @@ mod tests {
// Sequence extension: 00 00 01 B5 [ext data]
data.extend_from_slice(&[0x00, 0x00, 0x01, SEQ_EXT_CODE]);
data.extend_from_slice(&[0x14, 0x8A, 0x00, 0x01, 0x00, 0x00]); // ext payload
// Picture header follows.
// Picture header follows.
data.extend_from_slice(&make_picture_header(PICTURE_TYPE_I));
data.extend_from_slice(&[0xFF; 4]);
+1 -1
View File
@@ -4,7 +4,7 @@
//! Each PES packet contains one or more segments.
//! All segments are keyframes (no inter-segment dependencies).
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
pub struct PgsParser;
+122 -23
View File
@@ -1,17 +1,24 @@
//! Dolby TrueHD / Atmos elementary stream parser.
//!
//! TrueHD access units are 2560-byte fixed-size units (40 per major sync).
//! Each unit starts with a 4-byte header: [length_hi, length_lo, timestamp_hi, timestamp_lo].
//! Major sync: 0xF8726FBA appears within a unit.
//! Buffers across PES boundaries for complete unit delivery.
//! BD-TS TrueHD PES packets contain interleaved AC-3 + TrueHD access units.
//! Access units span PES boundaries — must buffer and reassemble.
//!
//! TrueHD access unit header (4 bytes):
//! [0..1] upper 4 bits = parity, lower 12 bits = length in 2-byte words
//! [2..3] timing value
//! [4..] substream data (major sync 0xF8726FBA may appear at offset 4)
//!
//! AC-3 frames (interleaved, same PID): start with sync word 0x0B77.
//! We skip AC-3 frames and only emit TrueHD access units.
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
/// TrueHD access unit size (fixed).
const TRUEHD_UNIT_SIZE: usize = 2560;
/// Duration of one TrueHD access unit in nanoseconds (1/1200 second).
const AU_DURATION_NS: i64 = 833_333;
pub struct TrueHdParser {
buf: Vec<u8>,
next_pts_ns: i64,
}
impl Default for TrueHdParser {
@@ -23,9 +30,58 @@ impl Default for TrueHdParser {
impl TrueHdParser {
pub fn new() -> Self {
Self {
buf: Vec::with_capacity(TRUEHD_UNIT_SIZE * 4),
buf: Vec::with_capacity(32768),
next_pts_ns: 0,
}
}
/// Skip an AC-3 frame starting at the current buffer position.
/// Returns number of bytes consumed, or 0 if not enough data.
fn skip_ac3_frame(&self) -> usize {
if self.buf.len() < 6 {
return 0;
}
// AC-3 frame size from frmsizcod + fscod
// Byte 4: [fscod:2][frmsizecod:6]
let fscod = (self.buf[4] >> 6) & 0x03;
let frmsizecod = (self.buf[4] & 0x3F) as usize;
// Frame size in 16-bit words per fscod (simplified table for common rates)
let frame_words = match fscod {
0 => {
// 48 kHz
static SIZES: [usize; 38] = [
64, 64, 80, 80, 96, 96, 112, 112, 128, 128, 160, 160, 192, 192, 224, 224, 256,
256, 320, 320, 384, 384, 448, 448, 512, 512, 640, 640, 768, 768, 896, 896,
1024, 1024, 1152, 1152, 1280, 1280,
];
SIZES.get(frmsizecod).copied().unwrap_or(0)
}
1 => {
// 44.1 kHz
static SIZES: [usize; 38] = [
69, 70, 87, 88, 104, 105, 121, 122, 139, 140, 174, 175, 208, 209, 243, 244,
278, 279, 348, 349, 417, 418, 487, 488, 557, 558, 696, 697, 835, 836, 975, 976,
1114, 1115, 1253, 1254, 1393, 1394,
];
SIZES.get(frmsizecod).copied().unwrap_or(0)
}
2 => {
// 32 kHz
static SIZES: [usize; 38] = [
96, 96, 120, 120, 144, 144, 168, 168, 192, 192, 240, 240, 288, 288, 336, 336,
384, 384, 480, 480, 576, 576, 672, 672, 768, 768, 960, 960, 1152, 1152, 1344,
1344, 1536, 1536, 1728, 1728, 1920, 1920,
];
SIZES.get(frmsizecod).copied().unwrap_or(0)
}
_ => 0,
};
let frame_bytes = frame_words * 2;
if frame_bytes == 0 || self.buf.len() < frame_bytes {
return 0;
}
frame_bytes
}
}
impl CodecParser for TrueHdParser {
@@ -33,39 +89,59 @@ impl CodecParser for TrueHdParser {
if pes.data.is_empty() {
return Vec::new();
}
let pts_ns = pes.pts.map(pts_to_ns).unwrap_or(0);
if let Some(pts) = pes.pts {
self.next_pts_ns = pts_to_ns(pts);
}
self.buf.extend_from_slice(&pes.data);
let mut frames = Vec::new();
// TrueHD units are variable-length but each starts with a 2-byte
// big-endian length field (in 16-bit words, includes the 4-byte header).
// Extract complete units from the buffer.
while self.buf.len() >= 4 {
let unit_words = ((self.buf[0] as usize) << 8) | self.buf[1] as usize;
loop {
if self.buf.len() < 4 {
break;
}
// AC-3 frame (interleaved): starts with sync word 0x0B77
if self.buf[0] == 0x0B && self.buf[1] == 0x77 {
let skip = self.skip_ac3_frame();
if skip == 0 {
break; // incomplete AC-3 frame, wait for more data
}
self.buf.drain(..skip);
continue;
}
// TrueHD access unit: lower 12 bits of first 2 bytes = length in words
let unit_words = (((self.buf[0] as usize) << 8) | self.buf[1] as usize) & 0xFFF;
if unit_words == 0 {
// Padding — skip 2 bytes
self.buf.drain(..2);
continue;
}
let unit_bytes = unit_words * 2;
if unit_bytes > 65536 {
// Invalid — skip 2 bytes to resync
if unit_bytes > 32768 {
// Likely misaligned — try to resync by scanning for AC-3 sync or
// a valid TrueHD length
self.buf.drain(..2);
continue;
}
if self.buf.len() < unit_bytes {
// Incomplete unit — wait for more data
break;
break; // incomplete access unit, wait for more data
}
let is_major_sync = unit_bytes >= 8
&& (u32::from_be_bytes([self.buf[4], self.buf[5], self.buf[6], self.buf[7]])
& 0xFFFF_FFFE)
== 0xF872_6FBA;
frames.push(Frame {
pts_ns,
keyframe: true,
pts_ns: self.next_pts_ns,
keyframe: is_major_sync,
data: self.buf[..unit_bytes].to_vec(),
});
self.buf.drain(..unit_bytes);
self.next_pts_ns += AU_DURATION_NS;
}
frames
@@ -93,11 +169,20 @@ mod tests {
fn make_truehd_unit(size_bytes: usize) -> Vec<u8> {
let words = size_bytes / 2;
let mut data = vec![0u8; size_bytes];
data[0] = (words >> 8) as u8;
data[0] = ((words >> 8) & 0x0F) as u8;
data[1] = (words & 0xFF) as u8;
data
}
fn make_ac3_frame() -> Vec<u8> {
// Minimal AC-3 frame: sync 0x0B77, fscod=0 (48kHz), frmsizecod=0 (64 words = 128 bytes)
let mut data = vec![0u8; 128];
data[0] = 0x0B;
data[1] = 0x77;
data[4] = 0x00; // fscod=0, frmsizecod=0
data
}
#[test]
fn parse_empty_pes() {
let mut parser = TrueHdParser::new();
@@ -131,7 +216,7 @@ mod tests {
}
#[test]
fn parse_multiple_units_in_one_pes() {
fn parse_multiple_units_incrementing_pts() {
let mut parser = TrueHdParser::new();
let mut data = make_truehd_unit(100);
data.extend_from_slice(&make_truehd_unit(120));
@@ -140,6 +225,20 @@ mod tests {
assert_eq!(frames.len(), 2);
assert_eq!(frames[0].data.len(), 100);
assert_eq!(frames[1].data.len(), 120);
assert_eq!(frames[1].pts_ns - frames[0].pts_ns, AU_DURATION_NS);
}
#[test]
fn skip_interleaved_ac3() {
let mut parser = TrueHdParser::new();
let ac3 = make_ac3_frame();
let truehd = make_truehd_unit(200);
let mut data = ac3;
data.extend_from_slice(&truehd);
let pes = make_pes(data, Some(90000));
let frames = parser.parse(&pes);
assert_eq!(frames.len(), 1);
assert_eq!(frames[0].data.len(), 200);
}
#[test]
+1 -1
View File
@@ -5,7 +5,7 @@
//! Frame start = Frame header start code (0x0D).
//! I-frames (keyframes) are identified from the frame header.
use super::{pts_to_ns, CodecParser, Frame, PesPacket};
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
const SC_SEQUENCE_HEADER: u8 = 0x0F;
const SC_ENTRY_POINT: u8 = 0x0E;
+408 -23
View File
@@ -5,18 +5,114 @@
//!
//! Read-only. For disc→ISO (raw sector copy), use `Disc::copy()`.
use crate::disc::{detect_max_batch_sectors, Disc, DiscTitle, Extent, ScanOptions};
use crate::sector::SectorReader;
use crate::disc::{Disc, DiscTitle, Extent};
use crate::drive::extract_scsi_context;
use crate::event::{BatchSizeReason, Event, EventKind};
use crate::halt::Halt;
use crate::sector::{DecryptingSectorSource, SectorReader, SectorSource};
use std::io;
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
/// Ramp back up to the preferred batch size after this many sectors
/// of clean reading at the current (reduced) size. 100 MiB = 51,200 sectors.
///
/// Chosen so that an isolated transient failure doesn't lock the rip at
/// size 1: once past the bad zone, we probe up after ~100 ms of good reads.
/// And so that noisy zones with occasional successes can't trigger a
/// premature probe — we need a sustained clean run.
const PROBE_THRESHOLD_SECTORS: u32 = 100 * 1024 * 1024 / 2048;
/// Halve a batch size, keeping 3-sector alignment when >= 6
/// (3-sector alignment = one AACS unit). At sizes < 6 we descend
/// through 3 → 1 without intermediate unaligned sizes.
fn halve_batch_size(size: u16) -> u16 {
let h = (size / 2).max(1);
if h >= 6 { h - (h % 3) } else { h }
}
/// Double a batch size toward a preferred max, keeping 3-sector alignment
/// when the result is >= 6.
fn double_batch_size(size: u16, preferred: u16) -> u16 {
let d = size.saturating_mul(2).min(preferred);
if d >= 6 { d - (d % 3) } else { d }
}
/// Adaptive batch sizer. Shrinks on read failure, grows after a sustained
/// clean streak. Amortizes the cost of entering a bad zone — descent happens
/// once, not once per bad sector.
#[derive(Debug)]
struct AdaptiveBatch {
preferred: u16,
current: u16,
streak_sectors: u32,
}
impl AdaptiveBatch {
fn new(preferred: u16) -> Self {
Self {
preferred,
current: preferred,
streak_sectors: 0,
}
}
fn current(&self) -> u16 {
self.current
}
/// Record a successful read of `sectors`. Returns an event if the
/// sizer probed up to a larger batch size.
fn on_success(&mut self, sectors: u16) -> Option<EventKind> {
self.streak_sectors = self.streak_sectors.saturating_add(sectors as u32);
if self.current < self.preferred && self.streak_sectors >= PROBE_THRESHOLD_SECTORS {
let new_size = double_batch_size(self.current, self.preferred);
if new_size != self.current {
self.current = new_size;
self.streak_sectors = 0;
return Some(EventKind::BatchSizeChanged {
new_size,
reason: BatchSizeReason::Probed,
});
}
}
None
}
/// Record a read failure. Returns an event if the sizer shrank.
/// Does nothing at size 1 (caller handles skip/error).
fn on_failure(&mut self) -> Option<EventKind> {
self.streak_sectors = 0;
if self.current <= 1 {
return None;
}
let new_size = halve_batch_size(self.current);
self.current = new_size;
Some(EventKind::BatchSizeChanged {
new_size,
reason: BatchSizeReason::Shrunk,
})
}
}
/// Disc stream. Reads sectors from any source → PES frames.
///
/// Sources: physical drive, ISO file, or any SectorReader.
/// Decrypt, demux, and codec parsing happen internally.
pub struct DiscStream {
reader: Box<dyn SectorReader>,
/// Underlying sector source wrapped in the 0.18
/// [`DecryptingSectorSource`] decorator. Every `read_sectors`
/// call yields plaintext, so `fill_extents` no longer needs an
/// inline `decrypt::decrypt_sectors` step. `DecryptKeys::None`
/// (raw / unencrypted disc) makes the decorator a pass-through.
reader: DecryptingSectorSource<Box<dyn SectorReader>>,
title: DiscTitle,
disc: Option<Disc>,
/// Mirror of the keys handed in at construction. The decorator
/// owns the cryptographic state; this field is kept for
/// metadata-side callers (`info()` and friends) that want to
/// know whether the disc was encrypted, without reaching through
/// the wrapper.
decrypt_keys: crate::decrypt::DecryptKeys,
// Extents to read
@@ -30,11 +126,29 @@ pub struct DiscStream {
read_buf: Vec<u8>,
buf_valid: usize,
// Batch size for reads
batch_sectors: u16,
// Adaptive batch sizer — preferred comes from the caller
// (detect_max_batch_sectors), shrinks/grows based on read outcomes.
adaptive: AdaptiveBatch,
pub errors: u64,
pub skip_errors: bool,
/// When set and the token is cancelled, fill_extents returns Err(Halted)
/// at the next retry boundary. Unlike skip_errors, this propagates the
/// error up so the rip terminates cleanly. Construct with
/// [`DiscStream::with_halt`] (preferred) or set post-hoc via the
/// deprecated [`DiscStream::set_halt`] bridge — both populate this same
/// field and either entry point yields one source of truth.
halt: Option<Halt>,
event_fn: Option<Box<dyn Fn(Event) + Send>>,
eof: bool,
// Cumulative bytes successfully read from the source. Drives
// EventKind::BytesRead emission and autorip's per-device progress.
bytes_read_total: u64,
// Pre-computed total of all extents in bytes (or 0 if extents are
// empty). Carried in EventKind::BytesRead.total so consumers can show
// a percent without a separate API call.
bytes_total_extents: u64,
// PES output
ts_demuxer: Option<super::ts::TsDemuxer>,
ps_demuxer: Option<super::ps::PsDemuxer>,
@@ -57,6 +171,7 @@ impl DiscStream {
content_format: crate::disc::ContentFormat,
) -> Self {
let extents = title.extents.clone();
let bytes_total_extents: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum();
let mut pids = Vec::new();
let mut parsers = Vec::new();
@@ -69,7 +184,7 @@ impl DiscStream {
};
pids.push(pid);
pid_to_track.push((pid, idx));
parsers.push((pid, super::codec::parser_for_codec(codec)));
parsers.push((pid, super::codec::parser_for_codec(codec, None)));
}
let mut ts_demuxer = None;
@@ -87,7 +202,11 @@ impl DiscStream {
}
Self {
reader,
// Wrap the input reader in DecryptingSectorSource so the
// internal fill_extents path sees plaintext bytes. For
// DecryptKeys::None (unencrypted / raw / test fixtures)
// the decorator is a pass-through.
reader: DecryptingSectorSource::new(reader, decrypt_keys.clone()),
title,
disc: None,
decrypt_keys,
@@ -96,9 +215,14 @@ impl DiscStream {
current_offset: 0,
read_buf: Vec::with_capacity(batch_sectors as usize * 2048),
buf_valid: 0,
batch_sectors,
adaptive: AdaptiveBatch::new(batch_sectors),
errors: 0,
skip_errors: false,
halt: None,
event_fn: None,
eof: false,
bytes_read_total: 0,
bytes_total_extents,
ts_demuxer,
ps_demuxer,
parsers,
@@ -107,9 +231,59 @@ impl DiscStream {
}
}
/// Skip decryption — return raw encrypted bytes.
/// Set event handler for sector-level events (binary search, skip, recover).
pub fn on_event(&mut self, f: impl Fn(Event) + Send + 'static) {
self.event_fn = Some(Box::new(f));
}
/// Constructor-time builder: attach a [`Halt`] token so that when
/// any clone is cancelled, the next read-retry boundary inside
/// `fill_extents` returns `Err(Halted)`. Required for Stop to work
/// during dense bad-sector regions (where the outer PES read() loop
/// can spend minutes inside fill_extents before emitting a frame).
///
/// Preferred over the post-hoc [`DiscStream::set_halt`] bridge —
/// pass the same `Halt` clone you hand to sweep / patch / mux so
/// every phase observes a single Stop signal.
pub fn with_halt(mut self, halt: Halt) -> Self {
self.halt = Some(halt);
self
}
/// Bridge for callers that haven't migrated to the
/// [`DiscStream::with_halt`] constructor-time path yet. Wraps the
/// supplied `Arc<AtomicBool>` as a [`Halt`] (`Halt::from_arc`) and
/// stores it in the same internal slot, so a halt installed via
/// either entry point goes through one halt-check inside
/// `fill_extents`. Calling `set_halt` after `with_halt` (or vice
/// versa) replaces the previous token with the new one.
#[deprecated(
since = "0.18.0",
note = "use `DiscStream::with_halt(Halt)` at construction instead"
)]
pub fn set_halt(&mut self, flag: Arc<AtomicBool>) {
self.halt = Some(Halt::from_arc(flag));
}
fn is_halted(&self) -> bool {
self.halt
.as_ref()
.map(|h| h.is_cancelled())
.unwrap_or(false)
}
fn emit(&self, kind: EventKind) {
if let Some(ref f) = self.event_fn {
f(Event { kind });
}
}
/// Skip decryption — return raw encrypted bytes. Updates both
/// the metadata-side key field and the wrapped reader's keys so
/// subsequent `read_sectors` calls become a pass-through.
pub fn set_raw(&mut self) {
self.decrypt_keys = crate::decrypt::DecryptKeys::None;
self.reader.set_keys(crate::decrypt::DecryptKeys::None);
}
/// Get the scanned Disc (for listing all titles).
@@ -125,22 +299,86 @@ impl DiscStream {
let ext_sectors = self.extents[self.current_extent].sector_count;
let remaining = ext_sectors.saturating_sub(self.current_offset);
let sectors = remaining.min(self.batch_sectors as u32) as u16;
let sectors = sectors - (sectors % 3);
if sectors == 0 {
if remaining == 0 {
self.current_extent += 1;
self.current_offset = 0;
return self.fill_extents();
}
let lba = ext_start + self.current_offset;
let bytes = sectors as usize * 2048;
self.read_buf.resize(bytes, 0);
self.reader
.read_sectors(lba, sectors, &mut self.read_buf[..bytes])?;
self.buf_valid = bytes;
self.current_offset += sectors as u32;
// Adaptive sizer: start at current (preferred until a failure), shrink
// on failure, advance on success. One 5s read attempt per try — no
// retry loops, no sleeps. On size-1 failure, skip or error.
//
// Halt is checked at the top of every iteration — in a dense bad zone
// this loop can spend minutes shrinking and skipping sectors; without
// the check, Stop wouldn't take effect until the outer PES read() loop
// finally emits a frame, which may never happen.
loop {
if self.is_halted() {
return Err(crate::error::Error::Halted.into());
}
let mut sectors = remaining.min(self.adaptive.current() as u32) as u16;
// Align to 3-sector AACS units when possible. Partial units at
// extent boundaries are safely handled by decrypt_sectors().
if sectors >= 3 {
sectors -= sectors % 3;
}
let bytes = sectors as usize * 2048;
self.read_buf.resize(bytes, 0);
let ok = self
.reader
.read_sectors(lba, sectors, &mut self.read_buf[..bytes], false)
.is_ok();
if ok {
if let Some(ev) = self.adaptive.on_success(sectors) {
self.emit(ev);
}
self.buf_valid = bytes;
self.current_offset += sectors as u32;
self.bytes_read_total = self.bytes_read_total.saturating_add(bytes as u64);
self.emit(EventKind::BytesRead {
bytes: self.bytes_read_total,
total: self.bytes_total_extents,
});
break;
}
if sectors == 1 {
// Bottomed out. Skip this sector or bail.
if self.skip_errors {
self.read_buf.resize(2048, 0);
self.read_buf[..2048].fill(0);
self.buf_valid = 2048;
self.errors += 1;
self.emit(EventKind::SectorSkipped { sector: lba as u64 });
self.current_offset += 1;
break;
} else {
let err = self
.reader
.read_sectors(lba, sectors, &mut self.read_buf[..2048], false)
.err();
let (status, sense) =
err.as_ref().map(extract_scsi_context).unwrap_or((0, None));
return Err(crate::error::Error::DiscRead {
sector: lba as u64,
status: Some(status),
sense,
}
.into());
}
}
// Shrink and retry at the same LBA with a smaller batch.
if let Some(ev) = self.adaptive.on_failure() {
self.emit(ev);
}
}
if self.current_offset >= ext_sectors {
self.current_extent += 1;
self.current_offset = 0;
@@ -149,6 +387,7 @@ impl DiscStream {
}
}
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSource/FrameSink in follow-up commit.
impl crate::pes::Stream for DiscStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
if let Some(frame) = self.pending_frames.pop_front() {
@@ -221,11 +460,10 @@ impl crate::pes::Stream for DiscStream {
}
let bytes = self.buf_valid;
if let Err(e) =
crate::decrypt::decrypt_sectors(&mut self.read_buf[..bytes], &self.decrypt_keys, 0)
{
return Err(e.into());
}
// Plaintext: the wrapped reader (DecryptingSectorSource)
// applied AACS / CSS in-place during fill_extents'
// read_sectors call. The pre-0.18 inline decrypt step
// lived here.
if let Some(ref mut demuxer) = self.ts_demuxer {
let packets = demuxer.feed(&self.read_buf[..bytes]);
@@ -327,3 +565,150 @@ impl crate::pes::Stream for DiscStream {
true
}
}
#[cfg(test)]
mod tests {
//! `DiscStream` is the only meaningful `FrameSource` impl in tree (every
//! other concrete `pes::Stream` impl in `mux/*` is a sink). The 0.18
//! round-1 blanket `impl<T: pes::Stream + Send> pes::FrameSource for T`
//! covers `DiscStream` for free as long as it is `Send`. These tests
//! lock that down: a static `Send` assertion plus a `Box<dyn FrameSource>`
//! round trip exercising every `FrameSource` method through the trait
//! object, so future Send-breaking edits to `DiscStream`'s interior
//! types fail at compile time and the trait-bridge dispatch is verified
//! at runtime.
#![allow(deprecated)] // exercising the 0.18 deprecation-window blanket bridge.
use super::*;
use crate::disc::{ContentFormat, DiscTitle};
use crate::pes::FrameSource;
/// Static-assert `DiscStream: Send`. The blanket
/// `impl<T: pes::Stream + Send> pes::FrameSource for T` only fires for
/// `Send` types — if a future field on `DiscStream` is non-`Send` (e.g.
/// a `Box<dyn Read>` instead of `Box<dyn SectorReader>`), this fails
/// at compile time, before the runtime trait-object test below.
fn _assert_disc_stream_is_send() {
fn requires_send<T: Send>() {}
requires_send::<DiscStream>();
}
/// Trivial `SectorReader` that yields zeroed sectors. Empty title means
/// the demuxer produces no PES frames, so `read()` walks the extents to
/// EOF and returns `Ok(None)`. That's enough to exercise the trait-object
/// dispatch — the goal here is the bridge, not the demuxer.
struct ZeroReader {
capacity: u32,
}
impl crate::sector::SectorReader for ZeroReader {
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> crate::error::Result<usize> {
let bytes = count as usize * 2048;
buf[..bytes].fill(0);
Ok(bytes)
}
fn capacity(&self) -> u32 {
self.capacity
}
}
fn synthetic_title(sector_count: u32) -> DiscTitle {
DiscTitle {
extents: vec![crate::disc::Extent {
start_lba: 0,
sector_count,
}],
..DiscTitle::empty()
}
}
/// Smallest credible witness that `DiscStream` flows through the
/// `FrameSource` blanket impl: build a `Box<dyn FrameSource>`, drive
/// `read()` to EOF, exercise `info()` / `headers_ready()` /
/// `codec_private()` through the trait object. The trait-bridge
/// correctness is what's being verified — not demuxer behaviour.
#[test]
fn frame_source_via_dyn_object() {
let reader = ZeroReader { capacity: 8 };
let title = synthetic_title(8);
let stream = DiscStream::new(
Box::new(reader),
title,
crate::decrypt::DecryptKeys::None,
8,
ContentFormat::BdTs,
);
let mut src: Box<dyn FrameSource> = Box::new(stream);
// Empty-title fixture has no streams configured, so headers are
// trivially ready and codec_private() yields nothing on track 0.
assert!(src.headers_ready());
assert!(src.codec_private(0).is_none());
let _ = src.info();
// Drive read() to EOF through the trait object — empty-title fixture
// produces no frames, but the call still routes through the blanket
// dispatch into Stream::read.
let mut frames = 0usize;
loop {
match src.read().expect("read") {
Some(_) => frames += 1,
None => break,
}
if frames > 1024 {
panic!("unexpected unbounded frame stream from empty title");
}
}
assert_eq!(frames, 0);
}
/// `is_halted()` must observe a cancellation signal regardless of
/// which entry point installed the token. The deprecated
/// `set_halt(Arc<AtomicBool>)` and the new `with_halt(Halt)` are
/// two views over one slot — flipping either bit must cause the
/// next `fill_extents` retry boundary to bail.
#[test]
fn halt_via_with_halt_observed_by_is_halted() {
let halt = Halt::new();
let stream = DiscStream::new(
Box::new(ZeroReader { capacity: 8 }),
synthetic_title(8),
crate::decrypt::DecryptKeys::None,
8,
crate::disc::ContentFormat::BdTs,
)
.with_halt(halt.clone());
assert!(!stream.is_halted());
halt.cancel();
assert!(
stream.is_halted(),
"with_halt token cancellation must be observed by is_halted()"
);
}
#[test]
fn halt_via_set_halt_bridge_observed_by_is_halted() {
let arc = Arc::new(AtomicBool::new(false));
let mut stream = DiscStream::new(
Box::new(ZeroReader { capacity: 8 }),
synthetic_title(8),
crate::decrypt::DecryptKeys::None,
8,
crate::disc::ContentFormat::BdTs,
);
stream.set_halt(arc.clone());
assert!(!stream.is_halted());
arc.store(true, std::sync::atomic::Ordering::Relaxed);
assert!(
stream.is_halted(),
"set_halt(Arc<AtomicBool>) bridge must observe Arc-side flips"
);
}
}
+1 -28
View File
@@ -87,11 +87,6 @@ pub fn write_uint(w: &mut impl Write, id: u32, val: u64) -> io::Result<()> {
}
}
/// Write a complete EBML signed integer element.
pub fn write_int(w: &mut impl Write, id: u32, val: i64) -> io::Result<()> {
write_uint(w, id, val as u64)
}
/// Write a complete EBML float element (8-byte double).
pub fn write_float(w: &mut impl Write, id: u32, val: f64) -> io::Result<()> {
write_id(w, id)?;
@@ -309,22 +304,6 @@ pub fn read_binary_val(r: &mut impl Read, len: usize) -> io::Result<Vec<u8>> {
Ok(buf)
}
/// Read a VINT (track number) from a SimpleBlock. Returns (value, bytes_consumed).
pub fn read_vint(r: &mut impl Read) -> io::Result<(u64, usize)> {
let mut first = [0u8; 1];
r.read_exact(&mut first)?;
let b0 = first[0];
if b0 & 0x80 != 0 {
return Ok(((b0 & 0x7F) as u64, 1));
}
if b0 & 0x40 != 0 {
let mut b = [0u8; 1];
r.read_exact(&mut b)?;
return Ok(((((b0 & 0x3F) as u64) << 8) | b[0] as u64, 2));
}
Err(crate::error::Error::MkvInvalid.into())
}
// ============================================================
// Matroska Element IDs
// ============================================================
@@ -342,12 +321,6 @@ pub const EBML_DOC_TYPE_READ_VERSION: u32 = 0x4285;
// Segment
pub const SEGMENT: u32 = 0x1853_8067;
// Seek Head
pub const SEEK_HEAD: u32 = 0x114D_9B74;
pub const SEEK: u32 = 0x4DBB;
pub const SEEK_ID: u32 = 0x53AB;
pub const SEEK_POSITION: u32 = 0x53AC;
// Segment Info
pub const INFO: u32 = 0x1549_A966;
pub const TIMESTAMP_SCALE: u32 = 0x2A_D7B1;
@@ -441,7 +414,7 @@ mod tests {
fn test_write_uint() {
let mut buf = Vec::new();
write_uint(&mut buf, 0x4286, 1).unwrap(); // EBML_VERSION = 1
// ID: 42 86, Size: 81 (1 byte), Data: 01
// ID: 42 86, Size: 81 (1 byte), Data: 01
assert_eq!(buf, [0x42, 0x86, 0x81, 0x01]);
}
+10 -5
View File
@@ -19,8 +19,7 @@ pub struct IsoSectorReader {
impl IsoSectorReader {
pub fn open(path: &str) -> std::io::Result<Self> {
let file = File::open(Path::new(path))
.map_err(|e| std::io::Error::new(e.kind(), format!("iso://{path}: {e}")))?;
let file = File::open(Path::new(path))?;
let size = file.metadata()?.len();
let sectors = size / SECTOR_SIZE;
if sectors > u32::MAX as u64 {
@@ -39,7 +38,13 @@ impl IsoSectorReader {
}
impl SectorReader for IsoSectorReader {
fn read_sectors(&mut self, lba: u32, count: u16, buf: &mut [u8]) -> Result<usize> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let bytes = count as usize * SECTOR_SIZE as usize;
self.file
.seek(SeekFrom::Start(lba as u64 * SECTOR_SIZE))
@@ -71,11 +76,11 @@ mod tests {
assert_eq!(reader.capacity(), 4);
let mut buf = [0u8; 2048];
reader.read_sectors(0, 1, &mut buf).unwrap();
reader.read_sectors(0, 1, &mut buf, true).unwrap();
assert_eq!(buf[0], 1);
assert_eq!(buf[2047], 100);
reader.read_sectors(2, 1, &mut buf).unwrap();
reader.read_sectors(2, 1, &mut buf, true).unwrap();
assert_eq!(buf[0], 3);
assert_eq!(buf[2047], 102);
-127
View File
@@ -1,127 +0,0 @@
//! LookaheadBuffer — generic pre-scan buffer for stream pipelines.
//!
//! Accumulates data up to a configurable limit. When the consumer finds
//! what it needs, the buffer can be drained (fast path, no re-read).
//! If the buffer fills before the consumer is satisfied, it signals
//! overflow — the caller should discard and re-read from the source.
//!
//! Used by MkvStream to collect SPS/PPS before writing the MKV header.
//! Reusable for any stream stage that needs to look ahead.
/// Default lookahead buffer size: 5 MB.
pub const DEFAULT_LOOKAHEAD_SIZE: usize = 5 * 1024 * 1024;
/// Lookahead buffer states.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum LookaheadState {
/// Still collecting data, haven't found what we need yet.
Collecting,
/// Found what we need, buffer has the data ready to drain.
Ready,
/// Buffer overflowed before finding what we need.
/// Caller should discard buffer, finish scanning without buffering,
/// then re-read from the source.
Overflow,
}
/// A bounded lookahead buffer.
pub struct LookaheadBuffer {
data: Vec<u8>,
max_size: usize,
state: LookaheadState,
}
impl LookaheadBuffer {
/// Create a new buffer with the given max size.
/// Pass 0 for no buffering (always overflows immediately).
pub fn new(max_size: usize) -> Self {
Self {
data: Vec::with_capacity(max_size.min(DEFAULT_LOOKAHEAD_SIZE)),
max_size,
state: LookaheadState::Collecting,
}
}
/// Push data into the buffer. Returns the new state.
/// If the buffer would overflow, transitions to Overflow state.
pub fn push(&mut self, chunk: &[u8]) -> LookaheadState {
if self.state != LookaheadState::Collecting {
return self.state;
}
if self.data.len() + chunk.len() > self.max_size {
self.state = LookaheadState::Overflow;
return self.state;
}
self.data.extend_from_slice(chunk);
self.state
}
/// Mark the buffer as ready — we found what we need.
pub fn mark_ready(&mut self) {
if self.state == LookaheadState::Collecting {
self.state = LookaheadState::Ready;
}
}
/// Get the buffered data (only valid in Ready state).
pub fn data(&self) -> &[u8] {
&self.data
}
/// Take ownership of the buffered data, clearing the buffer.
pub fn drain(&mut self) -> Vec<u8> {
self.state = LookaheadState::Collecting;
std::mem::take(&mut self.data)
}
/// Current state.
pub fn state(&self) -> LookaheadState {
self.state
}
/// How many bytes are buffered.
pub fn len(&self) -> usize {
self.data.len()
}
/// Is the buffer empty?
pub fn is_empty(&self) -> bool {
self.data.is_empty()
}
/// Max size this buffer can hold.
pub fn max_size(&self) -> usize {
self.max_size
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_basic_flow() {
let mut buf = LookaheadBuffer::new(100);
assert_eq!(buf.push(b"hello"), LookaheadState::Collecting);
assert_eq!(buf.push(b"world"), LookaheadState::Collecting);
assert_eq!(buf.len(), 10);
buf.mark_ready();
assert_eq!(buf.state(), LookaheadState::Ready);
assert_eq!(buf.data(), b"helloworld");
}
#[test]
fn test_overflow() {
let mut buf = LookaheadBuffer::new(5);
assert_eq!(buf.push(b"abc"), LookaheadState::Collecting);
assert_eq!(buf.push(b"def"), LookaheadState::Overflow);
}
#[test]
fn test_zero_size() {
let mut buf = LookaheadBuffer::new(0);
assert_eq!(buf.push(b"a"), LookaheadState::Overflow);
}
}
+33 -8
View File
@@ -18,10 +18,10 @@ const SCAN_SIZE: usize = 1024 * 1024;
enum Mode {
Write {
muxer: super::tsmux::TsMuxer<Box<dyn Write>>,
muxer: super::tsmux::TsMuxer<Box<dyn Write + Send>>,
},
Read {
reader: Box<dyn Read>,
reader: Box<dyn Read + Send>,
},
}
@@ -57,7 +57,7 @@ pub struct M2tsStream {
impl M2tsStream {
/// Create for writing PES frames → BD-TS output.
/// Writes FMKV metadata header, then muxes PES frames into BD transport stream.
pub fn create(mut writer: impl Write + 'static, title: &DiscTitle) -> io::Result<Self> {
pub fn create(mut writer: impl Write + Send + 'static, title: &DiscTitle) -> io::Result<Self> {
// Write FMKV metadata header
if !title.streams.is_empty() {
let m = meta::M2tsMeta::from_title(title);
@@ -72,7 +72,7 @@ impl M2tsStream {
DiscStream::Subtitle(s) => s.pid,
})
.collect();
let boxed: Box<dyn Write> = Box::new(writer);
let boxed: Box<dyn Write + Send> = Box::new(writer);
let mut muxer = super::tsmux::TsMuxer::new(boxed, &pids);
for (i, cp) in title.codec_privates.iter().enumerate() {
if let Some(data) = cp {
@@ -103,7 +103,7 @@ impl M2tsStream {
};
pids.push(pid);
pid_to_track.push((pid, i));
parsers.push((pid, super::codec::parser_for_codec(codec)));
parsers.push((pid, super::codec::parser_for_codec(codec, None)));
}
(pids, parsers, pid_to_track)
}
@@ -111,7 +111,7 @@ impl M2tsStream {
/// Open an M2TS stream for reading. Takes any Read source — file, pipe, socket.
///
/// Tries FMKV metadata header first. Falls back to PMT scan of first 1 MB.
pub fn open(mut reader: impl Read + 'static) -> io::Result<Self> {
pub fn open(mut reader: impl Read + Send + 'static) -> io::Result<Self> {
// Read first chunk — enough for FMKV header or PMT scan
let mut head = vec![0u8; SCAN_SIZE];
let head_len = read_fill(&mut reader, &mut head)?;
@@ -125,7 +125,7 @@ impl M2tsStream {
let (pids, parsers, pid_to_track) = Self::setup_pes(&title.streams);
// Chain: remaining head bytes + rest of reader
let remaining_head = &head[header_end..];
let chain: Box<dyn Read> =
let chain: Box<dyn Read + Send> =
Box::new(io::Cursor::new(remaining_head.to_vec()).chain(reader));
return Ok(Self {
disc_title: title.clone(),
@@ -150,7 +150,7 @@ impl M2tsStream {
let (pids, parsers, pid_to_track) = Self::setup_pes(&streams);
// Chain: full head (it's all TS data) + rest of reader
let chain: Box<dyn Read> = Box::new(io::Cursor::new(head).chain(reader));
let chain: Box<dyn Read + Send> = Box::new(io::Cursor::new(head).chain(reader));
Ok(Self {
disc_title: DiscTitle {
@@ -173,6 +173,7 @@ impl M2tsStream {
}
}
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSource/FrameSink in follow-up commit.
impl crate::pes::Stream for M2tsStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
if let Some(frame) = self.pending_frames.pop_front() {
@@ -283,3 +284,27 @@ impl crate::pes::Stream for M2tsStream {
true
}
}
/// FrameSink sibling to the deprecated Stream impl; both coexist during the
/// 0.18 deprecation window. Caller may pick either at the trait-object
/// boundary — `Box<dyn Stream>` (deprecated) or `Box<dyn FrameSink>` (new).
/// Use `M2tsStream::create(writer, title)` to construct the write half;
/// calling `FrameSink::write` on an `M2tsStream::open(reader)` instance
/// returns `StreamReadOnly`.
#[allow(deprecated)] // delegating to deprecated Stream during the 0.18 deprecation window so callers don't see the deprecation twice.
impl crate::pes::FrameSink for M2tsStream {
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
<Self as crate::pes::Stream>::write(self, frame)
}
fn finish(self: Box<Self>) -> io::Result<()> {
// Why: Stream::finish takes &mut self, FrameSink::finish takes Box<Self>.
// Re-borrow inside the box, call Stream::finish, drop the box.
let mut s: Self = *self;
<Self as crate::pes::Stream>::finish(&mut s)
}
fn info(&self) -> &crate::disc::DiscTitle {
<Self as crate::pes::Stream>::info(self)
}
}
+3 -1
View File
@@ -173,6 +173,7 @@ impl M2tsMeta {
.parse()
.unwrap_or(crate::disc::SampleRate::Unknown),
secondary: *secondary,
purpose: crate::disc::LabelPurpose::Normal,
label: label.clone(),
}),
MetaStream::Subtitle {
@@ -185,6 +186,7 @@ impl M2tsMeta {
codec: codec.parse().unwrap_or(crate::disc::Codec::Unknown(0)),
language: language.clone(),
forced: *forced,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
}),
})
@@ -211,7 +213,7 @@ impl M2tsMeta {
.iter()
.map(|s| {
if let MetaStream::Video {
codec_private: Some(ref b64),
codec_private: Some(b64),
..
} = s
{
+127 -74
View File
@@ -5,8 +5,10 @@
//! cues and seek head are finalized at the end.
use super::ebml;
use crate::disc::{AudioStream, Chapter, Codec, SubtitleStream, VideoStream};
use std::io::{self, Seek, SeekFrom, Write};
use crate::disc::{
AudioStream, Chapter, Codec, ColorSpace, HdrFormat, SubtitleStream, VideoStream,
};
use std::io::{self, Seek, Write};
/// MKV track definition (built from disc stream metadata).
pub struct MkvTrack {
@@ -20,6 +22,14 @@ pub struct MkvTrack {
// Video-specific
pub pixel_width: u32,
pub pixel_height: u32,
pub default_duration_ns: u64, // nanoseconds per frame (0 = unknown)
pub display_width: u32, // display aspect ratio width (0 = same as pixel)
pub display_height: u32, // display aspect ratio height (0 = same as pixel)
// HDR colour metadata
pub colour_matrix: u8, // MatrixCoefficients (9=bt2020nc)
pub colour_transfer: u8, // TransferCharacteristics (16=smpte2084/PQ)
pub colour_primaries: u8, // Primaries (9=bt2020)
pub colour_range: u8, // Range (1=tv/limited)
// Audio-specific
pub sample_rate: f64,
pub channels: u8,
@@ -36,16 +46,40 @@ impl MkvTrack {
_ => "V_MPEG2",
};
let (w, h) = v.resolution.pixels();
let (num, den) = v.frame_rate.as_fraction();
let default_duration_ns = if num > 0 {
(1_000_000_000u64 * den as u64) / num as u64
} else {
0
};
let (matrix, transfer, primaries, range) = match v.color_space {
ColorSpace::Bt2020 => (9, 16, 9, 1), // bt2020nc, PQ, bt2020, limited
ColorSpace::Bt709 => (1, 1, 1, 1), // bt709
ColorSpace::Unknown => (0, 0, 0, 0),
};
// Override transfer for non-PQ HDR
let transfer = match v.hdr {
HdrFormat::Hdr10 | HdrFormat::Hdr10Plus | HdrFormat::DolbyVision => 16, // PQ
HdrFormat::Hlg => 18,
_ => transfer,
};
Self {
track_type: ebml::TRACK_TYPE_VIDEO,
codec_id,
language: "und".into(),
name: v.label.clone(),
codec_private: None, // filled later by parser
codec_private: None,
is_default: !v.secondary,
is_forced: false,
pixel_width: w,
pixel_height: h,
default_duration_ns,
display_width: w,
display_height: h,
colour_matrix: matrix,
colour_transfer: transfer,
colour_primaries: primaries,
colour_range: range,
sample_rate: 0.0,
channels: 0,
bit_depth: 0,
@@ -63,16 +97,26 @@ impl MkvTrack {
};
let sr = a.sample_rate.hz();
let ch = a.channels.count();
let name = a.label.clone();
Self {
track_type: ebml::TRACK_TYPE_AUDIO,
codec_id,
language: a.language.clone(),
name: a.label.clone(),
name,
codec_private: None,
is_default: !a.secondary,
is_forced: false,
pixel_width: 0,
pixel_height: 0,
default_duration_ns: 0,
display_width: 0,
display_height: 0,
colour_matrix: 0,
colour_transfer: 0,
colour_primaries: 0,
colour_range: 0,
sample_rate: sr,
channels: ch,
bit_depth: 0,
@@ -94,6 +138,13 @@ impl MkvTrack {
is_forced: s.forced,
pixel_width: 0,
pixel_height: 0,
default_duration_ns: 0,
display_width: 0,
display_height: 0,
colour_matrix: 0,
colour_transfer: 0,
colour_primaries: 0,
colour_range: 0,
sample_rate: 0.0,
channels: 0,
bit_depth: 0,
@@ -116,12 +167,9 @@ pub struct MkvMuxer<W: Write + Seek> {
cluster_pos: u64,
cluster_size_pos: u64,
cluster_ts_ms: i64,
base_pts_ms: Option<i64>,
cues: Vec<CuePoint>,
frame_count: u64,
/// File positions of codecPrivate placeholders (track_idx → offset, max_size).
/// Used to seek back and fill in SPS/PPS after first keyframe.
codec_private_slots: Vec<Option<(u64, usize)>>,
codec_private_filled: Vec<bool>,
}
/// New cluster every 5 seconds.
@@ -130,16 +178,6 @@ const CLUSTER_DURATION_MS: i64 = 5000;
impl<W: Write + Seek> MkvMuxer<W> {
/// Create a new MKV muxer: writes EBML header, Segment start, Info, Tracks, Chapters.
pub fn new(
writer: W,
tracks: &[MkvTrack],
title: Option<&str>,
duration_secs: f64,
) -> io::Result<Self> {
Self::new_with_chapters(writer, tracks, title, duration_secs, &[])
}
/// Create a new MKV muxer with chapters: writes EBML header, Segment start, Info, Tracks, Chapters.
pub fn new_with_chapters(
mut writer: W,
tracks: &[MkvTrack],
title: Option<&str>,
@@ -177,8 +215,6 @@ impl<W: Write + Seek> MkvMuxer<W> {
ebml::end_master(&mut writer, info_pos)?;
// Tracks
let mut codec_private_slots: Vec<Option<(u64, usize)>> = Vec::new();
let mut codec_private_filled: Vec<bool> = Vec::new();
let tracks_pos = ebml::start_master(&mut writer, ebml::TRACKS)?;
for (i, track) in tracks.iter().enumerate() {
let entry_pos = ebml::start_master(&mut writer, ebml::TRACK_ENTRY)?;
@@ -201,19 +237,20 @@ impl<W: Write + Seek> MkvMuxer<W> {
if let Some(ref cp) = track.codec_private {
ebml::write_binary(&mut writer, ebml::CODEC_PRIVATE, cp)?;
codec_private_slots.push(None); // already filled
codec_private_filled.push(true);
} else if track.track_type == ebml::TRACK_TYPE_VIDEO {
// Reserve space for codecPrivate — will be filled after first keyframe
// Reserve 256 bytes (enough for SPS+PPS or VPS+SPS+PPS)
let cp_pos = writer.stream_position()?;
let placeholder = vec![0u8; 256];
ebml::write_binary(&mut writer, ebml::CODEC_PRIVATE, &placeholder)?;
codec_private_slots.push(Some((cp_pos, 256)));
codec_private_filled.push(false);
} else {
codec_private_slots.push(None);
codec_private_filled.push(true);
}
// Pre-0.13 a deferred codecPrivate path existed for video tracks
// (placeholder reserve + later seek-back fill via
// `fill_codec_private`). The PES pipeline hands codec_private
// up-front via the DiscTitle, so the deferred path was never
// exercised — removed in the 0.13 dead-code sweep.
// DefaultDuration — frame duration in nanoseconds
if track.default_duration_ns > 0 {
ebml::write_uint(
&mut writer,
ebml::DEFAULT_DURATION,
track.default_duration_ns,
)?;
}
// Video-specific
@@ -221,6 +258,31 @@ impl<W: Write + Seek> MkvMuxer<W> {
let vid_pos = ebml::start_master(&mut writer, ebml::VIDEO)?;
ebml::write_uint(&mut writer, ebml::PIXEL_WIDTH, track.pixel_width as u64)?;
ebml::write_uint(&mut writer, ebml::PIXEL_HEIGHT, track.pixel_height as u64)?;
if track.display_width > 0 && track.display_height > 0 {
ebml::write_uint(&mut writer, ebml::DISPLAY_WIDTH, track.display_width as u64)?;
ebml::write_uint(
&mut writer,
ebml::DISPLAY_HEIGHT,
track.display_height as u64,
)?;
}
// Colour metadata (HDR)
if track.colour_matrix > 0 || track.colour_transfer > 0 {
let col_pos = ebml::start_master(&mut writer, ebml::COLOUR)?;
ebml::write_uint(
&mut writer,
ebml::MATRIX_COEFFICIENTS,
track.colour_matrix as u64,
)?;
ebml::write_uint(
&mut writer,
ebml::TRANSFER_CHARACTERISTICS,
track.colour_transfer as u64,
)?;
ebml::write_uint(&mut writer, ebml::PRIMARIES, track.colour_primaries as u64)?;
ebml::write_uint(&mut writer, ebml::RANGE, track.colour_range as u64)?;
ebml::end_master(&mut writer, col_pos)?;
}
ebml::end_master(&mut writer, vid_pos)?;
}
@@ -265,10 +327,9 @@ impl<W: Write + Seek> MkvMuxer<W> {
cluster_pos: 0,
cluster_size_pos: 0,
cluster_ts_ms: 0,
base_pts_ms: None,
cues: Vec::new(),
frame_count: 0,
codec_private_slots,
codec_private_filled,
})
}
@@ -280,7 +341,9 @@ impl<W: Write + Seek> MkvMuxer<W> {
keyframe: bool,
data: &[u8],
) -> io::Result<()> {
let pts_ms = pts_ns / 1_000_000;
let raw_ms = pts_ns / 1_000_000;
let base = *self.base_pts_ms.get_or_insert(raw_ms);
let pts_ms = raw_ms - base;
// Start new cluster if needed
if !self.cluster_open || (pts_ms - self.cluster_ts_ms) >= CLUSTER_DURATION_MS {
@@ -336,30 +399,6 @@ impl<W: Write + Seek> MkvMuxer<W> {
Ok(())
}
/// Fill in a deferred codecPrivate for a track.
/// Seeks back to the placeholder, writes the actual data, restores position.
pub fn fill_codec_private(&mut self, track_idx: usize, data: &[u8]) -> io::Result<()> {
if track_idx >= self.codec_private_filled.len() || self.codec_private_filled[track_idx] {
return Ok(());
}
if let Some((pos, max_size)) = self.codec_private_slots[track_idx] {
if data.len() > max_size {
// Data too large for reserved space — can't fill in place
// This shouldn't happen with 256 bytes reserved
return Ok(());
}
let current = self.writer.stream_position()?;
self.writer.seek(SeekFrom::Start(pos))?;
// Rewrite: element ID + size + data + zero-pad remainder
let mut padded = data.to_vec();
padded.resize(max_size, 0);
ebml::write_binary(&mut self.writer, ebml::CODEC_PRIVATE, &padded)?;
self.writer.seek(SeekFrom::Start(current))?;
self.codec_private_filled[track_idx] = true;
}
Ok(())
}
fn start_cluster(&mut self, ts_ms: i64) -> io::Result<()> {
// Close previous cluster if open
if self.cluster_open {
@@ -450,6 +489,13 @@ mod tests {
is_forced: false,
pixel_width: 1920,
pixel_height: 1080,
default_duration_ns: 41708333,
display_width: 1920,
display_height: 1080,
colour_matrix: 0,
colour_transfer: 0,
colour_primaries: 0,
colour_range: 0,
sample_rate: 0.0,
channels: 0,
bit_depth: 0,
@@ -467,6 +513,13 @@ mod tests {
is_forced: false,
pixel_width: 0,
pixel_height: 0,
default_duration_ns: 0,
display_width: 0,
display_height: 0,
colour_matrix: 0,
colour_transfer: 0,
colour_primaries: 0,
colour_range: 0,
sample_rate: 48000.0,
channels: 6,
bit_depth: 0,
@@ -477,7 +530,7 @@ mod tests {
fn mkv_writes_ebml_header() {
let buf = Cursor::new(Vec::new());
let tracks = [make_video_track()];
let muxer = MkvMuxer::new(buf, &tracks, Some("Test"), 120.0).unwrap();
let muxer = MkvMuxer::new(buf, &tracks, Some("Test"), 120.0, &[]).unwrap();
let data = muxer.writer.into_inner();
// EBML header element ID: 0x1A45DFA3
assert!(data.len() >= 4);
@@ -488,7 +541,7 @@ mod tests {
fn mkv_writes_segment() {
let buf = Cursor::new(Vec::new());
let tracks = [make_video_track()];
let muxer = MkvMuxer::new(buf, &tracks, None, 0.0).unwrap();
let muxer = MkvMuxer::new(buf, &tracks, None, 0.0, &[]).unwrap();
let data = muxer.writer.into_inner();
// Segment element ID: 0x18538067
assert!(
@@ -501,7 +554,7 @@ mod tests {
fn mkv_write_frame_creates_cluster() {
let buf = Cursor::new(Vec::new());
let tracks = [make_video_track()];
let mut muxer = MkvMuxer::new(buf, &tracks, None, 60.0).unwrap();
let mut muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
muxer
.write_frame(0, 0, true, &[0xDE, 0xAD, 0xBE, 0xEF])
.unwrap();
@@ -540,7 +593,7 @@ mod tests {
let shared = Arc::new(Mutex::new(Cursor::new(Vec::new())));
let writer = SharedWriter(shared.clone());
let tracks = [make_video_track()];
let mut muxer = MkvMuxer::new(writer, &tracks, Some("Cue Test"), 60.0).unwrap();
let mut muxer = MkvMuxer::new(writer, &tracks, Some("Cue Test"), 60.0, &[]).unwrap();
muxer.write_frame(0, 0, true, &[0x01, 0x02, 0x03]).unwrap();
muxer.finish().unwrap();
@@ -555,7 +608,7 @@ mod tests {
fn mkv_multiple_tracks() {
let buf = Cursor::new(Vec::new());
let tracks = [make_video_track(), make_audio_track()];
let mut muxer = MkvMuxer::new(buf, &tracks, Some("Multi"), 120.0).unwrap();
let mut muxer = MkvMuxer::new(buf, &tracks, Some("Multi"), 120.0, &[]).unwrap();
// Write frames to both tracks
muxer.write_frame(0, 0, true, &[0x00, 0x00, 0x01]).unwrap();
muxer.write_frame(1, 0, false, &[0x0B, 0x77, 0x00]).unwrap();
@@ -574,7 +627,7 @@ mod tests {
fn mkv_keyframe_flag() {
let buf = Cursor::new(Vec::new());
let tracks = [make_video_track()];
let mut muxer = MkvMuxer::new(buf, &tracks, None, 10.0).unwrap();
let mut muxer = MkvMuxer::new(buf, &tracks, None, 10.0, &[]).unwrap();
// Record position before first frame
let pos_before_kf = muxer.writer.position();
@@ -606,7 +659,7 @@ mod tests {
// Track VINT: 1 byte (track 1 = 0x81)
let track_vint_pos = after_id + size_len;
let track_vint_len = 1; // track 1 encoded as 0x81
// 2-byte relative timestamp
// 2-byte relative timestamp
let ts_pos = track_vint_pos + track_vint_len;
// flags byte
let flags_pos = ts_pos + 2;
@@ -648,9 +701,7 @@ mod tests {
name: "Chapter 3".into(),
},
];
let muxer =
MkvMuxer::new_with_chapters(buf, &tracks, Some("Chapter Test"), 900.0, &chapters)
.unwrap();
let muxer = MkvMuxer::new(buf, &tracks, Some("Chapter Test"), 900.0, &chapters).unwrap();
let data = muxer.writer.into_inner();
// Chapters element ID: 0x1043A770
@@ -674,7 +725,7 @@ mod tests {
fn mkv_no_chapters_when_empty() {
let buf = Cursor::new(Vec::new());
let tracks = [make_video_track()];
let muxer = MkvMuxer::new(buf, &tracks, Some("No Chapters"), 60.0).unwrap();
let muxer = MkvMuxer::new(buf, &tracks, Some("No Chapters"), 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
assert!(
find_id(&data, ebml::CHAPTERS).is_none(),
@@ -693,7 +744,7 @@ mod tests {
let buf = Cursor::new(Vec::new());
let tracks = [video, audio1, audio2];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0).unwrap();
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
// FlagDefault ID is 0x88. When is_default is true, FlagDefault is NOT written
@@ -719,13 +770,14 @@ mod tests {
codec: Codec::Pgs,
language: "eng".into(),
forced: true,
qualifier: crate::disc::LabelQualifier::Forced,
codec_data: None,
});
assert!(forced_sub.is_forced);
let buf = Cursor::new(Vec::new());
let tracks = [video, forced_sub];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0).unwrap();
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
// FlagForced ID: 0x55AA (2-byte ID)
@@ -744,13 +796,14 @@ mod tests {
codec: Codec::Pgs,
language: "eng".into(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
});
assert!(!sub.is_forced);
let buf = Cursor::new(Vec::new());
let tracks = [video, sub];
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0).unwrap();
let muxer = MkvMuxer::new(buf, &tracks, None, 60.0, &[]).unwrap();
let data = muxer.writer.into_inner();
// FlagForced should NOT be written for non-forced tracks
+51 -9
View File
@@ -4,7 +4,7 @@
//! Write: PES frames in → MKV mux → Matroska container.
use super::mkv::{MkvMuxer, MkvTrack};
use super::{ebml, WriteSeek};
use super::{WriteSeek, ebml};
type MkvHeaderResult = io::Result<(crate::disc::DiscTitle, Vec<(u16, Vec<u8>)>)>;
@@ -18,7 +18,7 @@ use crate::disc::*;
use std::io::{self, Read};
struct ReadState {
reader: Box<dyn Read>,
reader: Box<dyn Read + Send>,
cluster_ts_ms: i64,
/// Codec private data per track (track_number, hvcC/avcC bytes).
codec_privates: Vec<(u16, Vec<u8>)>,
@@ -26,7 +26,7 @@ struct ReadState {
enum Mode {
Write {
muxer: Option<MkvMuxer<Box<dyn WriteSeek>>>,
muxer: Option<MkvMuxer<Box<dyn WriteSeek + Send>>>,
},
Read(ReadState),
}
@@ -40,21 +40,31 @@ pub struct MkvStream {
impl MkvStream {
/// Create for writing PES frames → MKV container.
/// Codec privates come from title.codec_privates (populated by input stream).
pub fn create(writer: Box<dyn WriteSeek>, title: &DiscTitle) -> io::Result<Self> {
pub fn create(writer: Box<dyn WriteSeek + Send>, title: &DiscTitle) -> io::Result<Self> {
let mut tracks = Vec::new();
let mut has_default_video = false;
let mut has_default_audio = false;
for (idx, s) in title.streams.iter().enumerate() {
let mut track = match s {
crate::disc::Stream::Video(v) => MkvTrack::video(v),
crate::disc::Stream::Audio(a) => MkvTrack::audio(a),
crate::disc::Stream::Subtitle(s) => MkvTrack::subtitle(s),
};
// Only first video and first audio are default
if track.is_default {
match track.track_type {
1 if !has_default_video => has_default_video = true,
2 if !has_default_audio => has_default_audio = true,
_ => track.is_default = false,
}
}
if let Some(cp) = title.codec_privates.get(idx).and_then(|c| c.as_ref()) {
track.codec_private = Some(cp.clone());
}
tracks.push(track);
}
let muxer = MkvMuxer::new_with_chapters(
let muxer = MkvMuxer::new(
writer,
&tracks,
Some(&title.playlist),
@@ -69,7 +79,7 @@ impl MkvStream {
}
/// Open an MKV file for reading → PES frames.
pub fn open(mut reader: impl Read + 'static) -> io::Result<Self> {
pub fn open(mut reader: impl Read + Send + 'static) -> io::Result<Self> {
let (disc_title, codec_privates) = parse_mkv_header(&mut reader)?;
Ok(Self {
disc_title,
@@ -82,6 +92,7 @@ impl MkvStream {
}
}
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSource/FrameSink in follow-up commit.
impl crate::pes::Stream for MkvStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
let rs = match self.mode {
@@ -140,9 +151,9 @@ impl crate::pes::Stream for MkvStream {
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
match &mut self.mode {
Mode::Write {
muxer: Some(ref mut m),
} => m.write_frame(frame.track, frame.pts, frame.keyframe, &frame.data),
Mode::Write { muxer: Some(m) } => {
m.write_frame(frame.track, frame.pts, frame.keyframe, &frame.data)
}
Mode::Write { muxer: None } => Ok(()),
Mode::Read(_) => Err(crate::error::Error::StreamReadOnly.into()),
}
@@ -178,6 +189,35 @@ impl crate::pes::Stream for MkvStream {
}
}
/// FrameSink sibling to the deprecated Stream impl; both coexist during the
/// 0.18 deprecation window. Caller may pick either at the trait-object
/// boundary — `Box<dyn Stream>` (deprecated) or `Box<dyn FrameSink>` (new).
/// Use `MkvStream::create(writer, title)` to construct the write half;
/// calling `FrameSink::write` on a `MkvStream::open(reader)` instance returns
/// `StreamReadOnly`. `finish` is where the Cues index is written, so it must
/// be called for the resulting MKV to be seekable.
#[allow(deprecated)] // delegating to deprecated Stream during the 0.18 deprecation window so callers don't see the deprecation twice.
impl crate::pes::FrameSink for MkvStream {
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
<Self as crate::pes::Stream>::write(self, frame)
}
fn finish(self: Box<Self>) -> io::Result<()> {
// Why: Stream::finish takes &mut self, FrameSink::finish takes Box<Self>.
// Re-borrow inside the box, call Stream::finish, drop the box.
// The inner `MkvMuxer` is owned via `Option`, and `Stream::finish`
// already takes it via `Option::take()` — moving `*self` out of the
// box hands it the same field by-value, so the muxer's own
// by-value `finish()` runs correctly.
let mut s: Self = *self;
<Self as crate::pes::Stream>::finish(&mut s)
}
fn info(&self) -> &crate::disc::DiscTitle {
<Self as crate::pes::Stream>::info(self)
}
}
// ── MKV header parsing (read side) ────────────────────────────
/// Returns (DiscTitle, codec_privates: Vec<(track_number, codec_private_bytes)>)
@@ -370,6 +410,7 @@ fn parse_track(
language: lang,
sample_rate: srs,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: name,
})),
17 => Some(crate::disc::Stream::Subtitle(SubtitleStream {
@@ -377,6 +418,7 @@ fn parse_track(
codec,
language: lang,
forced,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
})),
_ => None,
+31 -14
View File
@@ -15,22 +15,32 @@
//!
//! For disc→ISO (raw sector copy), use `Disc::copy()` instead.
// Public modules — types here are intentionally part of the consumable API.
pub mod codec;
pub mod disc;
pub mod ebml;
pub mod iso;
mod m2ts;
pub mod meta;
pub mod mkv;
mod mkvstream;
pub mod network;
pub mod null;
pub mod ps;
pub mod resolve;
pub mod stdio;
pub mod ts;
pub mod tsmux;
pub mod tsreader;
// Internal modules — implementation details. Their *types* are re-exported
// where appropriate (`MkvStream`, `M2tsStream`, etc. surface from `lib.rs`),
// but the module paths themselves are not part of the API. Pre-0.13 these
// were `pub`, leaking low-level EBML primitives, TS muxer internals, and
// network/stdio implementations that no external caller had business
// reaching for.
pub(crate) mod ebml;
pub(crate) mod m2ts;
/// FMKV metadata header (used by `M2tsStream` / `NetworkStream` / `StdioStream`
/// to round-trip codec_privates that don't fit inside the underlying format).
/// Exposed for integration tests that exercise the wire format directly.
pub mod meta;
pub(crate) mod mkv;
pub(crate) mod mkvstream;
pub(crate) mod network;
pub(crate) mod null;
pub(crate) mod ps;
pub(crate) mod stdio;
pub(crate) mod ts;
pub(crate) mod tsmux;
pub use disc::DiscStream;
pub use iso::IsoSectorReader;
@@ -38,11 +48,18 @@ pub use m2ts::M2tsStream;
pub use mkvstream::MkvStream;
pub use network::NetworkStream;
pub use null::NullStream;
pub use resolve::{input, output, parse_url, InputOptions, StreamUrl};
pub use resolve::{InputOptions, StreamUrl, input, output, parse_url};
pub use stdio::StdioStream;
use std::io::{Seek, Write};
// WriteSeek — used internally by MKV muxer (container format requires seeking).
/// Combined `Write + Seek` for sinks accepted by the MKV muxer.
///
/// Matroska's `SeekHead`, `Cues`, and `Cluster` size fields are written with
/// placeholder values during streaming and updated in-place at finalization,
/// so the output sink must support seeking. Provided as a single trait
/// alias so callers don't have to repeat `Write + Seek` everywhere; the
/// blanket impl below opts every `T: Write + Seek` in automatically
/// (`File`, `BufWriter<File>`, `Cursor<Vec<u8>>`).
pub trait WriteSeek: Write + Seek {}
impl<T: Write + Seek> WriteSeek for T {}
+28 -1
View File
@@ -70,6 +70,7 @@ impl NetworkStream {
}
}
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSource/FrameSink in follow-up commit.
impl crate::pes::Stream for NetworkStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
match &mut self.mode {
@@ -81,7 +82,7 @@ impl crate::pes::Stream for NetworkStream {
match &mut self.mode {
Mode::Write {
writer,
ref mut header_written,
header_written,
..
} => {
if !*header_written {
@@ -108,6 +109,30 @@ impl crate::pes::Stream for NetworkStream {
}
}
/// FrameSink sibling to the deprecated Stream impl; both coexist during the
/// 0.18 deprecation window. Caller may pick either at the trait-object
/// boundary — `Box<dyn Stream>` (deprecated) or `Box<dyn FrameSink>` (new).
/// Use `NetworkStream::connect(addr).meta(title)` to construct the write
/// half; calling `FrameSink::write` on `NetworkStream::listen(addr)` returns
/// `StreamReadOnly`.
#[allow(deprecated)] // delegating to deprecated Stream during the 0.18 deprecation window so callers don't see the deprecation twice.
impl crate::pes::FrameSink for NetworkStream {
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
<Self as crate::pes::Stream>::write(self, frame)
}
fn finish(self: Box<Self>) -> io::Result<()> {
// Why: Stream::finish takes &mut self, FrameSink::finish takes Box<Self>.
// Re-borrow inside the box, call Stream::finish, drop the box.
let mut s: Self = *self;
<Self as crate::pes::Stream>::finish(&mut s)
}
fn info(&self) -> &DiscTitle {
<Self as crate::pes::Stream>::info(self)
}
}
// NetworkStream is PES-only — no IOStream/Read/Write byte interface.
#[cfg(test)]
@@ -144,6 +169,7 @@ mod tests {
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: "English".into(),
}),
],
@@ -156,6 +182,7 @@ mod tests {
#[test]
#[ignore] // Requires TCP; may be flaky in CI environments
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSource/FrameSink in follow-up commit.
fn network_pes_roundtrip() {
use crate::pes;
+53
View File
@@ -16,6 +16,7 @@ impl NullStream {
}
}
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSink in follow-up commit.
impl crate::pes::Stream for NullStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
Ok(None)
@@ -30,3 +31,55 @@ impl crate::pes::Stream for NullStream {
&self.disc_title
}
}
/// FrameSink sibling to the deprecated Stream impl; both coexist during the
/// 0.18 deprecation window. Caller may pick either at the trait-object
/// boundary — `Box<dyn Stream>` (deprecated) or `Box<dyn FrameSink>` (new).
/// The deprecation-window callers eventually migrate; this impl exists so
/// new callers can target `FrameSink` without waiting for the rest of the
/// migration to complete.
#[allow(deprecated)] // delegating to deprecated Stream during the 0.18 deprecation window so callers don't see the deprecation twice.
impl crate::pes::FrameSink for NullStream {
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
<Self as crate::pes::Stream>::write(self, frame)
}
fn finish(self: Box<Self>) -> io::Result<()> {
// Why: Stream::finish takes &mut self, FrameSink::finish takes Box<Self>.
// Re-borrow inside the box, call Stream::finish, drop the box.
let mut s: Self = *self;
<Self as crate::pes::Stream>::finish(&mut s)
}
fn info(&self) -> &DiscTitle {
<Self as crate::pes::Stream>::info(self)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::pes::FrameSink;
/// Smallest credible witness that the new FrameSink impl on a concrete
/// `mux/*` sink works through the trait object: build a boxed
/// `dyn FrameSink`, write a frame, finish it. The trait-bridge correctness
/// is what's being verified — not NullStream-specific behaviour.
#[test]
fn frame_sink_via_dyn_object_writes_and_finishes() {
let title = DiscTitle::empty();
let mut sink: Box<dyn FrameSink> = Box::new(NullStream::new(&title));
let frame = crate::pes::PesFrame {
track: 0,
pts: 0,
keyframe: true,
data: vec![0x01, 0x02, 0x03],
};
sink.write(&frame).unwrap();
// info() routes through the trait object.
let _ = sink.info();
// finish() consumes the Box<Self> — must compile and run.
sink.finish().unwrap();
}
}
+26 -22
View File
@@ -167,21 +167,23 @@ pub struct InputOptions {
}
/// Open a PES input stream (produces PES frames).
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSource in follow-up commit.
pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::Stream>> {
let parsed = parse_url(url);
match parsed {
StreamUrl::Disc { .. } => {
// Disc sources should use DiscStream::new() directly.
// The caller opens the drive, inits, scans, then creates the stream.
Err(io::Error::new(
io::ErrorKind::Unsupported,
"Use Drive::open() + Disc::scan() + DiscStream::new() for disc sources",
))
// Disc sources require live SCSI state — caller must use
// `Drive::open() + Disc::scan() + DiscStream::new()` directly.
// Surfaced as a typed error (no English commentary in the
// library; the CLI/UI explains the right entry point).
Err(crate::error::Error::DiscUrlNotDirect.into())
}
StreamUrl::Iso { ref path } => {
validate_file_path(path, "iso")?;
let scan_opts = match &opts.keydb_path {
Some(p) => crate::disc::ScanOptions::with_keydb(p),
Some(p) => crate::disc::ScanOptions {
keydb_path: Some(p.into()),
},
None => crate::disc::ScanOptions::default(),
};
let mut reader = super::iso::IsoSectorReader::open(&path.to_string_lossy())?;
@@ -210,17 +212,13 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
}
StreamUrl::M2ts { ref path } => {
validate_file_path(path, "m2ts")?;
let file = std::fs::File::open(path).map_err(|e| {
io::Error::new(e.kind(), format!("m2ts://{}: {}", path.display(), e))
})?;
let file = std::fs::File::open(path)?;
let reader = std::io::BufReader::with_capacity(IO_BUF_SIZE, file);
Ok(Box::new(M2tsStream::open(reader)?))
}
StreamUrl::Mkv { ref path } => {
validate_file_path(path, "mkv")?;
let file = std::fs::File::open(path).map_err(|e| {
io::Error::new(e.kind(), format!("mkv://{}: {}", path.display(), e))
})?;
let file = std::fs::File::open(path)?;
let reader = std::io::BufReader::with_capacity(IO_BUF_SIZE, file);
Ok(Box::new(MkvStream::open(reader)?))
}
@@ -237,6 +235,7 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
}
/// Open a PES output stream (consumes PES frames).
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSink in follow-up commit.
pub fn output(
url: &str,
title: &crate::disc::DiscTitle,
@@ -245,19 +244,24 @@ pub fn output(
match parsed {
StreamUrl::Mkv { ref path } => {
validate_file_path(path, "mkv")?;
let file = std::fs::File::create(path).map_err(|e| {
io::Error::new(e.kind(), format!("mkv://{}: {}", path.display(), e))
})?;
let writer: Box<dyn super::WriteSeek> =
Box::new(std::io::BufWriter::with_capacity(IO_BUF_SIZE, file));
// Wrap the output in `crate::io::WritebackFile` (bounded-cache
// writeback) so a UHD-scale MKV mux to slow / network-attached
// staging doesn't hit the dirty-page burst pathology that
// sweep already side-steps. BufWriter sits on top to coalesce
// mux's many small EBML element writes.
let writer: Box<dyn super::WriteSeek + Send> =
Box::new(std::io::BufWriter::with_capacity(
IO_BUF_SIZE,
crate::io::WritebackFile::create(path)?,
));
Ok(Box::new(MkvStream::create(writer, title)?))
}
StreamUrl::M2ts { ref path } => {
validate_file_path(path, "m2ts")?;
let file = std::fs::File::create(path).map_err(|e| {
io::Error::new(e.kind(), format!("m2ts://{}: {}", path.display(), e))
})?;
let writer = std::io::BufWriter::with_capacity(IO_BUF_SIZE, file);
let writer = std::io::BufWriter::with_capacity(
IO_BUF_SIZE,
crate::io::WritebackFile::create(path)?,
);
Ok(Box::new(M2tsStream::create(writer, title)?))
}
StreamUrl::Network { ref addr } => {
+25 -1
View File
@@ -60,6 +60,7 @@ impl StdioStream {
}
}
#[allow(deprecated)] // 0.18 trait split: migrate to FrameSource/FrameSink in follow-up commit.
impl crate::pes::Stream for StdioStream {
fn read(&mut self) -> io::Result<Option<crate::pes::PesFrame>> {
self.ensure_header_read()?;
@@ -70,7 +71,7 @@ impl crate::pes::Stream for StdioStream {
}
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
match &mut self.writer {
Some(ref mut w) => {
Some(w) => {
if !self.header_written {
if !self.disc_title.streams.is_empty() {
let m = meta::M2tsMeta::from_title(&self.disc_title);
@@ -104,3 +105,26 @@ impl crate::pes::Stream for StdioStream {
self.header_read || self.writer.is_some()
}
}
/// FrameSink sibling to the deprecated Stream impl; both coexist during the
/// 0.18 deprecation window. Caller may pick either at the trait-object
/// boundary — `Box<dyn Stream>` (deprecated) or `Box<dyn FrameSink>` (new).
/// Use `StdioStream::output(title)` to construct the write half; calling
/// `FrameSink::write` on a `StdioStream::input()` returns `StreamReadOnly`.
#[allow(deprecated)] // delegating to deprecated Stream during the 0.18 deprecation window so callers don't see the deprecation twice.
impl crate::pes::FrameSink for StdioStream {
fn write(&mut self, frame: &crate::pes::PesFrame) -> io::Result<()> {
<Self as crate::pes::Stream>::write(self, frame)
}
fn finish(self: Box<Self>) -> io::Result<()> {
// Why: Stream::finish takes &mut self, FrameSink::finish takes Box<Self>.
// Re-borrow inside the box, call Stream::finish, drop the box.
let mut s: Self = *self;
<Self as crate::pes::Stream>::finish(&mut s)
}
fn info(&self) -> &DiscTitle {
<Self as crate::pes::Stream>::info(self)
}
}
+15 -115
View File
@@ -15,12 +15,6 @@ const TS_PACKET_SIZE: usize = 188;
/// TS sync byte.
const SYNC_BYTE: u8 = 0x47;
/// Size of head buffer for PTS/stream scanning (1 MB).
const SCAN_HEAD_SIZE: usize = 1024 * 1024;
/// Size of tail buffer for last-PTS scanning (2 MB).
const SCAN_TAIL_SIZE: usize = 2 * 1024 * 1024;
/// A reassembled PES packet with timestamp info.
#[derive(Debug)]
pub struct PesPacket {
@@ -104,19 +98,16 @@ pub struct TsDemuxer {
}
impl TsDemuxer {
/// Take the remainder bytes (leftover from last feed() that didn't
/// align to a 192-byte packet boundary).
pub fn take_remainder(&mut self) -> Vec<u8> {
std::mem::take(&mut self.remainder)
}
/// Set the remainder bytes. Used to transfer alignment state from
/// one demuxer to another without losing sync.
pub fn set_remainder(&mut self, data: Vec<u8>) {
self.remainder = data;
}
/// Create a new demuxer tracking the given PIDs.
///
/// Allocates a flat lookup table of `i16` slots — one per possible PID
/// up to `max(8192, max_pid + 1)`. The 8192 floor matches the BD-TS
/// 13-bit PID space (0..0x1FFF); the variable upper bound exists for
/// DVD program streams which may use 16-bit stream IDs above 8191.
/// Worst-case allocation is `u16::MAX × 2 bytes ≈ 128 KB` — bounded by
/// the type, so adversarial input can't drive this beyond predictable
/// limits. Empty `pids` yields max_pid 0; the floor still produces a
/// valid (wholly-unused) table.
pub fn new(pids: &[u16]) -> Self {
let max_pid = pids.iter().copied().max().unwrap_or(0) as usize;
let table_size = (max_pid + 1).max(8192);
@@ -446,6 +437,7 @@ pub fn scan_streams(data: &[u8]) -> Option<Vec<crate::disc::Stream>> {
language: "und".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
})),
0x83 => Some(Stream::Audio(AudioStream {
@@ -455,6 +447,7 @@ pub fn scan_streams(data: &[u8]) -> Option<Vec<crate::disc::Stream>> {
language: "und".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
})),
0x84 | 0xA1 => Some(Stream::Audio(AudioStream {
@@ -464,6 +457,7 @@ pub fn scan_streams(data: &[u8]) -> Option<Vec<crate::disc::Stream>> {
language: "und".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
})),
0x85 | 0x86 => Some(Stream::Audio(AudioStream {
@@ -473,6 +467,7 @@ pub fn scan_streams(data: &[u8]) -> Option<Vec<crate::disc::Stream>> {
language: "und".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
})),
0x82 => Some(Stream::Audio(AudioStream {
@@ -482,6 +477,7 @@ pub fn scan_streams(data: &[u8]) -> Option<Vec<crate::disc::Stream>> {
language: "und".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: crate::disc::LabelPurpose::Normal,
label: String::new(),
})),
0x90 => Some(Stream::Subtitle(SubtitleStream {
@@ -489,6 +485,7 @@ pub fn scan_streams(data: &[u8]) -> Option<Vec<crate::disc::Stream>> {
codec: Codec::Pgs,
language: "und".into(),
forced: false,
qualifier: crate::disc::LabelQualifier::None,
codec_data: None,
})),
_ => None,
@@ -511,103 +508,6 @@ pub fn scan_streams(data: &[u8]) -> Option<Vec<crate::disc::Stream>> {
}
}
// ============================================================
// PTS scanning utilities (for duration detection)
// ============================================================
/// Find the first PTS for a given PID in BD-TS data.
pub fn scan_first_pts(data: &[u8], target_pid: u16) -> Option<i64> {
let mut offset = 0;
while offset + BD_TS_PACKET_SIZE <= data.len() {
if data[offset + 4] != SYNC_BYTE {
offset += 1;
continue;
}
let pid = (((data[offset + 5] & 0x1F) as u16) << 8) | data[offset + 6] as u16;
let pusi = data[offset + 5] & 0x40 != 0;
if pid == target_pid && pusi {
let ts = &data[offset + 4..offset + BD_TS_PACKET_SIZE];
let afc = (ts[3] >> 4) & 0x03;
let payload_start = if afc == 3 { 5 + ts[4] as usize } else { 4 };
if payload_start < TS_PACKET_SIZE {
let payload = &ts[payload_start..];
if payload.len() >= 14 && payload[0] == 0 && payload[1] == 0 && payload[2] == 1 {
let pts_dts_flags = (payload[7] >> 6) & 0x03;
if pts_dts_flags >= 2 {
return parse_timestamp(&payload[9..14]);
}
}
}
}
offset += BD_TS_PACKET_SIZE;
}
None
}
/// Find the last PTS for a given PID in BD-TS data.
pub fn scan_last_pts(data: &[u8], target_pid: u16) -> Option<i64> {
let mut last_pts = None;
let mut offset = 0;
while offset + BD_TS_PACKET_SIZE <= data.len() {
if data[offset + 4] != SYNC_BYTE {
offset += 1;
continue;
}
let pid = (((data[offset + 5] & 0x1F) as u16) << 8) | data[offset + 6] as u16;
let pusi = data[offset + 5] & 0x40 != 0;
if pid == target_pid && pusi {
let ts = &data[offset + 4..offset + BD_TS_PACKET_SIZE];
let afc = (ts[3] >> 4) & 0x03;
let payload_start = if afc == 3 { 5 + ts[4] as usize } else { 4 };
if payload_start < TS_PACKET_SIZE {
let payload = &ts[payload_start..];
if payload.len() >= 14 && payload[0] == 0 && payload[1] == 0 && payload[2] == 1 {
let pts_dts_flags = (payload[7] >> 6) & 0x03;
if pts_dts_flags >= 2 {
last_pts = parse_timestamp(&payload[9..14]);
}
}
}
}
offset += BD_TS_PACKET_SIZE;
}
last_pts
}
/// Scan an m2ts file for duration by reading first and last PTS.
/// Returns duration in seconds, or None if PTS cannot be found.
/// The reader position is restored after scanning.
pub fn scan_duration<R: std::io::Read + std::io::Seek>(r: &mut R, video_pid: u16) -> Option<f64> {
use std::io::SeekFrom;
let start_pos = r.stream_position().ok()?;
// Read first 1MB for first PTS
let mut head_buf = vec![0u8; SCAN_HEAD_SIZE];
r.seek(SeekFrom::Start(0)).ok()?;
let head_n = r.read(&mut head_buf).ok()?;
let first_pts = scan_first_pts(&head_buf[..head_n], video_pid)?;
// Read last 2MB for last PTS (aligned to 192-byte boundary)
let file_size = r.seek(SeekFrom::End(0)).ok()?;
let tail_size: u64 = SCAN_TAIL_SIZE as u64;
let raw_pos = file_size.saturating_sub(tail_size);
let seek_pos = (raw_pos / BD_TS_PACKET_SIZE as u64) * BD_TS_PACKET_SIZE as u64;
r.seek(SeekFrom::Start(seek_pos)).ok()?;
let mut tail_buf = vec![0u8; tail_size as usize];
let tail_n = r.read(&mut tail_buf).ok()?;
let last_pts = scan_last_pts(&tail_buf[..tail_n], video_pid)?;
// Restore reader position
let _ = r.seek(SeekFrom::Start(start_pos));
if last_pts > first_pts {
Some((last_pts - first_pts) as f64 / 90000.0)
} else {
None
}
}
#[cfg(test)]
mod tests {
use super::*;
+6 -5
View File
@@ -15,6 +15,7 @@ pub struct TsMuxer<W: Write> {
continuity: Vec<u8>, // per-PID continuity counter (0-15)
codec_privates: Vec<Option<Vec<u8>>>, // per-track codec_private (for video parameter sets)
params_written: Vec<bool>, // per-track: have we written parameter sets?
base_pts_ns: Option<i64>,
}
impl<W: Write> TsMuxer<W> {
@@ -26,6 +27,7 @@ impl<W: Write> TsMuxer<W> {
continuity: vec![0u8; n],
codec_privates: vec![None; n],
params_written: vec![false; n],
base_pts_ns: None,
}
}
@@ -44,6 +46,9 @@ impl<W: Write> TsMuxer<W> {
if track >= self.pids.len() {
return Ok(()); // unknown track, skip
}
let base = *self.base_pts_ns.get_or_insert(pts_ns);
let pts_ns = pts_ns - base;
let pid = self.pids[track];
let is_video = (0x1011..=0x101F).contains(&pid);
@@ -222,11 +227,7 @@ fn hvcc_to_annex_b(hvcc: &[u8]) -> Option<Vec<u8>> {
}
}
if out.is_empty() {
None
} else {
Some(out)
}
if out.is_empty() { None } else { Some(out) }
}
/// Convert length-prefixed NALUs (4-byte BE length + NAL) to Annex B
-115
View File
@@ -1,115 +0,0 @@
//! TsDemuxReader — reads from any source, demuxes BD-TS, produces PES frames.
//!
//! Wraps any Read source with a TsDemuxer + CodecParsers.
//! One implementation used by M2TS, Network, Stdio, and any other BD-TS input.
use super::codec::{self, CodecParser};
use super::ts::TsDemuxer;
use crate::disc::Stream as DiscStream;
use crate::pes::PesFrame;
use std::collections::VecDeque;
use std::io::{self, Read};
const READ_BUF_SIZE: usize = 192 * 1024; // 1024 BD-TS packets
/// Generic BD-TS → PES frame reader.
pub struct TsDemuxReader<R: Read> {
reader: R,
demuxer: TsDemuxer,
parsers: Vec<(u16, Box<dyn CodecParser>)>,
pid_to_track: Vec<(u16, usize)>,
pending: VecDeque<PesFrame>,
buf: Vec<u8>,
eof: bool,
}
impl<R: Read> TsDemuxReader<R> {
/// Create from a reader and stream metadata.
pub fn new(reader: R, streams: &[DiscStream]) -> Self {
let mut pids = Vec::new();
let mut parsers: Vec<(u16, Box<dyn CodecParser>)> = Vec::new();
let mut pid_to_track = Vec::new();
for (i, s) in streams.iter().enumerate() {
let (pid, c) = match s {
DiscStream::Video(v) => (v.pid, v.codec),
DiscStream::Audio(a) => (a.pid, a.codec),
DiscStream::Subtitle(s) => (s.pid, s.codec),
};
pids.push(pid);
pid_to_track.push((pid, i));
parsers.push((pid, codec::parser_for_codec(c)));
}
Self {
reader,
demuxer: TsDemuxer::new(&pids),
parsers,
pid_to_track,
pending: VecDeque::new(),
buf: vec![0u8; READ_BUF_SIZE],
eof: false,
}
}
/// Get the next PES frame. Returns None at EOF.
pub fn next_frame(&mut self) -> io::Result<Option<PesFrame>> {
if let Some(frame) = self.pending.pop_front() {
return Ok(Some(frame));
}
if self.eof {
return Ok(None);
}
loop {
let n = self.reader.read(&mut self.buf)?;
if n == 0 {
self.eof = true;
return Ok(None);
}
let packets = self.demuxer.feed(&self.buf[..n]);
for pes in &packets {
if let Some((_, track)) = self.pid_to_track.iter().find(|(pid, _)| *pid == pes.pid)
{
if let Some((_, parser)) =
self.parsers.iter_mut().find(|(pid, _)| *pid == pes.pid)
{
for frame in parser.parse(pes) {
self.pending
.push_back(PesFrame::from_codec_frame(*track, frame));
}
}
}
}
if let Some(frame) = self.pending.pop_front() {
return Ok(Some(frame));
}
}
}
/// Codec private data for a track.
pub fn codec_private(&self, track: usize) -> Option<Vec<u8>> {
let pid = self
.pid_to_track
.iter()
.find(|(_, idx)| *idx == track)
.map(|(pid, _)| *pid)?;
self.parsers
.iter()
.find(|(p, _)| *p == pid)
.and_then(|(_, parser)| parser.codec_private())
}
/// True when all primary video tracks have codec_private.
pub fn headers_ready(&self, streams: &[DiscStream]) -> bool {
for (idx, s) in streams.iter().enumerate() {
if let DiscStream::Video(v) = s {
if !v.secondary && self.codec_private(idx).is_none() {
return false;
}
}
}
true
}
}
+303 -7
View File
@@ -78,31 +78,149 @@ impl PesFrame {
}
}
/// A stream. Read from it or write to it. Not both.
/// Deprecated; use [`FrameSource`] for read-only sources or [`FrameSink`]
/// for write-only sinks. The runtime direction-error semantics
/// (`StreamReadOnly` / `StreamWriteOnly` from a wrong-direction call) are
/// removed in 0.18 — direction is type-checked.
#[deprecated(
since = "0.18.0",
note = "use FrameSource (read-only) or FrameSink (write-only) instead"
)]
pub trait Stream {
/// Read the next frame. Returns None at end of stream.
/// Read the next frame, or `Ok(None)` at end of stream.
fn read(&mut self) -> std::io::Result<Option<PesFrame>>;
/// Write a frame.
/// Write a frame to the sink.
fn write(&mut self, frame: &PesFrame) -> std::io::Result<()>;
/// Finalize (flush, write index, close).
/// Finalize the stream: flush buffered frames, write any container
/// index (MKV `Cues`), close the underlying file/socket. Idempotent
/// for read-only streams (no-op).
fn finish(&mut self) -> std::io::Result<()>;
/// Stream metadata.
/// Stream metadata. Stable across reads — implementors must return a
/// consistent reference for the lifetime of the stream.
fn info(&self) -> &crate::disc::DiscTitle;
/// Codec initialization data for a track (SPS/PPS, etc).
/// Codec initialization data for a track (SPS/PPS, AC-3 fscod, etc.).
/// `None` for tracks that don't need codec_private (raw passthrough).
fn codec_private(&self, _track: usize) -> Option<Vec<u8>> {
None
}
/// True when codec_private is available for all video tracks.
/// True when `codec_private` is available for every video track —
/// callers buffer input frames until this flips, since some output
/// formats (MKV) can't write frames without codec init data.
fn headers_ready(&self) -> bool {
true
}
}
/// Read-only source of PES frames.
///
/// Replaces the read half of the deprecated [`Stream`] trait. Implementors
/// produce frames via [`read`](FrameSource::read) and never accept writes —
/// passing a `FrameSource` where a sink is expected is a compile error,
/// not a runtime `E9001`.
///
/// `info()` returns the source's `DiscTitle` metadata (track list, codec
/// info, duration). It must be stable across the lifetime of the source.
///
/// `codec_private(track)` exposes per-track codec initialization data
/// (H.264 SPS/PPS, HEVC VPS/SPS/PPS, AC-3 fscod, etc.) that downstream
/// muxers may need before any frame is written. `headers_ready()` returns
/// false until enough input frames have been seen to populate every video
/// track's codec-private blob — callers buffer frames they read until
/// `headers_ready()` returns true.
pub trait FrameSource: Send {
/// Read the next frame, or `Ok(None)` at end of stream.
fn read(&mut self) -> std::io::Result<Option<PesFrame>>;
/// Source metadata. Stable across reads — implementors must return a
/// consistent reference for the lifetime of the source.
fn info(&self) -> &crate::disc::DiscTitle;
/// Codec initialization data for a track (SPS/PPS, AC-3 fscod, etc.).
/// `None` for tracks that don't need codec_private (raw passthrough).
fn codec_private(&self, _track: usize) -> Option<Vec<u8>> {
None
}
/// True when `codec_private` is available for every video track —
/// callers buffer input frames until this flips, since some output
/// formats (MKV) can't write frames without codec init data.
fn headers_ready(&self) -> bool {
true
}
}
/// Write-only sink of PES frames.
///
/// Replaces the write half of the deprecated [`Stream`] trait. Implementors
/// accept frames via [`write`](FrameSink::write) and finalize via
/// [`finish`](FrameSink::finish) — passing a `FrameSink` where a source is
/// expected is a compile error, not a runtime `E9000`.
///
/// `finish` takes `Box<Self>` (rather than `&mut self` like `Stream::finish`)
/// so that finalization is a one-shot terminal operation: callers cannot use
/// the sink after `finish` returns. This is the standard idiom for terminal
/// methods on `dyn Trait` objects.
pub trait FrameSink: Send {
/// Write a frame to the sink.
fn write(&mut self, frame: &PesFrame) -> std::io::Result<()>;
/// Finalize the sink: flush buffered frames, write any container index
/// (e.g. MKV `Cues`), close the underlying file/socket. Consumes the
/// sink — callers cannot use it afterwards.
fn finish(self: Box<Self>) -> std::io::Result<()>;
/// Sink metadata. Stable across writes — implementors must return a
/// consistent reference for the lifetime of the sink.
fn info(&self) -> &crate::disc::DiscTitle;
}
// Bridge: any **`Send`** type implementing the deprecated `Stream` trait
// is also a `FrameSource`. This lets existing concrete `Stream` impls in
// `mux/*` satisfy `FrameSource` bounds without per-type migration during
// the 0.18 deprecation window.
//
// **Send caveat (read me before tightening `Stream` itself).** This
// blanket carries a `T: Send` bound rather than promoting `Send` to a
// supertrait of `Stream`, because not every concrete in-tree `Stream`
// impl is `Send`: `MkvStream` and `M2tsStream` carry `Box<dyn Read>`
// and `Box<dyn Write>` fields whose trait objects don't include `Send`.
// Adding `Stream: Send` would force a wider audit (every `Box<dyn Read>`
// becomes `Box<dyn Read + Send>`) than this commit is taking on, and
// the type-level migration target is `FrameSource` / `FrameSink`
// directly anyway. Consequence: coercing a non-Send `Box<dyn Stream>`
// (the return shape of `crate::mux::input` / `output`) to
// `Box<dyn FrameSource>` will fail with a `T: Send` trait-bound error.
// The fix on the consumer side is to construct a Send-compliant
// `FrameSource` / `FrameSink` directly rather than relying on this
// bridge for non-Send streams.
//
// Note: `FrameSink` cannot be blanket-impl'd from `Stream` because
// `Stream::finish` takes `&mut self` while `FrameSink::finish` takes
// `Box<Self>`; concrete types will be migrated in a follow-up commit.
#[allow(deprecated)]
impl<T: Stream + Send + ?Sized> FrameSource for T {
fn read(&mut self) -> std::io::Result<Option<PesFrame>> {
<Self as Stream>::read(self)
}
fn info(&self) -> &crate::disc::DiscTitle {
<Self as Stream>::info(self)
}
fn codec_private(&self, track: usize) -> Option<Vec<u8>> {
<Self as Stream>::codec_private(self, track)
}
fn headers_ready(&self) -> bool {
<Self as Stream>::headers_ready(self)
}
}
/// Wraps any output stream and counts bytes written.
///
/// Progress tracking is a CLI concern — streams don't know their size.
@@ -115,11 +233,13 @@ pub trait Stream {
/// let pct = output.bytes_written() as f64 / total as f64;
/// }
/// ```
#[allow(deprecated)]
pub struct CountingStream {
inner: Box<dyn Stream>,
written: u64,
}
#[allow(deprecated)]
impl CountingStream {
pub fn new(inner: Box<dyn Stream>) -> Self {
Self { inner, written: 0 }
@@ -131,6 +251,7 @@ impl CountingStream {
}
}
#[allow(deprecated)]
impl Stream for CountingStream {
fn read(&mut self) -> std::io::Result<Option<PesFrame>> {
self.inner.read()
@@ -157,3 +278,178 @@ impl Stream for CountingStream {
self.inner.headers_ready()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::disc::DiscTitle;
fn make_frame(track: usize, pts: i64) -> PesFrame {
PesFrame {
track,
pts,
keyframe: track == 0 && pts == 0,
data: vec![track as u8, (pts & 0xff) as u8, 0xAA],
}
}
/// Direct `FrameSource` impl (not via the deprecated `Stream` blanket).
struct MockSource {
frames: std::vec::IntoIter<PesFrame>,
title: DiscTitle,
}
impl MockSource {
fn new(frames: Vec<PesFrame>) -> Self {
Self {
frames: frames.into_iter(),
title: DiscTitle::empty(),
}
}
}
impl FrameSource for MockSource {
fn read(&mut self) -> std::io::Result<Option<PesFrame>> {
Ok(self.frames.next())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
/// Direct `FrameSink` impl (not via the deprecated `Stream` blanket).
struct MockSink {
collected: Vec<PesFrame>,
title: DiscTitle,
}
impl MockSink {
fn new() -> Self {
Self {
collected: Vec::new(),
title: DiscTitle::empty(),
}
}
}
impl FrameSink for MockSink {
fn write(&mut self, frame: &PesFrame) -> std::io::Result<()> {
self.collected.push(frame.clone());
Ok(())
}
fn finish(self: Box<Self>) -> std::io::Result<()> {
// Drop self; in real sinks this is where flush/fsync/close happens.
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
/// Variant of `MockSink` whose `finish` returns the collected frames so
/// the test can assert on them after the consuming `Box<Self>` call.
struct CollectingSink {
collected: Vec<PesFrame>,
title: DiscTitle,
}
impl CollectingSink {
fn new() -> Self {
Self {
collected: Vec::new(),
title: DiscTitle::empty(),
}
}
}
impl FrameSink for CollectingSink {
fn write(&mut self, frame: &PesFrame) -> std::io::Result<()> {
self.collected.push(frame.clone());
Ok(())
}
fn finish(self: Box<Self>) -> std::io::Result<()> {
// Real CollectingSink consumers would expose `take()` before
// finish; this trait method just confirms the boxed signature
// compiles and runs.
Ok(())
}
fn info(&self) -> &DiscTitle {
&self.title
}
}
#[test]
fn frame_source_yields_frames_then_eof() {
let frames = vec![make_frame(0, 0), make_frame(1, 1_000), make_frame(0, 2_000)];
let mut src = MockSource::new(frames.clone());
let f0 = src.read().unwrap().expect("first frame");
assert_eq!(f0.track, frames[0].track);
assert_eq!(f0.pts, frames[0].pts);
assert!(f0.keyframe);
let f1 = src.read().unwrap().expect("second frame");
assert_eq!(f1.track, frames[1].track);
assert_eq!(f1.pts, frames[1].pts);
let f2 = src.read().unwrap().expect("third frame");
assert_eq!(f2.track, frames[2].track);
assert_eq!(f2.pts, frames[2].pts);
assert!(src.read().unwrap().is_none());
assert!(src.read().unwrap().is_none()); // idempotent at EOF
}
#[test]
fn frame_sink_collects_then_finishes() {
let mut sink = MockSink::new();
let frames = [make_frame(0, 0), make_frame(1, 100), make_frame(2, 200)];
for f in &frames {
sink.write(f).unwrap();
}
assert_eq!(sink.collected.len(), 3);
assert_eq!(sink.collected[0].pts, 0);
assert_eq!(sink.collected[1].pts, 100);
assert_eq!(sink.collected[2].pts, 200);
// Box-and-finish — the `self: Box<Self>` shape must compile and run.
Box::new(sink).finish().unwrap();
}
#[test]
fn frame_sink_via_dyn_object() {
let frames = [make_frame(0, 0), make_frame(0, 33)];
let mut sink: Box<dyn FrameSink> = Box::new(CollectingSink::new());
for f in &frames {
sink.write(f).unwrap();
}
// info() routes through the trait object.
let _ = sink.info();
sink.finish().unwrap();
}
/// The deprecated blanket impl: any concrete `Stream` should also act as
/// a `FrameSource`. `NullStream` has the simplest constructor of every
/// concrete `Stream` impl in `mux/*`, so it's the smallest credible
/// witness that the bridge compiles and dispatches correctly.
#[test]
#[allow(deprecated)]
fn deprecated_stream_satisfies_frame_source() {
let title = DiscTitle::empty();
let mut null = crate::mux::NullStream::new(&title);
let src: &mut dyn FrameSource = &mut null;
// NullStream::read returns Ok(None) — it's a write-only sink.
assert!(src.read().unwrap().is_none());
// info() forwards through the blanket impl.
let _ = src.info();
assert!(src.headers_ready());
}
}
+6 -6
View File
@@ -96,8 +96,8 @@ impl Mt1959 {
if result.bytes_transferred != expected {
return Err(Error::ScsiError {
opcode: SCSI_READ_BUFFER,
status: 0xFF,
sense_key: 0,
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
Ok(result.bytes_transferred)
@@ -169,8 +169,8 @@ impl Mt1959 {
}
Err(Error::ScsiError {
opcode: SCSI_READ_BUFFER,
status: 0xFF,
sense_key: 0,
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
})
}
@@ -275,8 +275,8 @@ impl Mt1959 {
{
return Err(Error::ScsiError {
opcode: SCSI_READ_BUFFER,
status: 0xFF,
sense_key: 0,
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
addr = addr.wrapping_add(PROBE_STEP);
+125
View File
@@ -0,0 +1,125 @@
//! Pipeline-progress reporting for the rip pipeline.
//!
//! v0.13.16 architecture rule: ONE progress signal type. Every long-running
//! pipeline operation (`Disc::copy`, `Disc::patch`, `verify_title`) emits the same
//! `PassProgress` shape via the `Progress` trait. Consumers (autorip) compute
//! a single `PipelineStats` derived view and never reach into per-pass
//! internals.
//!
//! Why this matters: pre-0.13.16 the API leaked `pos`, `bytes_good`,
//! `work_done`, `bytes_pending`, `Finished/NonTrimmed` mapfile semantics —
//! and consumers reinvented the math each time they wanted a percentage.
//! UIs ended up reading one source while server-side computed from another,
//! producing wrong percentages without anyone noticing.
/// Identifies which pipeline phase the progress event belongs to.
///
/// Consumers can render a phase-specific label (e.g. "Sweep", "Trim
/// (reverse)", "Scrape", "Mux") or just use a generic "Pass N" label.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PassKind {
/// `Disc::copy` — initial sweep across the entire disc.
Sweep,
/// `Disc::patch` retry pass with `block_sectors >= 2`. `reverse=true`
/// means walking bad ranges from highest to lowest LBA.
Trim { reverse: bool },
/// `Disc::patch` final pass at 1 sector per block.
Scrape { reverse: bool },
/// Demux ISO → output (MKV / M2TS / network). Single phase that runs
/// after all rip passes complete.
Mux,
/// Sector verification — reads every sector and classifies health.
Verify,
}
/// One progress sample from a pipeline phase.
///
/// `work_done / work_total` is the per-pass percentage — always 0..=100%
/// regardless of which kind of pass is running. `bytes_good_total` is the
/// cumulative count of confirmed-clean bytes across the whole rip; useful
/// for the "data recovered" stat the user sees.
///
/// For `PassKind::Verify`, the fields map as follows:
/// - `work_done` = sectors read so far
/// - `work_total` = total sectors in title
/// - `bytes_good_total` = good + slow + recovered sectors × 2048
/// - `bytes_unreadable_total` = bad sectors × 2048
/// - `bytes_pending_total` = 0 (verify processes sequentially, nothing pending)
#[derive(Debug, Clone, Copy)]
pub struct PassProgress {
pub kind: PassKind,
pub work_done: u64,
pub work_total: u64,
pub bytes_good_total: u64,
pub bytes_unreadable_total: u64,
pub bytes_pending_total: u64,
pub bytes_total_disc: u64,
pub disc_duration_secs: Option<f64>,
/// How many bytes of the worst-case damage (unreadable + pending) fall
/// within the main title's extents. Zero means none of the damage
/// affects the main movie — it's all in extras/menus.
pub bytes_bad_in_main_title: u64,
/// Main title duration in seconds. Same as disc_duration_secs when the
/// disc has one dominant title, but separate so consumers can show both.
pub main_title_duration_secs: Option<f64>,
/// Main title size in bytes (sum of extent sizes).
pub main_title_size_bytes: Option<u64>,
}
impl PassProgress {
/// Percentage of work completed for this pass (0..=100).
///
/// Returns `100.0` if `work_total` is zero to avoid division by zero.
pub fn work_pct(&self) -> f64 {
if self.work_total == 0 {
return 100.0;
}
self.work_done as f64 / self.work_total as f64 * 100.0
}
/// Percentage of the disc that is confirmed clean (0..=100).
///
/// Computed from `bytes_good_total / bytes_total_disc`.
pub fn good_pct(&self) -> f64 {
if self.bytes_total_disc == 0 {
return 100.0;
}
self.bytes_good_total as f64 / self.bytes_total_disc as f64 * 100.0
}
/// Percentage of the disc that is unreadable (0..=100).
pub fn bad_pct(&self) -> f64 {
if self.bytes_total_disc == 0 {
return 0.0;
}
self.bytes_unreadable_total as f64 / self.bytes_total_disc as f64 * 100.0
}
/// Percentage of the disc that is still pending (not yet attempted or needs retry).
pub fn pending_pct(&self) -> f64 {
if self.bytes_total_disc == 0 {
return 0.0;
}
self.bytes_pending_total as f64 / self.bytes_total_disc as f64 * 100.0
}
}
/// A consumer of pipeline progress events. Library code calls
/// `Progress::report` once per inner-loop iteration (throttling is the
/// consumer's job — `report` is cheap; the library doesn't gate it).
///
/// Returns `true` to continue, `false` to request early stop.
///
/// No `Send`/`Sync` bound — `report` is always called from the same thread
/// running the pipeline, so closures with non-`Sync` captures (e.g.
/// `RefCell<PassProgressState>`) work directly. Blanket impl below lets
/// callers pass closures without explicit struct types.
pub trait Progress {
fn report(&self, p: &PassProgress) -> bool;
}
impl<F: Fn(&PassProgress) -> bool> Progress for F {
fn report(&self, p: &PassProgress) -> bool {
(self)(p)
}
}
+408 -143
View File
@@ -1,16 +1,32 @@
//! Linux SCSI transport via SG_IO ioctl.
//! Linux SCSI transport via synchronous blocking SG_IO ioctl.
//!
//! `execute()` is one syscall: `ioctl(fd, SG_IO, &hdr)` blocks until the
//! kernel completes the command (success, error, or its own timeout).
//! No userspace abort, no fd close+reopen, no SG_SCSI_RESET escalation —
//! the kernel SCSI mid-layer's `scsi_eh.rst` ladder
//! (ABORT TASK → LUN RESET → BUS RESET → HOST RESET) runs internally
//! when `hdr.timeout` expires, and by the time the ioctl returns the
//! kernel has already done what it can.
//!
//! This matches what every reference project does: MakeMKV (8 s sync
//! ioctl), sg_dd (60 s sync ioctl), the kernel default for SCSI block
//! devices (30 s `/sys/.../timeout`). See
//! the SCSI architecture audit (2026-04-26) for the full primary-source
//! references.
//!
//! Pre-0.13.20 we ran an async `write() + poll(1.5s) + close-on-timeout +
//! bg reopen` pattern. That abandoned slow-but-alive commands faster than
//! the drive could drain its internal queue, deepening the wedge
//! pattern on the LG BU40N. Reverted in 0.13.20.
use super::{DataDirection, ScsiResult, ScsiTransport};
use crate::error::{Error, Result};
use std::path::Path;
const SG_IO: u32 = 0x2285;
const SG_SCSI_RESET: u32 = 0x2284;
const SG_SCSI_RESET_DEVICE: i32 = 1;
const SG_DXFER_NONE: i32 = -1;
const SG_DXFER_TO_DEV: i32 = -2;
const SG_DXFER_FROM_DEV: i32 = -3;
const SG_FLAG_DIRECT_IO: u32 = 1;
const SG_FLAG_Q_AT_HEAD: u32 = 0x10;
#[repr(C)]
@@ -41,135 +57,61 @@ struct sg_io_hdr {
}
// Compile-time validation: sg_io_hdr must match the kernel's layout.
// 64 bytes on 64-bit, 44 bytes on 32-bit (pointer-size dependent).
// 88 bytes on 64-bit, 64 bytes on 32-bit (pointer-size dependent).
#[cfg(target_pointer_width = "64")]
const _: () = assert!(std::mem::size_of::<sg_io_hdr>() == 88);
#[cfg(target_pointer_width = "32")]
const _: () = assert!(std::mem::size_of::<sg_io_hdr>() == 64);
pub struct SgIoTransport {
fd: i32,
pub fd: i32,
device_path: std::path::PathBuf,
pub fd_recovery: std::sync::Arc<std::sync::atomic::AtomicI32>,
}
impl SgIoTransport {
/// Open a SCSI device for use. Resets the drive first to ensure
/// a known good state, then opens a fresh fd for commands.
/// Open a SCSI device for use.
pub fn open(device: &Path) -> Result<Self> {
let device = Self::resolve_to_sg(device);
Self::reset(&device)?;
let c_path = Self::to_c_path(&device);
let fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK,
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return Self::open_error(&device);
}
Ok(SgIoTransport { fd })
Ok(SgIoTransport {
fd,
device_path: device,
fd_recovery: std::sync::Arc::new(std::sync::atomic::AtomicI32::new(-1)),
})
}
/// Reset the drive to a known good state — equivalent to unplug/replug.
/// After reset, the drive is clean and no fd is held open.
/// Clean up kernel SG_IO state and unlock the tray. NOT a hardware
/// reset — purely software cleanup before this process opens the
/// device for real work.
///
/// ## Why each step exists
/// When a previous process is killed (SIGKILL) mid-SG_IO, the kernel
/// may hold queued commands against the dead fd, and `Drop` never
/// ran so the tray may still be locked via PREVENT MEDIUM REMOVAL.
/// This routine handles both: open + close flushes the kernel SG
/// queue (sg_release cancels commands tied to the fd), the 2 s sleep
/// gives the kernel time to finish that cleanup, then a fresh fd
/// sends ALLOW MEDIUM REMOVAL to clear any stale tray lock.
///
/// When a process is killed (SIGKILL/kill -9) mid-SG_IO ioctl, two things
/// go wrong: (1) the kernel's SG driver may have stale pending commands
/// queued for the dead process's fd, and (2) the drive firmware may still
/// be mid-operation (seeking, reading, processing a vendor command).
///
/// A new process opening the same /dev/sg* device gets a fresh fd, but the
/// kernel doesn't automatically abort the dead process's commands — the
/// drive can appear hung on the first SCSI command.
///
/// Additionally, killed processes skip Drop, so the tray may be locked
/// via PREVENT MEDIUM REMOVAL with no process alive to unlock it.
///
/// ## Sequence
///
/// 1. **open** — allocates kernel SG state for this fd
/// 2. **close** — triggers kernel cleanup: aborts any pending SG_IO
/// commands associated with this fd. The key operation —
/// the kernel's sg_release() cancels queued commands.
/// 3. **sleep 2s** — the drive firmware needs time to finish/abort whatever
/// it was doing when the previous process died. Without
/// this, the next command may block on drive-internal state.
/// 4. **open** — fresh fd with no stale commands in the kernel queue
/// 5. **unlock** — ALLOW MEDIUM REMOVAL (CDB 0x1E, prevent=0). Clears
/// any tray lock left by a killed process that never
/// ran its Drop/cleanup.
/// 6. **TUR** — TEST UNIT READY (CDB 0x00) with 3s timeout. If the
/// drive responds, it's in a good state.
/// 7. **escalate** — if TUR fails:
/// - SG_SCSI_RESET (device level) — kernel sends a SCSI
/// bus reset to the device, clearing all firmware state.
/// - STOP + START UNIT (CDB 0x1B) — power-cycles the
/// drive's logical unit, like pressing the eject button
/// and reinserting.
/// 8. **close** — release the fd. Drive is clean, nobody holds it.
pub fn reset(device: &Path) -> Result<()> {
let c_path = Self::to_c_path(device);
// Step 1-2: open + close — flush stale kernel SG_IO state
let probe_fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK,
)
};
if probe_fd >= 0 {
unsafe { libc::close(probe_fd) };
}
// Step 3: let drive settle
std::thread::sleep(std::time::Duration::from_secs(2));
// Step 4: open clean fd
let fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK,
)
};
if fd < 0 {
return Self::open_error(device);
}
// Step 5: unlock tray
let _ = Self::raw_command(fd, &[0x1E, 0, 0, 0, 0, 0], 3_000);
// Step 6: TUR — if drive responds, we're done
if Self::raw_command(fd, &[0, 0, 0, 0, 0, 0], 3_000).is_err() {
// Step 7: escalate — SG_SCSI_RESET
let mut reset_type: i32 = SG_SCSI_RESET_DEVICE;
unsafe { libc::ioctl(fd, SG_SCSI_RESET as _, &mut reset_type) };
std::thread::sleep(std::time::Duration::from_secs(3));
if Self::raw_command(fd, &[0, 0, 0, 0, 0, 0], 3_000).is_err() {
// STOP + START
let _ = Self::raw_command(fd, &[0x1B, 0, 0, 0, 0x00, 0], 3_000);
std::thread::sleep(std::time::Duration::from_secs(1));
let _ = Self::raw_command(fd, &[0x1B, 0, 0, 0, 0x01, 0], 3_000);
std::thread::sleep(std::time::Duration::from_secs(3));
let _ = Self::raw_command(fd, &[0, 0, 0, 0, 0, 0], 3_000);
}
}
// Step 8: close — drive is clean
unsafe { libc::close(fd) };
Ok(())
}
/// We do NOT verify the drive with TUR or escalate to SG_SCSI_RESET /
/// STOP+START UNIT. Both escalations were tried in 0.13.0–0.13.5
/// against the LG BU40N (Initio USB-SATA bridge); both failed to
/// recover wedged drives and made the wedge worse — see
/// the BU40N wedge recovery postmortem (2026-04-25).
fn open_error<T>(device: &Path) -> Result<T> {
let err = std::io::Error::last_os_error();
Err(if err.kind() == std::io::ErrorKind::PermissionDenied {
Error::DevicePermission {
path: format!(
"{}: permission denied (try running as root)",
device.display()
),
path: device.display().to_string(),
}
} else {
Error::DeviceNotFound {
@@ -195,7 +137,7 @@ impl SgIoTransport {
hdr.flags = SG_FLAG_Q_AT_HEAD;
let ret = unsafe { libc::ioctl(fd, SG_IO as _, &mut hdr as *mut sg_io_hdr) };
if ret < 0 || hdr.status != 0 {
if ret < 0 || hdr.status != 0 || hdr.host_status != 0 || hdr.driver_status != 0 {
Err(())
} else {
Ok(())
@@ -239,13 +181,42 @@ impl SgIoTransport {
impl Drop for SgIoTransport {
fn drop(&mut self) {
// Unlock tray before closing — don't leave it locked
let _ = Self::raw_command(self.fd, &[0x1E, 0, 0, 0, 0, 0], 3_000);
unsafe { libc::close(self.fd) };
if self.fd >= 0 {
// Unlock tray before closing — don't leave it locked.
let _ = Self::raw_command(self.fd, &[0x1E, 0, 0, 0, 0, 0], 3_000);
unsafe { libc::close(self.fd) };
}
}
}
impl ScsiTransport for SgIoTransport {
/// Execute a SCSI command via synchronous blocking SG_IO.
///
/// One syscall: `ioctl(fd, SG_IO, &hdr)`. The kernel honors
/// `hdr.timeout` and runs its own ABORT TASK → LUN RESET → BUS
/// RESET → HOST RESET escalation if the device times out (per
/// `Documentation/scsi/scsi_eh.rst`). By the time this returns,
/// the kernel has done its recovery work.
///
/// Errors we surface to caller (any of these = command failed):
///
/// - ioctl returned -1 → `Error::IoError` (kernel-level failure)
/// - `hdr.host_status` != 0 OR `(hdr.driver_status & ~DRIVER_SENSE)` != 0
/// → `Error::ScsiError { status: 0xFF, sense_key: 0, asc: 0, ascq: 0 }`
/// (real transport-layer failure: kernel timeout, bridge wedge, bus error)
/// - `hdr.status` != 0 (typically `0x02` CHECK CONDITION) →
/// `Error::ScsiError { status, sense_key, asc, ascq }` carrying the
/// drive's full SPC-4 sense triple. Callers route on
/// `is_medium_error()`, `is_unit_attention()`, etc.
///
/// Note: SG's `DRIVER_SENSE` (0x08) bit indicates *sense data is
/// attached* — it's set on every CHECK CONDITION reply. It is **not**
/// a transport failure; pre-0.13.23 we conflated it with one and
/// silently lost every drive-reported error reason. The mask in the
/// transport-error check below is the fix.
///
/// Caller's `data` buffer is mutated only on success; partial
/// transfers are reported via `bytes_transferred = data.len() - resid`.
fn execute(
&mut self,
cdb: &[u8],
@@ -253,24 +224,50 @@ impl ScsiTransport for SgIoTransport {
data: &mut [u8],
timeout_ms: u32,
) -> Result<ScsiResult> {
let mut sense = [0u8; 32];
let exec_t0 = std::time::Instant::now();
let opcode = cdb[0];
tracing::trace!(
target: "freemkv::scsi",
phase = "enter",
opcode = opcode,
timeout_ms,
data_len = data.len(),
fd = self.fd,
"SgIoTransport::execute"
);
// Check if a background recovery has produced a new fd.
let recovered = self
.fd_recovery
.swap(-1, std::sync::atomic::Ordering::Acquire);
if recovered >= 0 {
// Close the old fd if it's still valid.
if self.fd >= 0 {
unsafe { libc::close(self.fd) };
}
self.fd = recovered;
} else if self.fd < 0 {
return Err(Error::DeviceNotFound {
path: self.device_path.display().to_string(),
});
}
if data.len() > u32::MAX as usize {
return Err(Error::ScsiError {
opcode: cdb[0],
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
let dxfer_direction = match direction {
DataDirection::None => SG_DXFER_NONE,
DataDirection::FromDevice => SG_DXFER_FROM_DEV,
DataDirection::ToDevice => SG_DXFER_TO_DEV,
};
if data.len() > u32::MAX as usize {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
});
}
let cmd_len = cdb.len().min(16) as u8;
let mut sense = [0u8; 32];
let mut hdr: sg_io_hdr = unsafe { std::mem::zeroed() };
hdr.interface_id = b'S' as i32;
hdr.dxfer_direction = dxfer_direction;
@@ -281,38 +278,120 @@ impl ScsiTransport for SgIoTransport {
hdr.cmdp = cdb.as_ptr();
hdr.sbp = sense.as_mut_ptr();
hdr.timeout = timeout_ms;
if dxfer_direction == SG_DXFER_FROM_DEV
&& data.len() >= 4096
&& (data.as_ptr() as usize) % 4096 == 0
{
hdr.flags = SG_FLAG_DIRECT_IO | SG_FLAG_Q_AT_HEAD;
} else {
hdr.flags = SG_FLAG_Q_AT_HEAD;
}
hdr.flags = SG_FLAG_Q_AT_HEAD;
// The single blocking syscall. Returns when the device responds,
// when the kernel's timeout fires, or when the kernel's error
// recovery completes its escalation. On a healthy read this is
// <100 ms; on a slow-recovery bad sector it can be tens of
// seconds; on a hung drive it returns at `timeout_ms` with
// `host_status` flagged.
let ret = unsafe { libc::ioctl(self.fd, SG_IO as _, &mut hdr as *mut sg_io_hdr) };
let exec_elapsed_ms = exec_t0.elapsed().as_millis() as u64;
if ret < 0 {
return Err(Error::IoError {
source: std::io::Error::last_os_error(),
let errno = std::io::Error::last_os_error();
tracing::trace!(
target: "freemkv::scsi",
phase = "ioctl_err",
opcode = opcode,
errno = errno.raw_os_error().unwrap_or(0),
exec_elapsed_ms,
"ioctl(SG_IO) returned <0"
);
return Err(Error::IoError { source: errno });
}
// Transport-level failure (kernel timeout, USB bridge wedge,
// bus error). `hdr.status` may still be zero — the SCSI device
// never got to send a status byte. Surface as 0xFF so callers
// (e.g. `drive_has_disc`) can detect the wedge signature.
//
// 0.13.23: mask out `DRIVER_SENSE` (0x08) before treating
// `driver_status` as a transport failure. That bit is set on
// *every* CHECK CONDITION reply just to flag "sense data is
// attached in `sbp`" — it's not an error of its own. Pre-fix
// we collapsed every drive-reported error into a synthetic
// 0xFF wedge signature and discarded the sense data, which
// killed the rip's classification logic on damaged discs.
let driver_status_real = hdr.driver_status & !super::DRIVER_SENSE;
if hdr.host_status != 0 || driver_status_real != 0 {
tracing::trace!(
target: "freemkv::scsi",
phase = "transport_err",
opcode = opcode,
host_status = hdr.host_status,
driver_status = hdr.driver_status,
status = hdr.status,
exec_elapsed_ms,
"transport-level failure (timeout / bridge wedge)"
);
// Spawn recovery: close old fd, open new one in background.
// This prevents the main thread from blocking on close() while
// the kernel finishes the previous ioctl.
let old_fd = self.fd;
self.fd = -1;
let path = self.device_path.clone();
let recovery = self.fd_recovery.clone();
std::thread::spawn(move || {
if old_fd >= 0 {
unsafe { libc::close(old_fd) };
}
});
std::thread::spawn(move || {
let c_path = std::ffi::CString::new(path.as_os_str().as_encoded_bytes()).unwrap();
let new_fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
recovery.store(new_fd, std::sync::atomic::Ordering::Release);
});
return Err(Error::ScsiError {
opcode: cdb[0],
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
// SCSI-level failure: device responded, returned non-zero status
// (typically 0x02 CHECK CONDITION). Parse the full SPC-4 sense
// triple so callers can route on `ScsiSense::is_medium_error()`
// etc.
if hdr.status != 0 {
let parsed = super::parse_sense(&sense, hdr.sb_len_wr);
tracing::trace!(
target: "freemkv::scsi",
phase = "scsi_err",
opcode = opcode,
status = hdr.status,
sense_key = parsed.sense_key,
asc = parsed.asc,
ascq = parsed.ascq,
exec_elapsed_ms,
"SCSI status non-zero"
);
return Err(Error::ScsiError {
opcode: cdb[0],
status: hdr.status,
sense: Some(parsed),
});
}
let bytes_transferred = (data.len() as i32).saturating_sub(hdr.resid).max(0) as usize;
if hdr.status != 0 {
let sense_key = if hdr.sb_len_wr > 2 {
sense[2] & 0x0F
} else {
0
};
return Err(Error::ScsiError {
opcode: cdb[0],
status: hdr.status,
sense_key,
});
}
tracing::trace!(
target: "freemkv::scsi",
phase = "ok",
opcode = opcode,
bytes_transferred,
exec_elapsed_ms,
"execute() success"
);
Ok(ScsiResult {
status: hdr.status,
bytes_transferred,
@@ -320,3 +399,189 @@ impl ScsiTransport for SgIoTransport {
})
}
}
// ── Lightweight discovery + presence (Linux) ────────────────────────────────
//
// `list_drives` walks `/sys/class/scsi_generic/`, filters to type-5 (CD/DVD/BD),
// and runs one INQUIRY each for vendor/model/firmware. Falls back to a
// `/dev/sg0..15` probe when sysfs is unreadable (minimal containers).
//
// `drive_has_disc` issues a single TEST UNIT READY. On the wedge signature
// (kernel returns status `0xff` with no sense — synthesised by `execute()`
// from a non-zero `host_status`) the error bubbles directly to the caller;
// no in-library reset escalation. See the rationale block on
// `drive_has_disc` below.
/// SCSI peripheral type 5 = "CD-ROM device" (covers DVD, BD-ROM, BD-RE, etc.).
/// Stored in `/sys/class/scsi_generic/sgN/device/type` as ASCII decimal.
const SCSI_TYPE_OPTICAL: &str = "5";
/// Maximum sg index probed in the fallback path when sysfs is unavailable.
/// Linux assigns `/dev/sgN` sequentially per host adapter; 16 covers any
/// realistic homelab (typical PERC + USB optical = ≤8 nodes).
const SG_FALLBACK_MAX: u8 = 16;
pub(super) fn list_drives() -> Vec<super::DriveInfo> {
let mut out = Vec::new();
let names = enumerate_sg_names();
for name in names {
let path = format!("/dev/{name}");
if !std::path::Path::new(&path).exists() {
continue;
}
// Read sysfs-cached identity first. The kernel runs its own INQUIRY
// at device probe time and stashes vendor/model/rev under
// `/sys/class/scsi_generic/sgN/device/`. Those values survive even
// when the drive firmware is wedged below the USB bridge (our own
// INQUIRY times out but sysfs still has the pre-wedge answer), so
// the UI always has a human-readable identity to show.
let (sysfs_vendor, sysfs_model, sysfs_firmware) = sysfs_identity(&name);
// INQUIRY-only probe — open transport, run INQUIRY, drop. No
// identify, no init, no firmware reset preamble's secondary
// commands beyond what `SgIoTransport::open` already does.
let info = match SgIoTransport::open(std::path::Path::new(&path)) {
Ok(mut transport) => match super::inquiry(&mut transport) {
Ok(r) => super::DriveInfo {
path: path.clone(),
vendor: pick_identity(r.vendor_id, &sysfs_vendor),
model: pick_identity(r.model, &sysfs_model),
firmware: pick_identity(r.firmware, &sysfs_firmware),
},
Err(_) => super::DriveInfo {
path: path.clone(),
vendor: sysfs_vendor,
model: sysfs_model,
firmware: sysfs_firmware,
},
},
Err(_) => super::DriveInfo {
path: path.clone(),
vendor: sysfs_vendor,
model: sysfs_model,
firmware: sysfs_firmware,
},
};
out.push(info);
}
out
}
/// Prefer the live INQUIRY answer over the sysfs-cached one, but fall
/// back to sysfs when the live answer is empty (wedge / bridge bug).
fn pick_identity(live: String, sysfs: &str) -> String {
let trimmed = live.trim();
if trimmed.is_empty() {
sysfs.to_string()
} else {
live
}
}
/// Read the kernel's cached INQUIRY identity strings for `sgN` from
/// `/sys/class/scsi_generic/sgN/device/{vendor,model,rev}`. Empty strings
/// when sysfs is unavailable (minimal container, non-Linux filesystem).
fn sysfs_identity(name: &str) -> (String, String, String) {
let read = |field: &str| -> String {
std::fs::read_to_string(format!("/sys/class/scsi_generic/{name}/device/{field}"))
.map(|s| s.trim().to_string())
.unwrap_or_default()
};
(read("vendor"), read("model"), read("rev"))
}
/// Enumerate `sg*` names via `/sys/class/scsi_generic/`, filtered to
/// SCSI peripheral type 5 (optical). Falls back to a `sg0..15` probe
/// when sysfs is unreadable. Returns names sorted lexically so caller
/// iteration is deterministic.
fn enumerate_sg_names() -> Vec<String> {
let mut names = Vec::new();
if let Ok(entries) = std::fs::read_dir("/sys/class/scsi_generic") {
for entry in entries.flatten() {
let name = entry.file_name().to_string_lossy().to_string();
if !name.starts_with("sg") {
continue;
}
let type_path = format!("/sys/class/scsi_generic/{name}/device/type");
match std::fs::read_to_string(&type_path) {
Ok(s) if s.trim() == SCSI_TYPE_OPTICAL => names.push(name),
Ok(_) => {} // not optical
Err(_) => {}
}
}
} else {
// Sysfs missing — fall back to a brute-force probe. The INQUIRY
// step in `list_drives` filters non-optical responses naturally.
for i in 0..SG_FALLBACK_MAX {
let name = format!("sg{i}");
if std::path::Path::new(&format!("/dev/{name}")).exists() {
names.push(name);
}
}
}
names.sort();
names
}
/// Send TEST UNIT READY directly — no transport, no reset, no side effects.
pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
let device = SgIoTransport::resolve_to_sg(path);
let c_path = SgIoTransport::to_c_path(&device);
let fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return SgIoTransport::open_error(&device);
}
let cdb = [crate::scsi::SCSI_TEST_UNIT_READY, 0, 0, 0, 0, 0];
let mut sense = [0u8; 32];
let mut hdr: sg_io_hdr = unsafe { std::mem::zeroed() };
hdr.interface_id = b'S' as i32;
hdr.dxfer_direction = SG_DXFER_NONE;
hdr.cmd_len = cdb.len() as u8;
hdr.mx_sb_len = sense.len() as u8;
hdr.dxfer_len = 0;
hdr.dxferp = std::ptr::null_mut();
hdr.cmdp = cdb.as_ptr();
hdr.sbp = sense.as_mut_ptr();
hdr.timeout = crate::scsi::TUR_TIMEOUT_MS;
hdr.flags = SG_FLAG_Q_AT_HEAD;
let ret = unsafe { libc::ioctl(fd, SG_IO as _, &mut hdr as *mut sg_io_hdr) };
unsafe { libc::close(fd) };
if ret < 0 {
return Err(Error::IoError {
source: std::io::Error::last_os_error(),
});
}
let driver_status_real = hdr.driver_status & !super::DRIVER_SENSE;
if hdr.host_status != 0 || driver_status_real != 0 {
return Err(Error::ScsiError {
opcode: cdb[0],
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
if hdr.status == 0 {
return Ok(true);
}
let parsed = super::parse_sense(&sense, hdr.sb_len_wr);
if parsed.is_not_ready() {
Ok(false)
} else {
Err(Error::ScsiError {
opcode: cdb[0],
status: hdr.status,
sense: Some(parsed),
})
}
}
+106 -394
View File
@@ -1,174 +1,55 @@
//! macOS SCSI transport via IOKit SCSITaskDeviceInterface.
//! macOS SCSI transport: IOKit SCSITaskDeviceInterface with exclusive access.
//!
//! Sends SCSI commands to optical drives through IOKit's SCSI Architecture
//! Model family. Accepts BSD device paths like `/dev/disk2` or `/dev/rdisk2`.
//! Single dispatch path: **all** CDBs (INQUIRY, READ, REPORT KEY, etc.) go
//! through `SCSITaskDeviceInterface::ExecuteTaskSync` — 1:1 with the Linux
//! SG_IO backend. The C shim (`macos_shim.c`) handles:
//!
//! Requires exclusive access to the device — unmount the disc first:
//! `diskutil unmountDisk /dev/disk2`
//! 1. `diskutil unmountDisk force` on the target device only
//! 2. Find `IOBDServices` matching the requested BSD name (walks IOKit
//! registry: IOBDServices → IOBDBlockStorageDriver → IOMedia → BSD Name)
//! 3. Create `MMCDeviceInterface` → `SCSITaskDeviceInterface`
//! 4. `ObtainExclusiveAccess`
//! 5. Raw CDB dispatch via `CreateSCSITask` + `ExecuteTaskSync`
//!
//! Drive enumeration (`list_drives`) uses the IOKit registry directly via
//! `shim_list_drives` — no exclusive access, no SCSI commands, no unmounts.
use super::{DataDirection, ScsiResult, ScsiTransport};
use crate::error::{Error, Result};
use std::path::Path;
// ── IOKit / CoreFoundation type aliases ─────────────────────────────────────
type CFMutableDictionaryRef = *mut std::ffi::c_void;
type IOObject = u32;
type IOReturn = i32;
type MachPort = u32;
/// Opaque COM interface pointer — `*mut *mut VTable` (double-indirect).
/// IOKit plugins use COM-style vtables: the pointer points to a pointer
/// to the function table.
type ComRef = *mut *mut std::ffi::c_void;
const K_IO_RETURN_SUCCESS: IOReturn = 0;
// SCSI data transfer directions (SCSITaskLib.h)
const K_SCSI_DATA_TRANSFER_NO_DATA: u8 = 0;
const K_SCSI_DATA_TRANSFER_FROM_TARGET: u8 = 1;
const K_SCSI_DATA_TRANSFER_TO_TARGET: u8 = 2;
// SCSI task status values
const K_SCSI_TASK_STATUS_GOOD: u8 = 0x00;
const K_MAX_CDB_SIZE: usize = 16;
const K_SENSE_DATA_SIZE: usize = 32;
// ── IOKit plugin UUIDs ──────────────────────────────────────────────────────
// From IOKit/scsi/SCSITaskLib.h
/// kIOMMCDeviceUserClientTypeID — plugin type for MMC (optical) devices.
const K_IO_MMC_DEVICE_USER_CLIENT_TYPE_ID: [u8; 16] = [
0x97, 0xAB, 0xCF, 0x5C, 0x45, 0x71, 0x11, 0xD6, 0xB6, 0xA0, 0x00, 0x30, 0x65, 0xA4, 0x7A, 0xEE,
];
/// kIOCFPlugInInterfaceID — base IOCFPlugin interface.
const K_IO_CFPLUGIN_INTERFACE_ID: [u8; 16] = [
0xC2, 0x44, 0xE8, 0x58, 0x10, 0x9C, 0x11, 0xD4, 0x91, 0xD4, 0x00, 0x50, 0xE4, 0xC6, 0x42, 0x6F,
];
/// kIOSCSITaskDeviceInterfaceID — the interface we QueryInterface for.
const K_IO_SCSI_TASK_DEVICE_INTERFACE_ID: [u8; 16] = [
0x61, 0x3E, 0x48, 0xB0, 0x30, 0x01, 0x11, 0xD6, 0xA4, 0xC0, 0x00, 0x0A, 0x27, 0x05, 0x28, 0x61,
];
// ── Scatter/gather element ──────────────────────────────────────────────────
#[repr(C)]
struct SCSITaskSGElement {
address: u64,
length: u64,
#[derive(Copy, Clone)]
struct ShimDriveInfo {
bsd_name: [u8; 32],
vendor: [u8; 32],
model: [u8; 48],
firmware: [u8; 16],
}
// ── External IOKit / CoreFoundation functions ───────────────────────────────
extern "C" {
fn IOMasterPort(bootstrap: u32, master: *mut MachPort) -> IOReturn;
fn IOBSDNameMatching(
master: MachPort,
options: u32,
bsd_name: *const u8,
) -> CFMutableDictionaryRef;
fn IOServiceGetMatchingService(master: MachPort, matching: CFMutableDictionaryRef) -> IOObject;
fn IOObjectRelease(object: IOObject) -> IOReturn;
fn IORegistryEntryGetParentEntry(
entry: IOObject,
plane: *const u8,
parent: *mut IOObject,
) -> IOReturn;
fn IOObjectConformsTo(object: IOObject, class_name: *const u8) -> u8;
fn IOCreatePlugInInterfaceForService(
service: IOObject,
plugin_type: *const [u8; 16],
interface_type: *const [u8; 16],
the_interface: *mut ComRef,
the_score: *mut i32,
) -> IOReturn;
unsafe extern "C" {
fn shim_open_exclusive(bsd_name: *const u8) -> i32;
fn shim_close();
fn shim_execute(
cdb: *const u8,
cdb_len: u8,
buf: *mut u8,
buf_len: u32,
data_in: i32,
sense_out: *mut u8,
sense_len: u32,
task_status_out: *mut u8,
transfer_count: *mut u64,
) -> i32;
fn shim_list_drives(out: *mut ShimDriveInfo, max_entries: i32) -> i32;
}
// ── COM vtable helpers ──────────────────────────────────────────────────────
//
// IOKit plugin interfaces use COM-style vtables. A ComRef is **vtable —
// dereferencing once gives the vtable pointer, then index into it for
// individual function pointers.
//
// All vtable indices verified against Apple open source:
// IOSCSIArchitectureModelFamily/UserClientLib/SCSITaskLib.h
/// Read a function pointer from a COM vtable at the given index.
unsafe fn vtable_fn<T>(iface: ComRef, index: usize) -> T {
let vtable = *iface as *const *const std::ffi::c_void;
let fn_ptr = *vtable.add(index);
std::mem::transmute_copy(&fn_ptr)
}
/// Call Release (vtable index 3) on any COM interface.
fn com_release(iface: ComRef) {
type Fn = unsafe extern "C" fn(ComRef) -> u32;
unsafe {
let f: Fn = vtable_fn(iface, 3);
f(iface);
}
}
// ── SCSITaskDeviceInterface vtable ──────────────────────────────────────────
//
// Index Method
// 0 _reserved
// 1 QueryInterface
// 2 AddRef
// 3 Release
// 4 IsExclusiveAccessAvailable
// 5 AddCallbackDispatcherToRunLoop
// 6 RemoveCallbackDispatcherFromRunLoop
// 7 ObtainExclusiveAccess
// 8 ReleaseExclusiveAccess
// 9 CreateSCSITask
const VTIDX_OBTAIN_EXCLUSIVE: usize = 7;
const VTIDX_RELEASE_EXCLUSIVE: usize = 8;
const VTIDX_CREATE_TASK: usize = 9;
// ── SCSITaskInterface vtable ────────────────────────────────────────────────
//
// Index Method
// 0 _reserved
// 1 QueryInterface
// 2 AddRef
// 3 Release
// 4 IsTaskActive
// 5 SetTaskAttribute
// 6 GetTaskAttribute
// 7 GetTaskState
// 8 SetCommandDescriptorBlock
// 9 GetCommandDescriptorBlockSize
// 10 GetCommandDescriptorBlock
// 11 SetScatterGatherEntries
// 12 SetTimeoutDuration
// 13 GetTimeoutDuration
// 14 SetTaskCompletionCallback
// 15 ExecuteTaskSync
// 16 ExecuteTaskAsync
// 17 AbortTask
// 18 GetSCSIServiceResponse
// 19 GetTaskStatus
// 20 GetRealizedDataTransferCount
// 21 GetAutoSenseData
const VTIDX_SET_CDB: usize = 8;
const VTIDX_SET_SG: usize = 11;
const VTIDX_SET_TIMEOUT: usize = 12;
const VTIDX_EXECUTE_SYNC: usize = 15;
// ── Transport implementation ────────────────────────────────────────────────
pub struct MacScsiTransport {
device_iface: ComRef,
exclusive: bool,
_bsd_name: String,
}
// IOKit COM interface pointers are Mach port references — safe to send between threads.
unsafe impl Send for MacScsiTransport {}
impl MacScsiTransport {
@@ -177,7 +58,6 @@ impl MacScsiTransport {
path: device.display().to_string(),
})?;
// Strip /dev/ prefix to get BSD name (e.g. "disk2")
let bsd_name = if let Some(rest) = dev_str.strip_prefix("/dev/r") {
rest
} else if let Some(rest) = dev_str.strip_prefix("/dev/") {
@@ -186,98 +66,25 @@ impl MacScsiTransport {
dev_str
};
let service = find_scsi_service(bsd_name)?;
let mut bsd_c = bsd_name.as_bytes().to_vec();
bsd_c.push(0);
// Create IOKit plugin for the MMC device
let mut plugin: ComRef = std::ptr::null_mut();
let mut score: i32 = 0;
let kr = unsafe {
IOCreatePlugInInterfaceForService(
service,
&K_IO_MMC_DEVICE_USER_CLIENT_TYPE_ID,
&K_IO_CFPLUGIN_INTERFACE_ID,
&mut plugin,
&mut score,
)
};
unsafe { IOObjectRelease(service) };
if kr != K_IO_RETURN_SUCCESS || plugin.is_null() {
let rc = unsafe { shim_open_exclusive(bsd_c.as_ptr()) };
if rc != 0 {
return Err(Error::DeviceNotFound {
path: format!("{}: IOKit plugin creation failed (0x{:08x})", dev_str, kr),
});
}
// QueryInterface for SCSITaskDeviceInterface
let mut device_iface: ComRef = std::ptr::null_mut();
let hr = unsafe {
type QiFn = unsafe extern "C" fn(ComRef, *const [u8; 16], *mut ComRef) -> i32;
let qi: QiFn = vtable_fn(plugin, 1);
qi(
plugin,
&K_IO_SCSI_TASK_DEVICE_INTERFACE_ID,
&mut device_iface,
)
};
com_release(plugin);
if hr != 0 || device_iface.is_null() {
return Err(Error::DeviceNotFound {
path: format!("{}: SCSITaskDeviceInterface not available", dev_str),
});
}
// Obtain exclusive access
let kr = unsafe {
type Fn = unsafe extern "C" fn(ComRef) -> IOReturn;
let f: Fn = vtable_fn(device_iface, VTIDX_OBTAIN_EXCLUSIVE);
f(device_iface)
};
if kr != K_IO_RETURN_SUCCESS {
com_release(device_iface);
return Err(Error::DevicePermission {
path: format!(
"{}: exclusive access denied (0x{:08x}). Try: diskutil unmountDisk {}",
dev_str, kr, dev_str
),
path: bsd_name.to_string(),
});
}
Ok(MacScsiTransport {
device_iface,
exclusive: true,
_bsd_name: bsd_name.to_string(),
})
}
/// Reset the drive to a known good state.
/// On macOS, we open the device, release exclusive access, wait for
/// the system to reclaim it, then the next open() re-acquires.
/// IOKit's USB layer handles device-level resets internally when the
/// exclusive access is released and re-acquired.
///
/// NOTE: untested — macOS reset may need IOUSBDeviceInterface::ResetDevice()
/// for USB drives. This is a best-effort implementation.
pub fn reset(device: &Path) -> Result<()> {
// Opening and immediately dropping triggers release of exclusive access
// which forces IOKit to reset the device state.
if let Ok(transport) = Self::open(device) {
drop(transport); // Drop releases exclusive access + closes plugin
}
std::thread::sleep(std::time::Duration::from_secs(2));
Ok(())
}
}
impl Drop for MacScsiTransport {
fn drop(&mut self) {
if self.exclusive {
unsafe {
type Fn = unsafe extern "C" fn(ComRef) -> IOReturn;
let f: Fn = vtable_fn(self.device_iface, VTIDX_RELEASE_EXCLUSIVE);
f(self.device_iface);
}
}
com_release(self.device_iface);
unsafe { shim_close() };
}
}
@@ -287,197 +94,102 @@ impl ScsiTransport for MacScsiTransport {
cdb: &[u8],
direction: DataDirection,
data: &mut [u8],
timeout_ms: u32,
_timeout_ms: u32,
) -> Result<ScsiResult> {
// Create a SCSI task
let task: ComRef = unsafe {
type Fn = unsafe extern "C" fn(ComRef) -> ComRef;
let f: Fn = vtable_fn(self.device_iface, VTIDX_CREATE_TASK);
f(self.device_iface)
};
if task.is_null() {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
});
}
// Set CDB
let mut cdb_padded = [0u8; K_MAX_CDB_SIZE];
let cdb_len = cdb.len().min(K_MAX_CDB_SIZE);
cdb_padded[..cdb_len].copy_from_slice(&cdb[..cdb_len]);
unsafe {
type Fn = unsafe extern "C" fn(ComRef, *const u8, u8) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_SET_CDB);
f(task, cdb_padded.as_ptr(), cdb_len as u8);
}
// Set scatter/gather and transfer direction
let iokit_dir = match direction {
DataDirection::None => K_SCSI_DATA_TRANSFER_NO_DATA,
DataDirection::FromDevice => K_SCSI_DATA_TRANSFER_FROM_TARGET,
DataDirection::ToDevice => K_SCSI_DATA_TRANSFER_TO_TARGET,
let data_in = match direction {
DataDirection::FromDevice => 1,
DataDirection::ToDevice => 0,
DataDirection::None => 0,
};
if direction != DataDirection::None && !data.is_empty() {
let sg = SCSITaskSGElement {
address: data.as_mut_ptr() as u64,
length: data.len() as u64,
};
unsafe {
type Fn =
unsafe extern "C" fn(ComRef, *const SCSITaskSGElement, u8, u64, u8) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_SET_SG);
f(task, &sg, 1, data.len() as u64, iokit_dir);
}
} else {
unsafe {
type Fn =
unsafe extern "C" fn(ComRef, *const SCSITaskSGElement, u8, u64, u8) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_SET_SG);
f(task, std::ptr::null(), 0, 0, K_SCSI_DATA_TRANSFER_NO_DATA);
}
}
// Set timeout (IOKit SCSITask takes milliseconds)
unsafe {
type Fn = unsafe extern "C" fn(ComRef, u32);
let f: Fn = vtable_fn(task, VTIDX_SET_TIMEOUT);
f(task, timeout_ms);
}
// Execute synchronously
let mut sense = [0u8; K_SENSE_DATA_SIZE];
let mut task_status: u32 = 0;
let mut realized_count: u64 = 0;
let mut task_status: u8 = 0xFF;
let mut transfer_count: u64 = 0;
let kr = unsafe {
type Fn = unsafe extern "C" fn(ComRef, *mut u8, *mut u32, *mut u64) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_EXECUTE_SYNC);
f(
task,
shim_execute(
cdb.as_ptr(),
cdb.len() as u8,
data.as_mut_ptr(),
data.len() as u32,
data_in,
sense.as_mut_ptr(),
K_SENSE_DATA_SIZE as u32,
&mut task_status,
&mut realized_count,
&mut transfer_count,
)
};
com_release(task);
if kr != K_IO_RETURN_SUCCESS {
if kr != 0 {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
if task_status != K_SCSI_TASK_STATUS_GOOD as u32 {
let sense_key = if sense[2] != 0 { sense[2] & 0x0F } else { 0 };
if task_status != 0 {
let parsed = super::parse_sense(&sense, K_SENSE_DATA_SIZE as u8);
return Err(Error::ScsiError {
opcode: cdb[0],
status: task_status as u8,
sense_key,
status: task_status,
sense: Some(parsed),
});
}
Ok(ScsiResult {
status: task_status as u8,
bytes_transferred: realized_count as usize,
status: 0,
bytes_transferred: transfer_count as usize,
sense,
})
}
}
// ── IOKit service discovery ─────────────────────────────────────────────────
// ── Drive enumeration (registry-based, no exclusive access) ──────────────
/// BSD name → IOKit service for the SCSI device.
///
/// Walk: IOMedia (BSD name match) → parent chain → SCSIPeripheralDeviceNub.
fn find_scsi_service(bsd_name: &str) -> Result<IOObject> {
let mut master: MachPort = 0;
let kr = unsafe { IOMasterPort(0, &mut master) };
if kr != K_IO_RETURN_SUCCESS {
return Err(Error::DeviceNotFound {
path: format!("{}: IOMasterPort failed", bsd_name),
pub(super) fn list_drives() -> Vec<super::DriveInfo> {
let mut buf = [ShimDriveInfo {
bsd_name: [0; 32],
vendor: [0; 32],
model: [0; 48],
firmware: [0; 16],
}; 8];
let count = unsafe { shim_list_drives(buf.as_mut_ptr(), buf.len() as i32) };
let mut out = Vec::new();
for info in buf.iter().take((count as usize).min(buf.len())) {
let bsd_name = cstr_to_str(&info.bsd_name);
if bsd_name.is_empty() {
continue;
}
out.push(super::DriveInfo {
path: format!("/dev/{bsd_name}"),
vendor: cstr_to_str(&info.vendor).to_string(),
model: cstr_to_str(&info.model).to_string(),
firmware: cstr_to_str(&info.firmware).to_string(),
});
}
// IOBSDNameMatching creates a dictionary matching { "BSD Name" = bsd_name }
let mut bsd_c = bsd_name.as_bytes().to_vec();
bsd_c.push(0);
let matching = unsafe { IOBSDNameMatching(master, 0, bsd_c.as_ptr()) };
if matching.is_null() {
return Err(Error::DeviceNotFound {
path: format!("{}: IOBSDNameMatching failed", bsd_name),
});
}
// Find the single IOMedia service (consumes the matching dict)
let media = unsafe { IOServiceGetMatchingService(master, matching) };
if media == 0 {
return Err(Error::DeviceNotFound {
path: format!("{}: no IOMedia found", bsd_name),
});
}
// Walk up the IOService plane to find the authoring device.
// The chain is typically:
// IOMedia → IOPartitionScheme → IOMedia → IOBlockStorageDriver
// → IOSCSIPeripheralDeviceNub (this is what we want)
//
// We walk up until we find a service that IOCreatePlugInInterfaceForService
// accepts with kIOMMCDeviceUserClientTypeID, or until we hit the root.
let service = walk_to_authoring_device(media);
unsafe { IOObjectRelease(media) };
service.ok_or_else(|| Error::DeviceNotFound {
path: format!("{}: no SCSI authoring device in IORegistry", bsd_name),
})
out
}
/// Walk up the IOService plane from an IOMedia to the SCSI authoring device.
fn walk_to_authoring_device(start: IOObject) -> Option<IOObject> {
let mut current = start;
// Retain start so we can release uniformly in the loop
// (IORegistryEntryGetParentEntry retains the parent for us)
fn cstr_to_str(bytes: &[u8]) -> &str {
let end = bytes.iter().position(|&b| b == 0).unwrap_or(bytes.len());
std::str::from_utf8(&bytes[..end]).unwrap_or("")
}
// Target class names for authoring devices
let target_classes: &[&[u8]] = &[
b"IOSCSIPeripheralDeviceNub\0",
b"IOBDBlockStorageDevice\0",
b"IODVDBlockStorageDevice\0",
b"IOCDBlockStorageDevice\0",
b"IOBlockStorageDevice\0",
];
// Walk up to 10 levels (more than enough)
for _ in 0..10 {
let mut parent: IOObject = 0;
let kr =
unsafe { IORegistryEntryGetParentEntry(current, b"IOService\0".as_ptr(), &mut parent) };
if current != start {
unsafe { IOObjectRelease(current) };
}
if kr != K_IO_RETURN_SUCCESS || parent == 0 {
return None;
}
// Check if this parent matches any of our target classes
for class in target_classes {
if unsafe { IOObjectConformsTo(parent, class.as_ptr()) } != 0 {
return Some(parent);
}
}
current = parent;
}
if current != start {
unsafe { IOObjectRelease(current) };
pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
let mut transport = MacScsiTransport::open(path)?;
let cdb = [crate::scsi::SCSI_TEST_UNIT_READY, 0, 0, 0, 0, 0];
let mut buf = [0u8; 0];
match transport.execute(
&cdb,
crate::scsi::DataDirection::None,
&mut buf,
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
Err(ref e) if e.scsi_sense().is_some_and(|s| s.is_not_ready()) => Ok(false),
Err(e) => Err(e),
}
None
}
+387
View File
@@ -0,0 +1,387 @@
#include <IOKit/IOKitLib.h>
#include <IOKit/IOCFPlugIn.h>
#include <IOKit/scsi/SCSITaskLib.h>
#include <CoreFoundation/CoreFoundation.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
// ── Types ──────────────────────────────────────────────────────────────────
typedef struct {
IOCFPlugInInterface **plugin;
MMCDeviceInterface **mmc;
SCSITaskDeviceInterface **scsi;
int exclusive;
} ShimHandle;
typedef struct {
char bsd_name[32];
char vendor[32];
char model[48];
char firmware[16];
} ShimDriveInfo;
// ── Global handle (single-drive, same as before) ──────────────────────────
static ShimHandle g_handle = {NULL, NULL, NULL, 0};
// ── Registry helpers ──────────────────────────────────────────────────────
static int cfstring_to_cstr(CFStringRef cf, char *buf, size_t buflen) {
if (!cf) return 0;
if (!CFStringGetCString(cf, buf, buflen, kCFStringEncodingUTF8)) return 0;
return 1;
}
static int registry_entry_bsd_name(io_registry_entry_t entry, char *buf, size_t buflen) {
CFStringRef cf = IORegistryEntryCreateCFProperty(entry, CFSTR("BSD Name"),
kCFAllocatorDefault, 0);
if (!cf) return 0;
int ok = cfstring_to_cstr(cf, buf, buflen);
CFRelease(cf);
return ok;
}
static io_registry_entry_t find_iomedia_child(io_registry_entry_t parent) {
io_iterator_t iter;
kern_return_t kr = IORegistryEntryGetChildIterator(parent, kIOServicePlane, &iter);
if (kr != KERN_SUCCESS) return 0;
io_registry_entry_t child;
while ((child = IOIteratorNext(iter)) != 0) {
char cls[128];
kr = IOObjectGetClass(child, cls);
if (kr == KERN_SUCCESS) {
if (strcmp(cls, "IOMedia") == 0 || strcmp(cls, "IOBDMedia") == 0) {
IOObjectRelease(iter);
return child;
}
}
IOObjectRelease(child);
}
IOObjectRelease(iter);
return 0;
}
static io_registry_entry_t find_child_of_class(io_registry_entry_t parent, const char *target_class) {
io_iterator_t iter;
kern_return_t kr = IORegistryEntryGetChildIterator(parent, kIOServicePlane, &iter);
if (kr != KERN_SUCCESS) return 0;
io_registry_entry_t child;
while ((child = IOIteratorNext(iter)) != 0) {
char cls[128];
kr = IOObjectGetClass(child, cls);
if (kr == KERN_SUCCESS && strcmp(cls, target_class) == 0) {
IOObjectRelease(iter);
return child;
}
IOObjectRelease(child);
}
IOObjectRelease(iter);
return 0;
}
static io_registry_entry_t find_parent_of_class(io_registry_entry_t entry, const char *target_class) {
io_registry_entry_t parent;
kern_return_t kr = IORegistryEntryGetParentEntry(entry, kIOServicePlane, &parent);
if (kr != KERN_SUCCESS) return 0;
char cls[128];
kr = IOObjectGetClass(parent, cls);
if (kr == KERN_SUCCESS && strcmp(cls, target_class) == 0) {
return parent;
}
IOObjectRelease(parent);
return 0;
}
// Given an IOBDServices, find the BSD name of its IOMedia child.
// Chain: IOBDServices -> IOBDBlockStorageDriver -> IOMedia (has "BSD Name")
static int bdsvc_to_bsd_name(io_registry_entry_t bdsvc, char *buf, size_t buflen) {
io_registry_entry_t driver = find_child_of_class(bdsvc, "IOBDBlockStorageDriver");
if (!driver) return 0;
io_registry_entry_t media = find_iomedia_child(driver);
IOObjectRelease(driver);
if (!media) return 0;
int ok = registry_entry_bsd_name(media, buf, buflen);
IOObjectRelease(media);
return ok;
}
// Given an IOBDServices, extract Device Characteristics strings.
static void bdsvc_device_info(io_registry_entry_t bdsvc, ShimDriveInfo *info) {
CFDictionaryRef dc = IORegistryEntryCreateCFProperty(bdsvc,
CFSTR("Device Characteristics"), kCFAllocatorDefault, 0);
if (!dc) return;
CFStringRef val;
val = CFDictionaryGetValue(dc, CFSTR("Vendor Name"));
if (val) cfstring_to_cstr(val, info->vendor, sizeof(info->vendor));
val = CFDictionaryGetValue(dc, CFSTR("Product Name"));
if (val) cfstring_to_cstr(val, info->model, sizeof(info->model));
val = CFDictionaryGetValue(dc, CFSTR("Product Revision Level"));
if (val) cfstring_to_cstr(val, info->firmware, sizeof(info->firmware));
CFRelease(dc);
}
// Find the IOBDServices that owns the given BSD name.
// Returns a retained io_service_t (caller must release), or 0.
static io_service_t find_bdsvc_by_bsd_name(mach_port_t mp, const char *bsd_name) {
CFMutableDictionaryRef matching = IOServiceMatching("IOBDServices");
if (!matching) return 0;
io_iterator_t iter;
kern_return_t kr = IOServiceGetMatchingServices(mp, matching, &iter);
if (kr != KERN_SUCCESS) return 0;
io_service_t result = 0;
io_service_t svc;
while ((svc = IOIteratorNext(iter)) != 0) {
char name[64];
if (bdsvc_to_bsd_name(svc, name, sizeof(name))) {
if (strcmp(name, bsd_name) == 0) {
result = svc;
break;
}
}
IOObjectRelease(svc);
}
if (!result) {
IOIteratorReset(iter);
while ((svc = IOIteratorNext(iter)) != 0) {
IOObjectRelease(svc);
}
}
IOObjectRelease(iter);
return result;
}
// Find the IOBDServices that owns the given BSD name by walking from
// IOMedia upward. Used as fallback when bdsvc_to_bsd_name fails
// (e.g. disc under exclusive access, no IOMedia child).
// Chain: IOMedia -> IOBDBlockStorageDriver -> IOBDServices
static io_service_t find_bdsvc_from_iomedia(mach_port_t mp, const char *bsd_name) {
CFMutableDictionaryRef matching = IOServiceMatching("IOMedia");
if (!matching) return 0;
io_iterator_t iter;
kern_return_t kr = IOServiceGetMatchingServices(mp, matching, &iter);
if (kr != KERN_SUCCESS) return 0;
io_service_t result = 0;
io_service_t media;
while ((media = IOIteratorNext(iter)) != 0) {
char name[64];
if (registry_entry_bsd_name(media, name, sizeof(name))
&& strcmp(name, bsd_name) == 0)
{
io_registry_entry_t driver = find_parent_of_class(media, "IOBDBlockStorageDriver");
if (driver) {
io_registry_entry_t bdsvc = find_parent_of_class(driver, "IOBDServices");
IOObjectRelease(driver);
if (bdsvc) {
result = bdsvc;
IOObjectRelease(media);
break;
}
}
}
IOObjectRelease(media);
}
IOObjectRelease(iter);
return result;
}
// ── Public API ────────────────────────────────────────────────────────────
int shim_open_exclusive(const char *bsd_name) {
kern_return_t kr;
HRESULT hr;
SInt32 score = 0;
if (g_handle.exclusive && g_handle.scsi) {
return 0;
}
// Use a shell wrapper so the device path is not subject to buffer limits.
// snprintf into 128 bytes could truncate long BSD names (e.g. disk12s3s1),
// producing a broken command. sh -c with $1 passes the arg via argv.
const char *shell_fmt = "sh -c 'diskutil unmountDisk force \"$1\" >/dev/null 2>&1' _ %s";
char cmd[512];
int written = snprintf(cmd, sizeof(cmd), shell_fmt, bsd_name);
if (written < 0 || (size_t)written >= sizeof(cmd)) {
return -1;
}
system(cmd);
usleep(500000);
mach_port_t mp;
IOMainPort(0, &mp);
io_service_t svc = find_bdsvc_by_bsd_name(mp, bsd_name);
if (!svc) {
svc = find_bdsvc_from_iomedia(mp, bsd_name);
}
if (!svc) {
CFMutableDictionaryRef matching = IOServiceMatching("IOBDServices");
svc = IOServiceGetMatchingService(mp, matching);
}
if (!svc) return -1;
kr = IOCreatePlugInInterfaceForService(svc,
kIOMMCDeviceUserClientTypeID, kIOCFPlugInInterfaceID,
&g_handle.plugin, &score);
IOObjectRelease(svc);
if (kr != KERN_SUCCESS || !g_handle.plugin) return -2;
hr = (*g_handle.plugin)->QueryInterface(g_handle.plugin,
CFUUIDGetUUIDBytes(kIOMMCDeviceInterfaceID), (LPVOID *)&g_handle.mmc);
if (hr != S_OK || !g_handle.mmc) {
IODestroyPlugInInterface(g_handle.plugin);
g_handle.plugin = NULL;
return -3;
}
g_handle.scsi = (*g_handle.mmc)->GetSCSITaskDeviceInterface(g_handle.mmc);
if (!g_handle.scsi) {
(*g_handle.mmc)->Release(g_handle.mmc);
IODestroyPlugInInterface(g_handle.plugin);
g_handle.mmc = NULL;
g_handle.plugin = NULL;
return -4;
}
for (int retry = 0; retry < 10; retry++) {
kr = (*g_handle.scsi)->ObtainExclusiveAccess(g_handle.scsi);
if (kr == kIOReturnSuccess) break;
usleep(500000);
}
if (kr != kIOReturnSuccess) {
(*g_handle.scsi)->Release(g_handle.scsi);
(*g_handle.mmc)->Release(g_handle.mmc);
IODestroyPlugInInterface(g_handle.plugin);
g_handle.scsi = NULL;
g_handle.mmc = NULL;
g_handle.plugin = NULL;
return -5;
}
g_handle.exclusive = 1;
return 0;
}
void shim_close(void) {
if (g_handle.exclusive && g_handle.scsi) {
(*g_handle.scsi)->ReleaseExclusiveAccess(g_handle.scsi);
}
if (g_handle.scsi) {
(*g_handle.scsi)->Release(g_handle.scsi);
g_handle.scsi = NULL;
}
if (g_handle.mmc) {
(*g_handle.mmc)->Release(g_handle.mmc);
g_handle.mmc = NULL;
}
if (g_handle.plugin) {
IODestroyPlugInInterface(g_handle.plugin);
g_handle.plugin = NULL;
}
g_handle.exclusive = 0;
}
int shim_execute(const unsigned char *cdb, unsigned char cdb_len,
void *buf, unsigned int buf_len, int data_in,
unsigned char *sense_out, unsigned int sense_len,
unsigned char *task_status_out, unsigned long long *transfer_count) {
if (!g_handle.scsi) return -1;
SCSITaskInterface **task = (*g_handle.scsi)->CreateSCSITask(g_handle.scsi);
if (!task) return -2;
SCSICommandDescriptorBlock cdb_buf;
memset(&cdb_buf, 0, sizeof(cdb_buf));
memcpy(&cdb_buf, cdb, cdb_len);
(*task)->SetCommandDescriptorBlock(task, cdb_buf, cdb_len);
if (buf_len > 0 && buf) {
SCSITaskSGElement sg;
sg.address = (UInt64)(uintptr_t)buf;
sg.length = buf_len;
(*task)->SetScatterGatherEntries(task, &sg, 1, buf_len,
data_in ? kSCSIDataTransfer_FromTargetToInitiator
: kSCSIDataTransfer_FromInitiatorToTarget);
} else {
(*task)->SetScatterGatherEntries(task, NULL, 0, 0,
kSCSIDataTransfer_NoDataTransfer);
}
(*task)->SetTimeoutDuration(task, 30000);
SCSI_Sense_Data sense;
memset(&sense, 0, sizeof(sense));
SCSITaskStatus status = 0xFF;
UInt64 count = 0;
IOReturn kr = (*task)->ExecuteTaskSync(task, &sense, &status, &count);
if (sense_out && sense_len > 0) {
size_t copy = sense_len < sizeof(sense) ? sense_len : sizeof(sense);
memcpy(sense_out, &sense, copy);
}
if (task_status_out) *task_status_out = (unsigned char)status;
if (transfer_count) *transfer_count = count;
(*task)->Release(task);
return (int)kr;
}
// ── Registry-based drive enumeration ──────────────────────────────────────
//
// Walks IOBDServices entries in the IOKit registry. No exclusive access,
// no SCSI commands, no unmounts. Returns up to max_entries drives.
int shim_list_drives(ShimDriveInfo *out, int max_entries) {
mach_port_t mp;
IOReturn ret = IOMainPort(0, &mp);
if (ret != kIOReturnSuccess) return 0;
CFMutableDictionaryRef matching = IOServiceMatching("IOBDServices");
if (!matching) return 0;
io_iterator_t iter;
kern_return_t kr = IOServiceGetMatchingServices(mp, matching, &iter);
if (kr != KERN_SUCCESS) return 0;
int count = 0;
io_service_t svc;
while ((svc = IOIteratorNext(iter)) != 0 && count < max_entries) {
ShimDriveInfo *info = &out[count];
memset(info, 0, sizeof(*info));
bdsvc_device_info(svc, info);
bdsvc_to_bsd_name(svc, info->bsd_name, sizeof(info->bsd_name));
if (info->bsd_name[0]) {
count++;
}
IOObjectRelease(svc);
}
IOObjectRelease(iter);
return count;
}
+495 -12
View File
@@ -2,11 +2,11 @@
//!
//! Platform backends are in separate files:
//! - `linux.rs` — SG_IO ioctl
//! - `macos.rs` — IOKit SCSITaskDeviceInterface
//! - `macos.rs` — IOKit SCSITaskDeviceInterface (exclusive access)
//! - `windows.rs` — SPTI (SCSI Pass-Through Interface)
#[cfg(target_os = "linux")]
mod linux;
pub mod linux;
#[cfg(target_os = "macos")]
mod macos;
#[cfg(target_os = "windows")]
@@ -18,6 +18,10 @@ use std::path::Path;
// ── SCSI opcodes (SPC-4, MMC-6) ────────────────────────────────────────────
/// SPC-4 TEST UNIT READY — six-byte CDB, no data transfer. Used by
/// [`drive_has_disc`] as the cheapest "is the drive responsive / does
/// it have media?" probe.
pub const SCSI_TEST_UNIT_READY: u8 = 0x00;
pub const SCSI_INQUIRY: u8 = 0x12;
pub const SCSI_READ_CAPACITY: u8 = 0x25;
pub const SCSI_READ_10: u8 = 0x28;
@@ -33,6 +37,262 @@ pub const SCSI_READ_DISC_STRUCTURE: u8 = 0xAD;
/// AACS key class for REPORT KEY / SEND KEY commands.
pub const AACS_KEY_CLASS: u8 = 0x02;
/// Timeout for TEST UNIT READY probes used by [`drive_has_disc`].
/// TUR is the cheapest SCSI op (no data transfer); 5 s is generous
/// for any healthy bus and short enough that a hung device can't stall
/// a poll-loop tick.
pub(crate) const TUR_TIMEOUT_MS: u32 = 5_000;
/// Timeout for content READ commands (READ_10 / READ_12) on the fast
/// path — the [`disc::Disc::copy`] sweep that bisects-on-failure.
///
/// 10 s is calibrated from live empirical data on an LG BU40N + Initio
/// 1618L bridge ripping a UHD with marginal sectors:
///
/// - Sustained sequential reads: 3 – 7 ms
/// - Cold-start seek + read: up to ~1500 ms
/// - Successful ECC recovery: 1.6 – 2.6 sec
/// - Confirmed unreadable sector: 3.6 – 8.8 sec (kernel timeout)
///
/// 10 s catches every legitimate slow read with comfortable margin and
/// short-circuits truly bad sectors at ~10 s rather than letting the
/// kernel mid-layer escalate for 30 s+. See the SCSI architecture audit
/// (2026-04-26) for primary-source references.
///
/// Pre-0.13.21 this was 1.5 s, which forced the kernel mid-layer to
/// time out *normal* reads (cold-start often takes ~1.5 s) and run its
/// full ABORT TASK / LUN RESET / BUS RESET escalation while userspace
/// kept submitting fresh reads. The Initio bridge couldn't drain the
/// resulting command queue and entered a wedge state that only physical
/// replug recovered — proven by the v0.13.18 + v0.13.20 live tests.
pub(crate) const READ_TIMEOUT_MS: u32 = 10_000;
/// Timeout for content READ commands on the recovery path —
/// [`disc::Disc::patch`]'s targeted retries on bad ranges. Matches
/// `sg_dd`'s 60 s ceiling: long enough that any sector the drive can
/// recover at all gets the time to do so, short enough that an
/// unresponsive bus is detected before the per-range watchdog fires.
///
/// In practice failed reads return in 1–4 s (the drive itself gives up
/// on uncorrectable ECC before the timeout); the 60 s value is a
/// safety ceiling, not a steady-state cost.
///
/// Historical note (2026-05-08): briefly lowered to 2 s with a 5×
/// inline retry loop in `Disc::patch` to mimic the kernel `sr_mod`
/// driver's auto-retry pattern. The synthetic logic worked but on the
/// live drive each "2 s" read paid ~1.5 s of kernel SCSI mid-layer
/// error escalation on top, so 5× retries took ~17 s per LBA and
/// triggered MAX_RANGE_SECS after 4 sectors — pushing recovery to
/// 0/22 ranges (worse than the 0/22 baseline of v0.17.3 single-shot
/// at 60 s, since that at least visited every range). Reverted; the
/// kernel-auto-retry approach is being pursued via a `/dev/sr0` pread
/// fallback instead.
pub(crate) const READ_RECOVERY_TIMEOUT_MS: u32 = 60_000;
// ── SCSI status bytes (SPC-4 §4.5.5) ────────────────────────────────────────
/// Status byte 0x00 — `GOOD`. Command completed successfully.
pub const SCSI_STATUS_GOOD: u8 = 0x00;
/// Status byte 0x02 — `CHECK CONDITION`. Drive completed the command
/// reply and attached sense data describing the failure.
pub const SCSI_STATUS_CHECK_CONDITION: u8 = 0x02;
/// libfreemkv-synthesised sentinel: the transport never delivered a
/// SCSI status byte (kernel timeout, USB bridge wedge, IOKit service
/// failure). Distinct from any drive-returned value. Carriers
/// [`Error::ScsiError`] with `sense = None`.
pub const SCSI_STATUS_TRANSPORT_FAILURE: u8 = 0xFF;
// ── SPC-4 sense keys (§4.5.6 Table 28) ─────────────────────────────────────
//
// Broad failure category returned in a CHECK CONDITION reply's sense data.
// Names match the SCSI spec; predicate methods on [`ScsiSense`] (e.g.
// `is_medium_error`, `is_unit_attention`) read more fluently than raw
// constant comparisons at call sites.
pub const SENSE_KEY_NO_SENSE: u8 = 0x00;
pub const SENSE_KEY_RECOVERED_ERROR: u8 = 0x01;
pub const SENSE_KEY_NOT_READY: u8 = 0x02;
pub const SENSE_KEY_MEDIUM_ERROR: u8 = 0x03;
pub const SENSE_KEY_HARDWARE_ERROR: u8 = 0x04;
pub const SENSE_KEY_ILLEGAL_REQUEST: u8 = 0x05;
pub const SENSE_KEY_UNIT_ATTENTION: u8 = 0x06;
pub const SENSE_KEY_DATA_PROTECT: u8 = 0x07;
pub const SENSE_KEY_BLANK_CHECK: u8 = 0x08;
pub const SENSE_KEY_ABORTED_COMMAND: u8 = 0x0B;
// ── Sense parsing ───────────────────────────────────────────────────────────
/// Decoded SPC-4 sense triple — the precise reason a SCSI command failed.
///
/// Returned by [`parse_sense`] and embedded inside [`Error::ScsiError`]
/// (`sense: Option<ScsiSense>`). Predicate methods (`is_medium_error`,
/// `is_unit_attention`, `is_marginal`, …) read more fluently at call
/// sites than raw `sense_key` comparisons.
///
/// `Default::default()` and the [`ScsiSense::NONE`] constant both
/// produce the all-zero "no sense info" triple. Per SPC-4 §4.5.3, an
/// empty sense buffer is reported as NO SENSE (key 0); use the constant
/// for explicit intent at construction sites.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct ScsiSense {
/// Sense key — broad failure category (SPC-4 §4.5.6 Table 28).
/// See the `SENSE_KEY_*` constants for named values.
pub sense_key: u8,
/// Additional Sense Code — narrows the cause within a sense key
/// (SPC-4 §4.5.6 Table 29). E.g. `0x11` = UNRECOVERED READ ERROR.
pub asc: u8,
/// Additional Sense Code Qualifier — finest-grain disambiguation.
/// E.g. `0x05` (with `asc=0x11`) = L-EC UNCORRECTABLE.
pub ascq: u8,
}
impl ScsiSense {
/// Sense reply with all-zero fields — explicit "no sense info"
/// constructor for sites where `Default::default()` would be opaque.
pub const NONE: ScsiSense = ScsiSense {
sense_key: 0,
asc: 0,
ascq: 0,
};
/// `true` when the sense key indicates a *marginal-read* failure —
/// the kind of error where the same read at smaller granularity
/// (or a brief retry) sometimes succeeds:
///
/// - `MEDIUM ERROR` (3) — canonical bad-sector signal
/// - `NOT READY` (2) — on many drives (notably BU40N), this is the
/// dominant response for unreadable sectors (ASC 04/3E, 04/01, etc.)
/// - `ABORTED COMMAND` (B) — transient; retry usually works
/// - `RECOVERED ERROR` (1) / `NO SENSE` (0) — drive is healthy and
/// either recovered the data or has no specific fault to report
///
/// `false` for HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION,
/// ILLEGAL REQUEST, BLANK CHECK, and any unknown key. Used
/// by [`Error::is_marginal_read`] / `Disc::copy`'s hysteresis
/// dispatch.
pub fn is_marginal(&self) -> bool {
matches!(
self.sense_key,
SENSE_KEY_NO_SENSE
| SENSE_KEY_RECOVERED_ERROR
| SENSE_KEY_NOT_READY
| SENSE_KEY_MEDIUM_ERROR
| SENSE_KEY_ABORTED_COMMAND
)
}
/// `true` if `sense_key == MEDIUM ERROR (3)` — canonical "bad sector"
/// signal from the drive.
pub fn is_medium_error(&self) -> bool {
self.sense_key == SENSE_KEY_MEDIUM_ERROR
}
/// `true` if `sense_key == HARDWARE ERROR (4)` — drive itself is
/// failing. Not recoverable by retry.
pub fn is_hardware_error(&self) -> bool {
self.sense_key == SENSE_KEY_HARDWARE_ERROR
}
/// `true` if `sense_key == NOT READY (2)` — medium not present /
/// drive becoming ready / etc.
pub fn is_not_ready(&self) -> bool {
self.sense_key == SENSE_KEY_NOT_READY
}
/// `true` if `sense_key == UNIT ATTENTION (6)` — disc/drive state
/// changed since the prior command (media inserted/removed,
/// power-on reset, parameters changed). Caller should rescan rather
/// than retry the read.
pub fn is_unit_attention(&self) -> bool {
self.sense_key == SENSE_KEY_UNIT_ATTENTION
}
/// `true` if `sense_key == DATA PROTECT (7)` — read blocked by
/// AACS / region / write-protect. Retry won't help.
pub fn is_data_protect(&self) -> bool {
self.sense_key == SENSE_KEY_DATA_PROTECT
}
/// `true` if `sense_key == ILLEGAL REQUEST (5)` — typically a bug
/// in the CDB we sent (LBA out of range, reserved bit, etc.). Don't
/// retry.
pub fn is_illegal_request(&self) -> bool {
self.sense_key == SENSE_KEY_ILLEGAL_REQUEST
}
/// `true` if `sense_key == ABORTED COMMAND (B)` — transient; one
/// retry is usually safe.
pub fn is_aborted_command(&self) -> bool {
self.sense_key == SENSE_KEY_ABORTED_COMMAND
}
}
/// Decode an SPC-4 sense buffer into the structured triple
/// `(sense_key, asc, ascq)`.
///
/// Handles both response-code formats SPC-4 mandates:
///
/// - **Descriptor format** (response code `0x72` / `0x73`):
/// - sense key = `sense[1] & 0x0F`
/// - asc = `sense[2]`
/// - ascq = `sense[3]`
/// - **Fixed format** (response code `0x70` / `0x71` and any unknown
/// code per SPC-4 §4.5.3):
/// - sense key = `sense[2] & 0x0F`
/// - asc = `sense[12]`
/// - ascq = `sense[13]`
///
/// `sb_len_wr` is the number of bytes the transport actually wrote into
/// `sense`. When the buffer is too short for the relevant fields we
/// return [`ScsiSense::NONE`] for the missing pieces rather than reading
/// uninitialised memory. The minimum useful sense reply per SPC-4 is 8
/// bytes (descriptor) or 14 bytes (fixed, to reach ASC/ASCQ at offsets
/// 12/13).
///
/// Pure function — same parse on every platform backend (Linux SG_IO,
/// macOS IOKit, Windows SPTI) so a regression here would silently
/// mis-route SCSI errors on all three OSes simultaneously.
pub(crate) fn parse_sense(sense: &[u8], sb_len_wr: u8) -> ScsiSense {
let n = (sb_len_wr as usize).min(sense.len());
if n < 3 {
return ScsiSense::NONE;
}
let response_code = sense[0] & 0x7F;
let descriptor = response_code == 0x72 || response_code == 0x73;
if descriptor {
// Descriptor format: key/asc/ascq are at fixed offsets 1/2/3.
let asc = if n >= 3 { sense[2] } else { 0 };
let ascq = if n >= 4 { sense[3] } else { 0 };
ScsiSense {
sense_key: sense[1] & 0x0F,
asc,
ascq,
}
} else {
// Fixed format: key at byte 2, ASC/ASCQ at bytes 12/13.
let asc = if n >= 13 { sense[12] } else { 0 };
let ascq = if n >= 14 { sense[13] } else { 0 };
ScsiSense {
sense_key: sense[2] & 0x0F,
asc,
ascq,
}
}
}
// ── SG_IO driver_status bits ────────────────────────────────────────────────
/// `DRIVER_SENSE` (0x08) — bit set in `driver_status` to indicate that
/// sense data was attached to a CHECK CONDITION reply. **Not** a transport
/// failure on its own. Mask this off before deciding whether `driver_status`
/// represents a real bus/host problem.
///
/// Used by Linux SG_IO (`sg_io_hdr.driver_status`); macOS IOKit and
/// Windows SPTI carry the equivalent signal in different fields and
/// don't need the same masking — the misclassification was Linux-only.
#[cfg(target_os = "linux")]
pub(crate) const DRIVER_SENSE: u16 = 0x08;
// ── Types ───────────────────────────────────────────────────────────────────
#[derive(Debug, Clone, Copy, PartialEq)]
@@ -82,34 +342,144 @@ pub fn open(device: &Path) -> Result<Box<dyn ScsiTransport>> {
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
{
Err(Error::DeviceNotFound {
path: format!("{}: unsupported platform", device.display()),
Err(Error::UnsupportedPlatform {
target: std::env::consts::OS.to_string(),
})
}
}
/// Reset a SCSI device to a known good state. Platform-specific.
/// On Linux: open/close fd cycle + TUR + SG_SCSI_RESET escalation.
pub fn reset(device: &Path) -> Result<()> {
// Note: a top-level `scsi::reset()` used to live here, wrapping a
// platform reset in a thread+recv_timeout so a kernel-wedged ioctl
// couldn't hang the caller. Removed in 0.13.6 along with the
// SG_SCSI_RESET / STOP+START UNIT escalation that needed it. The
// remaining platform reset (Linux: SgIoTransport::reset, available
// for explicit opt-in) does pure userspace state cleanup with bounded
// sleeps — no escape-hatch wrapper required.
// ── USB-layer recovery: rolled back in 0.13.4 ───────────────────────────────
//
// 0.13.1 – 0.13.3 exposed `scsi::usb_reset()` (`USBDEVFS_RESET` on Linux,
// `IOUSBDeviceInterface::ResetDevice` on macOS) and chained it into
// `drive_has_disc` recovery. Production testing on the LG BU40N USB BD-RE
// confirmed the USB stack resets succeed — dmesg logs
// `usb 3-2: reset high-speed USB device` and the device re-authorises —
// but the drive firmware below the USB bridge stays locked: LUN never
// re-enumerates, TUR still times out, the drive is unusable until
// physical unplug-replug or host reboot. Additional approaches tried
// and discarded: `authorized` 0→1 toggle, usb-storage driver
// unbind/rebind, forced SCSI host rescan, `STOP` + `START UNIT`.
//
// The APIs were removed so no caller can be misled into thinking a
// software-only recovery exists for this class of wedge. If a future
// hardware class surfaces where USB-layer recovery actually helps, the
// code should live here again, gated on a wedge signature — see git
// tag `v0.13.3` for the full implementation.
// ── Lightweight discovery + presence probes ─────────────────────────────────
//
// These two are the *only* hardware-touching APIs autorip + freemkv CLI use
// outside the rip path itself. They're intentionally cheap:
//
// - `list_drives()` is a one-shot enumeration: filesystem walk for sg/cdrom
// nodes, type-5 filter, single INQUIRY per candidate. No firmware, no
// reset-on-open, no init. Caller caches the result.
// - `drive_has_disc(path)` is a single TEST UNIT READY (six-byte CDB, no
// data transfer) with internal wedge-recovery escalation. Callers in a
// poll loop don't need any other primitive to detect "disc inserted /
// removed" — and they never see the SCSI-vs-USB-reset escalation.
//
// `Drive::open` + `drive.init()` + `Disc::scan` remain heavy and on-demand;
// callers only invoke them once they've decided to actually rip / verify a
// specific drive.
/// One optical drive on the system. Returned by [`list_drives`]. The
/// fields are populated from a single INQUIRY at enumeration time —
/// no firmware reset, no init.
#[derive(Debug, Clone)]
pub struct DriveInfo {
/// Platform device path: `/dev/sgN` (Linux), `/dev/diskN` (macOS),
/// `\\.\CdRomN` (Windows).
pub path: String,
/// SCSI INQUIRY vendor identifier (e.g. `"HL-DT-ST"`).
pub vendor: String,
/// SCSI INQUIRY product identifier (e.g. `"BD-RE BU40N"`).
pub model: String,
/// SCSI INQUIRY firmware revision (e.g. `"1.04"`).
pub firmware: String,
}
/// Enumerate optical drives present on the system.
///
/// **What it does**: per-platform sysfs / IOKit / setupapi walk for SCSI
/// devices, filtered to type 5 (CD/DVD/BD), with a single INQUIRY each
/// for vendor/model/firmware. No firmware reset, no `Drive::init`, no
/// disc scan. Suitable for an autorip-style poll loop or a CLI's
/// drive-list command.
///
/// **What it doesn't do**: probe disc presence (use [`drive_has_disc`]),
/// open a `Drive` for ripping (use [`crate::Drive::open`]), or load
/// drive profiles. Those are heavier operations callers invoke once
/// they've selected a drive.
pub fn list_drives() -> Vec<DriveInfo> {
#[cfg(target_os = "linux")]
{
linux::SgIoTransport::reset(device)
linux::list_drives()
}
#[cfg(target_os = "macos")]
{
macos::MacScsiTransport::reset(device)
macos::list_drives()
}
#[cfg(target_os = "windows")]
{
windows::SptiTransport::reset(device)
windows::list_drives()
}
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
{
let _ = device;
Ok(())
Vec::new()
}
}
/// True if the drive at `path` currently has a disc inserted.
///
/// Issues a single TEST UNIT READY (cheapest SCSI op, no data transfer).
/// Sense-key 2 ("not ready, medium not present") → `Ok(false)`; any
/// other ready/not-ready response → `Ok(true)` or interpreted ready
/// state. Suitable for poll-loop tick (~50 ms / drive on a healthy bus).
///
/// **Internal wedge recovery.** When the kernel's response indicates a
/// wedged target — the `0xff` status pattern that means "no answer from
/// the device" — this function transparently escalates: SCSI bus reset
/// → if still wedged → USB device reset (`USBDEVFS_RESET` on Linux) →
/// retry TUR. Callers never see wedge errors and never need to know
/// about the escalation; if even the recovery path can't get a response,
/// `Err(DeviceResetFailed)` surfaces. **No SCSI primitive is exposed to
/// outside crates** — autorip / freemkv CLI / bdemu use this single
/// function for the entire "is there a disc?" decision.
pub fn drive_has_disc(path: &Path) -> Result<bool> {
#[cfg(target_os = "linux")]
{
linux::drive_has_disc(path)
}
#[cfg(target_os = "macos")]
{
macos::drive_has_disc(path)
}
#[cfg(target_os = "windows")]
{
windows::drive_has_disc(path)
}
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
{
let _ = path;
Err(Error::UnsupportedPlatform {
target: std::env::consts::OS.to_string(),
})
}
}
@@ -206,3 +576,116 @@ pub fn build_read10_raw(lba: u32, count: u16) -> [u8; 10] {
0x00,
]
}
#[cfg(test)]
mod parse_sense_tests {
//! Unit tests for `parse_sense_key`. Covers both SPC-4 sense data
//! formats (descriptor / fixed) and the short-buffer fallback. The
//! same helper runs on every platform backend so a regression here
//! would silently miscategorize SCSI errors on Linux, macOS, and
//! Windows simultaneously.
use super::parse_sense;
fn parse_sense_key(sense: &[u8], sb_len_wr: u8) -> u8 {
parse_sense(sense, sb_len_wr).sense_key
}
/// Helper: build a 32-byte sense buffer whose first three bytes are
/// the given prefix; the rest are zeroes (sense data area).
fn buf(b0: u8, b1: u8, b2: u8) -> [u8; 32] {
let mut s = [0u8; 32];
s[0] = b0;
s[1] = b1;
s[2] = b2;
s
}
#[test]
fn descriptor_format_72_picks_byte_1() {
// Response code 0x72 (current, descriptor): sense key is the
// low nibble of byte 1. Byte 2 here is 0x77 to prove it is NOT
// the byte the parser reads.
let s = buf(0x72, 0x05, 0x77); // ILLEGAL REQUEST
assert_eq!(parse_sense_key(&s, 8), 5);
}
#[test]
fn descriptor_format_73_picks_byte_1() {
// Response code 0x73 (deferred, descriptor): same parse rule
// as 0x72.
let s = buf(0x73, 0x06, 0xFF); // UNIT ATTENTION
assert_eq!(parse_sense_key(&s, 8), 6);
}
#[test]
fn fixed_format_70_picks_byte_2() {
// Response code 0x70 (current, fixed): sense key is the low
// nibble of byte 2. Byte 1 is 0x77 to prove it is NOT read.
let s = buf(0x70, 0x77, 0x05); // ILLEGAL REQUEST
assert_eq!(parse_sense_key(&s, 18), 5);
}
#[test]
fn fixed_format_71_picks_byte_2() {
// Response code 0x71 (deferred, fixed): same parse as 0x70.
let s = buf(0x71, 0x77, 0x02); // NOT READY
assert_eq!(parse_sense_key(&s, 18), 2);
}
#[test]
fn high_bit_in_byte_0_is_masked() {
// SPC-4 sets the top bit of byte 0 ("INFORMATION VALID" / "VALID")
// independently of the response code. parse_sense_key must mask
// it off before classifying the format.
let s = buf(0xF2, 0x05, 0x77);
assert_eq!(
parse_sense_key(&s, 8),
5,
"VALID-bit must not leak into format detection"
);
let s = buf(0xF0, 0x77, 0x02);
assert_eq!(parse_sense_key(&s, 18), 2);
}
#[test]
fn high_nibble_in_key_byte_is_masked() {
// Sense key is byte_n & 0x0F (low nibble). Top nibble holds
// FILEMARK / EOM / ILI / SDAT_OVFL flags, which must not bleed
// into the key value.
let s = buf(0x70, 0x00, 0xE5); // 0xE0 flags + key 5
assert_eq!(parse_sense_key(&s, 18), 5);
}
#[test]
fn sb_len_wr_zero_returns_no_sense() {
// Transport set status non-zero but wrote zero sense bytes —
// SPC-4 §4.5.3 says treat as NO SENSE (key 0).
let s = buf(0x72, 0x05, 0x05);
assert_eq!(parse_sense_key(&s, 0), 0);
}
#[test]
fn sb_len_wr_below_three_returns_no_sense() {
// Less than three bytes in the buffer means we can't safely
// read either format byte 0 or key byte 2 — return 0.
let s = buf(0x72, 0x05, 0x05);
assert_eq!(parse_sense_key(&s, 1), 0);
assert_eq!(parse_sense_key(&s, 2), 0);
}
#[test]
fn slice_below_three_returns_no_sense() {
// Defense-in-depth: even if a caller passes a too-short slice
// with a falsely-large sb_len_wr, we don't panic and we return 0.
let s = [0x72u8, 0x05];
assert_eq!(parse_sense_key(&s, 8), 0);
}
#[test]
fn unknown_response_code_falls_through_to_fixed() {
// SPC-4 mandates implementations tolerate unknown response
// codes and treat them as fixed format. Vendor-specific codes
// in the 0x74..0x7E range surface here.
let s = buf(0x7A, 0x77, 0x03); // MEDIUM ERROR via "fixed"
assert_eq!(parse_sense_key(&s, 18), 3);
}
}
+77 -14
View File
@@ -56,7 +56,7 @@ struct SptwbDirect {
// ── Windows FFI ────────────────────────────────────────────────────────────
extern "system" {
unsafe extern "system" {
fn CreateFileW(
lpFileName: *const u16,
dwDesiredAccess: u32,
@@ -87,6 +87,10 @@ pub struct SptiTransport {
handle: isize,
}
// SptiTransport's only field is the isize HANDLE — Send is auto-derived
// and intentional. Sync is NOT: handle mutation in execute() requires
// &mut, enforced by the trait object dispatch.
/// Normalize a device path to Windows \\.\X: format.
///
/// NOTE: A near-identical `normalize_path` exists in `drive::windows`.
@@ -129,8 +133,17 @@ impl SptiTransport {
};
if handle == INVALID_HANDLE_VALUE {
return Err(Error::DeviceNotFound {
path: format!("{}: cannot open device (run as administrator)", dev_str),
// Map last-os-error → Error variant; don't embed English hints
// in the path field (the CLI handles localization).
let err = std::io::Error::last_os_error();
return Err(if err.kind() == std::io::ErrorKind::PermissionDenied {
Error::DevicePermission {
path: dev_str.to_string(),
}
} else {
Error::DeviceNotFound {
path: dev_str.to_string(),
}
});
}
@@ -187,10 +200,45 @@ impl SptiTransport {
}
}
/// Enumerate optical drives on Windows via `find_drives()` (CdRom0..15
/// scan) and re-shape into `DriveInfo`. Existing implementation already
/// returns `(path, DriveId)`; mapped here to the public struct.
pub(super) fn list_drives() -> Vec<super::DriveInfo> {
crate::drive::windows::find_drives()
.into_iter()
.map(|(path, id)| super::DriveInfo {
path,
vendor: id.vendor_id.trim().to_string(),
model: id.product_id.trim().to_string(),
firmware: id.product_revision.trim().to_string(),
})
.collect()
}
/// TEST UNIT READY probe on Windows. No in-library recovery — see the
/// Linux `drive_has_disc` doc block for the rationale.
pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
let mut transport = SptiTransport::open(path)?;
let cdb = [crate::scsi::SCSI_TEST_UNIT_READY, 0, 0, 0, 0, 0];
let mut buf = [0u8; 0];
match transport.execute(
&cdb,
crate::scsi::DataDirection::None,
&mut buf,
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
Err(ref e) if e.scsi_sense().is_some_and(|s| s.is_not_ready()) => Ok(false),
Err(e) => Err(e),
}
}
impl Drop for SptiTransport {
fn drop(&mut self) {
unsafe {
CloseHandle(self.handle);
if self.handle != INVALID_HANDLE_VALUE {
unsafe {
CloseHandle(self.handle);
}
}
}
}
@@ -221,7 +269,11 @@ impl ScsiTransport for SptiTransport {
DataDirection::ToDevice => SCSI_IOCTL_DATA_OUT,
};
sptwb.spt.DataTransferLength = data.len() as u32;
sptwb.spt.TimeOutValue = (timeout_ms / 1000).max(1) as u32;
// Round up to the next whole second so a 1500ms request gets at
// least 2s, not 1s. SPTI's TimeOutValue is u32 seconds with no
// sub-second resolution; biasing toward "more time" is safer than
// truncating (truncation broke 1500ms fast-reads on Drive::read).
sptwb.spt.TimeOutValue = ((timeout_ms + 999) / 1000).max(1);
sptwb.spt.DataBuffer = if data.is_empty() {
std::ptr::null_mut()
} else {
@@ -247,23 +299,34 @@ impl ScsiTransport for SptiTransport {
};
if ok == 0 {
// Driver-level failure (timeout, handle gone, etc.). Bubble
// up; in-library handle recovery was removed in 0.13.20 along
// with Linux's async fd-recovery and macOS's `try_recover` —
// the kernel mid-layer already did its escalation by the time
// DeviceIoControl returned, and re-issuing reset/reopen here
// is at best redundant and at worst deepens the wedge. Caller
// surfaces the failure to UX.
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
if sptwb.spt.ScsiStatus != 0 {
let sense_key = if sptwb.sense[2] != 0 {
sptwb.sense[2] & 0x0F
} else {
0
};
// SPTI doesn't surface a "bytes written into sense buffer"
// count separate from SenseInfoLength (input). Pass the full
// K_SENSE_SIZE; parse_sense keys off byte 0's response code
// to handle descriptor (0x72/0x73) vs fixed (0x70/0x71).
//
// 0.13.23: carry the full SPC-4 sense triple in
// `Error::ScsiError::sense` so callers can route on
// `ScsiSense::is_medium_error()` etc.
let parsed = super::parse_sense(&sptwb.sense, K_SENSE_SIZE as u8);
return Err(Error::ScsiError {
opcode: cdb[0],
status: sptwb.spt.ScsiStatus,
sense_key,
sense: Some(parsed),
});
}
-64
View File
@@ -1,64 +0,0 @@
//! SectorReader — trait for reading 2048-byte disc sectors.
//!
//! Implemented by Drive (SCSI) and IsoFile (file-backed).
//! Used by UDF parser, disc scanner, label parsers — anything that
//! reads sectors doesn't need to know where they come from.
use crate::error::Result;
/// Read 2048-byte sectors from a disc or disc image.
pub trait SectorReader: Send {
/// Read `count` sectors starting at `lba` into `buf`.
/// `buf` must be at least `count * 2048` bytes.
fn read_sectors(&mut self, lba: u32, count: u16, buf: &mut [u8]) -> Result<usize>;
/// Total capacity in sectors, if known.
fn capacity(&self) -> u32 {
0
}
}
/// SectorReader backed by a file (ISO image).
/// Seeks to lba * 2048, reads count * 2048 bytes.
pub struct FileSectorReader {
file: std::io::BufReader<std::fs::File>,
capacity: u32,
}
impl FileSectorReader {
pub fn open(path: &str) -> std::io::Result<Self> {
let file = std::fs::File::open(path)?;
let len = file.metadata()?.len();
let sectors = len / 2048;
if sectors > u32::MAX as u64 {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
format!("{path}: image too large, max ~8 TB"),
));
}
let capacity = sectors as u32;
Ok(Self {
file: std::io::BufReader::with_capacity(4 * 1024 * 1024, file),
capacity,
})
}
}
impl SectorReader for FileSectorReader {
fn read_sectors(&mut self, lba: u32, count: u16, buf: &mut [u8]) -> Result<usize> {
use std::io::{Read, Seek, SeekFrom};
let offset = lba as u64 * 2048;
let bytes = count as usize * 2048;
self.file
.seek(SeekFrom::Start(offset))
.map_err(|e| crate::error::Error::IoError { source: e })?;
self.file
.read_exact(&mut buf[..bytes])
.map_err(|e| crate::error::Error::IoError { source: e })?;
Ok(bytes)
}
fn capacity(&self) -> u32 {
self.capacity
}
}
+200
View File
@@ -0,0 +1,200 @@
//! `DecryptingSectorSource` — wrap any [`SectorSource`] to apply
//! AACS / CSS in-place decryption on every read.
//!
//! This is the 0.18 single-source-of-truth for decrypt-on-read. The
//! actual cipher code lives in [`crate::aacs`] and [`crate::css`];
//! we just call the existing [`crate::decrypt::decrypt_sectors`]
//! helper that already drives both of them. In follow-up commits
//! `sweep_pipeline` and `DiscStream` migrate onto this decorator
//! and delete their duplicate decrypt call sites.
//!
//! Composition: `Drive` → `DecryptingSectorSource` → caller sees
//! plaintext. For `DecryptKeys::None` discs the decorator is a
//! pass-through, so callers can wire it unconditionally and keep
//! their pipeline shape uniform regardless of encryption state.
use crate::decrypt::{DecryptKeys, decrypt_sectors};
use crate::error::Result;
use super::SectorSource;
/// Decorator: read from `inner`, then run the configured
/// AACS / CSS decrypt over the bytes that landed in `buf`.
///
/// `unit_key_idx` selects the AACS unit key for the disc (0 for
/// the vast majority of titles; the rare multi-CPS-unit discs pick
/// the index that covers the title being read). For
/// [`DecryptKeys::None`] and [`DecryptKeys::Css`] the index is
/// ignored.
pub struct DecryptingSectorSource<S: SectorSource> {
inner: S,
keys: DecryptKeys,
unit_key_idx: usize,
}
impl<S: SectorSource> DecryptingSectorSource<S> {
/// Wrap `inner` with the given keys. The default unit-key
/// index is 0; use [`with_unit_key_idx`] for the multi-CPS-unit
/// case.
///
/// [`with_unit_key_idx`]: Self::with_unit_key_idx
pub fn new(inner: S, keys: DecryptKeys) -> Self {
Self {
inner,
keys,
unit_key_idx: 0,
}
}
/// Override the AACS unit-key index. Only meaningful for
/// [`DecryptKeys::Aacs`]; other variants ignore it.
pub fn with_unit_key_idx(mut self, idx: usize) -> Self {
self.unit_key_idx = idx;
self
}
/// Replace the configured keys without unwrapping the decorator.
/// Used by `DiscStream::set_raw()` to flip from encrypted-disc
/// decryption to a pass-through after the inner reader is already
/// owned by the wrapper. For new construction prefer [`new`].
///
/// [`new`]: Self::new
pub fn set_keys(&mut self, keys: DecryptKeys) {
self.keys = keys;
}
/// Borrow the inner source. Useful for tests and for adapters
/// that want to introspect the underlying drive / file without
/// unwrapping the decorator.
pub fn inner(&self) -> &S {
&self.inner
}
/// Mutable borrow of the inner source.
pub fn inner_mut(&mut self) -> &mut S {
&mut self.inner
}
/// Consume the decorator and return the underlying source.
pub fn into_inner(self) -> S {
self.inner
}
}
impl<S: SectorSource> SectorSource for DecryptingSectorSource<S> {
fn capacity_sectors(&self) -> u32 {
self.inner.capacity_sectors()
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
let n = self.inner.read_sectors(lba, count, buf, recovery)?;
// Reuse the existing crate-wide decrypt entry point — same
// path the 0.17 sweep_pipeline and DiscStream call, so we
// inherit their AACS / CSS / None semantics verbatim. The
// helper is a no-op for DecryptKeys::None.
decrypt_sectors(&mut buf[..n], &self.keys, self.unit_key_idx)?;
Ok(n)
}
fn set_speed(&mut self, kbs: u16) {
self.inner.set_speed(kbs)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::error::Result;
/// Synthetic SectorSource that yields a deterministic byte
/// pattern keyed by LBA. Used to verify the decorator's
/// pass-through behaviour for `DecryptKeys::None`.
struct PatternedSource {
capacity: u32,
}
impl PatternedSource {
fn fill(lba: u32, count: u16, buf: &mut [u8]) {
let bytes = count as usize * 2048;
for (i, slot) in buf[..bytes].iter_mut().enumerate() {
let abs = lba as u64 * 2048 + i as u64;
*slot = ((abs.wrapping_mul(2654435761) >> 16) & 0xff) as u8;
}
}
}
impl SectorSource for PatternedSource {
fn capacity_sectors(&self) -> u32 {
self.capacity
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Self::fill(lba, count, buf);
Ok(count as usize * 2048)
}
}
#[test]
fn passthrough_with_no_keys() {
let src = PatternedSource { capacity: 16 };
let mut wrapped = DecryptingSectorSource::new(src, DecryptKeys::None);
// capacity_sectors delegates.
assert_eq!(wrapped.capacity_sectors(), 16);
let mut got = vec![0u8; 4 * 2048];
let n = wrapped.read_sectors(3, 4, &mut got, false).unwrap();
assert_eq!(n, 4 * 2048);
let mut expected = vec![0u8; 4 * 2048];
PatternedSource::fill(3, 4, &mut expected);
assert_eq!(got, expected);
}
#[test]
fn passthrough_set_speed_delegates() {
struct SpeedRecorder {
last: Option<u16>,
}
impl SectorSource for SpeedRecorder {
fn capacity_sectors(&self) -> u32 {
0
}
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
Ok(0)
}
fn set_speed(&mut self, kbs: u16) {
self.last = Some(kbs);
}
}
let mut wrapped =
DecryptingSectorSource::new(SpeedRecorder { last: None }, DecryptKeys::None);
wrapped.set_speed(7200);
assert_eq!(wrapped.inner().last, Some(7200));
}
// TODO: AACS round-trip test — needs a fixture-encrypted unit
// (6144-byte aligned) plus the matching unit key. The cipher
// path itself is exercised by `crate::aacs` unit tests; here
// we only assert the decorator wires the existing helper, not
// that AES-128 is correct.
}
+243
View File
@@ -0,0 +1,243 @@
//! File-backed sector I/O — read and write 2048-byte sectors against
//! an ISO image on disk.
//!
//! [`FileSectorSource`] is the read side (open-only). [`FileSectorSink`]
//! is the write side (create or open-rw); writes go through
//! [`crate::io::WritebackFile`] so big sequential ISO writes share
//! the same bounded-cache writeback pipeline used by sweep / patch /
//! mux.
use std::fs::{File, OpenOptions};
use std::io::{Read, Seek, SeekFrom, Write};
use std::path::Path;
use crate::error::{Error, Result};
use super::{SectorReader, SectorSink};
/// SectorSource backed by a file (ISO image).
///
/// Seeks to `lba * 2048`, reads `count * 2048` bytes per call. The
/// file is held directly: every `read_sectors` call performs an
/// absolute seek, so a wrapping `BufReader` would have its buffer
/// invalidated on every call (its internal cursor moves with the
/// `Seek` impl) — pure overhead. Callers that benefit from buffered
/// reads should compose their own `BufReader` at the `read_sectors`
/// granularity they care about.
pub struct FileSectorSource {
file: File,
capacity: u32,
}
impl FileSectorSource {
/// Open an existing ISO file for reading. Capacity is derived
/// from `metadata().len() / 2048`. Returns
/// [`Error::IsoTooLarge`] if the file would exceed the 32-bit
/// LBA address space (~8 TB).
pub fn open(path: &Path) -> std::io::Result<Self> {
let file = File::open(path)?;
let len = file.metadata()?.len();
let sectors = len / 2048;
if sectors > u32::MAX as u64 {
return Err(Error::IsoTooLarge {
path: path.to_string_lossy().into_owned(),
}
.into());
}
let capacity = sectors as u32;
Ok(Self { file, capacity })
}
}
// Implement the legacy `SectorReader` trait. The blanket impl in
// `super` produces the `SectorSource` impl automatically — no need
// to write both, and writing both would conflict. This keeps the
// 0.17 method-resolution path intact (callers with `SectorReader`
// in scope can still write `fsr.read_sectors(..)` against a
// `FileSectorSource`).
impl SectorReader for FileSectorSource {
fn capacity(&self) -> u32 {
self.capacity
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let offset = lba as u64 * 2048;
let bytes = count as usize * 2048;
self.file
.seek(SeekFrom::Start(offset))
.map_err(|e| Error::IoError { source: e })?;
self.file
.read_exact(&mut buf[..bytes])
.map_err(|e| Error::IoError { source: e })?;
Ok(bytes)
}
}
/// SectorSink backed by a file (ISO image).
///
/// Writes go through [`crate::io::WritebackFile`], which on Linux drives
/// continuous `sync_file_range` + `posix_fadvise(DONTNEED)` to keep
/// the kernel dirty page cache bounded during multi-GB sequential
/// writes. macOS / Windows fall through to a no-op pipeline.
///
/// `finish` runs `sync_all` before dropping the underlying file.
pub struct FileSectorSink {
inner: crate::io::WritebackFile,
}
impl FileSectorSink {
/// Create a new ISO file at `path`, truncating any existing
/// file. The file is opened read-write so the same handle can
/// later be reused for verification reads if needed (sweep
/// doesn't, but it costs nothing here).
pub fn create(path: &Path) -> std::io::Result<Self> {
let file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(true)
.open(path)?;
Ok(Self {
inner: crate::io::WritebackFile::new(file)?,
})
}
/// Open an existing ISO file for in-place updates (e.g. patch
/// pass writing recovered sectors over zero-filled holes).
/// Does not truncate.
pub fn open(path: &Path) -> std::io::Result<Self> {
let file = OpenOptions::new().read(true).write(true).open(path)?;
Ok(Self {
inner: crate::io::WritebackFile::new(file)?,
})
}
}
impl SectorSink for FileSectorSink {
fn write_sectors(&mut self, lba: u32, buf: &[u8]) -> Result<()> {
debug_assert!(
buf.len() % 2048 == 0,
"FileSectorSink::write_sectors: buf len {} not a multiple of 2048",
buf.len()
);
let offset = lba as u64 * 2048;
self.inner
.seek(SeekFrom::Start(offset))
.map_err(|e| Error::IoError { source: e })?;
self.inner
.write_all(buf)
.map_err(|e| Error::IoError { source: e })?;
Ok(())
}
fn finish(mut self: Box<Self>) -> Result<()> {
self.inner
.sync_all()
.map_err(|e| Error::IoError { source: e })?;
Ok(())
}
}
#[cfg(test)]
mod tests {
// Bring the 0.18 trait into scope (not super::*: the super
// module also re-exports the legacy `SectorReader`, and
// having both `SectorReader::read_sectors` and
// `SectorSource::read_sectors` visible would force every
// call site to disambiguate). External consumers see the
// same surface this test exercises.
use super::{FileSectorSink, FileSectorSource};
use crate::sector::{SectorSink, SectorSource};
use tempfile::tempdir;
#[test]
fn round_trip_single_sector() {
let dir = tempdir().unwrap();
let path = dir.path().join("rt.iso");
let mut sink = FileSectorSink::create(&path).unwrap();
// Pre-extend the file to 4 sectors of zeros so we can write
// sector 2 in place. Easiest way: write zeros first.
let zeros = [0u8; 4 * 2048];
sink.write_sectors(0, &zeros).unwrap();
let mut payload = [0u8; 2048];
for (i, b) in payload.iter_mut().enumerate() {
*b = (i as u8).wrapping_mul(17);
}
sink.write_sectors(2, &payload).unwrap();
Box::new(sink).finish().unwrap();
let mut src = FileSectorSource::open(&path).unwrap();
assert_eq!(src.capacity_sectors(), 4);
let mut got = [0u8; 2048];
let n = src.read_sectors(2, 1, &mut got, false).unwrap();
assert_eq!(n, 2048);
assert_eq!(got, payload);
// Sectors 0,1,3 still zero.
let mut z = [0xffu8; 2048];
src.read_sectors(0, 1, &mut z, false).unwrap();
assert!(z.iter().all(|b| *b == 0));
}
#[test]
fn round_trip_multi_sector() {
let dir = tempdir().unwrap();
let path = dir.path().join("multi.iso");
let mut sink = FileSectorSink::create(&path).unwrap();
let mut payload = vec![0u8; 8 * 2048];
for (i, b) in payload.iter_mut().enumerate() {
*b = ((i * 31) ^ (i >> 7)) as u8;
}
sink.write_sectors(0, &payload).unwrap();
Box::new(sink).finish().unwrap();
let mut src = FileSectorSource::open(&path).unwrap();
assert_eq!(src.capacity_sectors(), 8);
let mut got = vec![0u8; 8 * 2048];
let n = src.read_sectors(0, 8, &mut got, false).unwrap();
assert_eq!(n, 8 * 2048);
assert_eq!(got, payload);
}
#[test]
fn open_existing_does_not_truncate() {
let dir = tempdir().unwrap();
let path = dir.path().join("open.iso");
// Create with 4 sectors of pattern A.
let mut sink = FileSectorSink::create(&path).unwrap();
let pat_a = [0xaau8; 4 * 2048];
sink.write_sectors(0, &pat_a).unwrap();
Box::new(sink).finish().unwrap();
// Reopen and overwrite sector 1 only.
let mut sink = FileSectorSink::open(&path).unwrap();
let pat_b = [0xbbu8; 2048];
sink.write_sectors(1, &pat_b).unwrap();
Box::new(sink).finish().unwrap();
let mut src = FileSectorSource::open(&path).unwrap();
assert_eq!(src.capacity_sectors(), 4);
let mut got = [0u8; 2048];
src.read_sectors(0, 1, &mut got, false).unwrap();
assert_eq!(got, [0xaau8; 2048]);
src.read_sectors(1, 1, &mut got, false).unwrap();
assert_eq!(got, [0xbbu8; 2048]);
src.read_sectors(2, 1, &mut got, false).unwrap();
assert_eq!(got, [0xaau8; 2048]);
}
}
+202
View File
@@ -0,0 +1,202 @@
//! Sector-level I/O traits.
//!
//! 0.18 splits the unidirectional read trait from a write trait at
//! the sector layer, so the type system catches "wrong direction"
//! mistakes at compile time instead of runtime. See
//! `(internal)/memory/0_18_redesign.md`.
//!
//! - [`SectorSource`] reads 2048-byte sectors. Implemented by
//! `Drive` (via the legacy [`SectorReader`] alias) and
//! [`FileSectorSource`] (ISO-backed).
//! - [`SectorSink`] writes 2048-byte sectors. Implemented by
//! [`FileSectorSink`] (ISO-backed) and, in later commits, by
//! sweep/patch consumer adapters.
//! - [`DecryptingSectorSource`] is a decorator that wraps any
//! `SectorSource` and applies the existing AACS / CSS in-place
//! decrypt to plaintext-out.
//!
//! [`SectorReader`] is the 0.17 read trait. It stays on through
//! the 0.18 migration window so existing call sites
//! (`Drive`, `IsoSectorReader`, `BufferedSectorReader`,
//! `DiscStream`, `verify`) compile unchanged. A blanket impl
//! forwards every `SectorReader` impl to `SectorSource`, so new
//! code should target `SectorSource` / `SectorSink` directly. The
//! formal `#[deprecated]` attribute lands once the internal
//! callers have migrated; see the comment on `SectorReader` for
//! why this commit holds it back.
pub mod decrypting;
pub mod file;
use crate::error::Result;
/// Read 2048-byte sectors from a disc, image, or composed source.
///
/// Direction-typed: a `SectorSource` cannot be written to. Wrap the
/// inner source in [`DecryptingSectorSource`] to get plaintext
/// sectors out of an encrypted disc.
pub trait SectorSource: Send {
/// Total capacity in sectors, if known. Returns 0 when unknown
/// (e.g. live drives that haven't completed `READ CAPACITY` yet).
fn capacity_sectors(&self) -> u32;
/// Read `count` sectors starting at `lba` into `buf`.
/// `buf` must be at least `count * 2048` bytes.
/// `recovery`: true = full retry/reset loop (ripping),
/// false = single attempt (verify). File-backed sources ignore
/// the flag.
///
/// Returns the number of bytes written into `buf` on success.
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize>;
/// Optional speed control for sources that map to a physical
/// drive. No-op for everything else.
fn set_speed(&mut self, _kbs: u16) {}
}
/// Write 2048-byte sectors to a disc image or composed sink.
///
/// Direction-typed: a `SectorSink` cannot be read from. The
/// terminal [`finish`] takes `Box<Self>` so it can run on `dyn
/// SectorSink` and consume the sink (`fsync` + close).
///
/// [`finish`]: SectorSink::finish
pub trait SectorSink: Send {
/// Write the sectors in `buf` starting at `lba`. `buf.len()`
/// must be a multiple of 2048; the implementation seeks to
/// `lba * 2048` before writing.
fn write_sectors(&mut self, lba: u32, buf: &[u8]) -> Result<()>;
/// Flush, fsync, and close. Consumes the sink. Always called
/// last; subsequent operations are not defined.
fn finish(self: Box<Self>) -> Result<()>;
}
/// 0.17 read trait. Slated for removal once internal call sites
/// migrate to [`SectorSource`] in follow-up commits; until then
/// it remains the trait that `Drive`, `IsoSectorReader`,
/// `BufferedSectorReader`, and existing `&mut dyn SectorReader`
/// signatures use unchanged.
///
/// New code should implement [`SectorSource`] directly. The
/// blanket impl below makes any `SectorReader` automatically
/// usable wherever a `SectorSource` is expected, so a one-way
/// migration off `SectorReader` is possible per-callsite without
/// touching the impls.
//
// NOTE: not marked `#[deprecated]` in this commit — `cargo clippy
// -- -D warnings` (the CI gauntlet) treats deprecation as an
// error, and the existing `Drive` / `udf::BufferedSectorReader` /
// `mux::DiscStream` / `verify` call sites all go through this
// trait. The deprecation attribute lands together with the
// migration commits that move those call sites to
// `SectorSource`. The behavioural contract — "this trait is
// going away in 0.18" — is documented above and tracked in
// `(internal)/memory/0_18_redesign.md`.
pub trait SectorReader: Send {
/// Read `count` sectors starting at `lba` into `buf`.
/// See [`SectorSource::read_sectors`] for semantics.
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize>;
/// Total capacity in sectors, if known.
fn capacity(&self) -> u32 {
0
}
fn set_speed(&mut self, _kbs: u16) {}
}
// Blanket impl: anything implementing the legacy `SectorReader`
// trait automatically satisfies `SectorSource`. This is what keeps
// existing impls (`Drive`, `IsoSectorReader`, `BufferedSectorReader`,
// etc.) compiling without source changes during the migration. The
// reverse direction (impl SectorReader for SectorSource) is
// intentionally NOT provided — new code targets the new trait.
impl<T: SectorReader + ?Sized> SectorSource for T {
fn capacity_sectors(&self) -> u32 {
<T as SectorReader>::capacity(self)
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
<T as SectorReader>::read_sectors(self, lba, count, buf, recovery)
}
fn set_speed(&mut self, kbs: u16) {
<T as SectorReader>::set_speed(self, kbs)
}
}
// Forwarding impls so callers can wrap `&mut dyn SectorReader` /
// `Box<dyn SectorReader>` in [`DecryptingSectorSource`] without
// having to unbox or re-borrow inside the lib's hot paths. The
// generic `&mut T` / `Box<T>` blankets would conflict with the
// `SectorReader → SectorSource` blanket above (a downstream crate
// could `impl SectorReader for &mut U`); the specific
// `dyn SectorReader` instantiations are unambiguous because
// `SectorReader` is the very trait whose `dyn` we're targeting.
impl SectorSource for &mut (dyn SectorReader + '_) {
fn capacity_sectors(&self) -> u32 {
<dyn SectorReader as SectorReader>::capacity(*self)
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
<dyn SectorReader as SectorReader>::read_sectors(*self, lba, count, buf, recovery)
}
fn set_speed(&mut self, kbs: u16) {
<dyn SectorReader as SectorReader>::set_speed(*self, kbs)
}
}
impl SectorSource for Box<dyn SectorReader> {
fn capacity_sectors(&self) -> u32 {
<dyn SectorReader as SectorReader>::capacity(&**self)
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
recovery: bool,
) -> Result<usize> {
<dyn SectorReader as SectorReader>::read_sectors(&mut **self, lba, count, buf, recovery)
}
fn set_speed(&mut self, kbs: u16) {
<dyn SectorReader as SectorReader>::set_speed(&mut **self, kbs)
}
}
pub use decrypting::DecryptingSectorSource;
pub use file::{FileSectorSink, FileSectorSource};
// Backwards-compat alias for the public API. `FileSectorReader` is
// the 0.17 name; new code uses `FileSectorSource`. Both point at
// the same type. The `#[deprecated]` attribute lands together with
// the migration commits that retire the alias from internal uses.
pub type FileSectorReader = FileSectorSource;
+52 -17
View File
@@ -101,7 +101,7 @@ impl UdfFs {
None => {
return Err(Error::UdfNotFound {
path: path.to_string(),
})
});
}
};
let entry = current
@@ -136,7 +136,7 @@ impl UdfFs {
None => {
return Err(Error::UdfNotFound {
path: path.to_string(),
})
});
}
};
let entry = current
@@ -278,10 +278,11 @@ impl UdfFs {
/// The data_lba is partition-relative.
fn read_icb_extent(&self, reader: &mut dyn SectorReader, meta_lba: u32) -> Result<(u32, u32)> {
let extents = self.read_icb_extents(reader, meta_lba)?;
extents
.first()
.copied()
.ok_or(Error::DiscRead { sector: 0 })
extents.first().copied().ok_or(Error::DiscRead {
sector: 0,
status: None,
sense: None,
})
}
/// Read ALL allocation extents for a file from its ICB.
@@ -307,6 +308,8 @@ impl UdfFs {
if ad_offset + l_ad > icb.len() {
return Err(Error::DiscRead {
sector: self.meta_to_abs(meta_lba) as u64,
status: None,
sense: None,
});
}
(ad_offset, l_ad)
@@ -319,11 +322,19 @@ impl UdfFs {
if ad_offset + l_ad > icb.len() {
return Err(Error::DiscRead {
sector: self.meta_to_abs(meta_lba) as u64,
status: None,
sense: None,
});
}
(ad_offset, l_ad)
}
_ => return Err(Error::DiscRead { sector: 0 }),
_ => {
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
};
let mut extents = Vec::new();
@@ -375,7 +386,7 @@ impl UdfFs {
None => {
return Err(Error::UdfNotFound {
path: path.to_string(),
})
});
}
};
let entry = current
@@ -414,7 +425,11 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
let tag_id = u16::from_le_bytes([avdp[0], avdp[1]]);
if tag_id != 2 {
return Err(Error::DiscRead { sector: 0 });
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
// Main VDS extent location: bytes [16:20] = LBA, [20:24] = length
@@ -455,14 +470,22 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
}
if partition_start == 0 {
return Err(Error::DiscRead { sector: 0 });
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
// Step 3: Parse partition maps from LVD to find metadata partition
// BD-ROM discs (UDF 2.50) use a metadata partition (Type 2 map with "*UDF Metadata Partition")
// The metadata file is stored at lba=0 of the physical partition
let metadata_start = if num_partition_maps >= 2 {
let lvd_sec = lvd_sector.ok_or(Error::DiscRead { sector: 0 })?;
let lvd_sec = lvd_sector.ok_or(Error::DiscRead {
sector: 0,
status: None,
sense: None,
})?;
// Read LVD to check partition map type
let mut lvd = [0u8; 2048];
@@ -497,6 +520,8 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
if ad_off + 8 > meta_icb.len() {
return Err(Error::DiscRead {
sector: meta_file_lba as u64,
status: None,
sense: None,
});
}
let ad_len = u32::from_le_bytes([
@@ -536,7 +561,11 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
let fsd_tag = u16::from_le_bytes([fsd[0], fsd[1]]);
if fsd_tag != 256 {
return Err(Error::DiscRead { sector: 0 });
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
// Root Directory ICB: long_ad at FSD offset 400
@@ -585,6 +614,8 @@ fn read_directory(
if ad_off + 8 > icb.len() {
return Err(Error::DiscRead {
sector: (meta_start + meta_lba) as u64,
status: None,
sense: None,
});
}
let len = u32::from_le_bytes([
@@ -607,6 +638,8 @@ fn read_directory(
if ad_off + 8 > icb.len() {
return Err(Error::DiscRead {
sector: (meta_start + meta_lba) as u64,
status: None,
sense: None,
});
}
let len = u32::from_le_bytes([
@@ -879,6 +912,7 @@ impl BufferedSectorReader<'_> {
start_lba + offset,
batch,
&mut self.cache[buf_off..buf_off + batch as usize * 2048],
true,
)
.is_err()
{
@@ -902,7 +936,7 @@ impl BufferedSectorReader<'_> {
let bytes = batch as usize * 2048;
if self
.inner
.read_sectors(start + offset, batch, &mut tmp[..bytes])
.read_sectors(start + offset, batch, &mut tmp[..bytes], true)
.is_err()
{
break;
@@ -924,6 +958,7 @@ impl SectorReader for BufferedSectorReader<'_> {
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> std::result::Result<usize, crate::error::Error> {
if count == 1 {
// Check permanent prefetch cache first (HashMap)
@@ -939,7 +974,7 @@ impl SectorReader for BufferedSectorReader<'_> {
}
let block = self.batch;
self.cache.resize(block as usize * 2048, 0);
match self.inner.read_sectors(lba, block, &mut self.cache) {
match self.inner.read_sectors(lba, block, &mut self.cache, true) {
Ok(_) => {
self.cache_start = lba;
self.cache_sectors = block as u32;
@@ -947,7 +982,7 @@ impl SectorReader for BufferedSectorReader<'_> {
Err(_) => {
// Near end of disc or error — single sector fallback
self.cache.resize(2048, 0);
self.inner.read_sectors(lba, 1, &mut self.cache)?;
self.inner.read_sectors(lba, 1, &mut self.cache, true)?;
self.cache_start = lba;
self.cache_sectors = 1;
}
@@ -956,12 +991,12 @@ impl SectorReader for BufferedSectorReader<'_> {
Ok(2048)
} else {
// Multi-sector read — pass through
self.inner.read_sectors(lba, count, buf)
self.inner.read_sectors(lba, count, buf, true)
}
}
}
fn read_sector(reader: &mut dyn SectorReader, lba: u32, buf: &mut [u8]) -> Result<()> {
reader.read_sectors(lba, 1, buf)?;
reader.read_sectors(lba, 1, buf, true)?;
Ok(())
}
+260
View File
@@ -0,0 +1,260 @@
//! Disc sector verification — read every sector and classify health.
use crate::disc::{Chapter, DiscTitle};
use crate::progress::Progress;
use crate::sector::SectorReader;
use std::time::Instant;
/// Health status of a single sector read.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum SectorStatus {
/// Read succeeded on first attempt, under 500ms
Good,
/// Read succeeded but took longer than expected (>500ms per sector)
Slow,
/// Read failed initially but succeeded on retry
Recovered,
/// Read failed all attempts — unrecoverable
Bad,
}
/// A contiguous range of bad or slow sectors.
#[derive(Debug, Clone)]
pub struct SectorRange {
pub start_lba: u32,
pub count: u32,
pub status: SectorStatus,
/// Byte offset from start of title
pub byte_offset: u64,
}
/// Result of a disc verification scan.
#[derive(Debug, Clone)]
pub struct VerifyResult {
pub total_sectors: u64,
pub good: u64,
pub slow: u64,
pub recovered: u64,
pub bad: u64,
/// Contiguous ranges of non-good sectors
pub ranges: Vec<SectorRange>,
pub elapsed_secs: f64,
}
impl VerifyResult {
/// Percentage of sectors that are fully readable (good + slow + recovered).
pub fn readable_pct(&self) -> f64 {
if self.total_sectors == 0 {
return 100.0;
}
(self.total_sectors - self.bad) as f64 / self.total_sectors as f64 * 100.0
}
/// True if every sector read successfully.
pub fn is_perfect(&self) -> bool {
self.bad == 0 && self.recovered == 0 && self.slow == 0
}
/// Map a bad sector range to a chapter timestamp.
pub fn chapter_at_offset(
chapters: &[Chapter],
byte_offset: u64,
duration_secs: f64,
total_bytes: u64,
) -> Option<(usize, f64)> {
if total_bytes == 0 || chapters.is_empty() {
return None;
}
let time_secs = byte_offset as f64 / total_bytes as f64 * duration_secs;
let mut chapter_idx = 0;
for (i, ch) in chapters.iter().enumerate() {
if ch.time_secs <= time_secs {
chapter_idx = i;
} else {
break;
}
}
Some((chapter_idx + 1, time_secs))
}
}
/// Verify all sectors in a title's extents.
/// Reads in batches for speed, falls back to single-sector on failure.
/// The progress callback returns false to request early stop.
pub fn verify_title(
reader: &mut dyn SectorReader,
title: &DiscTitle,
batch_sectors: u16,
on_progress: Option<&dyn Progress>,
) -> VerifyResult {
let start = Instant::now();
let mut good: u64 = 0;
let mut slow: u64 = 0;
let mut recovered: u64 = 0;
let mut bad: u64 = 0;
let mut ranges: Vec<SectorRange> = Vec::new();
let mut sectors_done: u64 = 0;
let mut byte_offset: u64 = 0;
let total_sectors: u64 = title.extents.iter().map(|e| e.sector_count as u64).sum();
let mut buf = vec![0u8; batch_sectors as usize * 2048];
'outer: for ext in &title.extents {
let mut offset: u32 = 0;
while offset < ext.sector_count {
let remaining = ext.sector_count - offset;
let count = remaining.min(batch_sectors as u32) as u16;
let lba = ext.start_lba + offset;
let bytes = count as usize * 2048;
let batch_start = Instant::now();
let batch_ok = reader
.read_sectors(lba, count, &mut buf[..bytes], false)
.is_ok();
let batch_ms = batch_start.elapsed().as_millis();
if batch_ok {
// Batch succeeded — classify by speed
let status = if batch_ms > count as u128 * 500 {
SectorStatus::Slow
} else {
SectorStatus::Good
};
match status {
SectorStatus::Good => good += count as u64,
SectorStatus::Slow => {
slow += count as u64;
ranges.push(SectorRange {
start_lba: lba,
count: count as u32,
status: SectorStatus::Slow,
byte_offset,
});
}
_ => {}
}
sectors_done += count as u64;
if let Some(cb) = on_progress {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Verify,
work_done: sectors_done,
work_total: total_sectors,
bytes_good_total: (good + slow + recovered) * 2048,
bytes_unreadable_total: bad * 2048,
bytes_pending_total: 0,
bytes_total_disc: total_sectors * 2048,
disc_duration_secs: Some(title.duration_secs),
bytes_bad_in_main_title: 0,
main_title_duration_secs: Some(title.duration_secs),
main_title_size_bytes: Some(total_sectors * 2048),
};
if !cb.report(&pp) {
break 'outer;
}
}
} else {
// Batch failed — test each sector individually
for i in 0..count {
let sector_lba = lba + i as u32;
let sector_offset = i as usize * 2048;
let sector_byte_offset = byte_offset + i as u64 * 2048;
let s1 = Instant::now();
let first_ok = reader
.read_sectors(
sector_lba,
1,
&mut buf[sector_offset..sector_offset + 2048],
false,
)
.is_ok();
let s1_ms = s1.elapsed().as_millis();
let status = if first_ok && s1_ms <= 500 {
good += 1;
SectorStatus::Good
} else if first_ok {
slow += 1;
SectorStatus::Slow
} else {
// Retry once more after brief pause
std::thread::sleep(std::time::Duration::from_secs(2));
if reader
.read_sectors(
sector_lba,
1,
&mut buf[sector_offset..sector_offset + 2048],
false,
)
.is_ok()
{
recovered += 1;
SectorStatus::Recovered
} else {
bad += 1;
SectorStatus::Bad
}
};
if status != SectorStatus::Good {
// Merge with previous range if contiguous and same status
if let Some(last) = ranges.last_mut() {
if last.status == status && last.start_lba + last.count == sector_lba {
last.count += 1;
} else {
ranges.push(SectorRange {
start_lba: sector_lba,
count: 1,
status,
byte_offset: sector_byte_offset,
});
}
} else {
ranges.push(SectorRange {
start_lba: sector_lba,
count: 1,
status,
byte_offset: sector_byte_offset,
});
}
}
sectors_done += 1;
if let Some(cb) = on_progress {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Verify,
work_done: sectors_done,
work_total: total_sectors,
bytes_good_total: (good + slow + recovered) * 2048,
bytes_unreadable_total: bad * 2048,
bytes_pending_total: 0,
bytes_total_disc: total_sectors * 2048,
disc_duration_secs: Some(title.duration_secs),
bytes_bad_in_main_title: 0,
main_title_duration_secs: Some(title.duration_secs),
main_title_size_bytes: Some(total_sectors * 2048),
};
if !cb.report(&pp) {
break 'outer;
}
}
}
}
offset += count as u32;
byte_offset += bytes as u64;
}
}
VerifyResult {
total_sectors,
good,
slow,
recovered,
bad,
ranges,
elapsed_secs: start.elapsed().as_secs_f64(),
}
}
+11 -6
View File
@@ -73,8 +73,8 @@ fn css_is_scrambled_detection() {
/// then decrypt with decrypt_unit() and verify the plaintext matches.
#[test]
fn aacs_decrypt_unit_roundtrip() {
use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit};
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
let unit_key = [0xAAu8; 16];
let aacs_iv: [u8; 16] = [
@@ -207,8 +207,8 @@ fn aacs_disc_hash_deterministic() {
/// decrypt_unit_key recovers the original.
#[test]
fn aacs_decrypt_unit_key_roundtrip() {
use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit};
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
let vuk = [
0x11u8, 0x14, 0x36, 0x0B, 0x10, 0xEE, 0x6E, 0xAC, 0x78, 0xAA, 0x4A, 0xC0, 0xB7, 0x52, 0xEA,
@@ -309,8 +309,8 @@ fn aacs_decrypt_unit_unencrypted_passthrough() {
/// Independent AES-128-ECB encrypt (uses `aes` crate directly, NOT our library).
fn ref_aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit};
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut block = GenericArray::clone_from_slice(data);
cipher.encrypt_block(&mut block);
@@ -321,8 +321,8 @@ fn ref_aes_ecb_encrypt(key: &[u8; 16], data: &[u8; 16]) -> [u8; 16] {
/// Independent AES-128-CBC encrypt (uses `aes` crate directly, NOT our library).
fn ref_aes_cbc_encrypt(key: &[u8; 16], iv: &[u8; 16], data: &mut [u8]) {
use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit};
use aes::Aes128;
use aes::cipher::{BlockEncrypt, KeyInit, generic_array::GenericArray};
let cipher = Aes128::new(GenericArray::from_slice(key));
let mut prev = *iv;
let num_blocks = data.len() / 16;
@@ -361,7 +361,9 @@ fn aacs_cross_validation_encrypt_then_decrypt() {
plaintext[off] = 0x47;
off += 192;
}
// Fill the rest with a recognisable pattern (prime modulus avoids artefacts)
// Fill the rest with a recognisable pattern (prime modulus avoids artefacts).
// Index-arithmetic — clearer with a counted loop than an enumerate chain.
#[allow(clippy::needless_range_loop)]
for i in 16..aacs::ALIGNED_UNIT_LEN {
if plaintext[i] == 0 {
plaintext[i] = (i % 251) as u8;
@@ -462,6 +464,7 @@ fn aacs_bus_decrypt_cross_validation() {
];
let mut plaintext = vec![0u8; aacs::ALIGNED_UNIT_LEN];
#[allow(clippy::needless_range_loop)]
for i in 0..aacs::ALIGNED_UNIT_LEN {
plaintext[i] = ((i * 3 + 17) & 0xFF) as u8;
}
@@ -494,6 +497,7 @@ fn css_roundtrip_with_snapshot() {
let mut sector = vec![0x00u8; 2048];
sector[0x14] = 0x30;
sector[0x54..0x59].copy_from_slice(&[0xDE, 0xAD, 0xBE, 0xEF, 0x42]);
#[allow(clippy::needless_range_loop)]
for i in 0x80..2048 {
sector[i] = ((i * 7 + 3) & 0xFF) as u8;
}
@@ -661,6 +665,7 @@ fn css_recover_title_key_with_exact_plaintext() {
sector[0x54..0x59].copy_from_slice(&seed);
let pes_header: [u8; 10] = [0x00, 0x00, 0x01, 0xE0, 0x00, 0x00, 0x80, 0x80, 0x05, 0x21];
sector[0x80..0x8A].copy_from_slice(&pes_header);
#[allow(clippy::needless_range_loop)]
for i in 0x8A..2048 {
sector[i] = ((i * 13 + 7) & 0xFF) as u8;
}
@@ -703,7 +708,7 @@ fn aacs_parse_unit_key_ro_minimal() {
data[0..4].copy_from_slice(&uk_pos.to_be_bytes());
// app_type
data[16] = 1; // BD-ROM
// num_bdmv_dir
// num_bdmv_dir
data[17] = 1;
// flags
data[18] = 0;
+17 -5
View File
@@ -1,7 +1,7 @@
//! Disc scanning pipeline tests.
use libfreemkv::error::Result;
use libfreemkv::SectorReader;
use libfreemkv::error::Result;
use libfreemkv::{Disc, DiscTitle, ScanOptions};
use std::collections::HashMap;
@@ -21,7 +21,13 @@ impl MockSectorReader {
}
impl SectorReader for MockSectorReader {
fn read_sectors(&mut self, lba: u32, count: u16, buf: &mut [u8]) -> Result<usize> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let total = count as usize * SECTOR_SIZE;
for i in 0..count as u32 {
let offset = i as usize * SECTOR_SIZE;
@@ -127,7 +133,9 @@ fn scan_options_default() {
#[test]
fn scan_options_with_keydb() {
let opts = ScanOptions::with_keydb("/tmp/KEYDB.cfg");
let opts = ScanOptions {
keydb_path: Some(("/tmp/KEYDB.cfg").into()),
};
assert_eq!(
opts.keydb_path.as_ref().unwrap().to_str().unwrap(),
"/tmp/KEYDB.cfg"
@@ -137,7 +145,9 @@ fn scan_options_with_keydb() {
#[test]
fn scan_options_with_keydb_pathbuf() {
let path = std::path::PathBuf::from("/home/user/.config/aacs/KEYDB.cfg");
let opts = ScanOptions::with_keydb(path.clone());
let opts = ScanOptions {
keydb_path: Some(path.clone()),
};
assert_eq!(opts.keydb_path.unwrap(), path);
}
@@ -505,7 +515,9 @@ fn resolve_encryption_no_keydb() {
build_udf_with_aacs_dir(&mut reader);
// No keydb configured and no standard keydb on the system
let opts = ScanOptions::with_keydb("/nonexistent/path/KEYDB.cfg");
let opts = ScanOptions {
keydb_path: Some(("/nonexistent/path/KEYDB.cfg").into()),
};
let disc = Disc::scan_image(&mut reader, 1000, &opts).unwrap();
// The disc detects encryption but can't resolve keys without a keydb
+802
View File
@@ -0,0 +1,802 @@
//! Integration tests for progress reporting, halt behavior, drop safety,
//! and the file-backed sector reader round trip.
// 0.18 trait split: this suite still drives the deprecated `pes::Stream`
// trait directly. It will be migrated to `FrameSource`/`FrameSink` in the
// follow-up that ports concrete impls.
#![allow(deprecated)]
use libfreemkv::disc::{CopyOptions, DiscRegion};
use libfreemkv::error::Result;
use libfreemkv::pes::Stream as PesStream;
use libfreemkv::{
ContentFormat, Disc, DiscFormat, DiscStream, DiscTitle, EventKind, Extent, FileSectorReader,
SectorReader,
};
use std::io::Write;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::time::{Duration, Instant};
const SECTOR_SIZE: usize = 2048;
// ── helpers ────────────────────────────────────────────────────────────────
/// Returns zeroed sectors. Always succeeds. Counts each call.
struct ZeroSectorReader {
capacity: u32,
calls: Arc<AtomicU64>,
}
impl ZeroSectorReader {
fn new(capacity: u32) -> Self {
Self {
capacity,
calls: Arc::new(AtomicU64::new(0)),
}
}
}
impl SectorReader for ZeroSectorReader {
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
self.calls.fetch_add(1, Ordering::Relaxed);
let bytes = count as usize * SECTOR_SIZE;
buf[..bytes].fill(0);
Ok(bytes)
}
fn capacity(&self) -> u32 {
self.capacity
}
}
/// Like ZeroSectorReader but sleeps a configurable duration per call.
/// Used by the halt test so the copy takes >1 s.
struct SlowZeroSectorReader {
capacity: u32,
sleep_per_call: Duration,
}
impl SlowZeroSectorReader {
fn new(capacity: u32, sleep_per_call: Duration) -> Self {
Self {
capacity,
sleep_per_call,
}
}
}
impl SectorReader for SlowZeroSectorReader {
fn read_sectors(
&mut self,
_lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
std::thread::sleep(self.sleep_per_call);
let bytes = count as usize * SECTOR_SIZE;
buf[..bytes].fill(0);
Ok(bytes)
}
fn capacity(&self) -> u32 {
self.capacity
}
}
/// Build a Disc instance with a known capacity, no titles, no encryption.
/// Sufficient for `Disc::copy` (which only uses capacity_sectors + decrypt keys).
fn synthetic_disc(capacity_sectors: u32) -> Disc {
Disc {
volume_id: String::new(),
meta_title: None,
format: DiscFormat::BluRay,
capacity_sectors,
capacity_bytes: capacity_sectors as u64 * SECTOR_SIZE as u64,
layers: 1,
titles: Vec::new(),
region: DiscRegion::Free,
aacs: None,
css: None,
encrypted: false,
content_format: ContentFormat::BdTs,
}
}
/// Build a DiscTitle with a single extent of `sector_count` sectors and no
/// streams (DiscStream still iterates sectors and would emit BytesRead).
fn synthetic_title(sector_count: u32) -> DiscTitle {
DiscTitle {
playlist: String::new(),
playlist_id: 0,
duration_secs: 0.0,
size_bytes: sector_count as u64 * SECTOR_SIZE as u64,
clips: Vec::new(),
streams: Vec::new(),
chapters: Vec::new(),
extents: vec![Extent {
start_lba: 0,
sector_count,
}],
content_format: ContentFormat::BdTs,
codec_privates: Vec::new(),
}
}
// ── 1. BytesRead events emitted during disc copy (TDD red) ────────────────
#[test]
fn test_bytes_read_emitted_during_disc_copy() {
// Build a tiny synthetic disc and stream it through DiscStream.
let reader = ZeroSectorReader::new(64);
let title = synthetic_title(64);
let keys = libfreemkv::DecryptKeys::None;
let mut stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
let count = Arc::new(AtomicU64::new(0));
let count_cb = count.clone();
stream.on_event(move |ev| {
if let EventKind::BytesRead { .. } = ev.kind {
count_cb.fetch_add(1, Ordering::Relaxed);
}
});
// Drive the stream to EOF. With no streams configured, read() returns
// Ok(None) once all extents are exhausted.
loop {
match stream.read() {
Ok(Some(_frame)) => {}
Ok(None) => break,
Err(e) => panic!("stream read failed: {e:?}"),
}
}
let n = count.load(Ordering::Relaxed);
// EXPECTED TO FAIL until BytesRead emission is wired up. TDD red.
assert!(
n > 0,
"expected at least one BytesRead event, got {n} (lib does not yet emit BytesRead)"
);
}
// ── 2. Disc::copy on_progress callback fires (regression guard) ───────────
#[test]
fn test_disc_copy_progress_callback_fires() {
let disc = synthetic_disc(64);
let mut reader = ZeroSectorReader::new(64);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp); // we want the path, not the file handle
let calls = Arc::new(AtomicU64::new(0));
let last_bytes = Arc::new(AtomicU64::new(0));
struct CountingReporter {
calls: Arc<AtomicU64>,
last_bytes: Arc<AtomicU64>,
}
impl libfreemkv::progress::Progress for CountingReporter {
fn report(&self, p: &libfreemkv::progress::PassProgress) -> bool {
self.calls.fetch_add(1, Ordering::Relaxed);
self.last_bytes.store(p.bytes_good_total, Ordering::Relaxed);
true
}
}
let reporter = CountingReporter {
calls: calls.clone(),
last_bytes: last_bytes.clone(),
};
let opts = CopyOptions {
decrypt: false,
progress: Some(&reporter),
..Default::default()
};
let result = disc.copy(&mut reader, &iso_path, &opts).expect("copy ok");
// Cleanup any sidecar mapfile + ISO before assertions.
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
assert!(result.complete, "copy should be complete");
let n = calls.load(Ordering::Relaxed);
let last = last_bytes.load(Ordering::Relaxed);
assert!(n > 0, "on_progress should fire at least once, got {n}");
assert!(
last > 0,
"final progress bytes should be non-zero, got {last}"
);
}
// ── 3. Halt aborts disc copy promptly ─────────────────────────────────────
#[test]
fn test_halt_aborts_disc_copy_promptly() {
// 6000 sectors, 60-sector batches → 100 read_sectors() calls.
// 10 ms sleep per call → ~1 s total without halt.
let capacity_sectors: u32 = 6000;
let mut reader = SlowZeroSectorReader::new(capacity_sectors, Duration::from_millis(10));
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let halt = Arc::new(AtomicBool::new(false));
let halt_for_thread = halt.clone();
let iso_path_for_thread = iso_path.clone();
let join = std::thread::spawn(move || {
let opts = CopyOptions {
decrypt: false,
halt: Some(halt_for_thread),
..Default::default()
};
let t0 = Instant::now();
let res = disc.copy(&mut reader, &iso_path_for_thread, &opts);
(res, t0.elapsed())
});
// Let copy run, then halt.
std::thread::sleep(Duration::from_millis(200));
halt.store(true, Ordering::Relaxed);
// Bound the join: should exit far before the full 1 s otherwise needed.
let started = Instant::now();
let mut joined = None;
while started.elapsed() < Duration::from_millis(2000) {
if join.is_finished() {
joined = Some(join.join().expect("thread join"));
break;
}
std::thread::sleep(Duration::from_millis(20));
}
let (result, elapsed) = joined.expect("copy thread did not exit within 2s of halt");
// Cleanup
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
let copy_result = result.expect("copy returns Ok with halted=true on halt");
assert!(
copy_result.halted,
"copy_result.halted should be true after halt"
);
assert!(
!copy_result.complete,
"copy_result.complete should be false when halted"
);
assert!(
elapsed < Duration::from_millis(2000),
"copy thread exit elapsed {elapsed:?} exceeded 2s"
);
}
// ── 4. DiscStream Drop does not panic or block ────────────────────────────
#[test]
fn test_drop_impls_do_not_panic_or_block() {
let reader = ZeroSectorReader::new(64);
let title = synthetic_title(64);
let keys = libfreemkv::DecryptKeys::None;
let stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
// Drop on a worker thread; main thread enforces the timeout.
let handle = std::thread::spawn(move || {
drop(stream);
});
let started = Instant::now();
while started.elapsed() < Duration::from_millis(100) {
if handle.is_finished() {
handle.join().expect("drop thread join");
return;
}
std::thread::sleep(Duration::from_millis(5));
}
panic!("DiscStream drop did not complete within 100ms");
}
// ── 5. FileSectorReader round trip ────────────────────────────────────────
#[test]
fn test_file_sector_reader_round_trip() {
// Build 8 sectors of pseudo-random bytes (sector-aligned).
const N_SECTORS: usize = 8;
let mut data = vec![0u8; N_SECTORS * SECTOR_SIZE];
for (i, b) in data.iter_mut().enumerate() {
// Cheap PRNG: just a multiplicative pattern, deterministic for asserts.
*b = ((i as u64).wrapping_mul(2654435761) >> 16) as u8;
}
let mut tmp = tempfile::NamedTempFile::new().expect("tempfile create");
tmp.write_all(&data).expect("write data");
tmp.flush().expect("flush");
let path = tmp.path().to_path_buf();
let mut fsr = FileSectorReader::open(&path).expect("open FileSectorReader");
assert_eq!(fsr.capacity(), N_SECTORS as u32, "capacity mismatch");
// Read each sector individually and compare.
let mut buf = vec![0u8; SECTOR_SIZE];
for lba in 0..N_SECTORS as u32 {
let n = fsr
.read_sectors(lba, 1, &mut buf, false)
.expect("read_sectors");
assert_eq!(n, SECTOR_SIZE);
let off = lba as usize * SECTOR_SIZE;
assert_eq!(
&buf[..],
&data[off..off + SECTOR_SIZE],
"sector {lba} mismatch"
);
}
// Read all sectors at once and compare.
let mut all = vec![0u8; N_SECTORS * SECTOR_SIZE];
let n = fsr
.read_sectors(0, N_SECTORS as u16, &mut all, false)
.expect("read all sectors");
assert_eq!(n, N_SECTORS * SECTOR_SIZE);
assert_eq!(all, data, "bulk read mismatch");
}
// ── 6. Pass 1 sweeps the entire disc even when every read fails ───────────
//
// Per RIP_DESIGN.md §2.1 + §3: Disc::copy must reach the end of the disc
// regardless of how many reads fail. The only legitimate early exit is the
// halt flag. With `skip_on_error` and a reader that returns
// Err for every read, Pass 1 must:
// - mark every sector NonTrimmed (so Pass 2 can retry them)
// - return cleanly (no panic, no hang)
// - bytes_good = 0
// - bytes_pending = total_bytes (NonTrimmed counts as pending in mapfile
// accounting; see disc/mapfile.rs::stats)
// - bytes_unreadable = 0 (only Pass 2 marks Unreadable)
// - complete = false (work remains for Pass 2)
// - halted = false (no user stop)
// - ISO file is `total_bytes` size on disk (sparse zeros)
/// Reader that returns Err for every read. Optionally signals a halt
/// flag on the first read so tests can exercise the halt-during-skip-forward
/// path deterministically (no wallclock dependency).
struct FailingSectorReader {
capacity: u32,
/// If set, signals halt on the first `read_sectors` call. Cleared after
/// the first signal so subsequent reads are plain Err.
halt_on_first_read: Option<Arc<AtomicBool>>,
}
impl FailingSectorReader {
fn new(capacity: u32) -> Self {
Self {
capacity,
halt_on_first_read: None,
}
}
fn with_halt_on_first_read(capacity: u32, halt: Arc<AtomicBool>) -> Self {
Self {
capacity,
halt_on_first_read: Some(halt),
}
}
}
impl SectorReader for FailingSectorReader {
fn read_sectors(
&mut self,
_lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
if let Some(h) = self.halt_on_first_read.take() {
h.store(true, Ordering::Relaxed);
}
// Model what a real damaged-disc read returns: CHECK CONDITION +
// MEDIUM ERROR (sense_key 3, ASC 0x11 UNRECOVERED READ ERROR,
// ASCQ 0x05 L-EC UNCORRECTABLE). Disc::copy's hysteresis must
// engage on this — `Error::DiscRead` is libfreemkv's own
// post-classification signal, not what a real reader emits.
Err(libfreemkv::error::Error::ScsiError {
opcode: libfreemkv::scsi::SCSI_READ_10,
status: libfreemkv::scsi::SCSI_STATUS_CHECK_CONDITION,
sense: Some(libfreemkv::ScsiSense {
sense_key: libfreemkv::scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x05,
}),
})
}
fn capacity(&self) -> u32 {
self.capacity
}
}
#[test]
fn test_disc_copy_completes_full_disc_with_failing_reader() {
// 1024 sectors = 2 MB. Reader fails every read. With skip_on_error +
// skip_on_error, Pass 1 must mark every sector NonTrimmed and return
// cleanly — no bail, no hang.
let capacity_sectors: u32 = 1024;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
let mut reader = FailingSectorReader::new(capacity_sectors);
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let t0 = Instant::now();
let result = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok");
let elapsed = t0.elapsed();
// Cleanup
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
// Hard bound — even at 0 ms per read, 1024 sectors with skip-forward
// should complete in well under a second on any host.
assert!(
elapsed < Duration::from_secs(5),
"Pass 1 took {elapsed:?} on a 2 MB synthetic disc — expected < 5 s"
);
// Per RIP_DESIGN.md §2.1: Pass 1 must reach end of disc regardless of
// read outcomes.
assert_eq!(
result.bytes_total, total_bytes,
"bytes_total must match disc capacity"
);
assert_eq!(
result.bytes_good, 0,
"no reads succeeded, bytes_good must be 0"
);
assert_eq!(
result.bytes_unreadable, 0,
"Pass 1 does not mark Unreadable; only Pass 2 (Disc::patch) does"
);
assert_eq!(
result.bytes_pending, total_bytes,
"every sector must be NonTrimmed → counted as pending. \
Got bytes_pending={} of total {}",
result.bytes_pending, total_bytes
);
assert!(
!result.complete,
"complete=false because NonTrimmed regions remain (work for Pass 2)"
);
assert!(!result.halted, "no halt was set; halted must be false");
// ISO file should be the full disc size on disk (sparse zeros where
// reads failed).
// Note: tempfile was dropped above; the file may or may not still exist
// depending on cleanup ordering. We only assert what we can observe in
// the CopyResult.
}
// ── 7. Halt during Pass 1 skip-forward path returns promptly (deterministic) ─
//
// Per RIP_DESIGN.md §3: halt is the only legitimate early exit from Pass 1.
// Even when every read is failing (skip-forward path), a halt must be
// honored within a small bounded time.
//
// Deterministic fixture: the reader signals halt on its FIRST read. The
// inner copy loop's halt check fires on the next iteration, breaking out
// of 'outer. This avoids any wallclock race on fast CI runners (where a
// 2 GB synthetic disc can sweep skip-forward in <100 ms).
#[test]
fn test_disc_copy_halts_promptly_on_failing_reader() {
let capacity_sectors: u32 = 1024 * 1024; // 2 GB synthetic disc
let halt = Arc::new(AtomicBool::new(false));
let mut reader = FailingSectorReader::with_halt_on_first_read(capacity_sectors, halt.clone());
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let opts = CopyOptions {
decrypt: false,
multipass: true,
halt: Some(halt),
..Default::default()
};
let t0 = Instant::now();
let result = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok on halt");
let elapsed = t0.elapsed();
// Cleanup
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
assert!(
elapsed < Duration::from_secs(2),
"halt must return within 2 s; took {elapsed:?}"
);
assert!(result.halted, "result.halted must be true");
assert!(
!result.complete,
"halted run cannot be complete (bytes_pending > 0 expected)"
);
assert!(
result.bytes_pending > 0,
"halt fired before sweep completed; bytes_pending must be > 0"
);
}
// ── 8. Hysteresis recovers data the drive can read individually ──────────
//
// Pass 1 reads in batch (32 sectors = 1 ECC block). Failed blocks are marked
// NonTrimmed for Pass 2 recovery. This test verifies that a reader where every
// multi-sector read fails produces all NonTrimmed output with zero bytes_good.
struct BlockSizeFailingReader {
capacity: u32,
}
impl SectorReader for BlockSizeFailingReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
if count == 1 {
for chunk in buf.chunks_mut(SECTOR_SIZE) {
chunk.fill((lba & 0xff) as u8);
}
Ok(buf.len())
} else {
Err(libfreemkv::error::Error::ScsiError {
opcode: libfreemkv::scsi::SCSI_READ_10,
status: libfreemkv::scsi::SCSI_STATUS_CHECK_CONDITION,
sense: Some(libfreemkv::ScsiSense {
sense_key: libfreemkv::scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
}),
})
}
}
fn capacity(&self) -> u32 {
self.capacity
}
}
#[test]
fn test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed() {
let capacity_sectors: u32 = 256;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
let mut reader = BlockSizeFailingReader {
capacity: capacity_sectors,
};
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let result = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok");
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
// Pass 1 reads every batch at bpt=32 (no batch reduction, no skip-ahead).
// BlockSizeFailingReader fails on multi-sector reads but succeeds on single-sector.
// Bridge degradation handling retries failed batches as individual sectors,
// so all data is recovered as Finished. bytes_good == total_bytes is correct.
assert_eq!(
result.bytes_good, total_bytes,
"Pass 1 at bpt=32 recovers all sectors via single-sector retry on MEDIUM_ERROR"
);
assert_eq!(
result.bytes_pending, 0,
"no pending sectors after full recovery"
);
assert!(result.complete, "complete=true when all sectors recovered");
}
// ── 9. PassProgress carries separate unreadable vs pending byte counts ─────
//
// The video-damage-time display needs bytes_unreadable_total (confirmed dead)
// separate from bytes_pending_total (might still recover). This test verifies
// that a Pass 2 with some confirmed failures produces correct field values.
#[test]
fn test_pass_progress_separates_unreadable_from_pending() {
let capacity_sectors: u32 = 128;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
let mut reader = FailingSectorReader::new(capacity_sectors);
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let pass1 = disc.copy(&mut reader, &iso_path, &opts).expect("pass1 ok");
assert_eq!(pass1.bytes_good, 0, "pass1: no good sectors");
assert_eq!(pass1.bytes_unreadable, 0, "pass1: no confirmed unreadable");
assert_eq!(
pass1.bytes_pending, total_bytes,
"pass1: all sectors NonTrimmed"
);
let last_unreadable = Arc::new(AtomicU64::new(0));
let last_pending = Arc::new(AtomicU64::new(0));
let last_good = Arc::new(AtomicU64::new(0));
let last_dur = Arc::new(AtomicU64::new(0));
struct SnapshotReporter {
unreadable: Arc<AtomicU64>,
pending: Arc<AtomicU64>,
good: Arc<AtomicU64>,
dur: Arc<AtomicU64>,
}
impl libfreemkv::progress::Progress for SnapshotReporter {
fn report(&self, p: &libfreemkv::progress::PassProgress) -> bool {
self.unreadable
.store(p.bytes_unreadable_total, Ordering::Relaxed);
self.pending.store(p.bytes_pending_total, Ordering::Relaxed);
self.good.store(p.bytes_good_total, Ordering::Relaxed);
if let Some(d) = p.disc_duration_secs {
self.dur.store((d * 1000.0) as u64, Ordering::Relaxed);
}
true
}
}
let reporter = SnapshotReporter {
unreadable: last_unreadable.clone(),
pending: last_pending.clone(),
good: last_good.clone(),
dur: last_dur.clone(),
};
let pass2_opts = CopyOptions {
decrypt: false,
multipass: true,
progress: Some(&reporter),
..Default::default()
};
let pass2 = disc
.copy(&mut reader, &iso_path, &pass2_opts)
.expect("pass2 ok");
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
assert_eq!(
pass2.bytes_good, 0,
"pass2: still no good sectors (reader always fails)"
);
assert!(
pass2.bytes_unreadable > 0,
"pass2: some sectors confirmed unreadable"
);
assert!(
pass2.bytes_pending < pass1.bytes_pending,
"pass2: fewer pending sectors than pass1"
);
let observed_unreadable = last_unreadable.load(Ordering::Relaxed);
let observed_pending = last_pending.load(Ordering::Relaxed);
assert!(
observed_unreadable > 0,
"progress should report confirmed unreadable bytes"
);
assert!(
observed_pending == 0 || observed_pending < total_bytes,
"pending should shrink as sectors are confirmed unreadable"
);
// Video damage time: unreadable / total * duration
// With no titles on synthetic disc, disc_duration_secs = None
assert_eq!(
last_dur.load(Ordering::Relaxed),
0,
"synthetic disc has no titles, duration should be None/0"
);
}
// ── 10. Damage time calculation (unit test) ────────────────────────────────
//
// Verifies the formula: damage_secs = bytes_unreadable / bytes_total * duration
// This mirrors the CLI's print_disc_progress logic.
#[test]
fn test_damage_time_calculation() {
// 78.8 GB disc, 2h45m movie (9900s), 74 KB unreadable
let disc_bytes: u64 = 78_800_000_000;
let duration_secs: f64 = 9900.0;
let cases: Vec<(u64, &str)> = vec![
(74 * 1024, "~10ms"), // 74 KB → ~9ms, negligible
(10 * 1024 * 1024, "~1.3s"), // 10 MB → ~1.3s
(100 * 1024 * 1024, "~13s"), // 100 MB → ~13s
(1024 * 1024 * 1024, "~134s"), // 1 GB → ~134s
];
for (bad_bytes, label) in cases {
let damage_secs = bad_bytes as f64 / disc_bytes as f64 * duration_secs;
match label {
"~10ms" => assert!(damage_secs < 0.05, "{label}: {damage_secs:.3}s"),
"~1.3s" => assert!(
(damage_secs - 1.3).abs() < 0.2,
"{label}: {damage_secs:.2}s"
),
"~13s" => assert!(
(damage_secs - 13.0).abs() < 1.0,
"{label}: {damage_secs:.1}s"
),
"~134s" => assert!(
(damage_secs - 134.0).abs() < 2.0,
"{label}: {damage_secs:.0}s"
),
_ => {}
}
}
// 0.25s threshold: how many bad bytes = 0.25s of damage?
let threshold_bytes = (0.25 / duration_secs * disc_bytes as f64) as u64;
assert!(
threshold_bytes > 0,
"0.25s damage threshold should be > 0 bytes"
);
// At 9900s / 78.8 GB ≈ 0.25s = ~2 MB
let expected_mb = threshold_bytes as f64 / (1024.0 * 1024.0);
assert!(
(expected_mb - 2.0).abs() < 0.5,
"0.25s ≈ {expected_mb:.2} MB (expected ~2 MB)"
);
}
+119
View File
@@ -0,0 +1,119 @@
//! Regression tests for Pass N (patch) fix — decrypt key inversion bug.
//!
//! Tests that decrypt_sectors is invoked correctly when opts.decrypt=true.
//! The 2026-05-03 bug at `libfreemkv/src/disc/mod.rs:1938-1942` inverted
//! the decrypt key arms, causing patch to pass DecryptKeys::None on encrypted discs.
use libfreemkv::{aacs, decrypt::DecryptKeys};
/// Test: decrypt_sectors with AACS keys actually decrypts units.
#[test]
fn decrypt_sectors_with_aacs_keys_works() {
// Build an encrypted aligned unit
let mut unit = vec![0xFFu8; aacs::ALIGNED_UNIT_LEN];
// Set encryption flag (bits 6-7 of byte 0)
unit[0] |= 0xC0;
// Fill with recognizable pattern
for (i, byte) in unit
.iter_mut()
.enumerate()
.take(aacs::ALIGNED_UNIT_LEN)
.skip(1)
{
*byte = ((i * 3 + 7) & 0xFF) as u8;
}
let unit_key: [u8; 16] = [0xAAu8; 16];
// Encrypt the unit using AACS algorithm
aacs::decrypt_unit(&mut unit, &unit_key); // decrypt_unit is idempotent on already-encrypted data
// Now we have encrypted data - create DecryptKeys with actual keys
let keys = DecryptKeys::Aacs {
unit_keys: vec![(0u32, unit_key)],
read_data_key: None,
};
// decrypt_sectors should handle this without error
let result = libfreemkv::decrypt::decrypt_sectors(&mut unit, &keys, 0);
assert!(
result.is_ok(),
"decrypt_sectors with AACS keys should not error"
);
}
/// Test: decrypt_sectors with DecryptKeys::None is a no-op.
#[test]
fn decrypt_sectors_with_none_keys_is_noop() {
let mut sector = vec![0x42u8; 2048];
let keys = DecryptKeys::None;
let result = libfreemkv::decrypt::decrypt_sectors(&mut sector, &keys, 0);
assert!(result.is_ok());
assert_eq!(
&sector[..],
&[0x42u8; 2048][..],
"DecryptKeys::None should not modify buffer"
);
}
/// Test: decrypt_sectors with CSS keys descrambles sectors.
#[test]
fn decrypt_sectors_with_css_keys_works() {
let mut sector = vec![0xFFu8; 2048];
// Set CSS scramble flag (bits 4-5 of byte 0x14)
sector[0x14] |= 0x30;
let title_key: [u8; 5] = [0x42, 0x13, 0x37, 0xBE, 0xEF]; // Not used - defined later
let keys = DecryptKeys::Css { title_key };
// Descramble (CSS uses same operation for encrypt/decrypt)
libfreemkv::decrypt::decrypt_sectors(&mut sector, &keys, 0).unwrap();
// Flag should be cleared
assert_eq!(sector[0x14] & 0x30, 0x00, "CSS flag should be cleared");
}
/// Test: AACS unit encryption detection works.
#[test]
fn aacs_encryption_flag_detection() {
let mut unit = vec![0u8; aacs::ALIGNED_UNIT_LEN];
// No encryption flag
assert!(!aacs::is_unit_encrypted(&unit));
// Set bit 6
unit[0] |= 0x40;
assert!(aacs::is_unit_encrypted(&unit));
// Set bit 7
unit[0] = 0x80;
assert!(aacs::is_unit_encrypted(&unit));
// Both bits set
unit[0] = 0xC0;
assert!(aacs::is_unit_encrypted(&unit));
}
/// Test: DecryptKeys::is_encrypted() correctly identifies encrypted state.
#[test]
fn decrypt_keys_is_encrypted_variants() {
let none = DecryptKeys::None;
assert!(!none.is_encrypted());
let aacs = DecryptKeys::Aacs {
unit_keys: vec![],
read_data_key: None,
};
assert!(aacs.is_encrypted());
let css = DecryptKeys::Css {
title_key: [0u8; 5],
};
assert!(css.is_encrypted());
}
+399
View File
@@ -0,0 +1,399 @@
//! Pass N (Disc::patch) size-aware-skip targeted tests.
//!
//! The user's failure mode (2026-05-07): "what if we have a 100 sector zone
//! and its really 2 25 sector zones and we keep jumping over the good in
//! the middle." Today's pre-fix patch escalates skip-distance based on
//! `consecutive_skips_without_recovery` with hardcoded 32 → 4096 sector
//! caps. A 100-sector bad range whose actual layout is 25 bad + 50 good +
//! 25 bad would have the patch skip 32-4096 sectors after a couple of
//! failures, leaping over the entire range AND the good middle.
//!
//! The fix: cap each skip at `range_remaining/4`. These tests exercise
//! that boundary.
use libfreemkv::disc::CopyOptions;
use libfreemkv::disc::DiscRegion;
use libfreemkv::disc::mapfile::{Mapfile, SectorStatus};
use libfreemkv::error::Result;
use libfreemkv::{ContentFormat, Disc, DiscFormat, SectorReader};
use std::collections::HashSet;
use std::sync::{Arc, Mutex};
const SECTOR_SIZE: usize = 2048;
/// Reader where you specify exactly which LBAs return Err. Everything else
/// returns Ok with the LBA encoded in each byte for verification.
struct PatternedSectorReader {
capacity: u32,
bad_lbas: HashSet<u32>,
/// Trace every read so tests can assert what was actually attempted.
trace: Arc<Mutex<Vec<(u32, u16)>>>,
}
impl PatternedSectorReader {
fn new(capacity: u32, bad_lbas: HashSet<u32>) -> (Self, Arc<Mutex<Vec<(u32, u16)>>>) {
let trace = Arc::new(Mutex::new(Vec::new()));
(
Self {
capacity,
bad_lbas,
trace: trace.clone(),
},
trace,
)
}
}
impl SectorReader for PatternedSectorReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
self.trace.lock().unwrap().push((lba, count));
// Whole-batch fails if ANY sector in the batch is bad. (Models a
// real drive: a multi-sector READ aborts on the first ECC failure.)
for offset in 0..count as u32 {
if self.bad_lbas.contains(&(lba + offset)) {
return Err(libfreemkv::error::Error::ScsiError {
opcode: libfreemkv::scsi::SCSI_READ_10,
status: libfreemkv::scsi::SCSI_STATUS_CHECK_CONDITION,
sense: Some(libfreemkv::ScsiSense {
sense_key: libfreemkv::scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
}),
});
}
}
for chunk in buf.chunks_mut(SECTOR_SIZE) {
chunk.fill((lba & 0xff) as u8);
}
Ok(buf.len())
}
fn capacity(&self) -> u32 {
self.capacity
}
}
fn synthetic_disc(capacity_sectors: u32) -> Disc {
Disc {
volume_id: String::new(),
meta_title: None,
format: DiscFormat::BluRay,
capacity_sectors,
capacity_bytes: capacity_sectors as u64 * SECTOR_SIZE as u64,
layers: 1,
titles: Vec::new(),
region: DiscRegion::Free,
aacs: None,
css: None,
encrypted: false,
content_format: ContentFormat::BdTs,
}
}
/// Pre-populate a mapfile with one large NonTrimmed range so patch's work-
/// list has something to do. Caller pre-allocates the ISO at `total_bytes`
/// so seeks don't fail.
fn prep_iso_and_mapfile(
iso_path: &std::path::Path,
total_bytes: u64,
finished_ranges: &[(u64, u64)],
nontrimmed_ranges: &[(u64, u64)],
) {
use std::fs::OpenOptions;
use std::io::{Seek, SeekFrom, Write};
let mut f = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(iso_path)
.unwrap();
f.set_len(total_bytes).unwrap();
f.seek(SeekFrom::Start(0)).unwrap();
f.write_all(&[]).unwrap();
let map_path = libfreemkv::disc::mapfile_path_for(iso_path);
let mut mf = Mapfile::create(&map_path, total_bytes, "test").unwrap();
for &(pos, size) in finished_ranges {
mf.record(pos, size, SectorStatus::Finished).unwrap();
}
for &(pos, size) in nontrimmed_ranges {
mf.record(pos, size, SectorStatus::NonTrimmed).unwrap();
}
}
/// THE critical test. A 100-sector "bad" range hides 50 good sectors in
/// the middle (LBAs 125-174). Pre-fix patch would skip-escalate at 32+
/// sectors and leap over the whole range. Post-fix: skip is capped at
/// range_remaining/4 (=25 sectors initially), which forces convergence.
#[test]
fn patch_recovers_good_middle_of_a_bad_range() {
let capacity_sectors: u32 = 1024;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
// Bad range layout: LBAs 100-124 bad, 125-174 GOOD, 175-199 bad.
let mut bad_lbas = HashSet::new();
for lba in 100..125 {
bad_lbas.insert(lba);
}
for lba in 175..200 {
bad_lbas.insert(lba);
}
let (mut reader, _trace) = PatternedSectorReader::new(capacity_sectors, bad_lbas);
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().unwrap();
let iso_path = tmp.path().to_path_buf();
drop(tmp);
// Pre-populate: 0..100 already Finished from an imagined Pass 1,
// 100..200 NonTrimmed (the range we want patch to retry),
// 200..1024 already Finished.
let finished = [
(0, 100 * 2048),
(200 * 2048, (capacity_sectors as u64 - 200) * 2048),
];
let nontrimmed = [(100 * 2048, 100 * 2048)];
prep_iso_and_mapfile(&iso_path, total_bytes, &finished, &nontrimmed);
// Run patch.
// disc.copy() with multipass=true auto-dispatches to patch when the
// mapfile already covers the disc and has retryable ranges.
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let pr = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok");
// Re-load mapfile and inspect.
let map_path = libfreemkv::disc::mapfile_path_for(&iso_path);
let map = Mapfile::load(&map_path).unwrap();
// The good middle (125..175) MUST end up Finished. If size-aware skip
// is not enabled, patch would skip 32+ sectors after a few failures
// and leap clean over LBA 125 → middle stays NonTrimmed/Unreadable.
let finished_ranges = map.ranges_with(&[SectorStatus::Finished]);
let total_finished_in_middle: u64 = finished_ranges
.iter()
.map(|&(pos, sz)| {
let start = pos.max(125 * 2048);
let end = (pos + sz).min(175 * 2048);
end.saturating_sub(start)
})
.sum();
// Allow 2 sectors (4 KB) of boundary slop — patch's bisection may
// not converge exactly on the good/bad boundary in a single pass,
// and that's acceptable. The pre-fix behaviour would have left the
// entire good middle as NonTrimmed (~0 bytes recovered).
let good_middle_bytes: u64 = 50 * 2048;
let min_acceptable: u64 = good_middle_bytes - 2 * 2048;
// Cleanup before assertions
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(&map_path);
assert!(
total_finished_in_middle >= min_acceptable,
"size-aware skip should have discovered most of the 50 good sectors in the middle. \
Recovered {} of {} good middle bytes (min acceptable {}). bytes_good={} bytes_total={}",
total_finished_in_middle,
good_middle_bytes,
min_acceptable,
pr.bytes_good,
pr.bytes_total,
);
}
/// A second test: a bad range that's actually 4 small bad sub-zones
/// separated by good sectors. Demonstrates the bisection behaviour
/// converges when zones are non-uniform.
#[test]
fn patch_recovers_multiple_good_middles() {
let capacity_sectors: u32 = 2048;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
// Bad pattern: 1000-1024 bad, 1025-1099 good, 1100-1124 bad,
// 1125-1199 good, 1200-1224 bad, 1225-1299 good.
let mut bad_lbas = HashSet::new();
for lba in 1000..1025 {
bad_lbas.insert(lba);
}
for lba in 1100..1125 {
bad_lbas.insert(lba);
}
for lba in 1200..1225 {
bad_lbas.insert(lba);
}
let (mut reader, _trace) = PatternedSectorReader::new(capacity_sectors, bad_lbas);
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().unwrap();
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let finished = [
(0, 1000 * 2048),
(1300 * 2048, (capacity_sectors as u64 - 1300) * 2048),
];
let nontrimmed = [(1000 * 2048, 300 * 2048)];
prep_iso_and_mapfile(&iso_path, total_bytes, &finished, &nontrimmed);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let pr = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok");
let map_path = libfreemkv::disc::mapfile_path_for(&iso_path);
let map = Mapfile::load(&map_path).unwrap();
let finished_ranges = map.ranges_with(&[SectorStatus::Finished]);
let recovered: u64 = finished_ranges
.iter()
.map(|&(pos, sz)| {
let start = pos.max(1000 * 2048);
let end = (pos + sz).min(1300 * 2048);
end.saturating_sub(start)
})
.sum();
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(&map_path);
// Three good middles of 75 sectors each = 225 good sectors in the
// bad range. Total bad = 75. So we want at least most of 225 sectors
// (= 460800 bytes) to be Finished after patch.
let target = 200 * 2048; // be generous — anything over 200 sectors is convincing
assert!(
recovered >= target,
"size-aware skip should find most of the 3 good middles. \
Recovered {} bytes; expected ≥ {}. bytes_good={} bytes_total={}",
recovered,
target,
pr.bytes_good,
pr.bytes_total,
);
}
/// 0.18 Pass N pipeline split: exercises the new producer/consumer
/// path end-to-end on a synthetic patterned reader. Bad range layout
/// is small (5 bad LBAs surrounded by good middle) so the producer
/// emits a mix of `Recovered` and `Unreadable` items and the consumer
/// thread must apply both kinds. Verifies:
///
/// - `bytes_good` advances (good sectors flow producer→consumer→file
/// →mapfile with the data preserved).
/// - The recovered LBAs end up Finished; the bad LBAs end up Unreadable.
/// - Bytes written at the recovered offsets match what the producer
/// read from the patterned source (proves the channel hand-off
/// didn't drop or reorder buffers, and the consumer's seek+write
/// landed at the right offsets).
#[test]
fn patch_pipeline_split_recovers_and_records_correctly() {
let capacity_sectors: u32 = 512;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
// Layout: LBAs 200-204 inclusive are bad (5 sectors), 205-249 good.
// The pre-existing range is LBAs 200-249 NonTrimmed (100 KB).
let mut bad_lbas = HashSet::new();
for lba in 200..205 {
bad_lbas.insert(lba);
}
let (mut reader, _trace) = PatternedSectorReader::new(capacity_sectors, bad_lbas.clone());
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().unwrap();
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let finished = [
(0, 200 * 2048),
(250 * 2048, (capacity_sectors as u64 - 250) * 2048),
];
let nontrimmed = [(200 * 2048, 50 * 2048)];
prep_iso_and_mapfile(&iso_path, total_bytes, &finished, &nontrimmed);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let pr = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok");
// Bytes_good_total should advance — the good LBAs in the bad range
// (205-249, 45 sectors) are all reachable via per-sector retry.
// Initial bytes_good = 200 * 2048 + (512-250) * 2048 = 462 sectors.
// After patch, bytes_good should be ≥ 462 + 45 = 507 sectors worth.
let initial_good_sectors: u64 = 200 + (capacity_sectors as u64 - 250);
let min_expected_good_bytes = (initial_good_sectors + 30) * 2048;
assert!(
pr.bytes_good >= min_expected_good_bytes,
"patch should have recovered most good LBAs in the bad range via the pipeline. \
bytes_good={} (expected ≥ {}); bytes_total={}",
pr.bytes_good,
min_expected_good_bytes,
pr.bytes_total,
);
// Verify the mapfile records: every good LBA is Finished, every
// bad LBA is either Unreadable or NonTrimmed (not Finished).
let map_path = libfreemkv::disc::mapfile_path_for(&iso_path);
let map = Mapfile::load(&map_path).unwrap();
let finished_ranges = map.ranges_with(&[SectorStatus::Finished]);
let in_finished = |lba: u32| -> bool {
let pos = lba as u64 * 2048;
finished_ranges
.iter()
.any(|&(p, sz)| pos >= p && pos < p + sz)
};
for lba in 205..250 {
assert!(
in_finished(lba),
"good LBA {lba} should be Finished after pipeline patch run"
);
}
for lba in 200..205 {
assert!(
!in_finished(lba),
"bad LBA {lba} should NOT be Finished after pipeline patch run"
);
}
// Verify the consumer wrote the producer's bytes at the right
// offsets. PatternedSectorReader fills each sector with `(lba & 0xff)
// as u8` — picking LBA 220 (well inside the recovered region) gives
// a clean signature byte to check.
use std::io::{Read, Seek, SeekFrom};
let mut iso = std::fs::File::open(&iso_path).unwrap();
iso.seek(SeekFrom::Start(220 * 2048)).unwrap();
let mut sector = [0u8; 2048];
iso.read_exact(&mut sector).unwrap();
let expected_byte = (220u32 & 0xff) as u8;
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(&map_path);
assert!(
sector.iter().all(|&b| b == expected_byte),
"consumer should have written PatternedSectorReader's pattern \
(byte {expected_byte:#x} for LBA 220) to the recovered offset; \
got first 8 bytes = {:?}",
&sector[..8]
);
}
+394
View File
@@ -0,0 +1,394 @@
//! Integration tests for the SCSI error-decoding contract.
//!
//! v0.13.20 rewrote `scsi/linux.rs` to a synchronous blocking SG_IO and
//! consolidated sense parsing into the `parse_sense` helper that every
//! platform backend now shares. v0.13.23 replaced the `Error::ScsiError`
//! flat-fields shape with `{ opcode, status, sense: Option<ScsiSense> }`
//! so callers can route on structured sense data (key + ASC + ASCQ) via
//! [`Error::scsi_sense`] / [`Error::is_marginal_read`] /
//! [`ScsiSense::is_*`].
//!
//! The actual `ioctl(SG_IO, ...)` call is impossible to mock without a
//! kernel — see
//! `(internal)/docs/audits/2026-04-26-scsi-architecture-research.md`
//! for why the audit recommends against libc shims here. These tests
//! therefore pin the *contract* every backend must satisfy via a mock
//! `ScsiTransport`:
//!
//! 1. Healthy result → `Ok(ScsiResult { bytes_transferred = data.len() - resid })`.
//! 2. Transport-level failure (no SCSI status delivered: kernel
//! timeout, USB bridge wedge, IOKit service error) →
//! `Error::ScsiError { status: SCSI_STATUS_TRANSPORT_FAILURE, sense: None }`.
//! `Error::is_scsi_transport_failure()` returns `true`. Used by
//! `drive_has_disc` to detect the wedge signature.
//! 3. SCSI-level failure (drive replied CHECK CONDITION with sense) →
//! `Error::ScsiError { status: 0x02, sense: Some(ScsiSense {…}) }`
//! with the parsed key/ASC/ASCQ.
//! 4. `Error::is_marginal_read()` is `true` for MEDIUM ERROR /
//! ABORTED COMMAND / RECOVERED ERROR / NO SENSE; `false` for
//! HARDWARE / DATA PROTECT / UNIT ATTENTION / NOT READY / ILLEGAL
//! REQUEST and for transport failures.
//!
//! Inline `parse_sense_tests` in `src/scsi/mod.rs` cover the pure parse
//! logic (descriptor 0x72/0x73 vs fixed 0x70/0x71, short-buffer, VALID
//! bit masking, unknown response codes, ASC/ASCQ offsets); this file
//! covers the consumer side — a real transport feeding a real Error
//! variant to a real call site (`scsi::inquiry`).
use libfreemkv::error::Error;
use libfreemkv::scsi::{
DataDirection, SCSI_STATUS_CHECK_CONDITION, SCSI_STATUS_TRANSPORT_FAILURE,
SENSE_KEY_ABORTED_COMMAND, SENSE_KEY_DATA_PROTECT, SENSE_KEY_HARDWARE_ERROR,
SENSE_KEY_ILLEGAL_REQUEST, SENSE_KEY_MEDIUM_ERROR, SENSE_KEY_NOT_READY,
SENSE_KEY_RECOVERED_ERROR, SENSE_KEY_UNIT_ATTENTION, ScsiResult, ScsiSense, ScsiTransport,
};
/// A scripted ScsiTransport. Each `execute()` consumes the next entry
/// from `script` and returns the corresponding outcome.
///
/// Outcomes mirror what each backend's `execute()` should produce after
/// the v0.13.23 sense plumbing: `Option<ScsiSense>` carrying the full
/// SPC-4 triple for drive-reported failures, `None` for transport-level
/// failures.
struct MockTransport {
script: Vec<MockOutcome>,
next: usize,
}
#[derive(Clone)]
enum MockOutcome {
/// Healthy completion. `data` is what the transport wrote into the
/// caller's data buffer (truncated to the buffer length); `resid`
/// is reported back as `data.len() - bytes_transferred`.
Ok { data: Vec<u8>, resid: i32 },
/// Transport-level failure: `hdr.host_status = DID_TIME_OUT` on
/// Linux, `kIOReturnError` on macOS, `DeviceIoControl` returning 0
/// on Windows. Backends synthesise `SCSI_STATUS_TRANSPORT_FAILURE`
/// with `sense = None`.
TransportFailure,
/// Drive replied with sense data (typically `SCSI_STATUS_CHECK_CONDITION`
/// + a populated sense buffer).
ScsiFailure { status: u8, sense: ScsiSense },
}
impl MockTransport {
fn new(script: Vec<MockOutcome>) -> Self {
Self { script, next: 0 }
}
}
impl ScsiTransport for MockTransport {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> libfreemkv::error::Result<ScsiResult> {
let i = self.next;
self.next += 1;
let outcome = self
.script
.get(i)
.cloned()
.expect("MockTransport script ran out — test wrote fewer outcomes than calls");
match outcome {
MockOutcome::Ok { data: d, resid } => {
let n = d.len().min(data.len());
data[..n].copy_from_slice(&d[..n]);
let bytes = (data.len() as i32).saturating_sub(resid).max(0) as usize;
Ok(ScsiResult {
status: 0,
bytes_transferred: bytes,
sense: [0u8; 32],
})
}
MockOutcome::TransportFailure => Err(Error::ScsiError {
opcode: cdb[0],
status: SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
}),
MockOutcome::ScsiFailure { status, sense } => Err(Error::ScsiError {
opcode: cdb[0],
status,
sense: Some(sense),
}),
}
}
}
/// Helper — build a CHECK CONDITION outcome with a given sense key
/// and zero ASC/ASCQ. Most consumer-side tests only care about the key.
fn check_cond(sense_key: u8) -> MockOutcome {
MockOutcome::ScsiFailure {
status: SCSI_STATUS_CHECK_CONDITION,
sense: ScsiSense {
sense_key,
asc: 0,
ascq: 0,
},
}
}
// ── 1. Healthy read ───────────────────────────────────────────────────────
#[test]
fn test_healthy_inquiry_returns_ok_with_full_transfer() {
// Build a fake INQUIRY response (96 bytes per scsi::inquiry).
// Vendor "TEST-VND", model "TEST-MODEL ", firmware "1.00".
let mut payload = vec![0u8; 96];
payload[8..16].copy_from_slice(b"TEST-VND");
payload[16..32].copy_from_slice(b"TEST-MODEL ");
payload[32..36].copy_from_slice(b"1.00");
let mut transport = MockTransport::new(vec![MockOutcome::Ok {
data: payload,
resid: 0,
}]);
let r = libfreemkv::scsi::inquiry(&mut transport).expect("inquiry should succeed");
assert_eq!(r.vendor_id, "TEST-VND");
assert_eq!(r.model, "TEST-MODEL");
assert_eq!(r.firmware, "1.00");
}
// ── 2. Transport-level failure: no SCSI status, no sense data ─────────────
#[test]
fn test_transport_failure_surfaces_with_sense_none() {
let mut transport = MockTransport::new(vec![MockOutcome::TransportFailure]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"TransportFailure must satisfy is_scsi_transport_failure()"
);
assert!(
err.scsi_sense().is_none(),
"transport failure has no sense data"
);
assert!(
!err.is_marginal_read(),
"transport failure must not be classified as marginal-read"
);
match err {
Error::ScsiError { status, sense, .. } => {
assert_eq!(status, SCSI_STATUS_TRANSPORT_FAILURE);
assert!(sense.is_none());
}
other => panic!("expected ScsiError, got {other:?}"),
}
}
// ── 3. CHECK CONDITION + ILLEGAL REQUEST ──────────────────────────────────
#[test]
fn test_check_cond_illegal_request_carries_sense() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_ILLEGAL_REQUEST)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert_eq!(sense.sense_key, SENSE_KEY_ILLEGAL_REQUEST);
assert!(sense.is_illegal_request());
assert!(!err.is_marginal_read(), "ILLEGAL REQUEST is not marginal");
assert!(!err.is_scsi_transport_failure());
}
// ── 4. CHECK CONDITION + NOT READY (drive_has_disc relies on this) ────────
#[test]
fn test_check_cond_not_ready_predicate() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_NOT_READY)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_not_ready(), "sense_key 2 ⇒ is_not_ready");
assert!(
err.is_marginal_read(),
"NOT READY is marginal (BU40N returns this for bad sectors)"
);
}
// ── 5. CHECK CONDITION + MEDIUM ERROR (canonical marginal-read) ───────────
#[test]
fn test_check_cond_medium_error_is_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_MEDIUM_ERROR)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_medium_error());
assert!(sense.is_marginal());
assert!(
err.is_marginal_read(),
"MEDIUM ERROR is the canonical marginal-read signal"
);
}
// ── 6. CHECK CONDITION + ABORTED COMMAND (also marginal) ──────────────────
#[test]
fn test_check_cond_aborted_command_is_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_ABORTED_COMMAND)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_aborted_command());
assert!(err.is_marginal_read(), "ABORTED COMMAND is marginal");
}
// ── 7. CHECK CONDITION + RECOVERED ERROR (drive recovered; marginal) ──────
#[test]
fn test_check_cond_recovered_error_is_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_RECOVERED_ERROR)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
assert!(
err.is_marginal_read(),
"RECOVERED ERROR is treated as marginal (drive recovered, retry-friendly class)"
);
}
// ── 8. CHECK CONDITION + HARDWARE ERROR (NOT marginal — bail) ─────────────
#[test]
fn test_check_cond_hardware_error_not_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_HARDWARE_ERROR)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_hardware_error());
assert!(
!err.is_marginal_read(),
"HARDWARE ERROR must not be marginal — drive failing, retry can't help"
);
}
// ── 9. CHECK CONDITION + DATA PROTECT (NOT marginal — bail) ───────────────
#[test]
fn test_check_cond_data_protect_not_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_DATA_PROTECT)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_data_protect());
assert!(
!err.is_marginal_read(),
"DATA PROTECT (AACS / region) must not be marginal — retry can't help"
);
}
// ── 10. CHECK CONDITION + UNIT ATTENTION (NOT marginal — caller rescans) ──
#[test]
fn test_check_cond_unit_attention_not_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_UNIT_ATTENTION)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_unit_attention());
assert!(
!err.is_marginal_read(),
"UNIT ATTENTION must not be marginal — caller should rescan, not retry"
);
}
// ── 11. ASC/ASCQ propagate from sense buffer to ScsiError.sense ───────────
#[test]
fn test_asc_ascq_round_trip_through_error() {
let outcome = MockOutcome::ScsiFailure {
status: SCSI_STATUS_CHECK_CONDITION,
sense: ScsiSense {
sense_key: SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x05, // L-EC UNCORRECTABLE
},
};
let mut transport = MockTransport::new(vec![outcome]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("must carry sense");
assert_eq!(sense.sense_key, SENSE_KEY_MEDIUM_ERROR);
assert_eq!(sense.asc, 0x11);
assert_eq!(sense.ascq, 0x05);
}
// ── 12. Healthy short transfer (resid > 0) ────────────────────────────────
#[test]
fn test_healthy_short_transfer_reports_partial_bytes() {
// Transport reported back fewer bytes than requested (resid = 16).
// For a 96-byte buffer that means bytes_transferred = 80.
let payload = vec![0u8; 96];
let mut transport = MockTransport::new(vec![MockOutcome::Ok {
data: payload,
resid: 16,
}]);
let cdb = [libfreemkv::scsi::SCSI_INQUIRY, 0, 0, 0, 0x60, 0];
let mut buf = [0u8; 96];
let r = transport
.execute(&cdb, DataDirection::FromDevice, &mut buf, 1_000)
.expect("ok");
assert_eq!(r.status, 0);
assert_eq!(
r.bytes_transferred, 80,
"bytes_transferred must equal data.len() - resid"
);
}
// ── 13. Error::Display does not leak English in the SCSI variant ──────────
//
// Library rule (project docs): no English in error display, only "E{code}"
// + structured data. Regression guard for the SCSI variant specifically
// because it's the most-emitted error in the rip path.
#[test]
fn test_scsi_error_display_format_is_codes_only() {
let err = Error::ScsiError {
opcode: 0x12,
status: SCSI_STATUS_CHECK_CONDITION,
sense: Some(ScsiSense {
sense_key: SENSE_KEY_ILLEGAL_REQUEST,
asc: 0x24,
ascq: 0x00,
}),
};
let s = err.to_string();
assert!(
s.starts_with("E4000:"),
"ScsiError must lead with E4000: {s}"
);
assert!(
s.contains("0x12") && s.contains("0x02") && s.contains("0x05") && s.contains("0x24"),
"ScsiError must show opcode/status/key/asc in hex: {s}"
);
// Crude English filter — same as the inline error.rs::display test.
for word in s.split(|c: char| !c.is_ascii_alphabetic()) {
assert!(
word.len() <= 4,
"ScsiError display contains suspicious word `{word}`: {s}"
);
}
}
// ── 14. Transport-failure Display omits sense fields ──────────────────────
#[test]
fn test_scsi_transport_failure_display_short_form() {
let err = Error::ScsiError {
opcode: 0x28,
status: SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
};
let s = err.to_string();
assert!(s.starts_with("E4000:"));
assert!(s.contains("0x28") && s.contains("0xff"));
// No sense triple should appear in the no-sense form.
assert!(
!s.contains("0x00/0x00/0x00"),
"transport failure must not carry phantom sense: {s}"
);
}
+117
View File
@@ -0,0 +1,117 @@
//! Tests for SgIoTransport timeout recovery (Fix 2).
//!
//! When `execute()` detects a transport-level failure (kernel timeout /
//! USB bridge wedge — `host_status != 0`), it spawns two background
//! threads to (1) close the old fd and (2) open a fresh one, then
//! stores the new fd in `fd_recovery`. The next `execute()` call picks
//! up the recovered fd without blocking on close().
//!
//! These tests require a real /dev/sg* device and are therefore #[ignore].
use std::path::Path;
#[test]
#[ignore]
fn test_sgio_transport_timeout_does_not_kill_transport() {
let device = "/dev/sg2";
let device = std::env::var("FREEMKV_TEST_SG_DEVICE").unwrap_or(device.to_string());
let _path = Path::new(&device);
#[cfg(target_os = "linux")]
{
use libfreemkv::scsi::ScsiTransport;
use libfreemkv::scsi::linux::SgIoTransport;
use libfreemkv::scsi::{DataDirection, SCSI_STATUS_TRANSPORT_FAILURE};
use std::sync::Arc;
use std::sync::atomic::{AtomicI32, Ordering};
use std::time::Duration;
let mut transport = SgIoTransport::open(_path).expect("open device");
let fd_before = transport.fd;
// READ_10 with 1 ms timeout to force kernel timeout.
let cdb = [0x28, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00];
let mut data = vec![0u8; 2048];
let start = std::time::Instant::now();
let result = transport.execute(&cdb, DataDirection::FromDevice, &mut data, 1);
let _elapsed = start.elapsed();
assert!(result.is_err(), "expected Err on timeout, got {:?}", result);
let err = result.unwrap_err();
assert!(
matches!(&err, libfreemkv::Error::ScsiError { status, .. } if *status == SCSI_STATUS_TRANSPORT_FAILURE),
"expected ScsiError(TRANSPORT_FAILURE), got {:?}",
err
);
assert_eq!(transport.fd, -1, "fd should be -1 after timeout");
// Wait for recovery thread to store new fd.
let recovered = Arc::new(AtomicI32::new(-1));
let recovery_ref = transport.fd_recovery.clone();
for _ in 0..100 {
let v = recovery_ref.load(Ordering::Acquire);
if v >= 0 {
recovered.store(v, Ordering::Release);
break;
}
std::thread::sleep(Duration::from_millis(50));
}
let new_fd = recovered.load(Ordering::Acquire);
assert!(new_fd >= 0, "recovery thread should have produced a new fd");
// Next execute() must pick up recovered fd quickly.
let mut data2 = vec![0u8; 2048];
let start2 = std::time::Instant::now();
let result2 = transport.execute(&cdb, DataDirection::FromDevice, &mut data2, 5000);
let elapsed2 = start2.elapsed();
assert!(
result2.is_ok(),
"execute() with recovered fd should succeed, got {:?}",
result2
);
assert!(
elapsed2 < Duration::from_secs(2),
"should return quickly, took {:?}",
elapsed2
);
assert_ne!(transport.fd, -1, "fd should be valid after recovery");
assert_ne!(transport.fd, fd_before, "fd should be fresh after recovery");
}
#[cfg(not(target_os = "linux"))]
{
eprintln!("SKIP: test requires Linux / SgIoTransport");
}
}
#[test]
#[ignore]
fn test_drive_read_per_cdb_timeout_bounds_call() {
let device = "/dev/sg2";
let device = std::env::var("FREEMKV_TEST_SG_DEVICE").unwrap_or(device.to_string());
let _path = Path::new(&device);
#[cfg(target_os = "linux")]
{
use std::time::Duration;
let mut drive = libfreemkv::Drive::open(_path).expect("open drive");
let timeout_ms: u32 = 5_000;
let start = std::time::Instant::now();
let _ = drive.read(0, 1, &mut [0u8; 2048], false);
let elapsed = start.elapsed();
let overhead = Duration::from_millis(500);
assert!(
elapsed < Duration::from_millis(timeout_ms as u64) + overhead,
"Drive::read should return within timeout_ms + overhead, took {:?}",
elapsed
);
}
#[cfg(not(target_os = "linux"))]
{
eprintln!("SKIP: test requires Linux / SgIoTransport");
}
}
+38 -19
View File
@@ -1,5 +1,10 @@
//! Integration tests for the PES stream pipeline.
// 0.18 trait split: this suite still drives the deprecated `pes::Stream`
// trait directly. It will be migrated to `FrameSource`/`FrameSink` in the
// follow-up that ports concrete impls.
#![allow(deprecated)]
use libfreemkv::mux::meta::M2tsMeta;
use libfreemkv::pes::Stream as PesStream;
use libfreemkv::*;
@@ -30,6 +35,7 @@ fn sample_disc_title() -> DiscTitle {
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: libfreemkv::LabelPurpose::Normal,
label: "English Atmos".into(),
}),
Stream::Audio(AudioStream {
@@ -39,6 +45,7 @@ fn sample_disc_title() -> DiscTitle {
language: "fra".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: libfreemkv::LabelPurpose::Normal,
label: "French".into(),
}),
Stream::Subtitle(SubtitleStream {
@@ -46,6 +53,7 @@ fn sample_disc_title() -> DiscTitle {
codec: Codec::Pgs,
language: "eng".into(),
forced: false,
qualifier: libfreemkv::LabelQualifier::None,
codec_data: None,
}),
],
@@ -299,12 +307,12 @@ fn m2ts_stream_write_read() {
keyframe: i == 0,
data: vec![i; 100],
};
stream.write(&frame).unwrap();
PesStream::write(&mut stream, &frame).unwrap();
}
stream.finish().unwrap();
PesStream::finish(&mut stream).unwrap();
// Verify the info is correct
let info = stream.info();
let info = PesStream::info(&stream);
assert_eq!(info.streams.len(), 4);
assert_eq!(info.duration_secs, 7200.0);
}
@@ -353,7 +361,7 @@ fn m2ts_read_returns_error_on_write_stream() {
let dt = sample_disc_title();
let output = Cursor::new(Vec::new());
let mut stream = M2tsStream::create(output, &dt).unwrap();
assert!(stream.read().is_err());
assert!(PesStream::read(&mut stream).is_err());
}
// ── DiscTitle::empty ──────────────────────────────────────────
@@ -404,6 +412,7 @@ fn meta_codec_roundtrip() {
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: libfreemkv::LabelPurpose::Normal,
label: String::new(),
}));
}
@@ -413,6 +422,7 @@ fn meta_codec_roundtrip() {
codec,
language: "eng".into(),
forced: false,
qualifier: libfreemkv::LabelQualifier::None,
codec_data: None,
}));
}
@@ -501,6 +511,7 @@ fn meta_all_stream_types() {
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: libfreemkv::LabelPurpose::Normal,
label: "Primary Audio".into(),
}),
Stream::Subtitle(SubtitleStream {
@@ -508,6 +519,7 @@ fn meta_all_stream_types() {
codec: Codec::Pgs,
language: "fra".into(),
forced: true,
qualifier: libfreemkv::LabelQualifier::None,
codec_data: None,
}),
Stream::Audio(AudioStream {
@@ -517,6 +529,7 @@ fn meta_all_stream_types() {
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: true,
purpose: libfreemkv::LabelPurpose::Normal,
label: "Commentary".into(),
}),
],
@@ -570,7 +583,7 @@ fn meta_all_stream_types() {
#[test]
fn mkvstream_write_finish() {
let dt = sample_disc_title();
let writer: Box<dyn libfreemkv::mux::WriteSeek> = Box::new(Cursor::new(Vec::new()));
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let mut stream = MkvStream::create(writer, &dt).unwrap();
// Write some fake PES frames (they won't produce valid MKV content
@@ -582,20 +595,20 @@ fn mkvstream_write_finish() {
keyframe: i == 0,
data: vec![i; 100],
};
stream.write(&frame).unwrap();
PesStream::write(&mut stream, &frame).unwrap();
}
// finish should not panic even without valid codec data
stream.finish().unwrap();
PesStream::finish(&mut stream).unwrap();
}
#[test]
fn mkvstream_meta_sets_title() {
let dt = sample_disc_title();
let writer: Box<dyn libfreemkv::mux::WriteSeek> = Box::new(Cursor::new(Vec::new()));
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let stream = MkvStream::create(writer, &dt).unwrap();
let info = stream.info();
let info = PesStream::info(&stream);
assert_eq!(info.playlist, "Test Movie");
assert_eq!(info.duration_secs, 7200.0);
assert_eq!(info.streams.len(), 4);
@@ -623,6 +636,7 @@ fn mkvstream_roundtrip_bdts() {
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: libfreemkv::LabelPurpose::Normal,
label: "English".into(),
})],
chapters: Vec::new(),
@@ -631,7 +645,7 @@ fn mkvstream_roundtrip_bdts() {
codec_privates: Vec::new(),
};
let writer: Box<dyn libfreemkv::mux::WriteSeek> = Box::new(Cursor::new(Vec::new()));
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let mut stream = MkvStream::create(writer, &dt).unwrap();
// Write PES frames targeting the audio track
@@ -642,13 +656,13 @@ fn mkvstream_roundtrip_bdts() {
keyframe: true,
data: vec![i; 100],
};
stream.write(&frame).unwrap();
PesStream::write(&mut stream, &frame).unwrap();
}
stream.finish().unwrap();
PesStream::finish(&mut stream).unwrap();
// Verify the info is correct
let info = stream.info();
let info = PesStream::info(&stream);
assert_eq!(info.streams.len(), 1);
assert_eq!(info.playlist, "Audio Only");
}
@@ -679,6 +693,7 @@ fn mkvstream_meta_preserves_all_streams() {
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: libfreemkv::LabelPurpose::Normal,
label: "English".into(),
}),
Stream::Audio(AudioStream {
@@ -688,6 +703,7 @@ fn mkvstream_meta_preserves_all_streams() {
language: "fra".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: libfreemkv::LabelPurpose::Normal,
label: "French".into(),
}),
Stream::Subtitle(SubtitleStream {
@@ -695,6 +711,7 @@ fn mkvstream_meta_preserves_all_streams() {
codec: Codec::Pgs,
language: "eng".into(),
forced: false,
qualifier: libfreemkv::LabelQualifier::None,
codec_data: None,
}),
Stream::Subtitle(SubtitleStream {
@@ -702,6 +719,7 @@ fn mkvstream_meta_preserves_all_streams() {
codec: Codec::Pgs,
language: "fra".into(),
forced: true,
qualifier: libfreemkv::LabelQualifier::None,
codec_data: None,
}),
],
@@ -711,10 +729,10 @@ fn mkvstream_meta_preserves_all_streams() {
codec_privates: Vec::new(),
};
let writer: Box<dyn libfreemkv::mux::WriteSeek> = Box::new(Cursor::new(Vec::new()));
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> = Box::new(Cursor::new(Vec::new()));
let stream = MkvStream::create(writer, &dt).unwrap();
let info = stream.info();
let info = PesStream::info(&stream);
assert_eq!(info.streams.len(), 5, "all 5 streams should be preserved");
assert_eq!(info.playlist, "Stream Test");
assert_eq!(info.duration_secs, 3600.0);
@@ -819,7 +837,8 @@ fn mkvstream_e2e_h264_produces_valid_mkv() {
}
}
let writer: Box<dyn libfreemkv::mux::WriteSeek> = Box::new(SharedWriter(output2.clone()));
let writer: Box<dyn libfreemkv::mux::WriteSeek + Send> =
Box::new(SharedWriter(output2.clone()));
let mut stream2 = MkvStream::create(writer, &dt).unwrap();
// Write the ES data (SPS+PPS+IDR) as a keyframe PES frame.
@@ -829,7 +848,7 @@ fn mkvstream_e2e_h264_produces_valid_mkv() {
keyframe: true,
data: es_data,
};
stream2.write(&frame1).unwrap();
PesStream::write(&mut stream2, &frame1).unwrap();
// Write a second non-IDR frame
let frame2 = libfreemkv::pes::PesFrame {
@@ -838,8 +857,8 @@ fn mkvstream_e2e_h264_produces_valid_mkv() {
keyframe: false,
data: es_data2,
};
stream2.write(&frame2).unwrap();
stream2.finish().unwrap();
PesStream::write(&mut stream2, &frame2).unwrap();
PesStream::finish(&mut stream2).unwrap();
let data = output2.lock().unwrap().clone().into_inner();
+12 -6
View File
@@ -1,7 +1,7 @@
//! UDF parser tests using a MockSectorReader.
use libfreemkv::error::Result;
use libfreemkv::{read_filesystem, SectorReader};
use libfreemkv::{SectorReader, read_filesystem};
use std::collections::HashMap;
const SECTOR_SIZE: usize = 2048;
@@ -39,7 +39,13 @@ impl MockSectorReader {
}
impl SectorReader for MockSectorReader {
fn read_sectors(&mut self, lba: u32, count: u16, buf: &mut [u8]) -> Result<usize> {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let total = count as usize * SECTOR_SIZE;
assert!(buf.len() >= total, "buffer too small");
for i in 0..count as u32 {
@@ -215,7 +221,7 @@ fn mock_sector_reader_roundtrip() {
// Read it back
let mut buf = vec![0u8; SECTOR_SIZE];
let n = reader.read_sectors(100, 1, &mut buf).unwrap();
let n = reader.read_sectors(100, 1, &mut buf, true).unwrap();
assert_eq!(n, SECTOR_SIZE);
assert_eq!(buf[0], 0xAB);
assert_eq!(buf[1], 0xCD);
@@ -223,7 +229,7 @@ fn mock_sector_reader_roundtrip() {
// Reading an unmapped sector returns zeros
let mut buf2 = vec![0xFFu8; SECTOR_SIZE];
let n2 = reader.read_sectors(999, 1, &mut buf2).unwrap();
let n2 = reader.read_sectors(999, 1, &mut buf2, true).unwrap();
assert_eq!(n2, SECTOR_SIZE);
assert_eq!(buf2[0], 0);
assert_eq!(buf2[2047], 0);
@@ -243,7 +249,7 @@ fn mock_sector_reader_multi_sector() {
// Read 2 consecutive sectors
let mut buf = vec![0u8; SECTOR_SIZE * 2];
let n = reader.read_sectors(10, 2, &mut buf).unwrap();
let n = reader.read_sectors(10, 2, &mut buf, true).unwrap();
assert_eq!(n, SECTOR_SIZE * 2);
assert_eq!(buf[0], 10);
assert_eq!(buf[SECTOR_SIZE], 11);
@@ -463,7 +469,7 @@ fn sector_reader_is_object_safe() {
let dyn_reader: &mut dyn SectorReader = &mut reader;
let mut buf = vec![0u8; SECTOR_SIZE];
let n = dyn_reader.read_sectors(0, 1, &mut buf).unwrap();
let n = dyn_reader.read_sectors(0, 1, &mut buf, true).unwrap();
assert_eq!(n, SECTOR_SIZE);
assert_eq!(buf[0], 42);
}