Commit Graph
942 Commits
Author SHA1 Message Date
MattJackson 385c9f094c v0.18.3: canonical_title_order — main feature first on branching UHDs 2026-05-09 20:08:56 -07:00
MattJackson 9f2a13739d Disc title order: main feature first on branching UHDs
Disc::titles previously sorted purely by duration_secs descending,
which puts a play-all virtual playlist at index 0 on UHDs that ship
one. Such playlists reference the same source clips multiple times
for seamless alternate-angle / alternate-ending playback and report
inflated duration AND inflated size_bytes that exceeds the disc's
physical capacity.

Concrete observed case (The Amateur 2025 4K UHD, 58.5 GB BD-100):
  Title 1 — 00020.mpls — 4h13m — 92.4 GB — 253 clips  ← impossible
  Title 2 — 00800.mpls — 2h02m — 57.2 GB — 1 clip      ← the movie

92.4 GB > 58.5 GB capacity is proof of clip double-counting. With
the duration-only sort, freemkv -t 1 / disc.titles.first() / autorip's
main-feature picker all selected the 4-hour composite instead of the
2-hour movie.

New canonical_title_order:
  1. Real titles (size_bytes <= capacity_bytes) before virtual
     composites — capacity gate is hard physical truth.
  2. Among real titles, fewer clips first (1-clip wins as the
     canonical main feature; multi-clip is either chapter-stitched
     or composite).
  3. Tiebreak on longer duration first.

Behaviour:
- Non-branching discs: unchanged. The longest 1-clip title is
  already the movie.
- Branching UHDs: virtual composite drops to the back, the real
  movie surfaces at index 0.

Comparator exposed as Disc::canonical_title_order for downstream
consumers that need the same logic on custom title sets.

Three regression tests (disc::tests::canonical_order_*):
- pushes_oversize_play_all_behind_real_main (The Amateur)
- preserves_natural_ranking_on_normal_disc
- fewer_clips_wins_tiebreak
2026-05-09 19:57:07 -07:00
MattJackson d9ce69bc9d v0.18.2: fix AACS nav-file scramble + sweep progress non-regression
decrypt::decrypt_sectors now restores chunks when decrypt_unit_full's
TS-sync verification fails, preventing 0.18.1's silent corruption of
MPLS/CLPI navigation files when DecryptingSectorSource decorates the
sweep reader. Fixes E6009 NoStreams on info iso:// for AACS-encrypted
UHDs ripped without --raw.

Disc::sweep progress takes max(snapshot.bytes_good, bytes_done) so
the user-visible counter never regresses below what the producer has
already sent.
2026-05-09 17:19:47 -07:00
MattJackson 7cd2c937ed 0.18.1 docs: refresh README, CHANGELOG, and docs/ for the trait split
The library's public-facing docs were sitting on the 0.17 trait
surface — Disc::copy, pes::Stream, SectorReader, etc. — even though
all in-tree callers migrated in 0.18 rounds 1-3. With 0.18.1 about
to ship, a user copy-pasting the README sample from crates.io would
have hit a compile error.

This commit is purely doc-side:

- README.md: Quick Start rewritten onto Disc::sweep + Disc::patch
  with caller-orchestrated multipass; Streams table footnote and
  Architecture row reference FrameSource / FrameSink.
- CHANGELOG.md: 0.18.1 entry describing the redesign — primitives,
  trait splits, deprecations (kept alive through 0.18.x, deletion
  target 0.18.2), throughput numbers.
- docs/{rip-recovery,api-design,architecture,disc-to-rip,
  drive-access,udf}.md: every Disc::copy / pes::Stream /
  SectorReader reference updated to the 0.18 trait surface.
- FEATURES.md: deleted (8+ versions stale; capabilities live in
  README.md and CHANGELOG.md now, matching the workspace-top
  FEATURES.md removal in 84acd65).
- examples/iso_dump.rs: verified compiles against 0.18.1.

No code changes.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 12:13:51 -07:00
MattJackson 107524da5e 0.18.1: bump version 2026-05-09 11:38:24 -07:00
MattJackson 055a3c5276 0.18 round 3: make Disc::sweep + Disc::patch pub (was pub(crate))
Round 3 step 1: lift the visibility on the two flat rip-phase verbs
so consumers (autorip + freemkv CLI) can call them directly instead
of going through Disc::copy's multipass dispatcher. Also lift their
option/outcome types and re-export at crate root.

- fn sweep -> pub fn sweep (with rustdoc explaining its role)
- fn patch -> pub fn patch (ditto)
- pub(crate) struct SweepOptions -> pub struct SweepOptions
- pub(crate) struct PatchOpts -> pub struct PatchOptions (renamed
  for consistency — both are 'Options')
- pub(crate) struct PatchOutcome -> pub struct PatchOutcome
- libfreemkv::{SweepOptions, PatchOptions, PatchOutcome} re-exports
  at crate root.

Disc::copy still exists and still calls Disc::sweep / Disc::patch
through the now-private sweep_internal / patch_internal wrappers.
Migration of the two autorip callers + the freemkv CLI's
disc_to_iso to direct sweep/patch is a follow-up; once those land
Disc::copy + CopyOptions + CopyResult delete in the same commit.

See (internal)/memory/0_18_redesign.md and
0_18_round3_migration_audit.md.

Single contributor: MattJackson.
2026-05-09 11:19:46 -07:00
MattJackson 50b04790ce 0.18 round 2: thread Halt through DiscStream construction
# Conflicts:
#	src/mux/disc.rs
2026-05-09 11:04:07 -07:00
MattJackson a00b2c884b 0.18 round 2: adopt DecryptingSectorSource decorator at sweep + patch + DiscStream 2026-05-09 11:03:43 -07:00
MattJackson 44e2c64d7e 0.18 round 2 (Halt threading): DiscStream accepts Halt at construction
Adds a Halt field to DiscStream, populated via the new
`with_halt(halt)` builder. The internal recovery / fill_extents
loops check `halt.is_cancelled()` directly. The existing
`set_halt(Arc<AtomicBool>)` method stays through the deprecation
window for callers (autorip mux) that haven't migrated; marked
#[deprecated] with a pointer to the constructor-time path.

Both signals are unified inside DiscStream: either Halt or the
legacy Arc<AtomicBool> triggers cancellation, so callers can mix
during the deprecation window without breaking stop behaviour.

See (internal)/memory/0_18_redesign.md and
0_18_round3_migration_audit.md.

Single contributor: MattJackson.
2026-05-09 10:58:54 -07:00
MattJackson d6535b8f57 0.18 round 2 (decrypt dedup): adopt DecryptingSectorSource at the two
existing call sites — sweep producer and DiscStream demux

Round 1 shipped the DecryptingSectorSource decorator
(libfreemkv/src/sector/decrypting.rs) but the existing decrypt
sites kept calling crate::decrypt::decrypt_sectors inline. This
commit migrates both:

- Disc::sweep (disc/mod.rs): producer wraps the input reader
  in DecryptingSectorSource::new(reader, keys) before the read loop.
  The inline decrypt_sectors call goes away — read_sectors yields
  plaintext directly.

- DiscStream (mux/disc.rs): constructor wraps the underlying
  Box<dyn SectorReader> in DecryptingSectorSource so the internal
  fill_extents / read path sees plaintext bytes. The DecryptKeys
  field stays on DiscStream for metadata-side use; it just no
  longer drives decryption.

Disc::patch carried the same inline decrypt step at three call
sites (main read, backtrack read, non-NOT_READY retry read). All
three migrated onto the same wrapping for a single audit surface.

Two small support changes carry the migration without touching
the round-1 decorator shape:
- sector/mod.rs gains specific SectorSource impls for
  &mut dyn SectorReader and Box<dyn SectorReader>, mirroring
  std's Read forwarding pattern. Generic blankets would conflict
  with the existing SectorReader → SectorSource blanket under the
  orphan rule (downstream could impl SectorReader for &mut U), so
  the impls are scoped to the dyn-trait shape we actually consume.
- sector/decrypting.rs gains DecryptingSectorSource::set_keys so
  DiscStream::set_raw() can flip the wrapped reader to a
  DecryptKeys::None pass-through without rebuilding the decorator
  (which would require moving the inner Box out from behind &mut self).

After this commit, grep `decrypt_sectors` in src/ shows the
function definition, its single use inside DecryptingSectorSource,
plus comments only. One audit surface for AACS / CSS / passthrough
correctness.

Behaviour-preserving: same plaintext bytes flow through; the only
difference is which type owns the decrypt step.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:49:53 -07:00
MattJackson 90ab00ed45 0.18 round 2: cargo fmt after sweep+patch+DiscStream merge
Auto-fmt nit on the multi-line map.flush() expression that landed
when the sweep + patch + DiscStream-FrameSource branches were merged
together. No semantic change.
2026-05-09 10:36:40 -07:00
MattJackson 6bfd7dfd13 0.18 round 2: confirm DiscStream as FrameSource via blanket impl 2026-05-09 10:32:56 -07:00
MattJackson b9a7f601d4 0.18 round 2: refactor Disc::patch onto Pipeline + PatchSink
# Conflicts:
#	src/io/mod.rs
#	src/io/pipeline.rs
2026-05-09 10:32:51 -07:00
MattJackson cf0a61f8b4 0.18 round 2: refactor Disc::sweep onto Pipeline + SweepSink (delete sweep_pipeline.rs) 2026-05-09 10:32:05 -07:00
MattJackson f98f07b2d3 0.18 round 2: refactor Disc::sweep onto Pipeline + SweepSink
Sweep was the original producer/consumer split that motivated the
generic Pipeline primitive (round 1, commit 198268b). Now that
Pipeline + Sink exist, sweep stops shipping its own bespoke
threading.

- New SweepSink: Sink<WorkItem> impl in src/disc/sweep.rs. Owns
  WritebackFile + Mapfile + ProgressSnapshot back-channel. apply()
  carries the file-write + mapfile.record per WorkItem; close()
  drains writeback, fsyncs, flushes mapfile.
- Disc::sweep: constructs SweepSink, calls Pipeline::spawn_named
  (so the consumer thread keeps showing up as
  freemkv-sweep-consumer), sends WorkItems, calls pipe.finish().
  The producer-side ReadCtx state machine, decrypt, set_speed,
  halt — all unchanged.
- Pipeline gains spawn_named(name, depth, sink) so callers can
  preserve identifiable thread names without the primitive baking
  one in. Also adds Pipeline::try_send for the throttled
  StatsRequest path that must not block the producer.
- Deleted src/disc/sweep_pipeline.rs entirely. WorkItem,
  ProgressSnapshot, ConsumerSummary moved into disc/sweep.rs as
  module-private types. WorkItem::Finish dropped — dropping the
  channel is the end-of-stream signal Pipeline already uses.

Behaviour-preserving: the sweep algorithm, mapfile invariants,
back-pressure via channel depth (DEFAULT_PIPELINE_DEPTH = 4) all
match the 0.17.13 implementation. New synthetic regression test
(sweep_pipeline_full_good_100_batches) exercises ~100 batches of
clean reads end-to-end through the new Pipeline path and verifies
bytes_good and ISO file size.

See (internal)/memory/0_18_redesign.md.
2026-05-09 10:31:05 -07:00
MattJackson b53454fa09 0.18 round 2: refactor Disc::patch onto Pipeline + PatchSink
Patch was strictly serial (per-sector recovery: read → seek+write
→ mapfile.record → next). Lifting the write+record onto a consumer
thread lets the drive issue the next per-sector retry while the
previous block's recovered bytes are being committed — small but
real win on damaged discs with many bad sectors, and uniform with
sweep's threading model.

- New PatchSink: Sink<PatchItem> impl in src/disc/patch.rs. Owns
  WritebackFile + Mapfile. apply() seeks+writes recovered bytes
  and records mapfile state per item; close() runs sync_all and
  mapfile.flush.
- Channel depth: WRITE_THROUGH_DEPTH (1). Patch wants minimum
  buffering — back-pressure should kick in immediately so the
  drive's per-sector retry budget isn't ahead of the writer.
- Disc::patch: keeps every existing recovery decision on the
  producer (reverse walk, damage-window skip, NOT_READY pauses,
  bridge-degradation handling, wedge exit, range watchdog).
  WritebackFile ownership moves to the sink.

Behaviour-preserving: per-sector single-shot read budget unchanged
(BU40N+Initio bridge wedge concern still respected); recovery
algorithm bit-identical.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:28:14 -07:00
MattJackson f28b6ee6d9 0.18 round 2: re-export Pipeline + Sink + Flow at the crate root
Round 2 #1 landed `Pipeline` / `Sink` / `Flow` / `DEFAULT_PIPELINE_DEPTH`
in `crate::io::pipeline` but only re-exported them through
`crate::io` (which is `pub(crate)`), so no out-of-tree consumer could
reach them. autorip's round 2 #2 (lifting the mux loop onto Pipeline +
MuxSink) is the first such consumer; surface the primitives at the
crate root for ergonomic access.

No behaviour change — the items themselves are unchanged from round
2 #1; this is just `pub use` plumbing.

Single contributor: MattJackson.
2026-05-09 10:22:44 -07:00
MattJackson d9d778fa64 0.18 round 2: confirm DiscStream as FrameSource
Mirror of the FrameSink concrete migrations slice (f52e4c5) on the
read side. DiscStream is the only meaningful source impl in tree;
all the mux/* impls are sinks.

The round-1 blanket impl<T: Stream + Send> FrameSource for T
already covers DiscStream if it's Send. This slice:

- Audits DiscStream's interior types for Send (its Box<dyn
  SectorReader> already requires Send via the trait's super-bound;
  verify nothing else interior breaks Send).
- Adds a synthetic-input test that constructs Box<dyn FrameSource>
  over a DiscStream, reads frames through the trait object, and
  exercises info() / headers_ready() / codec_private().
- (Conditional) Adds a direct FrameSource impl on DiscStream only
  if call-site ergonomics demand it; otherwise relies on the
  blanket.

No caller migrated. mux::resolve::input still returns
Box<dyn Stream>; autorip / CLI consumers still call Stream::read.
Per-caller migration is a later slice.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:13:14 -07:00
MattJackson bbfb887a35 0.18 round 1+2 integration fixes
Two clippy issues surfaced when round 1 polish + round 2 FrameSink
migrations both landed on libfreemkv main:

- src/halt.rs: clippy::new_without_default fires when a public new()
  exists without Default. The polish pass dropped the derive thinking
  it was redundant — clippy disagrees, so add a manual impl that
  forwards to new(). Doc-comment notes why both exist.

- src/disc/read_error.rs:372: pre-existing
  assert_eq!(.., true) trips clippy::bool_assert_comparison. Pre-0.18
  precommits passed because that lint sat outside the gate; the
  round-2 commits brought enough new clippy surface that it now
  shows up. Trivial cleanup: assert!(...) instead of assert_eq!.

Single contributor: MattJackson.
2026-05-09 10:00:06 -07:00
MattJackson 9884346c24 0.18 round 2: FrameSink concrete migrations for mux/* sinks 2026-05-09 09:53:12 -07:00
MattJackson 766b7c6636 0.18 round 1 polish: address libfreemkv code-review findings 2026-05-09 09:53:06 -07:00
MattJackson 925c30686b 0.18 round 1 polish: address libfreemkv code-review findings
Applies must-fix + in-scope should-fix items from the round-1 code
review:

- M1: FileSectorSource::open takes &Path (was &str — non-UTF-8 panic)
- M2: drop FileSectorSource's BufReader (defeated by absolute seeks)
- M3: WritebackFile Drop impl finalises the writeback pipeline
- M4: Pipeline::finish preserves panic payload in error message
- M5: pes::Stream is left without a : Send supertrait — concrete
  in-tree impls (MkvStream, M2tsStream) hold Box<dyn Read> /
  Box<dyn Write> trait objects that aren't Send, so the simple
  trait tightening would cascade into a wider Send audit. Per the
  review's escape clause the FrameSource blanket impl keeps its
  T: Send bound and the constraint is documented loudly there.
- S6: document Pipeline::send post-Flow::Stop semantics
- S9: truncate stale Stream docs (E9001/E9000 was runtime-only)
- S10: document WritebackPipeline.fd lifetime invariant
- S11: pub use pes::Stream as PesStream to disambiguate from
  disc::Stream codec enum at crate root
- S12: rename DEFAULT_DEPTH → DEFAULT_PIPELINE_DEPTH; add
  WRITE_THROUGH_DEPTH constant
- N14: drop Halt's Default derive (redundant with Halt::new)
- N17: Pipeline::spawn propagates thread-spawn error instead of expect
- N19: deprecation since = "0.18.0" (was "0.18.0-dev", non-conventional)
- N21: rename Apply enum to Flow

Deferred to follow-up commits: SectorReader/SectorSource competition
(migration commit), WritebackFile::create/open orphans (migration
commit), AACS round-trip test (design doc defers), various nits.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 09:52:25 -07:00
MattJackson f52e4c5d22 0.18 round 2: add FrameSink impls to concrete mux sinks
Per-impl migration of MkvStream / M2tsStream / NetworkStream /
NullStream / StdioStream from the deprecated pes::Stream trait
to the typed pes::FrameSink trait. Both impls coexist during
the 0.18 deprecation window — the existing Stream impls are
unchanged.

The FrameSink::finish signature differs (Box<Self> vs &mut self),
which is why this couldn't be a blanket impl. Each migration
re-borrows the box and delegates to the underlying Stream::finish
body.

FrameSink: Send forced two struct fields (M2tsStream's boxed
Write/Read, MkvStream's boxed WriteSeek/Read) to gain `+ Send`
bounds — minimum surface needed to make the Send-bounded trait
impl-able. mux::resolve::output's local Box<dyn WriteSeek>
construction picks up the same `+ Send`. tests/streams.rs's
shared `stream.write/.finish/.info/.read` calls were
disambiguated to `PesStream::*` to resolve the now-multiple
candidates from coexisting trait impls.

Caller migration (mux::resolve::output return type, autorip,
CLI) is a later slice. This commit only adds new impls; nothing
removed.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 09:49:18 -07:00
MattJackson 8d67790d2e 0.18: thread WritebackFile rename through FileSectorSink
The SectorSource/Sink agent and the WritebackFile-rename agent both
branched from main concurrently; the sector branch wrote against the
0.17 Writer name and only the rename branch knew about WritebackFile.
This integration commit reconciles the two: FileSectorSink::create /
::open / the inner-field type all use WritebackFile directly, and the
module-level + struct-level docs are corrected.
2026-05-09 09:17:39 -07:00
MattJackson 246a439990 0.18: FrameSource/FrameSink trait split (deprecate Stream) 2026-05-09 09:13:39 -07:00
MattJackson bd19041648 0.18: SectorSource/SectorSink trait split + DecryptingSectorSource decorator 2026-05-09 09:13:34 -07:00
MattJackson 198268b725 0.18: add crate::io::Pipeline + Sink trait
# Conflicts:
#	src/io/mod.rs
2026-05-09 09:13:30 -07:00
MattJackson 31424c8203 0.18: add crate::halt::Halt cancellation token 2026-05-09 09:13:07 -07:00
MattJackson c747ecc589 0.18: rename crate::io::Writer → WritebackFile 2026-05-09 09:12:59 -07:00
MattJackson 283a561c12 0.18 primitive: SectorSource/SectorSink trait split + DecryptingSectorSource
Splits the unidirectional read trait from a (planned) write trait at
the sector level, eliminating runtime "wrong direction" potential.
Keeps SectorReader alive as a pre-deprecation alias via blanket impl
so existing callers compile unchanged through the migration window.

Adds DecryptingSectorSource decorator: wrap any SectorSource in this
to get plaintext sectors out. Replaces the duplicate decrypt code
paths in sweep_pipeline and DiscStream (those migrations are
follow-up commits).

The formal #[deprecated] attribute on SectorReader is held back to a
follow-up commit because internal call sites in disc/, udf/, mux/,
and verify/ still go through the legacy trait, and the CI gauntlet
treats deprecation lints as errors. Behavioural intent — "this trait
is going away" — is documented on the trait itself.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 09:04:16 -07:00
MattJackson 2667d68675 0.18 primitive: FrameSource/FrameSink trait split (deprecate Stream)
Splits the bidirectional pes::Stream into one-direction traits so
calling read() on a write-only sink is a compile error instead of
runtime E9001. Keeps Stream alive as deprecated through 0.18 with a
blanket FrameSource impl so existing concrete types compile unchanged.

FrameSink can't be blanket-impl'd from Stream (different finish
signature), so concrete impls migrate per-type in a follow-up.

Concrete `impl pes::Stream for X` blocks in mux/* and the existing
tests gain a one-line `#[allow(deprecated)]` to keep `-D warnings`
clean during the deprecation window — no behavior changes.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 08:57:48 -07:00
MattJackson 57f2c22e30 0.18 primitive: crate::io::Pipeline + Sink trait
Generic bounded producer/consumer pipeline. The same shape applies to
sweep, patch, and mux today via three near-duplicate implementations
(or, in patch's and mux's case, no implementation at all). 0.18
collapses them onto one primitive. See
(internal)/memory/0_18_redesign.md for full context.

Single contributor: MattJackson.
2026-05-09 08:56:31 -07:00
MattJackson 5f3545d244 0.18 primitive: rename crate::io::Writer → WritebackFile
The type's job is the bounded-cache writeback pipeline (sync_file_range
+ posix_fadvise(DONTNEED)) — not generic writing. The 0.17 name was
ambiguous; reading `Writer::new(file)` gave no hint about what was
special. New name makes the role obvious at every call site.

Adds `WritebackFile::create(path)` and `WritebackFile::open(path)`
constructors so callers don't have to assemble a `File` first.

No alias kept; this is a clean 0.18 rename. See
(internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 08:53:17 -07:00
MattJackson 6e17ef0859 0.18 primitive: crate::halt::Halt cancellation token
One-bit cooperative cancellation flag. Replaces ad-hoc Arc<AtomicBool>
patterns scattered across libfreemkv (DiscStream::set_halt) and the
HALT_FLAGS global registry in autorip. See
(internal)/memory/0_18_redesign.md.

Single contributor: MattJackson (no attribution trailers).
2026-05-09 08:50:13 -07:00
MattJackson 6ec97af104 v0.17.13: thread Writer through patch + mux for big-write consistency
The bounded-cache writeback wrapper (crate::io::Writer) was added in
0.17.10 and wired into Disc::sweep in 0.17.11, but the other two
paths in the crate that write large amounts of data sequentially —
Disc::patch and the MKV/M2TS mux — were still operating on raw
std::fs::File. That meant the dirty-page burst pathology the wrapper
exists to prevent could still bite on slow / network-attached staging
during recovery and mux phases.

This release plugs those gaps:

- Disc::patch (disc/mod.rs:1981) now wraps the reopened ISO in
  Writer before any seek / write. sync_all on Writer cleanly drains
  the in-flight chunk before the existing fsync.
- mux/resolve.rs MKV and M2TS branches wrap the output File in
  Writer underneath BufWriter. UHD MKV mux routinely produces 70+ GB
  of sequential output; the page cache no longer absorbs that as a
  single hot blast on slow targets.

Mapfile, log, settings, history, and stream-pipeline byte buffers
remain unchanged: those are either small one-shot writes (where
the wrapper has zero benefit and adds a stream_position syscall) or
already use bounded persistence (mapfile time-batched in 0.17.12).
The principle: any path that writes substantial sequential data to
a single file uses Writer; trivial writes don't.
2026-05-09 06:32:08 -07:00
MattJackson 3a6c1aa5a3 v0.17.12: mapfile time-batched persistence — unblock NFS staging
Pre-0.17.12 every Mapfile::record() persisted the full mapfile via
tempfile-create + write + atomic-rename. On local LVM that's
microseconds; on NFS each rename is multiple RPCs through the
unraid user-share fuse layer, dragging a Black Mass UHD rip from
~11 MB/s on local to ~1.5 MB/s on NFS — the mapfile path alone burned
multiple seconds of wall time per real-world second of work.

Mapfile now batches the rename to once per second:

- record() always updates in-memory state and stats; only fires
  write_to_disk when last_flushed.elapsed() >= FLUSH_INTERVAL (1 s).
- New flush() API forces a persist; called by sweep_pipeline's
  consumer at end-of-sweep and by Disc::patch at end-of-patch,
  after the file's sync_all.
- Drop impl best-effort flushes so an early-return / unwind doesn't
  silently lose pending state.

Crash-safety changes from "lose at most one block" to "lose at most
1 s of recorded progress" — the ISO file's payload bytes are unaffected;
only the mapfile's authority over which sectors are already-good is at
risk, and a resume re-reads anything Pass 1 had already covered.

Measured on the BU40N test bed against Black Mass UHD inner zone:
- NFS staging: 1.5 MB/s → 16.48 MB/s (10.9× recovery)
- Local LVM staging: 11.09 MB/s → 11.83 MB/s (+6.7 % bonus)

Internal round_trip_load test now flushes before reading back from
disk. External patch / copy tests are unaffected: patch and
sweep_pipeline flush at completion before returning.
2026-05-08 23:00:52 -07:00
MattJackson 1ba3264747 v0.17.11: sweep producer/consumer split — overlap drive read with file write
Pre-0.17.11 sweep ran strictly serialised: SCSI read → decrypt → seek
+ write → mapfile.record → next read. Drive idled for the post-read
work; throughput capped at the sum of both costs. On a healthy disc
that's ~7-12 ms read + ~5-15 ms write/record per 64 KB batch, limiting
sustained throughput to ~10-12 MB/s on the test bed (BU40N + UHD inner
zone), well below the ~14-16 MB/s drive ceiling.

Decouples them: producer thread (caller's) owns SectorReader +
read_error state + decrypt + set_speed + halt; consumer thread (one
spawn) owns Writer + Mapfile, receives WorkItem messages, applies
file write + mapfile record. Bounded mpsc::sync_channel(4) gives
natural back-pressure. While the consumer writes batch N, the
producer is already reading batch N+1 — steady-state throughput is
now bound by the slower of the two pipelines (drive on healthy
discs), not their sum.

Side effects:
- Bisect path now decrypts. Pre-0.17.11 the bisect inner loop wrote
  raw cyphertext for single-sector recoveries on encrypted discs —
  quiet correctness bug exercised only by batch-fail-then-
  bisect-succeed on encrypted media. New producer-side decrypt
  covers main + bisect success paths uniformly.
- All read_ctx state stays single-threaded on producer (damage
  window, jump multiplier, etc.). No locking added.
- Mapfile remains single-writer on consumer. No locking.
- Halt latency: producer breaks loop, sends Finish, consumer drains
  ≤4 in-flight items + sync_all. ~1 batch (~12 ms) typical.
- BU40N + Initio bridge wedge concern unchanged: still single SCSI
  command in flight, error-path timing identical, no new retries.

New module: src/disc/sweep_pipeline.rs (WorkItem, ProgressSnapshot,
ConsumerInputs, spawn_consumer, consumer_loop, helpers). Public API
unchanged — Disc::copy / CopyOptions / CopyResult identical.

Patch (Pass N) is NOT changed; it's bound by drive recovery time, not
the read/write serialisation.
2026-05-08 21:32:11 -07:00
MattJackson ae2909fe8d v0.17.10: bounded-cache writeback pipeline for big sequential writes
Pass 1 sweep speed on a healthy disc previously dipped from ~15 MB/s
to ~1 MB/s every ~30 s on a host with default Linux dirty-page
settings. Empirical cause: the kernel's vm.dirty_ratio (~20% of RAM)
lets hundreds of MB of dirty pages accumulate, then bursts a flush at
99% disk utilisation that blocks app writes for ~1 s. Confirmed on
the BU40N test bed — dirty pages grew 112 → 563 MB between bursts;
lowering vm.dirty_bytes to 64 MB at the host sysctl level eliminated
the dips. Shipping the equivalent inside libfreemkv so users do not
need to tune the host kernel.

- New crate::io::Writer: drop-in File wrapper (impl Write + Seek).
  Wraps a per-platform WritebackPipeline that on Linux schedules
  sync_file_range(WRITE) + lagging sync_file_range(WAIT_AFTER) +
  posix_fadvise(DONTNEED) in 32 MB chunks, bounding dirty cache at
  ~64 MB. macOS and Windows ship a no-op stub.
- Disc::sweep wraps its output File in Writer. Loop body unchanged.
- Module is purpose-built so any large sequential output (patch,
  mux) can adopt the same wrapper as a one-line change later.
2026-05-08 19:54:25 -07:00
MattJackson 6a3d19a453 v0.17.7: sync release — no functional changes
Version bump to keep the four freemkv crates at unified versioning
after autorip's v0.17.6 + v0.17.7 work today. No libfreemkv code
changes; republished to crates.io so downstream consumers stay
aligned on the latest patch version.
2026-05-08 16:20:28 -07:00
MattJackson a35596d2d1 v0.17.5: Pass N kernel block-device fallback + per-range fixes
Direct-SATA BU40N + Dune Part Two UHD live testing exposed that the
v0.17.3 single-shot SCSI READ path matched 0/22 of the small bad-
sector LBAs that dd if=/dev/sr0 recovers on the same drive. This
release closes that gap and fixes adjacent bugs silently capping
recovery.

- /dev/sr0 pread fallback in Drive::read (Linux only): on SCSI READ
  Err, fall back to posix_fadvise(DONTNEED) + pread() against the
  corresponding block device. Kernel sr_mod runs ~5 internal retries
  with no per-attempt mid-layer escalation overhead — the mechanism
  behind dd's recovery advantage. End-to-end byte verification
  confirms the fallback path returns real disc data.

- Disc::patch per-range watchdog fix: MAX_RANGE_SECS was breaking
  'outer (one slow range killed the entire patch). Now skips to the
  next range. Pre-fix patch died after 4 sectors of range 1 of 47.

- Per-sector range budget: range_budget = sectors × 25 s, capped at
  1800 s. Replaces the flat 180 s/range that was unfair to medium
  ranges and pointlessly generous to single-sector ones.

- consecutive_failures resets per range. The wedge-exit detector is
  for stuck-on-one-range, not many-small-ranges-with-one-fail-each.

- Reverted inline 5× retry experiment (was hurting: each retry paid
  kernel SCSI escalation overhead). Restored READ_RECOVERY_TIMEOUT_MS
  to 60 s. The kernel-auto-retry pattern is now provided by sr0
  fallback.

Empirical: pass 1 recovered 94.6 MB / 11 s of main title (33 sr0
saves). Pass 2 added 0.6 MB. Remaining ~233 MB on the test disc
appears physically unrecoverable on this hardware.
2026-05-08 12:58:48 -07:00
MattJackson 318f654fed v0.17.3: fix scsi_recovery.rs Linux compile (missing Duration import)
tests/scsi_recovery.rs:
- Add `use std::time::Duration` inside both `#[cfg(target_os = "linux")]`
  blocks. Locally on macOS the linux blocks are cfg-out so the missing
  import was invisible to precommit on macOS.
- Bug pre-dated this branch but only surfaced when v0.17.2 release CI
  ran the test compile on Linux.

Cargo.toml: 0.17.2 -> 0.17.3.
2026-05-07 19:16:21 -07:00
MattJackson 5e82ef65de v0.17.2: bump version (skip 0.17.1 — stray tag exists)
Cargo.toml: 0.17.1 -> 0.17.2. Functionally identical to the prior
commit; 0.17.1 was never published to crates.io but a tag exists on
the remote pointing at an unrelated commit. Bumping past it.
2026-05-07 19:10:39 -07:00
MattJackson 8534607329 v0.17.1: cache priming, NonTrimmed marking, decrypt regression test
src/disc/mod.rs:
- Cache priming (3-sector lookback) before patch's single-sector reads.
  Drive read-ahead pulls in adjacent pages so the target may already be
  cached when we ask for it. Throwaway reads — failures here don't
  update mapfile state.
- When patch hits skip-limit on a range, leave remaining sectors
  NonTrimmed instead of marking Unreadable. We never tried to read those
  sectors, so don't give them terminal status — drive state evolves
  between passes (cache, mechanical settle), and a later pass may
  succeed.

tests/pass_n_patch_fix.rs:
- New regression test for the decrypt key inversion bug at
  src/disc/mod.rs:1938-1942. Asserts decrypt_sectors is invoked with
  the correct key when opts.decrypt=true.

tests/pass_n_size_aware_skip.rs:
- rustfmt-only changes.

Cargo.toml: 0.17.0 -> 0.17.1.
2026-05-07 19:09:17 -07:00
MattJackson 97e1a4cad3 unified read-error handler + pass N size-aware skip
New disc/read_error.rs as the single entry point all read failures flow
through. Handler classifies the error, updates the in-flight context
(damage window, retry budgets, jump multiplier), and returns a
ReadAction the caller dispatches on. Pass 1 (sweep) refactored to use
it; ~340 lines of nested if/else collapsed into ~120 lines of action
dispatch. Adding a new error class = one match arm. Logging is in one
place. Bisect inner failures don't poison the damage window. Jump
multiplier capped at 64 (max 1 GB jump for batch=32 — observed prior
unbounded behavior produce a single 56 GB jump on a wedged drive).

Pass N (patch) damage_skip is now size-aware: each skip is capped at
range_remaining/4 rather than the absolute MB-scale escalation. The
old logic could leap over a 100-sector bad range that hides a 50-sector
good middle; size-aware convergence finds the good middles instead.

Tests in tests/pass_n_size_aware_skip.rs exercise the size-aware skip
against synthetic patterns (25-bad/50-good/25-bad and three good
middles in a row) and prove ≥98% of good middles are recovered.
Existing test test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed
updated to reflect that MEDIUM_ERROR now triggers single-sector
bisect (which the BlockSizeFailingReader succeeds at).
2026-05-07 09:08:43 -07:00
MattJackson 3f6bf33de5 Format code 2026-05-04 20:06:10 -07:00
MattJackson 359301e8fc Pass 1 transport failure recovery without user intervention 2026-05-04 20:02:01 -07:00
MattJackson bfa527162a 0.17.0: unwrap safety fix, patch pass algorithm, clippy compliance
- Fix unwrap in disc/mod.rs sweep() hot path using pattern matching
- Patch pass excludes Unreadable sectors from work list
- Expose bytes_bad_in_title for accurate UI reporting
- All 256 tests pass, cargo clippy clean with -D warnings
2026-05-04 09:42:07 -07:00
MattJackson 45defd47f3 fix patch pass: exclude Unreadable from work list; expose bytes_bad_in_title; clippy 1.86 fixes
- patch(): only process NonTrimmed + NonScraped ranges (Unreadable=terminal, NonTried=not-yet-swept)
- bytes_bad_in_title: pub fn for autorip main-movie lost_ms computation
- Clippy 1.86: saturating_sub, unused assignments, unused variable
- fmt: rustfmt formatting
2026-05-03 16:35:06 -07:00
MattJackson fe8933dad2 v0.16.3: i18n damage display
- Add rip.damage_lost, rip.damage_lost_movie, rip.damage_lost_simple, rip.damage_none keys
- Update all 6 locale files (en, es, fr, de, it, pt, nl)
- CLI progress shows 'Xs lost (Y in movie)' instead of 'Xs unreadable'
- Perfect rips show '0s' instead of '0s unreadable'
2026-04-30 20:51:02 -07:00
MattJackson 0a3c1b7b70 fix: scsi_recovery test variable shadowing and missing imports for Linux CI 2026-04-30 19:57:16 -07:00