Commit Graph
59 Commits
Author SHA1 Message Date
Matthew Jackson 980eeb3de9 mux: track skipped bytes for accurate loss estimation
DiscStream skips a whole AACS unit (3 sectors = 6144 bytes) per
read-error event, but only the skip-event count was exposed. Loss
estimates built from errors*2048 therefore undercounted AACS loss ~3x.

Add a lost_bytes field that accumulates the actual zero-filled byte
count at each skip, expose it via a new Stream::lost_bytes() accessor
(default 0; DiscStream and CountingStream override), so consumers can
scale lost-video time by real bytes lost rather than the event count.

Regression tests assert the AACS path records 6144 B/event (and
exceeds the errors*2048 undercount) while the align=1 path records
2048 B/event.
2026-06-23 00:12:56 -07:00
Matthew Jackson f863e9a4be mux/disc: abort single-pass rip on transport failure (USB-bridge crash)
A direct disc://→mkv:// single-pass rip drives fill_extents in
skip_errors mode. On a read failure it shrank the batch, retried, and
once bottomed out zero-filled + skipped the unit and continued. A SCSI
transport failure (status=0xFF) is a USB-bridge crash, NOT a skippable
bad sector: the bridge is wedged and every subsequent read fails the
same way. So the loop marched the entire disc at one ~15s bridge-
recovery per probe, producing no MKV — the user-reported 'hundreds of
0x28/0xff warnings, runs forever, Movie.mkv never created'.

Fix: short-circuit to an error on transport failure before any
shrink/skip, even under skip_errors — mirroring the multipass sweep's
transport-failure rule in read_error::handle_read_error. The CLI
surfaces it so the user power-cycles the drive or switches to multipass
recovery. Regression test asserts exactly one read is issued and no skip
is counted (no infinite march).
2026-06-22 15:18:52 -07:00
Matthew Jackson 5941c059c6 v1.0.0-rc.1
CSS keyless decrypt (Stevenson), AACS 1.0/2.0/2.1, MPEG-2 DVD, multi-OS SCSI, multipass recovery, mux highway, audit hardening
2026-06-21 21:06:07 -07:00
Matthew Jackson 061f68594a 0.31.0: hardening and correctness pass across mux, codec, AACS/CSS, UDF/MPLS/CLPI, recovery, drive/SCSI, labels, and I/O
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
2026-06-07 17:37:38 -07:00
MattJackson cfc12774f2 mux: fix DVD subtitle/audio track collision, PGS/VobSub flush, unify TS codec table
Subtitle/DVD output-corruption + stream-mapping coverage fixes.

1. DVD subtitle/audio track-mapping collision (CRITICAL). The PS path
   routed 0xBD private-stream packets to a track via (sub_id & 0x1F)+1,
   so VobSub subtitle sub-id 0x20+j aliased audio track j+1: subtitle
   PES was fed to the AC-3 parser and the real subtitle track got
   nothing. Route by the canonical DVD PID instead via a new
   PsPacket::dvd_pid() that mirrors scan_dvd_titles' PID assignment
   (video 0xE0, audio 0xBD00+i, subtitle 0x20+j), then look up the
   track in pid_to_track. Fixed identically at all three sites
   (pipelined_stream consume_ps, disc.rs live feed, disc.rs EOF flush).
   Unmappable/unmapped packets now WARN instead of silently dropping.

2. PGS flush() missing. PgsParser inherited the no-op default flush, so
   the last subtitle of every PGS track (emitted only when a following
   PCS arrives) was dropped at EOF. Implemented flush() to drain the
   pending display set (duration_ns: None for the trailing block).

3. DVD VobSub multi-PES SPU not reassembled. A subpicture unit larger
   than one PES spans multiple PES (only the head carries a PTS).
   DvdSubParser is now stateful: it buffers per sub-stream until the
   leading 2-byte SPU_size is satisfied, inherits the head PTS, and
   emits one Frame. flush() drains a truncated trailing SPU at EOF.

4. One-table hygiene. scan_streams had a duplicate stream_type->Codec
   table that had drifted from Codec::from_coding_type (missing 0x80
   LPCM, 0x85 mapped to DTS-HD MA vs HR, etc.). scan_streams now uses
   from_coding_type plus a new Codec::kind()/CodecKind category split,
   so the two mappings can never diverge. Silent drops in
   scan_streams and bluray STN parsing now WARN with PID + type.

Tests: dvd_pid mapping + subtitle/audio collision regression, PGS
final-subtitle flush, VobSub multi-PES reassembly + EOF flush,
scan_streams 0x80 LPCM via from_coding_type.
2026-06-06 21:33:11 -07:00
MattJackson 6be5198886 libfreemkv: v1.0 hardening — codec/EBML/TS robustness + DTS parser fixes
Audit-driven fixes (rounds 1–3):
- hevc: correct hvcC profile/level SPS offsets (HEVC has a 2-byte NAL header)
- mkv: map all DTS variants to the registered A_DTS codec id; force a new
  cluster before the i16 cluster-relative timestamp can overflow
- ebml/mkvstream: bound untrusted EBML sizes (no multi-GB allocs); reject
  uint>8 (was an OOB panic) and non-{0,4,8} float widths (were a desync)
- ts: skip PES-header bytes that span a TS packet boundary; add the PMT
  section_len/prog_info_len bounds the PAT parser already had
- ac3: preserve a 0x0B77 syncword split across a PES boundary; cap buffer
- dts: validate each next-core boundary by decoded core size (a 0x7FFE8001
  pattern inside XLL payload no longer false-splits/drops the lossless
  extension); reject sub-minimum core frames; fix forced-emit PTS base
- lpcm: DVD program-stream PCM no longer double-strips the BD LPCM header
- vc1/mpeg2: do not emit a parameter-set-only PES as a standalone frame
- pgs/truehd: cap the pending reassembly buffer (parity with ac3/dts)
- aacs: ts_syncs_intact uses the exact packet count
- prefetched: capacity-guard the recycled-buffer set_len
- Cargo.toml: exclude project docs from the published crate

Convergence: a third independent audit pass found no remaining material
(CRITICAL/HIGH/MEDIUM) issues. Full precommit (fmt + clippy -D + tests,
Rust 1.86) green.
2026-06-05 16:23:39 -07:00
MattJackson 3b7bee9ed4 Fix DTS-HD MA muxing as lossy core only (#10)
DTS-HD MA/HRA access units on Blu-ray are a DTS core frame (sync
0x7FFE8001) followed by one or more DTS extension substreams (sync
0x64582025) carrying the lossless audio. Ground-truthing the Dunkirk
ISO showed the m2ts demuxer hands these out as SEPARATE PES packets on
the same PID: one core PES (exactly core-sized, nothing trailing), then
the extension substreams in following PES packets with their own later
PTS.

The old DtsParser emitted one frame per PES the moment a core frame was
complete, and dropped any PES with no core sync. So every core became a
core-only (lossy) frame and the extension PES packets were discarded as
junk -- silently downgrading the track to lossy DTS core (1557 kb/s CBR,
16-bit) instead of DTS-HD MA (VBR, 24-bit lossless).

Rewrite the parser to assemble across PES boundaries: an access unit
runs from its core sync up to (but not including) the NEXT core sync, so
the core plus every following extension substream stays together. Add a
CodecParser::flush() (default empty) called at end-of-stream by both the
pipelined and inline DiscStream mux paths to drain the final buffered
unit. A 64 KiB cap guarantees forward progress and never stalls if a
boundary can't be found.

Validated on the rip1 testbed: Dunkirk eng+ger and Fight Club eng main
audio now ffprobe as profile=DTS-HD MA (Fight Club eng at 24-bit), with
VBR packet sizes (~2716-2788 B) well above the old fixed 2012 B lossy
core. Genuinely-lossy DTS dub tracks are left untouched.
2026-06-05 06:40:27 -07:00
MattJackson 1b95193517 v0.25.2: DTS-HD codec ID + PGS BlockDuration
- MkvTrack::audio emits A_DTS/MA, A_DTS/HR, A_DTS per the DTS family
  instead of mislabelling everything as A_DTS. Plex transcoder and
  strict hardware decoders reject DTS-HD MA payload under a plain
  A_DTS track.
- PgsParser is now stateful: pairs display PCS with the following
  empty PCS to compute a duration. Frame::duration_ns + PesFrame::duration_ns
  carry it through; MkvMuxer::write_frame gains a final Option<u64>
  parameter that emits BlockGroup + BlockDuration when set. Fixes
  subtitle bitmaps lingering past their intended end-time.
2026-05-19 16:11:54 -07:00
MattJackson 4da559e39f mux: PrefetchedSectorSource event_fn + delete DiscStream::new_pipeline
* `PrefetchedSectorSource::new_with_events` adds an optional
  `event_fn` callback that fires `BytesRead` after every successful
  batch from the producer thread. The original `new()` becomes a
  thin no-events wrapper. Lets autorip wire the highway and still
  get UI progress events without polling the consumer side.

* `build_iso_pipeline` grows an `event_fn` arg so the autorip
  multipass mux can pipe BytesRead straight through to its progress
  UI.

* Stream trait gains a default `errors() -> u64` method (= 0) so
  Box<dyn Stream> callers (autorip's mux loop) can read the
  skip-on-error counter without downcasting. `DiscStream` overrides
  to return its `errors` field.

* Delete `DiscStream::new_pipeline` and the pipeline-mode fields
  (`demux_thread`, `demux_rx`) plus the `read_pipeline` helper.
  All pipeline construction now goes through
  `PipelinedPesStream` via `build_iso_pipeline`; `DiscStream`
  becomes the single-thread-only inline path used by the drive
  single-pass read.

* `lib.rs` re-exports `build_iso_pipeline`.
2026-05-19 14:19:37 -07:00
MattJackson c51b3181f2 mux: pipelined PES highway — read+decrypt → demux → parse on 3 threads
Introduces the freemkv mux throughput highway: a three-stage thread
pipeline that replaces the inline single-thread read path for any
file-backed source (ISO and m2ts file URLs both route through it).

  Thread A: read + decrypt  (PrefetchedSectorSource / BytePrefetcher)
  Thread B: M2TS demux      (DemuxThread)
  Thread C: codec parse     (PipelinedPesStream, on caller thread)

Each handoff uses a bounded crossbeam channel with a recycled buffer
pool — no allocations or memcpys in the steady-state hot loop.

Component map:

* io/byte_prefetcher.rs (new) — std::io::Read producer thread with
  recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector
  side) so demux_thread::spawn_zero_copy can wire either upstream.
* sector/prefetched.rs — recycled buffer pool added; into_channels()
  peels off the rx/recycle_tx/shell triple for zero-copy demux.
* mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs
  feed() on its thread, ships Vec<PesPacket> batches.
* mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls
  packets from the demux thread and runs codec parse on the caller.
* mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline
  (private) assemble the three stages; iso:// and m2ts:// both
  return PipelinedPesStream.
* mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted;
  the read direction lives on the highway now).
* mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find.
* mux/codec/hevc.rs — tightened frame_data initial capacity.
* mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB
  remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page
  first-touch fault tax that the previous 256 KiB pre-alloc paid on
  every PES boundary.
* mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as
  the legacy autorip ingress (drive + multipass paths still need
  on_event / skip_errors before they migrate to the highway).
* io/file_sector_source/* — per-OS prefetch() syscall hook
  (Linux readahead, macOS F_RDADVISE, Windows/other no-op).
* decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS;
  pool sized to all cores by default.

Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://):

  60 → 322 MB/s warm cache (old new_pipeline path)
  60 → 660 MB/s warm cache (highway path, this commit)
  60 → 126 MB/s sustained disk-bound

The IsoSectorReader baseline reader was deleted in favour of
FileSectorSource so the freemkv CLI and autorip exercise the same
read path.
2026-05-19 13:35:32 -07:00
MattJackson f1926c38dc v0.20.1: delete SectorReader, extract Disc::patch, doc/stub cleanup
WO-2 (delete SectorReader trait):
- The 0.18 trait split into SectorSource (read-only) and SectorSink
  (write-only) is final; the legacy SectorReader alias was a bridge.
- Renames every internal &mut dyn SectorReader (~25 sites) to
  &mut dyn SectorSource. The trait method capacity() becomes
  capacity_sectors() with a default of 0 (preserves SectorReader's
  default-0 behavior).
- Deletes the SectorReader trait, its blanket-to-Source bridge, and
  the FileSectorReader type alias. Adds explicit forwarding impls
  for Box<dyn SectorSource> and &mut dyn SectorSource so generic
  decorators like DecryptingSectorSource<S: SectorSource> compose.

WO-3a (extract Disc::patch):
- Moves Disc::patch (1230 lines) and bytes_bad_in_title from
  disc/mod.rs into disc/patch.rs as a split inherent impl. Zero
  behavior change — pure mechanical relocation. disc/mod.rs drops
  from 3,945 to 2,714 LOC.

WO-6 (partial):
- Deletes src/labels/png_filenames.rs — was a 72-LOC stub with
  detect() returning false, never wired into the PARSERS registry.

project docs doc drift fixes (audited 2026-05-13):
- JUMP_BASE_SECTORS: 256→1024 (64 MB base for UHD, not 8 MB)
- PASSN_DAMAGE_THRESHOLD_PCT: 12→6
- PASSN_SKIP_SECTORS_BASE: 64→32
- MAX_RANGE_SECS=180: replaced by proportional range_sectors × 25,
  capped at RANGE_BUDGET_CAP_SECS=1800.
2026-05-13 11:36:55 -07:00
MattJackson 4709a73c80 v0.20.0: delete FrameSource/FrameSink, keep single Stream trait
The 0.18 trait split into FrameSource (read-only) and FrameSink
(write-only) was an over-engineered API. Consumers don't think
"frame source backed by MKV" — they think "open MKV for reading".
The split paid a real API-complexity cost (two trait names, two
re-exports, dual impls per bidirectional type, deprecation bridge)
for one marginal property: compile-time direction-safety at the
trait-object boundary. The runtime error path on a wrong-direction
call (StreamReadOnly / StreamWriteOnly) is unambiguous and rare in
practice.

Deletions:
- pes::Stream is no longer #[deprecated]
- pes::FrameSource trait + its blanket-from-Stream bridge
- pes::FrameSink trait + the trampoline impls on every concrete type
- The compile-time-direction-safety test scaffolding
- Crate-root FrameSource / FrameSink re-exports

Additions:
- Stream is now Send-bounded (Stream: Send supertrait). Every
  concrete impl was already Send-compliant — Box<dyn Read + Send>
  and Box<dyn Write + Send> were already in place on the trait
  objects MkvStream / M2tsStream / etc hold internally. Promoting
  Send into the trait makes Box<dyn Stream> Send too, which lets
  autorip drop its SendStream unsafe newtype.

The public API is now: one Stream trait, one concrete type per
format, two constructors (open/create or input/output). Bidirectional
types route through internal Mode { Read | Write } discriminants.

Net: -347 lines libfreemkv, -38 lines autorip, -5 lines freemkv.
2026-05-13 08:42:14 -07:00
MattJackson 55bd1ee868 v0.19.1: repair Cargo.toml after botched 0.19.0 bump
v0.19.0 was tagged with a search-and-replace gone wrong:
rust-version, serde, and zip all had their version strings
replaced with "0.19.0". Edition 2024 rejected rust-version
0.19.0 (< 1.85), failing every CI build. No artifacts shipped
to crates.io.

Repair:
- rust-version: 0.19.0 → 1.86 (CI pin)
- serde: 0.19.0 → 1
- zip: 0.19.0 → 2

Also drops an unused start_lba binding in mux/disc.rs that
clippy 1.86 catches.
2026-05-12 21:29:07 -07:00
MattJackson ed73c1ab68 v0.18.25: add mux debug logging for reader type and stall detection 2026-05-12 16:27:16 -07:00
MattJackson 99236ffd55 labels: add class_reader, hand-rolled JVM .class file parser
Foundation for label parsers that need structured access to .class
files inside /BDMV/JAR/<x>.jar. Replaces noak (~3KLOC dep) with a
~1000-line std-only reader.

Public API:
- ClassFile::parse(&[u8]) -> Result<ClassFile>
- ConstantPool::{get, utf8, class_name, string, integer, member_ref, iter}
- Member::code(&pool) -> Option<CodeAttribute>
- CodeAttribute::instructions() -> Instructions iterator
- Instruction::{name, operand_u8, operand_u16, cp_index}
- Opcode constants (LDC, AASTORE, NEW, GETSTATIC, INVOKESPECIAL, ...)

Spec coverage:
- Constant pool: all 17 tag types incl. Long/Double 2-slot quirk
- Modified UTF-8 incl. 0xC0 0x80 -> U+0000 special case
- Bytecode iteration with full opcode size table
- Variable-length tableswitch / lookupswitch / wide

12 unit tests cover the opcode table edge cases (padded switch tables,
wide-iinc 6-byte form), modified-UTF-8 decoder, and iterator
stop-on-truncated behaviour.

Module is currently #![allow(dead_code)] — the public API is staged
for labels::deluxe (Phases A-E bytecode walker) and a labels::dbp
refactor onto the constant-pool iterator. Tests exercise the API
in isolation. The allow comes off as those callers land.

Also fixes two pre-existing clippy lints that 1.86's stricter checks
flagged after I touched the labels module:
- src/mux/disc.rs: while-let-loop in test fixture
- tests/pass_n_size_aware_skip.rs: type_complexity in helper signature

Precommit (cargo +1.86 fmt + clippy + test) green.
2026-05-10 15:06:34 -07:00
MattJackson 50b04790ce 0.18 round 2: thread Halt through DiscStream construction
# Conflicts:
#	src/mux/disc.rs
2026-05-09 11:04:07 -07:00
MattJackson 44e2c64d7e 0.18 round 2 (Halt threading): DiscStream accepts Halt at construction
Adds a Halt field to DiscStream, populated via the new
`with_halt(halt)` builder. The internal recovery / fill_extents
loops check `halt.is_cancelled()` directly. The existing
`set_halt(Arc<AtomicBool>)` method stays through the deprecation
window for callers (autorip mux) that haven't migrated; marked
#[deprecated] with a pointer to the constructor-time path.

Both signals are unified inside DiscStream: either Halt or the
legacy Arc<AtomicBool> triggers cancellation, so callers can mix
during the deprecation window without breaking stop behaviour.

See (internal)/memory/0_18_redesign.md and
0_18_round3_migration_audit.md.

Single contributor: MattJackson.
2026-05-09 10:58:54 -07:00
MattJackson d6535b8f57 0.18 round 2 (decrypt dedup): adopt DecryptingSectorSource at the two
existing call sites — sweep producer and DiscStream demux

Round 1 shipped the DecryptingSectorSource decorator
(libfreemkv/src/sector/decrypting.rs) but the existing decrypt
sites kept calling crate::decrypt::decrypt_sectors inline. This
commit migrates both:

- Disc::sweep (disc/mod.rs): producer wraps the input reader
  in DecryptingSectorSource::new(reader, keys) before the read loop.
  The inline decrypt_sectors call goes away — read_sectors yields
  plaintext directly.

- DiscStream (mux/disc.rs): constructor wraps the underlying
  Box<dyn SectorReader> in DecryptingSectorSource so the internal
  fill_extents / read path sees plaintext bytes. The DecryptKeys
  field stays on DiscStream for metadata-side use; it just no
  longer drives decryption.

Disc::patch carried the same inline decrypt step at three call
sites (main read, backtrack read, non-NOT_READY retry read). All
three migrated onto the same wrapping for a single audit surface.

Two small support changes carry the migration without touching
the round-1 decorator shape:
- sector/mod.rs gains specific SectorSource impls for
  &mut dyn SectorReader and Box<dyn SectorReader>, mirroring
  std's Read forwarding pattern. Generic blankets would conflict
  with the existing SectorReader → SectorSource blanket under the
  orphan rule (downstream could impl SectorReader for &mut U), so
  the impls are scoped to the dyn-trait shape we actually consume.
- sector/decrypting.rs gains DecryptingSectorSource::set_keys so
  DiscStream::set_raw() can flip the wrapped reader to a
  DecryptKeys::None pass-through without rebuilding the decorator
  (which would require moving the inner Box out from behind &mut self).

After this commit, grep `decrypt_sectors` in src/ shows the
function definition, its single use inside DecryptingSectorSource,
plus comments only. One audit surface for AACS / CSS / passthrough
correctness.

Behaviour-preserving: same plaintext bytes flow through; the only
difference is which type owns the decrypt step.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:49:53 -07:00
MattJackson d9d778fa64 0.18 round 2: confirm DiscStream as FrameSource
Mirror of the FrameSink concrete migrations slice (f52e4c5) on the
read side. DiscStream is the only meaningful source impl in tree;
all the mux/* impls are sinks.

The round-1 blanket impl<T: Stream + Send> FrameSource for T
already covers DiscStream if it's Send. This slice:

- Audits DiscStream's interior types for Send (its Box<dyn
  SectorReader> already requires Send via the trait's super-bound;
  verify nothing else interior breaks Send).
- Adds a synthetic-input test that constructs Box<dyn FrameSource>
  over a DiscStream, reads frames through the trait object, and
  exercises info() / headers_ready() / codec_private().
- (Conditional) Adds a direct FrameSource impl on DiscStream only
  if call-site ergonomics demand it; otherwise relies on the
  blanket.

No caller migrated. mux::resolve::input still returns
Box<dyn Stream>; autorip / CLI consumers still call Stream::read.
Per-caller migration is a later slice.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 10:13:14 -07:00
MattJackson 2667d68675 0.18 primitive: FrameSource/FrameSink trait split (deprecate Stream)
Splits the bidirectional pes::Stream into one-direction traits so
calling read() on a write-only sink is a compile error instead of
runtime E9001. Keeps Stream alive as deprecated through 0.18 with a
blanket FrameSource impl so existing concrete types compile unchanged.

FrameSink can't be blanket-impl'd from Stream (different finish
signature), so concrete impls migrate per-type in a follow-up.

Concrete `impl pes::Stream for X` blocks in mux/* and the existing
tests gain a one-line `#[allow(deprecated)]` to keep `-D warnings`
clean during the deprecation window — no behavior changes.

See (internal)/memory/0_18_redesign.md.

Single contributor: MattJackson.
2026-05-09 08:57:48 -07:00
MattJackson b5ed97801b libfreemkv: extend DiscRead with SCSI status/sense for 30% wedge diagnostics 2026-04-27 16:47:27 -07:00
MattJackson fc8eca44e1 docs: rewrite rip-recovery + drive-access for 0.13.6 single-shot model
Updates docs/ to reflect the recovery-loop strip:
- rip-recovery.md: drops Phase 1/2/3 description, replaces with three-layer
  model (Disc::patch multi-pass / DiscStream batch halving / Drive::read
  single-shot). Notes that no SCSI resets fire from any retry path.
- drive-access.md: removes SG_SCSI_RESET + STOP/START UNIT escalation
  references; SgIoTransport::reset is now kernel SG_IO flush + ALLOW
  MEDIUM REMOVAL only.
- src/mux/disc.rs + tests/: cargo fmt cleanup.
2026-04-24 21:35:33 -07:00
MattJackson 43836865be v0.13.6: strip Drive::read inline recovery + reset escalation; emit BytesRead
Drive::read is now single-shot. Phase 1/2/3 retries + scsi::reset+reopen
removed (~80 lines). recovery=true bumps timeout to 30s; recovery=false
stays at 1.5s. On any failure returns Err(DiscRead) immediately — caller
(Disc::patch outer loop, DiscStream batch halver) handles retries.

Inline reset+reopen WAS the wedge primitive on the LG BU40N. Per prior
post-mortem, every USB/SCSI reset path tested fails to recover the
wedged Initio bridge — the inline retry was pure cost.

SgIoTransport::reset (Linux) trimmed to kernel SG_IO state flush +
ALLOW MEDIUM REMOVAL. SG_SCSI_RESET ioctl + STOP/START UNIT escalation
removed. macOS reset removed (no-op). scsi::reset() top-level family
removed (no callers).

EventKind::BytesRead { bytes, total } now actually emitted from
DiscStream::fill_extents after each successful sector read. Was
declared in 0.13.0, never fired. Drives autorip per-device progress
in direct mode.

EventKind::Retry / SectorRecovered no longer emitted (variants kept
for forward compat). SpeedChange still emitted via Drive::set_speed
public path.

Tests: new tests/integration_progress_and_halt.rs (5 tests). 233 unit
tests + 5 integration green.
2026-04-24 21:32:45 -07:00
MattJackson 3dea679dac style: cargo fmt 2026-04-24 12:23:42 -07:00
MattJackson 166fc4bf8c v0.11.18: DiscStream halt flag — Stop works in dense bad-sector regions
DiscStream::fill_extents loops internally while the demuxer waits for
enough clean data to emit a PES frame. In a dense bad zone that loop
can run for minutes without returning to the outer read() call, so
the caller's Stop signal never gets serviced until a frame is finally
emitted — which may be very far away.

Add DiscStream::set_halt(Arc<AtomicBool>) — typically wired to
Drive::halt_flag() for unified Stop across drive recovery phases and
stream sector processing. fill_extents checks the flag at the top of
every retry iteration; raising it returns Err(Error::Halted) within
one SCSI round-trip.

No behavior change for callers that don't call set_halt. Unblocks the
architectural fix for the "Stop doesn't stop" bug observed on a
damaged UHD disc.
2026-04-24 07:41:13 -07:00
MattJackson 9aaddaa9b8 v0.11.17: adaptive batch sizer — no per-sector descent
Replace read_with_binary_search + 3×5s light recovery with an adaptive
sizer that shrinks on failure (halve, 3-aligned ≥6) and probes back up
after 100 MiB (51,200 sectors) of clean reads. Descent cost is paid
once per bad region, not once per bad sector.

Emit BatchSizeChanged { new_size, reason } on shrink and probe-up.
Remove BinarySearch event — no longer produced.

Side fix: scsi/macos.rs one-liner for manual_c_str_literals clippy
lint that surfaced on a newer toolchain.
2026-04-23 20:24:40 -07:00
Matt Jackson 40ec1e1eba v0.11.16: API cleanup — one method per action 2026-04-21 19:17:50 +00:00
Matt Jackson 8843833ea7 v0.11.15: lint cleanup — fmt + clippy clean 2026-04-21 18:52:55 +00:00
Matt Jackson d8089ec491 v0.11.14: audit fixes — trailing sectors, verify stop, SCSI sense, O_CLOEXEC
Fix trailing sectors dropped at extent boundaries when sector_count % 3 != 0.
Add verify_title stop support via progress callback returning bool.
Add O_CLOEXEC on all SCSI fd opens to prevent leak to child processes.
Fix SCSI sense descriptor format detection (0x72/0x73 vs 0x70/0x71).
2026-04-21 18:40:04 +00:00
Matt Jackson 6ba958fe8a Fix: initial batch read uses fast read, not full recovery 2026-04-21 02:02:13 +00:00
Matt Jackson 6971f00be2 Drive halt flag, sector events, binary search light recovery (3x5s) 2026-04-21 00:18:46 +00:00
Matt Jackson ad86f92d85 v0.11.11: binary search error recovery in fill_extents 2026-04-20 21:51:46 +00:00
Matt Jackson 081ad4f619 DiscStream skip_errors, read_sectors_recover(recovery) API 2026-04-20 03:38:32 +00:00
Matt Jackson fba1eb189c Fix truncated rips on dual-layer discs, propagate read errors
Use UDF file_extents() to read actual allocation descriptors instead
of assuming m2ts files are contiguous from file_start_lba. Dual-layer
UHD discs split large files across 70+ extents (~1 GB each) — the old
code created one extent from packet count which only covered the first
chunk, causing silent truncation at ~37%.

Also changed fill_extents() to return io::Result<bool> so read errors
propagate instead of being silently treated as EOF.
2026-04-18 02:08:34 +00:00
Matt Jackson 29f9e916d6 DiscStream::new() replaces open_drive/open_iso/from_reader
A stream is a stream. DiscStream::new() takes reader + title + keys +
batch + format — same pattern as every other stream constructor.

Deleted: open_drive(), open_iso(), from_reader() — these were helper
functions that chained multiple operations. Library provides primitives,
callers decide the sequence.

Removed disc:// case from input() — callers use Drive::open() +
Disc::scan() + DiscStream::new() directly for disc sources.
2026-04-17 15:32:48 +00:00
MattJackson 8820f7a460 Fix cargo fmt formatting 2026-04-16 17:51:45 +00:00
MattJackson 8e907393a3 Route DVD PS demuxer through codec parsers, fix CSS test expectations
- DVD PS path now calls parser.parse() like BD-TS path does
- MPEG-2 sequence headers extracted for codec_private
- Keyframe detection from parser instead of always-true
- Fix CSS roundtrip tests: descramble uses TAB1 permutation, not pure XOR
2026-04-16 04:49:46 +00:00
MattJackson e0f40583c4 Fix cargo fmt formatting 2026-04-15 22:32:53 +00:00
MattJackson 9bb1d02245 Fix batch overflow in Disc::copy(), DVD PGC parsing, demuxer flush at EOF
- Disc::copy() hardcoded batch=64 sectors, exceeding BU40N's 60-sector
  hw limit. Now accepts batch_sectors param, defaults to 60.
- IFO PGC: playback time at offset 0x04 not 0x02, cell time at cell+4
- DiscStream: set demuxer from content_format (TS for BD, PS for DVD)
- Flush TS/PS demuxers at EOF to avoid losing last PES frame
- M2tsStream: flush demuxer at EOF
- StdioStream: FMKV metadata header for roundtrip compatibility
2026-04-15 22:26:07 +00:00
MattJackson 0f18906ede v0.10.1: Streams are PES, Disc::copy() for sector dumps, zero English
Architecture:
- One stream per format, bidirectional PES (read/write on same type)
- IsoStream merged into DiscStream (one type, any SectorReader)
- Disc::copy() for disc→ISO raw sector dump
- IOStream trait deleted, all byte-level Read/Write removed
- ContentReader/OpenDisc/open_title/open_input/open_output deleted
- CountingStream wrapper for progress tracking

Error codes:
- All io::Error English strings replaced with Error enum variants
- From<Error> for io::Error conversion
- Unused variants removed, new stream/mux variants added

Deleted: mkvout.rs, pesout.rs, isowriter.rs, mkv-muxer-plan.md
Updated: all docs, README stream table, CHANGELOG

238 tests, 0 clippy warnings.
2026-04-15 19:46:01 +00:00
MattJackson 8bbf630d82 Fix all PES pipeline audit findings (20 issues)
Critical:
- C1: PES serialize validates track < 256 and data < 4GB
- C2: PES deserialize caps frame size at 256MB (OOM protection)
- C3: TsMuxer stuffing uses static buffer, no per-packet alloc
- C4: PES length uses unbounded (0x0000) for audio >65535 bytes
- C6: TsDemuxer validates AF length <= 183

Warning:
- W1: parse_timestamp validates marker bits, returns Option
- W2: PES header data_start clamped to data.len()
- W3: TsMuxer PTS conversion uses saturating_mul, rejects negative
- W4: AC3/DTS replace debug_assert with runtime bounds check
- W6: MKV block_vint handles 3-4 byte VINTs
- W7: meta.rs to_title() uses unwrap_or fallbacks instead of panic
- W8: MKV reader skips frames for non-existent tracks
- W9: DVD PTS uses higher-precision conversion (1e9/90000)
- FMKV read_header caps JSON at 10MB
- PAT section_len underflow guard

Suggestion:
- S2: TsMuxer uses static STUFF_FF buffer
- S3: HEVC parser single-pass NAL scan (was duplicated)
- S4: TsDemuxer caps remainder at one packet
- S5: PTS 90kHz→ns uses round-to-nearest
2026-04-15 16:22:28 +00:00
MattJackson d983985faa Fix all clippy warnings: dead code, match patterns, type complexity, docs
- Remove unused pes_buf field from M2tsStream and unused TS_PACKET/BD_TS_PACKET constants
- Replace match-with-single-pattern with if let (3 instances in drive/mod.rs)
- Replace match-can-be-? with ? operator for scsi::open call
- Add type aliases PesSetup and MkvHeaderResult to reduce type complexity
- Collapse identical if/else branches in tsmux.rs build_pes_header
- Use RangeInclusive::contains instead of manual range checks
- Make WriteSeek trait pub (was pub(crate) but leaked through pub fn)
- Remove empty line after doc comment in disc.rs
- Fix doc list item indentation in scsi/linux.rs (12 instances)
2026-04-15 04:09:56 +00:00
MattJackson 56fe26c9b8 All streams complete — DVD PS demux, network/stdio PES, MKV input
- DiscStream: BD (TsDemuxer) or DVD (PsDemuxer) auto-detected
- NetworkStream: Stream impl with PES serialize/deserialize
- StdioStream: Stream impl with PES serialize/deserialize
- MkvStream: Stream read returns PesFrame from EBML blocks
- M2tsStream: Stream read via TsDemuxReader
- PesFrame: serialize/deserialize for wire format
- TsDemuxReader: shared BD-TS demux helper
- All inputs and outputs support PES
2026-04-15 04:03:56 +00:00
MattJackson ff6004a567 Unified Stream trait: read() and write() on one type
Stream trait: read() returns PesFrame, write() accepts PesFrame.
A stream is a stream — you read from it or write to it.
No separate Input/Output traits.

API: libfreemkv::input(url) and libfreemkv::output(url, title, codecs)
Returns Box<dyn Stream>.
2026-04-15 03:33:29 +00:00
MattJackson ab63950ef1 PES pipeline: InputStream on DiscStream + IsoStream, MkvOutputStream
- DiscStream: next_frame() reads sectors → decrypts → demuxes → returns PesFrame
- IsoStream: same pattern, reads from ISO file
- MkvOutputStream: accepts PesFrame, writes MKV via MkvMuxer
- InputStream trait: next_frame(), info(), codec_private(), headers_ready()
- OutputStream trait: write_frame(), finish()
- FileSectorReader: SectorReader backed by a file
- PesFrame: track + pts + keyframe + data

Old Read/Write IOStream impls preserved for backward compatibility.
Next: replace pipe() in CLI to use PES pipeline.
2026-04-15 03:03:45 +00:00
MattJackson cab87ffdf4 DiscStream implements InputStream — produces PES frames
DiscStream now has TsDemuxer + CodecParsers internally.
next_frame() reads sectors → decrypts → demuxes → parses → returns PesFrame.
Old Read/Write impls preserved alongside for backward compatibility.
2026-04-15 02:54:08 +00:00
MattJackson e4ffe7cc91 Wire --raw through InputOptions to streams
set_raw() on IsoStream and DiscStream sets keys to None.
open_input passes raw flag to streams via InputOptions.
Streams skip decrypt when raw=true.
2026-04-15 01:38:18 +00:00
MattJackson fbdfccdd83 Add keys() to IOStream trait — streams know their own decrypt keys
- IOStream::keys() default returns DecryptKeys::None
- IsoStream and DiscStream override with real AACS/CSS keys
- Pipeline calls input.keys() instead of separate scan_keys()
- Streams are self-contained: read bytes + provide keys
2026-04-14 23:42:14 +00:00
MattJackson 2f52188f77 Drive recovery, reset on open, simplified DiscStream
- SgIoTransport::reset() — open/close/TUR/escalate on every open
- Drive::read() — single read method with error recovery (min speed,
  sleep 30s, retry, phase 1/2/3 escalation)
- Removed read_timeout, read_sectors, read_range — one read() method
- DiscStream simplified — no on_error/on_success/Recovery, delegates
  all error handling to Drive::read()
- IsoStream no longer decrypts — streams return raw bytes, pipeline
  handles decryption
- reset() on all platforms (Linux real, Windows/macOS stubs)
- Watchdog thread removed — kernel handles USB timeouts
2026-04-14 23:32:22 +00:00
MattJackson 3b96976a7a Add decrypt module, merge to one drive.read(), Disc::decrypt_keys()
- New decrypt.rs: DecryptKeys enum (AACS/CSS/None) + decrypt_sectors()
- Single drive.read() replaces read_disc/read_content (same SCSI READ(10))
- ContentReader and DiscStream use decrypt_sectors() (no duplicated crypto)
- Disc::decrypt_keys() exposes resolved keys for disc-to-ISO
2026-04-13 02:08:58 +00:00