New primitives for two-stage rip workflows: fast forward pass with
zero-fill on failures, then targeted retries of bad ranges via a
ddrescue-compatible mapfile.
- Disc::copy now takes &CopyOptions (breaking change from positional
args). Always writes a sidecar .mapfile. Opt-in skip_on_error +
skip_forward give ddrescue-style fast sweep: 64 KB blocks,
exponential skip-forward on failure, zero-fill bad blocks. Defaults
preserve pre-0.11.21 behavior (recovery reads, abort on bad sector).
- Disc::patch is new and idempotent. Reads the mapfile, re-reads every
non-finished range with full drive recovery, patches good bytes back
into the ISO at exact offsets. Call N times for N retry attempts.
- disc::mapfile is a new module. ddrescue text format, crash-safe
(flushed on every record()), greppable, human-editable, tool-compatible.
Status chars match ddrescue: ? / * / / / - / +.
- Re-exports FileSectorReader from the crate root.
- freemkv CLI caller (pipe.rs) updated to the new Disc::copy signature
in lockstep — shipped in the 0.11.21 freemkv CLI release.
Part of the 0.11.21 ecosystem sync (libfreemkv + freemkv + bdemu +
autorip all on 0.11.21).
DiscStream::fill_extents loops internally while the demuxer waits for
enough clean data to emit a PES frame. In a dense bad zone that loop
can run for minutes without returning to the outer read() call, so
the caller's Stop signal never gets serviced until a frame is finally
emitted — which may be very far away.
Add DiscStream::set_halt(Arc<AtomicBool>) — typically wired to
Drive::halt_flag() for unified Stop across drive recovery phases and
stream sector processing. fill_extents checks the flag at the top of
every retry iteration; raising it returns Err(Error::Halted) within
one SCSI round-trip.
No behavior change for callers that don't call set_halt. Unblocks the
architectural fix for the "Stop doesn't stop" bug observed on a
damaged UHD disc.
Replace read_with_binary_search + 3×5s light recovery with an adaptive
sizer that shrinks on failure (halve, 3-aligned ≥6) and probes back up
after 100 MiB (51,200 sectors) of clean reads. Descent cost is paid
once per bad region, not once per bad sector.
Emit BatchSizeChanged { new_size, reason } on shrink and probe-up.
Remove BinarySearch event — no longer produced.
Side fix: scsi/macos.rs one-liner for manual_c_str_literals clippy
lint that surfaced on a newer toolchain.
Fix trailing sectors dropped at extent boundaries when sector_count % 3 != 0.
Add verify_title stop support via progress callback returning bool.
Add O_CLOEXEC on all SCSI fd opens to prevent leak to child processes.
Fix SCSI sense descriptor format detection (0x72/0x73 vs 0x70/0x71).
Replace blocking ioctl(SG_IO) with the sg driver's async interface.
Commands are submitted via write(), waited on via poll() with a hard
wall-clock timeout, and completed via read(). If poll() times out,
the fd is abandoned and a fresh one opened — the kernel can no longer
hold us hostage during USB error recovery.
- write() submits command, returns immediately
- poll() enforces exact timeout (EINTR-safe with deadline tracking)
- read() retrieves result + copies data to caller's buffer
- On timeout: old fd closed in background thread, new fd opened
- No SG_FLAG_DIRECT_IO — kernel buffers for safe timeout abandonment
- Store device_path for fd reopen after timeout
- Drop guards fd=-1 (abandoned fd)
Use UDF file_extents() to read actual allocation descriptors instead
of assuming m2ts files are contiguous from file_start_lba. Dual-layer
UHD discs split large files across 70+ extents (~1 GB each) — the old
code created one extent from packet count which only covered the first
chunk, causing silent truncation at ~37%.
Also changed fill_extents() to return io::Result<bool> so read errors
propagate instead of being silently treated as EOF.
A stream is a stream. DiscStream::new() takes reader + title + keys +
batch + format — same pattern as every other stream constructor.
Deleted: open_drive(), open_iso(), from_reader() — these were helper
functions that chained multiple operations. Library provides primitives,
callers decide the sequence.
Removed disc:// case from input() — callers use Drive::open() +
Disc::scan() + DiscStream::new() directly for disc sources.
- DTS: buffer with core sync detection + frame size from header
- TrueHD: buffer with unit length field parsing
- Same pattern as AC3 fix: incomplete frames held for next PES
- When PES boundaries align (normal case), buffering is a no-op
- Add state to Ac3Parser (was stateless, split frames at PES boundaries)
- Buffer leftover bytes from incomplete frames for next PES packet
- Calculate exact AC3 frame size from fscod/frmsizecod table
- Calculate EAC3 frame size from frmsiz field
- Skip invalid frame sizes (0 or >8192)
- Eliminates all AC3 decode errors on BD and UHD output
- DVD PS path now calls parser.parse() like BD-TS path does
- MPEG-2 sequence headers extracted for codec_private
- Keyframe detection from parser instead of always-true
- Fix CSS roundtrip tests: descramble uses TAB1 permutation, not pure XOR
- Implement complete CSS key chain: bus auth → disc key → title key
- Add 31 player keys for disc key decryption
- Read disc key via READ DVD STRUCTURE format 0x02
- Read title key via REPORT KEY format 0x04
- Fix CryptKey round 1: use original scratch for term, not modified tmp1
- Fix decrypt_key: use TAB5 for LFSR1 output, TAB4 for LFSR0^invert
- Fix descramble_sector: use TAB5 for LFSR1, TAB4 for LFSR0 (no invert),
and apply TAB1 permutation to ciphertext before XOR
- Fix title key bus XOR: forward order (bus_key[i]), not reversed
- Two-session auth: disc key and title key need separate AGID sessions
- Fix crack_key: scan across extents for scrambled sectors
- Fix TsDemuxer: dynamic PID table size for DVD PIDs
- Set max read speed after scan for DVD riplock removal
- Disc::copy() hardcoded batch=64 sectors, exceeding BU40N's 60-sector
hw limit. Now accepts batch_sectors param, defaults to 60.
- IFO PGC: playback time at offset 0x04 not 0x02, cell time at cell+4
- DiscStream: set demuxer from content_format (TS for BD, PS for DVD)
- Flush TS/PS demuxers at EOF to avoid losing last PES frame
- M2tsStream: flush demuxer at EOF
- StdioStream: FMKV metadata header for roundtrip compatibility
Architecture:
- One stream per format, bidirectional PES (read/write on same type)
- IsoStream merged into DiscStream (one type, any SectorReader)
- Disc::copy() for disc→ISO raw sector dump
- IOStream trait deleted, all byte-level Read/Write removed
- ContentReader/OpenDisc/open_title/open_input/open_output deleted
- CountingStream wrapper for progress tracking
Error codes:
- All io::Error English strings replaced with Error enum variants
- From<Error> for io::Error conversion
- Unused variants removed, new stream/mux variants added
Deleted: mkvout.rs, pesout.rs, isowriter.rs, mkv-muxer-plan.md
Updated: all docs, README stream table, CHANGELOG
238 tests, 0 clippy warnings.
Streams are streams — they take impl Read, not Read+Seek or File.
- MkvStream::open takes impl Read (was Read+Seek)
- EBML element skipping uses skip_bytes() instead of seek(Current)
- Byte position tracking uses remaining-bytes counter, not stream_position()
- Removed file_size from open (progress is CLI concern)
- M2tsStream::open takes impl Read (was Read+Seek)
- Buffers first 1MB for FMKV header / PMT scan
- Uses chain reader (buffered head + rest) for sequential reading
- Duration unknown without seeking (0.0) — CLI can set from metadata
- Removed ReadSeek trait (no longer needed)
- WriteSeek kept (MKV muxer container format requires seeking internally)
Design fix: codec_privates are now a field on DiscTitle, not a separate
parameter passed through the pipeline. This eliminates the root cause of
the network codec_private bug (forgot to pass the separate param).
API changes:
- output() takes (url, &DiscTitle) — no separate codec_privates param
- MkvOutputStream::create, M2tsOutputStream::create, NetworkOutputStream::connect
all read codec_privates from title.codec_privates
- M2tsMeta::from_title() takes only &DiscTitle — reads privates from title
- Deleted from_title_with_privates (was the wrong-name duplicate)
- Merged read_header + read_header_from_stream into one read_header(impl Read)
- Deleted finish(self) from TsMuxer, keep only finish(&mut self)
Rule: ONE public method per action. No _with_X, _from_Y, _ref variants.
Bug 1: M2TS roundtrip dropped frames — TsMuxer converts length-prefixed
NALs to Annex B, prepends VPS/SPS/PPS from HEVCDecoderConfigurationRecord.
Bug 2: MKV remux lost codec_private — MkvStream.codec_private() now returns
data from EBML header.
FMKV header carries codec_private (base64) per video stream for lossless
M2TS roundtrip.
- M2tsOutputStream writes FMKV metadata header before TS data
- NetworkOutputStream sends FMKV header on connect
- Enables M2TS→MKV and network roundtrip to work correctly
- Remove IOStream, Read, Write impls from NetworkStream
- PES write sends FMKV header before first frame (protocol fix)
- PES finish sends TCP shutdown for clean EOF
- Update tests to use PES roundtrip instead of byte-level
- Remove NetworkStream from open_input/open_output (use input/output)
- Remove unused pes_buf field from M2tsStream and unused TS_PACKET/BD_TS_PACKET constants
- Replace match-with-single-pattern with if let (3 instances in drive/mod.rs)
- Replace match-can-be-? with ? operator for scsi::open call
- Add type aliases PesSetup and MkvHeaderResult to reduce type complexity
- Collapse identical if/else branches in tsmux.rs build_pes_header
- Use RangeInclusive::contains instead of manual range checks
- Make WriteSeek trait pub (was pub(crate) but leaked through pub fn)
- Remove empty line after doc comment in disc.rs
- Fix doc list item indentation in scsi/linux.rs (12 instances)
- DiscStream: BD (TsDemuxer) or DVD (PsDemuxer) auto-detected
- NetworkStream: Stream impl with PES serialize/deserialize
- StdioStream: Stream impl with PES serialize/deserialize
- MkvStream: Stream read returns PesFrame from EBML blocks
- M2tsStream: Stream read via TsDemuxReader
- PesFrame: serialize/deserialize for wire format
- TsDemuxReader: shared BD-TS demux helper
- All inputs and outputs support PES
- Unified Stream trait: read() and write() on one type
- PesFrame serialize/deserialize for wire format
- TsDemuxReader: shared BD-TS demux for any Read source
- MkvStream: PES read from MKV (EBML → PesFrame)
- M2tsStream: PES read via TsDemuxReader
- Network/Stdio output: PES serialization directly (no BD-TS wrap)
- Network/Stdio input: deferred (needs PES deserialization protocol)
- TsMuxer for M2TS output from PES frames
- input() and output() functions return Box<dyn Stream>
Stream trait: read() returns PesFrame, write() accepts PesFrame.
A stream is a stream — you read from it or write to it.
No separate Input/Output traits.
API: libfreemkv::input(url) and libfreemkv::output(url, title, codecs)
Returns Box<dyn Stream>.
- TsMuxer: PES frames → BD-TS packets (new, reverse of TsDemuxer)
- M2tsOutputStream: PES → TsMuxer → file
- NetworkOutputStream: PES → TsMuxer → TCP
- StdioOutputStream: PES frames → stdout
- NullOutputStream: discard
- MkvOutputStream: PES → MKV mux
- All outputs via open_pes_output()
- All inputs via open_pes_input() (ISO, disc)
- No byte-level fallback — everything is PES
- pes.rs: PesFrame, InputStream, OutputStream traits
- sector.rs: SectorReader now public, added FileSectorReader (ISO = file)
- Foundation for unified DiscStream that handles both disc and ISO
When the MKV muxer transitions from Scanning to Streaming phase,
it creates a fresh TsDemuxer. The old demuxer's remainder bytes
(partial 192-byte BD-TS packets) were lost, causing the new demuxer
to lose sync. All subsequent feed() calls found 0 packets.
Fix: transfer remainder via take_remainder/set_remainder so the
new demuxer maintains packet alignment.
Previously returned HandshakeResult with zeros when all host certs
failed. Now returns None. Also propagates volume_id read failure
instead of silently using zeros.
Previously used all-zeros AES key when unit_key_idx was out of range,
producing silently corrupted output. Now returns DecryptFailed error.
Also wired --raw flag through InputOptions → set_raw() on streams.
IsoStream decrypts in its read path using keys from scan.
IOStream trait gets keys() method with default DecryptKeys::None.
Pipeline no longer handles decrypt — just reads decrypted bytes.
- IOStream::keys() default returns DecryptKeys::None
- IsoStream and DiscStream override with real AACS/CSS keys
- Pipeline calls input.keys() instead of separate scan_keys()
- Streams are self-contained: read bytes + provide keys
- Rename DriveSession → Drive across entire codebase
- find_drives() returns Vec<Drive>, find_drive() returns Option<Drive>
- resolve_device() now pub(crate) — internal only
- StreamUrl is now a typed enum (Disc, Mkv, M2ts, Iso, Network, Stdio, Null)
with scheme() and path_str() accessors, replacing struct of Strings
- Add lock_tray() / unlock_tray() for safe disc access during rips
- Improve reset() with eject cycle that clears LibreDrive stuck state
- Add Send bounds to ScsiTransport and PlatformDriver traits
- DiscOptions uses PathBuf instead of String for device/keydb paths
- Update doc example to use new Drive API
- DriveStatus enum: TrayOpen, NoDisc, DiscPresent, NotReady, Unknown
- drive_status(): GET EVENT STATUS NOTIFICATION with TUR fallback
- reset(): PREVENT ALLOW → START STOP → init() escalation
- wait_ready(): tries reset on Illegal Request, falls back to drive_status
- Remaining: standard READ(10) still fails in LibreDrive stuck state
- wait_ready: detects MMkv vendor probe when TUR returns Illegal Request
- scan: capacity fallback to 0 when READ CAPACITY fails
- Still needs: re-init to restore standard SCSI commands, or use raw reads for UDF
- Doc comments on DriveSession, find_drives, all Error variants, Result type
- 24 format! strings inlined (clippy pedantic)
- 25 long hex literals with separators (0xFFFFFFFF → 0xFFFF_FFFF)
- README install example updated to 0.8
MattJackson