0.18.1 docs: refresh README, CHANGELOG, and docs/ for the trait split
The library's public-facing docs were sitting on the 0.17 trait
surface — Disc::copy, pes::Stream, SectorReader, etc. — even though
all in-tree callers migrated in 0.18 rounds 1-3. With 0.18.1 about
to ship, a user copy-pasting the README sample from crates.io would
have hit a compile error.
This commit is purely doc-side:
- README.md: Quick Start rewritten onto Disc::sweep + Disc::patch
with caller-orchestrated multipass; Streams table footnote and
Architecture row reference FrameSource / FrameSink.
- CHANGELOG.md: 0.18.1 entry describing the redesign — primitives,
trait splits, deprecations (kept alive through 0.18.x, deletion
target 0.18.2), throughput numbers.
- docs/{rip-recovery,api-design,architecture,disc-to-rip,
drive-access,udf}.md: every Disc::copy / pes::Stream /
SectorReader reference updated to the 0.18 trait surface.
- FEATURES.md: deleted (8+ versions stale; capabilities live in
README.md and CHANGELOG.md now, matching the workspace-top
FEATURES.md removal in 84acd65).
- examples/iso_dump.rs: verified compiles against 0.18.1.
No code changes.
See freemkv-private/memory/0_18_redesign.md.
Single contributor: MattJackson.
This commit is contained in:
@@ -1,5 +1,83 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 0.18.1 (2026-05-09)
|
||||||
|
|
||||||
|
### I/O stack redesign — primitives over orchestration
|
||||||
|
|
||||||
|
0.18 reshapes the read/write surface from "library does the multipass dance"
|
||||||
|
to "library hands the caller flat verbs and a few composable primitives." All
|
||||||
|
in-tree consumers (autorip, the `freemkv` CLI) drove their own multipass
|
||||||
|
loops in 0.18 rounds 1-3; 0.18.1 lands the cleanup with the deprecated names
|
||||||
|
still alive for one minor-version window.
|
||||||
|
|
||||||
|
The 0.18 design notes are in
|
||||||
|
`freemkv-private/memory/0_18_redesign.md` (private) — this entry sticks to
|
||||||
|
what changed at the public surface.
|
||||||
|
|
||||||
|
#### Flat verbs
|
||||||
|
|
||||||
|
`Disc::sweep` is the forward Pass 1. `Disc::patch` is one retry pass over the
|
||||||
|
mapfile. Neither knows about pass index, retry budget, or accept-loss policy
|
||||||
|
— the caller invokes them in whatever sequence its use case dictates. The
|
||||||
|
old multipass-aware `Disc::copy` dispatcher is deprecated and slated for
|
||||||
|
deletion in 0.18.2; no in-tree caller still uses it.
|
||||||
|
|
||||||
|
#### Trait splits — direction-typed at compile time
|
||||||
|
|
||||||
|
- `pes::Stream` (combined read+write) is split into `FrameSource` and
|
||||||
|
`FrameSink`. Calling `read()` on a write-only sink is now a compile error,
|
||||||
|
not the runtime `E9001` (`StreamWriteOnly`) it used to be.
|
||||||
|
- `SectorReader` is split into `SectorSource` (read) and `SectorSink`
|
||||||
|
(write). `Drive` impls `SectorSource` only; `FileSectorSource` /
|
||||||
|
`FileSectorSink` replace `FileSectorReader` for ISO-backed I/O.
|
||||||
|
- A blanket impl bridges legacy `SectorReader` callers onto the new
|
||||||
|
`SectorSource` so existing code keeps compiling through the deprecation
|
||||||
|
window.
|
||||||
|
|
||||||
|
#### New primitives
|
||||||
|
|
||||||
|
- `Halt` — one cancellation token (cloneable, `Arc<AtomicBool>` under the
|
||||||
|
hood) replaces the three near-duplicate halt flags scattered through the
|
||||||
|
workspace. Threaded through every long-running loop.
|
||||||
|
- `Pipeline<I, R>` + `Sink<I>` — generic producer/consumer primitive in
|
||||||
|
`crate::io`. Replaces the bespoke `disc/sweep_pipeline.rs` and now also
|
||||||
|
drives `Disc::patch` and the autorip mux loop. `DEFAULT_PIPELINE_DEPTH`
|
||||||
|
for streaming reads (`4`); `WRITE_THROUGH_DEPTH` for write-through patch
|
||||||
|
semantics (`1`).
|
||||||
|
- `WritebackFile` — was `crate::io::Writer`. The renamed type makes its job
|
||||||
|
explicit: a `File` wrapper that runs continuous `sync_file_range` +
|
||||||
|
`posix_fadvise(DONTNEED)` to keep the kernel dirty-page cache bounded on
|
||||||
|
long sequential writes.
|
||||||
|
- `DecryptingSectorSource<S>` — a single decorator wrapping any
|
||||||
|
`SectorSource` to yield plaintext sectors. One audit surface for AACS /
|
||||||
|
CSS / passthrough; the previous two-site decrypt (sweep producer +
|
||||||
|
`DiscStream` demux) is gone.
|
||||||
|
|
||||||
|
#### Throughput
|
||||||
|
|
||||||
|
The round-2 producer/consumer split is now applied uniformly to sweep,
|
||||||
|
patch, and mux. Mux on NFS-staged UHD measured ~16 MB/s sustained on the
|
||||||
|
test bed (was ~12 MB/s pre-round-2).
|
||||||
|
|
||||||
|
#### Module reorg
|
||||||
|
|
||||||
|
- `sector/` and `io/` are now module directories.
|
||||||
|
- `disc/sweep.rs`, `disc/patch.rs`, and `disc/mapfile.rs` split out of the
|
||||||
|
monolithic `disc/mod.rs`.
|
||||||
|
|
||||||
|
#### Renames (no behavior change)
|
||||||
|
|
||||||
|
- `crate::io::Writer` → `crate::io::WritebackFile`.
|
||||||
|
- `Apply` → `Flow` (`Sink::apply` return value).
|
||||||
|
- `DEFAULT_DEPTH` → `DEFAULT_PIPELINE_DEPTH`.
|
||||||
|
- `PatchOpts` → `PatchOptions`.
|
||||||
|
|
||||||
|
#### Deprecated (alive in 0.18.1, deletion target 0.18.2)
|
||||||
|
|
||||||
|
`Disc::copy`, `pes::Stream`, `SectorReader`, `FileSectorReader`,
|
||||||
|
`CopyOptions`, `CopyResult`, `DiscStream::set_halt`. Each compiles with a
|
||||||
|
deprecation warning; all in-tree call sites have migrated.
|
||||||
|
|
||||||
## 0.17.13 (2026-05-09)
|
## 0.17.13 (2026-05-09)
|
||||||
|
|
||||||
### Use `crate::io::Writer` uniformly for all binary file output
|
### Use `crate::io::Writer` uniformly for all binary file output
|
||||||
|
|||||||
-34
@@ -1,34 +0,0 @@
|
|||||||
# libfreemkv — Feature List
|
|
||||||
|
|
||||||
## v0.10.10 (current)
|
|
||||||
|
|
||||||
### Done
|
|
||||||
- [x] Drive access: open, identify, unlock, firmware upload, speed calibration, eject
|
|
||||||
- [x] 206 bundled drive profiles (MediaTek MT1959 A + B variants)
|
|
||||||
- [x] SCSI transport: Linux SG_IO, macOS IOKit, Windows SPTI
|
|
||||||
- [x] UDF 2.50 filesystem parser (metadata partitions, Blu-ray profile)
|
|
||||||
- [x] MPLS playlist parser (play items, STN table, secondary streams)
|
|
||||||
- [x] CLPI clip info parser (EP map, sector extents)
|
|
||||||
- [x] AACS 1.0 decryption (4 VUK paths: KEYDB, media key, processing key, device key)
|
|
||||||
- [x] AACS 2.0 SCSI handshake (P-256/SHA-256 ECDH, bus decryption, read data key)
|
|
||||||
- [x] DVD IFO parser (VMG, VTS, PGC chains, cell addresses)
|
|
||||||
- [x] DVD CSS decryption (bus auth, disc key via player keys, title key, sector descramble)
|
|
||||||
- [x] CSS Stevenson plaintext attack for ISO key recovery
|
|
||||||
- [x] MPEG-2 PS demuxer (DVD Program Stream with PES extraction)
|
|
||||||
- [x] MPEG-2 video codec parser (sequence header, quantizer matrices, keyframe detection)
|
|
||||||
- [x] KEYDB.cfg download, verify, save (raw TCP, zero HTTP deps)
|
|
||||||
- [x] Content reading with adaptive batch sizing, error recovery, 12+ MB/s
|
|
||||||
- [x] Stream labels: 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
|
|
||||||
- [x] MKV muxer: 14 codec parsers (H.264, HEVC, MPEG-2, AC-3, DTS, TrueHD, PGS, DVD Sub, VC-1, LPCM, +4 more)
|
|
||||||
- [x] SectorReader trait: decouples disc scanning from SCSI
|
|
||||||
- [x] 7 stream types: Disc, ISO, MKV, M2TS, Network, Stdio, Null
|
|
||||||
- [x] PES pipeline with unified Stream trait (any source → any dest)
|
|
||||||
- [x] FMKV metadata header for M2TS and network streams
|
|
||||||
- [x] Numeric error codes only (no English text in library)
|
|
||||||
- [x] Event system for progress callbacks
|
|
||||||
|
|
||||||
### Planned
|
|
||||||
- [ ] Windows testing on real hardware
|
|
||||||
- [ ] Pioneer Renesas platform support (48 drives, need GET_CONFIG 010C)
|
|
||||||
- [ ] TranscodeStream (ffmpeg integration)
|
|
||||||
- [ ] ISO write with BD-compliant UDF structure
|
|
||||||
@@ -53,27 +53,48 @@ output.finish()?;
|
|||||||
|
|
||||||
### Multi-pass recovery rip
|
### Multi-pass recovery rip
|
||||||
|
|
||||||
For damaged discs, the library offers a two-stage rip model: fast sweep with zero-fill and a ddrescue-format mapfile, then targeted retry of bad ranges. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for the full architecture.
|
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
|
||||||
|
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
|
||||||
|
loops; the multipass policy is the caller's job. See
|
||||||
|
[`docs/rip-recovery.md`](docs/rip-recovery.md) and the design notes in
|
||||||
|
`freemkv-private/memory/0_18_redesign.md`.
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
use libfreemkv::disc::{CopyOptions, PatchOptions};
|
use libfreemkv::{SweepOptions, PatchOptions};
|
||||||
|
use libfreemkv::disc::{mapfile, mapfile_path_for};
|
||||||
|
use std::path::Path;
|
||||||
|
|
||||||
// Pass 1: disc → ISO. Fast 64 KB reads, skip-forward on failure,
|
let iso = Path::new("disc.iso");
|
||||||
// zero-fill bad blocks, write a sidecar .mapfile.
|
|
||||||
let mut result = disc.copy(
|
|
||||||
&mut drive,
|
|
||||||
Path::new("disc.iso"),
|
|
||||||
&CopyOptions { skip_on_error: true, skip_forward: true, ..Default::default() },
|
|
||||||
)?;
|
|
||||||
|
|
||||||
// Pass 2..N: retry bad ranges with full drive recovery.
|
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
|
||||||
// Idempotent — call as many times as you want.
|
disc.sweep(&mut drive, iso, &SweepOptions {
|
||||||
while result.bytes_unreadable + result.bytes_pending > 0 {
|
decrypt: true,
|
||||||
let pr = disc.patch(&mut drive, Path::new("disc.iso"), &PatchOptions::default())?;
|
resume: false,
|
||||||
if pr.bytes_recovered_this_pass == 0 { break; }
|
batch_sectors: None,
|
||||||
|
skip_on_error: true,
|
||||||
|
progress: None,
|
||||||
|
halt: None,
|
||||||
|
})?;
|
||||||
|
|
||||||
|
// Pass 2..N: retry every non-finished range. Idempotent.
|
||||||
|
loop {
|
||||||
|
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
|
||||||
|
let stats = map.stats();
|
||||||
|
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
|
||||||
|
|
||||||
|
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
|
||||||
|
decrypt: true,
|
||||||
|
block_sectors: None,
|
||||||
|
full_recovery: true,
|
||||||
|
reverse: true,
|
||||||
|
wedged_threshold: 50,
|
||||||
|
progress: None,
|
||||||
|
halt: None,
|
||||||
|
})?;
|
||||||
|
if outcome.bytes_recovered_this_pass == 0 { break; }
|
||||||
}
|
}
|
||||||
|
|
||||||
// Then mux from the ISO via the normal stream pipeline (no drive involvement).
|
// Mux from the ISO via the normal stream pipeline (no drive involvement).
|
||||||
```
|
```
|
||||||
|
|
||||||
## What It Does
|
## What It Does
|
||||||
@@ -92,14 +113,14 @@ while result.bytes_unreadable + result.bytes_pending > 0 {
|
|||||||
| Stream | Input | Output | Transport |
|
| Stream | Input | Output | Transport |
|
||||||
|--------|-------|--------|-----------|
|
|--------|-------|--------|-----------|
|
||||||
| DiscStream | Yes | -- | Optical drive via SCSI |
|
| DiscStream | Yes | -- | Optical drive via SCSI |
|
||||||
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::copy()`) |
|
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
|
||||||
| MkvStream | Yes | Yes | Matroska container |
|
| MkvStream | Yes | Yes | Matroska container |
|
||||||
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
|
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
|
||||||
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
|
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
|
||||||
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
|
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
|
||||||
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
|
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
|
||||||
|
|
||||||
Streams implement `pes::Stream` (frame-level). `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
|
Streams implement `FrameSource` (read) and/or `FrameSink` (write); direction is type-checked. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
|
||||||
|
|
||||||
AACS decryption requires a KEYDB.cfg file. If available at `~/.config/aacs/KEYDB.cfg` or passed via `ScanOptions`, the library handles everything — handshake, key derivation, and per-sector decryption — without the application needing to know anything about encryption.
|
AACS decryption requires a KEYDB.cfg file. If available at `~/.config/aacs/KEYDB.cfg` or passed via `ScanOptions`, the library handles everything — handshake, key derivation, and per-sector decryption — without the application needing to know anything about encryption.
|
||||||
|
|
||||||
@@ -122,7 +143,8 @@ Disc — scan titles, streams, AACS/CSS state
|
|||||||
└── KEYDB — download + verify + save
|
└── KEYDB — download + verify + save
|
||||||
|
|
||||||
Streams — unified PES pipeline
|
Streams — unified PES pipeline
|
||||||
├── pes::Stream — read()/write() PES frames
|
├── FrameSource — read() PES frames (direction-typed)
|
||||||
|
├── FrameSink — write() PES frames (direction-typed)
|
||||||
├── DiscStream — sectors → decrypt → TS demux → PES
|
├── DiscStream — sectors → decrypt → TS demux → PES
|
||||||
├── IsoStream — ISO file → decrypt → TS demux → PES
|
├── IsoStream — ISO file → decrypt → TS demux → PES
|
||||||
├── MkvStream — MKV mux/demux
|
├── MkvStream — MKV mux/demux
|
||||||
|
|||||||
+39
-37
@@ -44,35 +44,30 @@ while let Ok(Some(frame)) = input.read() {
|
|||||||
output.finish()?;
|
output.finish()?;
|
||||||
```
|
```
|
||||||
|
|
||||||
The `pes::Stream` trait:
|
The `FrameSource` and `FrameSink` traits — direction is type-checked, so
|
||||||
|
calling `read()` on a write-only sink (or `write()` on a read-only source)
|
||||||
|
is a compile error rather than a runtime fault:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
pub trait Stream {
|
pub trait FrameSource: Send {
|
||||||
fn read(&mut self) -> io::Result<Option<PesFrame>>;
|
fn read(&mut self) -> Result<Option<PesFrame>, Error>;
|
||||||
fn write(&mut self, frame: &PesFrame) -> io::Result<()>;
|
|
||||||
fn finish(&mut self) -> io::Result<()>;
|
|
||||||
fn info(&self) -> &DiscTitle;
|
fn info(&self) -> &DiscTitle;
|
||||||
fn codec_private(&self, track: usize) -> Option<Vec<u8>>;
|
fn codec_private(&self, track: usize) -> Option<Vec<u8>> { None }
|
||||||
fn headers_ready(&self) -> bool;
|
fn headers_ready(&self) -> bool { true }
|
||||||
}
|
}
|
||||||
```
|
|
||||||
|
|
||||||
## IOStream (byte-level API)
|
pub trait FrameSink: Send {
|
||||||
|
fn write(&mut self, frame: &PesFrame) -> Result<(), Error>;
|
||||||
For raw byte copies (disc→ISO, resume, benchmarks). Lower level than PES.
|
fn finish(self: Box<Self>) -> Result<(), Error>;
|
||||||
|
fn info(&self) -> &DiscTitle;
|
||||||
```rust
|
}
|
||||||
let opts = InputOptions::default();
|
|
||||||
let mut input = open_input("iso://Disc.iso", &opts)?;
|
|
||||||
let mut output = open_output("mkv://Movie.mkv", input.info())?;
|
|
||||||
io::copy(&mut *input, &mut *output)?;
|
|
||||||
output.finish()?;
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Streams
|
## Streams
|
||||||
|
|
||||||
All streams implement `IOStream` (byte-level) and/or `pes::Stream` (frame-level).
|
All streams implement `FrameSource` (read) and/or `FrameSink` (write); the
|
||||||
URL-based resolvers open any stream by string.
|
directional split prevents runtime "wrong-direction" errors. URL-based
|
||||||
|
resolvers open any stream by string.
|
||||||
|
|
||||||
| Stream | Input | Output | URL | Transport |
|
| Stream | Input | Output | URL | Transport |
|
||||||
|--------|-------|--------|-----|-----------|
|
|--------|-------|--------|-----|-----------|
|
||||||
@@ -87,21 +82,14 @@ URL-based resolvers open any stream by string.
|
|||||||
All URLs require a `scheme://path` format. Bare paths are rejected.
|
All URLs require a `scheme://path` format. Bare paths are rejected.
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
// PES pipeline (frame-level)
|
// PES pipeline (frame-level) — input() returns Box<dyn FrameSource>,
|
||||||
|
// output() returns Box<dyn FrameSink>.
|
||||||
let input = libfreemkv::input("disc:///dev/sg4", &opts)?; // DiscStream
|
let input = libfreemkv::input("disc:///dev/sg4", &opts)?; // DiscStream
|
||||||
let input = libfreemkv::input("iso://Dune.iso", &opts)?; // IsoStream
|
let input = libfreemkv::input("iso://Dune.iso", &opts)?; // IsoStream
|
||||||
let output = libfreemkv::output("mkv://Dune.mkv", &title)?; // MkvOutputStream
|
let output = libfreemkv::output("mkv://Dune.mkv", &title)?; // MkvOutputStream
|
||||||
let output = libfreemkv::output("m2ts://Dune.m2ts", &title)?; // M2tsOutputStream
|
let output = libfreemkv::output("m2ts://Dune.m2ts", &title)?; // M2tsOutputStream
|
||||||
let output = libfreemkv::output("network://10.1.7.11:9000", &title)?; // NetworkOutputStream
|
let output = libfreemkv::output("network://10.1.7.11:9000", &title)?; // NetworkOutputStream
|
||||||
let output = libfreemkv::output("null://", &title)?; // NullOutputStream
|
let output = libfreemkv::output("null://", &title)?; // NullOutputStream
|
||||||
|
|
||||||
// IOStream (byte-level)
|
|
||||||
let input = open_input("disc://", &opts)?; // DiscStream
|
|
||||||
let input = open_input("iso://Dune.iso", &opts)?; // IsoStream
|
|
||||||
let output = open_output("iso://Copy.iso", &meta)?; // IsoStream (write)
|
|
||||||
let output = open_output("mkv://Dune.mkv", &meta)?; // MkvStream
|
|
||||||
let output = open_output("m2ts://Dune.m2ts", &meta)?; // M2tsStream
|
|
||||||
let output = open_output("null://", &meta)?; // NullStream
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### FMKV Metadata Header
|
### FMKV Metadata Header
|
||||||
@@ -175,20 +163,34 @@ libfreemkv/src/
|
|||||||
├── lib.rs Public exports
|
├── lib.rs Public exports
|
||||||
├── error.rs Error codes (no English)
|
├── error.rs Error codes (no English)
|
||||||
├── event.rs Event types for callbacks
|
├── event.rs Event types for callbacks
|
||||||
|
├── halt.rs Halt cancellation token (Arc<AtomicBool> wrapper)
|
||||||
|
├── io/ Pipeline + WritebackFile primitives
|
||||||
|
│ ├── mod.rs Re-exports WritebackFile, Pipeline, Sink, Flow
|
||||||
|
│ ├── pipeline.rs Generic Pipeline<I, R> + Sink trait
|
||||||
|
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
|
||||||
|
│ └── writeback.rs sync_file_range pipeline
|
||||||
├── drive/ Drive (open, init, single-shot read)
|
├── drive/ Drive (open, init, single-shot read)
|
||||||
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
|
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
|
||||||
│ ├── capture.rs Drive profile capture for contribution
|
│ ├── capture.rs Drive profile capture for contribution
|
||||||
│ ├── linux.rs Linux drive discovery
|
│ ├── linux.rs Linux drive discovery
|
||||||
│ ├── macos.rs macOS drive discovery
|
│ ├── macos.rs macOS drive discovery
|
||||||
│ └── windows.rs Windows drive discovery
|
│ └── windows.rs Windows drive discovery
|
||||||
├── disc/ Disc (scan, titles, AACS setup)
|
├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
|
||||||
|
│ ├── mod.rs Disc struct, scan, titles, formats
|
||||||
|
│ ├── sweep.rs Disc::sweep (Pass 1 forward sweep)
|
||||||
|
│ ├── patch.rs Disc::patch (Pass N retry over mapfile)
|
||||||
|
│ ├── mapfile.rs ddrescue-format mapfile
|
||||||
|
│ └── read_error.rs ReadCtx / ReadAction state machine
|
||||||
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
|
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
|
||||||
├── platform/ Drive unlock (MT1959 A/B)
|
├── platform/ Drive unlock (MT1959 A/B)
|
||||||
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
|
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
|
||||||
├── css/ DVD CSS cipher
|
├── css/ DVD CSS cipher
|
||||||
├── decrypt.rs Unified decrypt dispatcher (AACS/CSS/None)
|
├── decrypt.rs Unified decrypt dispatcher (AACS/CSS/None)
|
||||||
├── pes.rs PES frame types, Stream trait
|
├── pes.rs PES frame types, FrameSource / FrameSink traits
|
||||||
├── sector.rs SectorReader trait
|
├── sector/ Sector I/O (was sector.rs in 0.17)
|
||||||
|
│ ├── mod.rs SectorSource, SectorSink traits
|
||||||
|
│ ├── file.rs FileSectorSource, FileSectorSink (ISO-backed)
|
||||||
|
│ └── decrypting.rs DecryptingSectorSource decorator
|
||||||
├── udf.rs UDF 2.50 filesystem parser
|
├── udf.rs UDF 2.50 filesystem parser
|
||||||
├── mpls.rs MPLS playlist parser
|
├── mpls.rs MPLS playlist parser
|
||||||
├── clpi.rs CLPI clip info parser
|
├── clpi.rs CLPI clip info parser
|
||||||
@@ -199,15 +201,15 @@ libfreemkv/src/
|
|||||||
├── profile.rs Bundled drive profiles
|
├── profile.rs Bundled drive profiles
|
||||||
├── speed.rs DriveSpeed enum
|
├── speed.rs DriveSpeed enum
|
||||||
├── mux/
|
├── mux/
|
||||||
│ ├── mod.rs IOStream trait, public exports
|
│ ├── mod.rs Public mux exports
|
||||||
│ ├── resolve.rs URL parser + open_input/open_output + input/output
|
│ ├── resolve.rs URL parser + input/output (Box<dyn FrameSource/Sink>)
|
||||||
│ ├── meta.rs FMKV header format
|
│ ├── meta.rs FMKV header format
|
||||||
│ ├── disc.rs DiscStream (optical drive → PES)
|
│ ├── disc.rs DiscStream (optical drive → PES)
|
||||||
│ ├── iso.rs IsoStream (ISO image read/write)
|
│ ├── iso.rs IsoStream (ISO image read)
|
||||||
│ ├── isowriter.rs ISO image writer (UDF, AVDP, multi-extent)
|
│ ├── isowriter.rs ISO image writer (UDF, AVDP, multi-extent)
|
||||||
│ ├── mkvstream.rs MkvStream (bidirectional Matroska, IOStream)
|
│ ├── mkvstream.rs MkvStream (bidirectional Matroska)
|
||||||
│ ├── mkvout.rs MkvOutputStream (PES → MKV)
|
│ ├── mkvout.rs MkvOutputStream (PES → MKV)
|
||||||
│ ├── m2ts.rs M2tsStream (BD-TS, IOStream)
|
│ ├── m2ts.rs M2tsStream (BD-TS)
|
||||||
│ ├── pesout.rs PES output streams (M2ts, Network, Stdio, Null)
|
│ ├── pesout.rs PES output streams (M2ts, Network, Stdio, Null)
|
||||||
│ ├── network.rs NetworkStream (TCP + FMKV header)
|
│ ├── network.rs NetworkStream (TCP + FMKV header)
|
||||||
│ ├── stdio.rs StdioStream (stdin/stdout pipe)
|
│ ├── stdio.rs StdioStream (stdin/stdout pipe)
|
||||||
|
|||||||
@@ -65,8 +65,12 @@ libfreemkv (lib.rs)
|
|||||||
│
|
│
|
||||||
├── Streaming
|
├── Streaming
|
||||||
│ ├── mux/ Stream implementations (Disc, ISO, MKV, M2TS, Network, Stdio, Null)
|
│ ├── mux/ Stream implementations (Disc, ISO, MKV, M2TS, Network, Stdio, Null)
|
||||||
│ ├── pes PES frame types, Stream trait (read/write frames)
|
│ ├── pes PES frame types; FrameSource / FrameSink direction-typed traits
|
||||||
│ └── sector SectorReader trait — abstracts disc vs ISO vs file
|
│ └── sector/ SectorSource / SectorSink traits, FileSector{Source,Sink}, DecryptingSectorSource
|
||||||
|
│
|
||||||
|
├── I/O Primitives
|
||||||
|
│ ├── halt Halt cancellation token (one Arc<AtomicBool>, cloneable)
|
||||||
|
│ └── io/ Pipeline<I, R> + Sink trait + WritebackFile (bounded-cache writer)
|
||||||
│
|
│
|
||||||
├── Support
|
├── Support
|
||||||
│ ├── keydb KEYDB.cfg download, parse, verify, save
|
│ ├── keydb KEYDB.cfg download, parse, verify, save
|
||||||
|
|||||||
+8
-4
@@ -97,7 +97,9 @@ drive.probe_disc()?;
|
|||||||
// Scan disc (UDF + playlists + AACS — all automatic)
|
// Scan disc (UDF + playlists + AACS — all automatic)
|
||||||
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
|
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
|
||||||
|
|
||||||
// Stream pipeline — PES frames from any source to any output
|
// Stream pipeline — PES frames from any source to any output.
|
||||||
|
// 0.18: input() returns Box<dyn FrameSource>, output() returns Box<dyn FrameSink>;
|
||||||
|
// direction is type-checked, so calling .write() on an input is a compile error.
|
||||||
let opts = InputOptions::default();
|
let opts = InputOptions::default();
|
||||||
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
|
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
|
||||||
let title = input.info().clone();
|
let title = input.info().clone();
|
||||||
@@ -121,11 +123,13 @@ output.finish()?;
|
|||||||
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
|
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
|
||||||
| css/ | -- | DVD CSS cipher |
|
| css/ | -- | DVD CSS cipher |
|
||||||
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
|
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
|
||||||
| disc/ | -- | High-level scan + read API |
|
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
|
||||||
| labels/ | -- | BD-J stream labels (5 format parsers) |
|
| labels/ | -- | BD-J stream labels (5 format parsers) |
|
||||||
| mux/ | -- | Stream implementations (7 stream types) |
|
| mux/ | -- | Stream implementations (7 stream types) |
|
||||||
| pes.rs | -- | PES frame types + Stream trait |
|
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
|
||||||
| sector.rs | -- | SectorReader trait |
|
| sector/ | -- | SectorSource / SectorSink + DecryptingSectorSource decorator |
|
||||||
|
| io/ | -- | Pipeline<I, R> + Sink trait + WritebackFile |
|
||||||
|
| halt.rs | -- | Halt cancellation token |
|
||||||
| keydb.rs | -- | KEYDB download, parse, save |
|
| keydb.rs | -- | KEYDB download, parse, save |
|
||||||
| error.rs | -- | Error codes (E1xxx-E8xxx) |
|
| error.rs | -- | Error codes (E1xxx-E8xxx) |
|
||||||
| event.rs | -- | Drive event system |
|
| event.rs | -- | Drive event system |
|
||||||
|
|||||||
@@ -52,8 +52,8 @@ selects the per-CDB timeout:
|
|||||||
|
|
||||||
| `recovery` | Timeout | Used by |
|
| `recovery` | Timeout | Used by |
|
||||||
|------------|----------|------------------------------------------|
|
|------------|----------|------------------------------------------|
|
||||||
| `false` | 1.5 s | `Disc::copy` fast skip-forward sweep, `DiscStream::fill_extents` |
|
| `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
|
||||||
| `true` | 30 s | `Disc::patch` multi-pass over the mapfile |
|
| `true` | 30 s | `Disc::patch` retry pass over the mapfile |
|
||||||
|
|
||||||
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
|
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
|
||||||
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
|
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
|
||||||
|
|||||||
+43
-22
@@ -12,21 +12,24 @@ reach into the others.
|
|||||||
|
|
||||||
| Layer | Where it lives | What it does |
|
| Layer | Where it lives | What it does |
|
||||||
|-------|---------------|--------------|
|
|-------|---------------|--------------|
|
||||||
| 1 — Bad-range retry | `Disc::patch` (multi-pass over the mapfile) | Re-reads non-`+` ranges with the long timeout. Idempotent; call N times. |
|
| 1 — Bad-range retry | `Disc::patch` (one pass over the mapfile per call) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
|
||||||
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
|
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
|
||||||
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
|
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
|
||||||
|
|
||||||
The caller orchestrates layer 1. Autorip's `rip_disc` loops `copy` then
|
The library exposes flat verbs; the caller drives the multipass loop. Autorip
|
||||||
N × `patch` per the `MAX_RETRIES` config, then hands the ISO off to the
|
runs `Disc::sweep` once, then loops `Disc::patch` until either the mapfile is
|
||||||
existing mux pipeline. Layer 3 runs inside any consumer of `DiscStream`
|
clean or the configured retry budget is exhausted, then hands the ISO off to
|
||||||
(direct PES pipeline, ISO playback, etc.) without caller involvement.
|
the mux pipeline. The `freemkv` CLI does the same shape with a
|
||||||
|
terminal-output progress sink. Layer 3 runs inside any consumer of
|
||||||
|
`DiscStream` (direct PES pipeline, ISO playback, etc.) without caller
|
||||||
|
involvement.
|
||||||
|
|
||||||
Three primitives compose the disc-side flow:
|
Three primitives compose the disc-side flow:
|
||||||
|
|
||||||
| Primitive | What it does |
|
| Primitive | What it does |
|
||||||
|---------------------------|-----------------------------------------------------------------------|
|
|---------------------------|-----------------------------------------------------------------------|
|
||||||
| `Disc::copy` | disc → ISO. Writes a sidecar `.mapfile`. Opt-in skip-forward on failure. |
|
| `Disc::sweep` | disc → ISO, one forward pass. Writes a sidecar `.mapfile`. Opt-in skip-on-error. |
|
||||||
| `Disc::patch` | Re-reads bad ranges from the drive. Idempotent; call N times. |
|
| `Disc::patch` | Re-reads bad ranges from the drive. One pass per call; caller invokes N times. |
|
||||||
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
|
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
|
||||||
|
|
||||||
## Data model
|
## Data model
|
||||||
@@ -60,28 +63,46 @@ Status characters match ddrescue:
|
|||||||
|
|
||||||
Position and size are hex byte offsets into the ISO.
|
Position and size are hex byte offsets into the ISO.
|
||||||
|
|
||||||
### `CopyOptions` and `PatchOptions`
|
### `SweepOptions` and `PatchOptions`
|
||||||
|
|
||||||
`Disc::copy()` auto-detects the pass from mapfile state:
|
The library no longer dispatches between sweep and patch internally — the
|
||||||
|
caller picks the verb explicitly per pass. The two option structs are flat
|
||||||
|
and have no overlap:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
CopyOptions {
|
SweepOptions {
|
||||||
decrypt: true, // decrypt AACS/CSS sectors
|
decrypt: true,
|
||||||
multipass: true, // enable mapfile + skip-on-error + damage-jump
|
resume: false,
|
||||||
progress: Some(&reporter), // progress callback
|
batch_sectors: None,
|
||||||
halt: Some(flag), // halt flag for graceful stop
|
skip_on_error: true, // damage-jump + zero-fill on read failure
|
||||||
|
progress: Some(&reporter),
|
||||||
|
halt: Some(flag),
|
||||||
|
}
|
||||||
|
|
||||||
|
PatchOptions {
|
||||||
|
decrypt: true,
|
||||||
|
block_sectors: None,
|
||||||
|
full_recovery: true,
|
||||||
|
reverse: true, // walk bad ranges high → low LBA
|
||||||
|
wedged_threshold: 50,
|
||||||
|
progress: Some(&reporter),
|
||||||
|
halt: Some(flag),
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Dispatch logic:
|
Caller-orchestrated dispatch (the policy `Disc::copy` used to embed):
|
||||||
- No mapfile → sweep (fresh Pass 1)
|
|
||||||
- Mapfile with NonTried (`?`) → sweep with resume
|
- No mapfile → `sweep` (fresh Pass 1).
|
||||||
- Mapfile covering full disc, only NonTrimmed/NonScraped/Unreadable → patch
|
- Mapfile with `?` ranges → `sweep` with `resume: true`.
|
||||||
- Mapfile clean → no-op
|
- Mapfile covers full disc, only `*` / `/` / `-` ranges → `patch`.
|
||||||
|
- Mapfile clean → done; no further pass needed.
|
||||||
|
|
||||||
|
Each consumer (autorip, `freemkv` CLI) implements the loop in roughly five
|
||||||
|
lines of `Mapfile::stats()` checks.
|
||||||
|
|
||||||
## Algorithm
|
## Algorithm
|
||||||
|
|
||||||
### Pass 1 — fast sweep (`Disc::copy` → `sweep_internal`)
|
### Pass 1 — fast sweep (`Disc::sweep`)
|
||||||
|
|
||||||
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
|
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
|
||||||
2. On success: write data to ISO, mark `+`, advance.
|
2. On success: write data to ISO, mark `+`, advance.
|
||||||
@@ -95,7 +116,7 @@ Dispatch logic:
|
|||||||
|
|
||||||
Pass 1 completes when every byte has been visited (either `+` or `*`).
|
Pass 1 completes when every byte has been visited (either `+` or `*`).
|
||||||
|
|
||||||
### Pass 2+ — patch (`Disc::copy` → `patch_internal`)
|
### Pass 2+ — patch (`Disc::patch`)
|
||||||
|
|
||||||
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
|
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
|
||||||
(walks ranges from highest LBA to lowest, within each range from end to start).
|
(walks ranges from highest LBA to lowest, within each range from end to start).
|
||||||
@@ -177,4 +198,4 @@ scrape vs. retry with direction reversal) if there's measured benefit.
|
|||||||
|
|
||||||
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
|
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
|
||||||
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
|
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
|
||||||
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/mod.rs`](../src/disc/mod.rs) (`Disc::copy`, `Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
|
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/sweep.rs`](../src/disc/sweep.rs) (`Disc::sweep`), [`src/disc/patch.rs`](../src/disc/patch.rs) (`Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
|
||||||
|
|||||||
+1
-1
@@ -126,7 +126,7 @@ USB optical drives have ~500ms round-trip latency per SCSI command. Since `read_
|
|||||||
|
|
||||||
`Disc::scan()` wraps the drive in a `BufferedSectorReader` before reading. On a single-sector read, the buffer prefetches a batch of sectors (sized from the kernel's `max_hw_sectors_kb` for the device) and caches them. Subsequent reads to nearby LBAs return from cache with zero SCSI overhead. After parsing the UDF directory structure, the entire metadata partition is pre-read into the cache, so all ICB lookups during title scanning and encryption resolution are instant.
|
`Disc::scan()` wraps the drive in a `BufferedSectorReader` before reading. On a single-sector read, the buffer prefetches a batch of sectors (sized from the kernel's `max_hw_sectors_kb` for the device) and caches them. Subsequent reads to nearby LBAs return from cache with zero SCSI overhead. After parsing the UDF directory structure, the entire metadata partition is pre-read into the cache, so all ICB lookups during title scanning and encryption resolution are instant.
|
||||||
|
|
||||||
The buffer is transparent -- `read_filesystem()`, `read_file()`, and all downstream code still call `read_sectors(lba, 1, buf)` as before. The batching happens inside the `SectorReader` implementation.
|
The buffer is transparent -- `read_filesystem()`, `read_file()`, and all downstream code still call `read_sectors(lba, 1, buf)` as before. The batching happens inside the `SectorSource` implementation.
|
||||||
|
|
||||||
### UDF Filename Encoding
|
### UDF Filename Encoding
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user