pq/libfreemkv
Watch
1
Fork
You've already forked libfreemkv
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Compare commits

base: pq:v0.26.1
Branches Tags
pq:dev
pq:main
pq:qa
..
compare: pq:v0.8.2
Branches Tags
pq:dev
pq:main
pq:qa
161 changed files with 6895 additions and 41119 deletions
Download patch file Download diff file Expand all files Collapse all files

22
.github/workflows/ci.yml vendored
View file

@ -6,37 +6,23 @@ on:
pull_request:
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
with:
components: clippy, rustfmt
- run: cargo fmt --check
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
- run: cargo clippy -- -D warnings
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- run: cargo test --tests
- uses: dtolnay/rust-toolchain@stable
- run: cargo test
check-macos:
runs-on: macos-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: dtolnay/rust-toolchain@stable
- run: cargo check
check-windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: dtolnay/rust-toolchain@stable
- run: cargo check

6
.github/workflows/release.yml vendored
View file

@ -27,9 +27,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
- uses: dtolnay/rust-toolchain@stable
- run: cargo test
publish:
@ -37,7 +35,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: dtolnay/rust-toolchain@stable
- name: Publish to crates.io
run: cargo publish
env:

1
.gitignore vendored
View file

@ -4,4 +4,3 @@ Cargo.lock
*.swo
.DS_Store
.cargo/
.claude/worktrees/

2221
CHANGELOG.md
View file

File diff suppressed because it is too large Load diff

33
Cargo.toml
View file

@ -1,8 +1,7 @@
[package]
name = "libfreemkv"
version = "0.26.1"
edition = "2024"
rust-version = "1.86"
version = "0.8.2"
edition = "2021"
license = "AGPL-3.0-only"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
@ -24,27 +23,6 @@ rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# under project docs ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
# Bounded MPSC channel with kernel-wakeup send_timeout. Used by `io::pipeline`
# so the halt-aware send/finish loops can BLOCK on consumer drain instead of
# polling — the 50 ms poll cadence of the previous mpsc-based impl capped mux
# throughput at ~1 MB/s (see (internal)/memory/
# feedback_send_with_halt_poll_throttle.md, 0.21.7).
crossbeam-channel = "0.5"
# Persistent work-stealing thread pool for parallel AACS unit
# decryption. Per-call std::thread::scope spawned fresh OS threads
# and that overhead dominated for typical batch sizes (60 units).
# rayon's global pool initialises once on first use.
rayon = "1"
# SIMD-accelerated bytestring search. Drives the HEVC/H.264 start-code
# scan in `mux::codec::h264::find_start_code` — naive byte-by-byte
# walk is ~500 MB/s single-thread on x86_64; memchr's vectorised
# `memmem::find` for the 3-byte `00 00 01` needle hits ~5 GB/s on
# AVX2-capable hosts.
memchr = "2"
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
@ -52,10 +30,3 @@ libc = "0.2"
[target.'cfg(target_os = "macos")'.dependencies]
libc = "0.2"
[dev-dependencies]
tempfile = "3"
[[bench]]
name = "sgio_read"
harness = false

30
FEATURES.md Normal file
View file

@ -0,0 +1,30 @@
# libfreemkv — Feature List
## v0.7.1 (current)
### Done
- [x] Drive access: open, identify, unlock, firmware upload, speed calibration, eject
- [x] 206 bundled drive profiles (MediaTek MT1959 A + B variants)
- [x] SCSI transport: Linux SG_IO, macOS IOKit, Windows SPTI
- [x] UDF 2.50 filesystem parser (metadata partitions, Blu-ray profile)
- [x] MPLS playlist parser (play items, STN table, secondary streams)
- [x] CLPI clip info parser (EP map, sector extents)
- [x] AACS 1.0 decryption (4 VUK paths: KEYDB, media key, processing key, device key)
- [x] AACS 2.0 SCSI handshake (P-256/SHA-256 ECDH, bus decryption, read data key)
- [x] KEYDB.cfg download, verify, save (raw TCP, zero HTTP deps)
- [x] Content reading with adaptive batch sizing, error recovery, 12+ MB/s
- [x] Stream labels: 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
- [x] MKV muxer: 8 codec parsers (H.264, HEVC, AC-3, DTS, TrueHD, PGS, VC-1, LPCM)
- [x] SectorReader trait: decouples disc scanning from SCSI
- [x] 7 stream types: Disc, ISO, MKV, M2TS, Network, Stdio, Null
- [x] IOStream trait with URL-based resolver (scheme://path)
- [x] FMKV metadata header for M2TS and network streams
- [x] Numeric error codes only (no English text in library)
- [x] Event system for progress callbacks
### Planned
- [ ] Windows testing on real hardware
- [ ] Pioneer Renesas platform support (48 drives, need GET_CONFIG 010C)
- [ ] DVD CSS decryption
- [ ] TranscodeStream (ffmpeg integration)
- [ ] ISO write with BD-compliant UDF structure

20
Makefile
View file

@ -1,20 +0,0 @@
# libfreemkv — local dev helper.
# Mirrors the cross-crate scripts in (internal)/scripts/test-all.sh
# but scoped to this single crate.
.PHONY: test build check ci clean
test:
cargo test --tests
build:
cargo build --release
check:
cargo fmt --check
cargo clippy --all-targets -- -D warnings
ci: check build test
clean:
cargo clean

108
README.md
View file

@ -4,9 +4,7 @@
# libfreemkv
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Bundled drive profiles — no external files needed.
Built-in keys cover DVDs and Blu-rays (AACS 1.0). For UHD (AACS 2.0 / 2.1) discs, an optional `keydb.cfg` supplies disc-specific volume unique keys.
Rust library for 4K UHD / Blu-ray optical drives. Drive access, disc scanning, stream labels, AACS decryption, KEYDB updates, and content reading in one crate. Bundled drive profiles — no external files needed.
**12+ MB/s** sustained read speeds on BD. Full init: unlock, firmware upload, speed calibration — all from pure Rust.
@ -20,82 +18,33 @@ Part of the [freemkv](https://github.com/freemkv) project.
```toml
[dependencies]
libfreemkv = "0.25"
libfreemkv = "0.6"
```
## Quick Start
```rust
use libfreemkv::{Drive, Disc, ScanOptions};
use libfreemkv::{DriveSession, Disc, ScanOptions};
use std::path::Path;
// Open drive — profiles are bundled, auto-identified
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?; // wait for disc
drive.init()?; // unlock + firmware upload
drive.probe_disc()?; // probe disc surface for optimal speeds
let mut session = DriveSession::open(Path::new("/dev/sr0"))?;
session.wait_ready()?; // wait for disc
session.init()?; // unlock + firmware upload
session.probe_disc()?; // probe disc surface for optimal speeds
// Scan disc — UDF, playlists, streams, AACS (all automatic)
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
let disc = Disc::scan(&mut session, &ScanOptions::default())?;
for title in &disc.titles {
println!("{} — {} streams", title.duration_display(), title.streams.len());
}
// Stream pipeline — read PES frames from any source, write to any output
let opts = libfreemkv::InputOptions::default();
let mut input = libfreemkv::input("iso://Disc.iso", &opts)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
output.write(&frame)?;
// Read content (decrypted transparently if AACS keys available)
let mut reader = disc.open_title(&mut session, 0)?;
while let Some(unit) = reader.read_unit()? {
// 6144 bytes of content per aligned unit
}
output.finish()?;
```
### Multi-pass recovery rip
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md).
```rust
use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
## What It Does
@ -106,7 +55,7 @@ loop {
- **Stream labels** — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
- **AACS decryption** — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
- **KEYDB updates** — download, verify, save from any HTTP URL (zero deps, raw TCP)
- **Content reading** — adaptive batch reads with automatic decryption
- **Content reading** — adaptive batch reads with automatic decryption and error recovery
- **Stream I/O** — unified stream pipeline for reading and writing any format
### Streams
@ -114,49 +63,32 @@ loop {
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| IsoStream | Yes | -- | Blu-ray ISO image file |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
Streams implement `FrameSource` (read) and/or `FrameSink` (write); direction is type-checked. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
All streams implement the `IOStream` trait. `open_input()` and `open_output()` resolve URL strings to stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
### Keys
DVDs (CSS) decrypt out of the box — the 1999-era public player keys are compiled into the library.
Blu-rays and UHD (AACS) require a `keydb.cfg` at `~/.config/freemkv/keydb.cfg` (or passed via `ScanOptions`). The file holds all DKs, PKs, host certs, and per-disc VUKs. No AACS key material is compiled into the binary.
AACS decryption requires a KEYDB.cfg file. If available at `~/.config/aacs/KEYDB.cfg` or passed via `ScanOptions`, the library handles everything — handshake, key derivation, and per-sector decryption — without the application needing to know anything about encryption.
## Architecture
```text
Drive — open, identify, init, unlock, single-shot read
├── ScsiTransport — SG_IO (Linux), IOKit (macOS), SPTI (Windows)
DriveSession — open any drive, identify, init (optional), read sectors
├── ScsiTransport — SG_IO (Linux), IOKit (macOS)
├── DriveProfile — per-drive unlock parameters (bundled)
└── PlatformDriver — MediaTek (supported), Renesas (planned)
Disc — scan titles, streams, AACS/CSS state
Disc — scan titles, streams, AACS state
├── UDF reader — Blu-ray UDF 2.50 with metadata partitions
├── MPLS parser — playlists → titles + clips + streams
├── CLPI parser — clip info → EP map → sector extents
├── IFO parser — DVD title sets, PGC chains, cell addresses
├── Labels — 5 BD-J format parsers (detect + parse)
├── AACS — key resolution + content decryption
├── CSS — DVD CSS cipher (table-driven, no keys needed)
└── KEYDB — download + verify + save
Streams — unified PES pipeline
├── FrameSource — read() PES frames (direction-typed)
├── FrameSink — write() PES frames (direction-typed)
├── DiscStream — sectors → decrypt → TS demux → PES
├── IsoStream — ISO file → decrypt → TS demux → PES
├── MkvStream — MKV mux/demux
├── M2tsStream — BD transport stream
├── NetworkStream — TCP with FMKV metadata header
├── StdioStream — stdin/stdout pipe
└── NullStream — discard sink
```
See [docs/](docs/) for detailed technical documentation on each module.
@ -186,7 +118,7 @@ All errors are structured with numeric codes. No user-facing English text — ap
## Contributing
Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to contribute your drive's profile.
Run `freemkv drive-info --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to contribute your drive's profile.
## License

312
TROUBLESHOOTING.md
View file

@ -1,312 +0,0 @@
# Troubleshooting Guide
Common problems and solutions for optical drive ripping with freemkv.
---
## 1. USB-SATA Bridge Issues
This is the single most common source of problems when ripping discs over USB.
### Symptoms
- The drive disappears mid-rip. The ripping tool reports the device is gone, and `ls /dev/sg*` no longer shows it.
- The device re-enumerates under a different name: `sg4` becomes `sg5`, then `sg7`, then `sg11` after each USB port reset.
- `dmesg` shows USB port resets: `usb X-Y: reset high-speed USB device`, `xhci_hcd 0000:00:14.0: Cannot enable. Maybe the USB cable is bad?`, or `usb-storage: device reset failed`.
- The SCSI layer reports `host_status=7` (Linux USB transport error) in sense data.
- The drive works fine for reading data discs or burning, but crashes when hitting damaged sectors during a rip.
- After the crash, the drive is completely invisible until physically unplugged and reconnected.
### Root Cause
USB-SATA bridges translate between the USB Mass Storage protocol (BOT or UAS) and the drive's native SATA interface. When the optical drive encounters an unreadable sector, it returns a SCSI CHECK CONDITION with sense key 0x03 (MEDIUM ERROR). Some bridge chipsets -- particularly the Initio INIC-36xx family -- have buggy firmware that mishandles this error response.
Specific failure modes:
- **Incorrect residue reporting.** The bridge claims a different number of bytes transferred than what actually occurred. The Linux USB storage driver sees this discrepancy as a protocol violation and resets the port to recover. The `US_FL_IGNORE_RESIDUE` quirk exists specifically for this class of bug (see `drivers/usb/storage/transport.c` in the Linux kernel).
- **Bridge firmware crash.** On some Initio bridges, a malformed SCSI error response from the drive causes the bridge MCU to hang entirely. The USB controller sees the device stop responding and initiates a port reset. The bridge recovers (it re-enumerates), but the rip is dead -- all state is lost.
- **Sense data corruption.** The bridge forwards garbled or truncated sense data to the host, which the SCSI midlayer cannot parse, leading to a transport reset.
This is a hardware + firmware problem, not a software bug. The same drive connected via direct SATA does not exhibit these symptoms.
### Known Affected Bridges
| Chipset | USB IDs | Notes |
|---------|---------|-------|
| Initio INIC-3609 | `13fd:3609` | Very common in cheap SATA-to-USB enclosures. Highly problematic. |
| Initio INIC-3619 | `13fd:3940` | Same firmware family as INIC-3609. |
| Initio INIC-3069 | `13fd:0840` | Older variant, same residue bug. |
| ASMedia ASM1051 | `174c:5106` | Early ASM SATA bridge. Residue issues on error paths. |
| JMicron JMB36x | `152d:0561` | Some firmware versions. Not all JMicroon chips are affected. |
If your drive came in a pre-built external enclosure (Vantec, Sabrent, OWC, etc.), it almost certainly uses one of these bridge chips internally.
### The Fix: USB Storage Quirk
Apply the `US_FL_IGNORE_RESIDUE` kernel quirk for your bridge. This tells the Linux USB storage driver to ignore the residue field in SCSI response frames, preventing the port reset on mismatched byte counts.
**Step 1: Identify your bridge's vendor:product ID.**
```bash
lsusb
```
Look for your drive's entry. Example output:
```
Bus 002 Device 005: ID 13fd:0840 Initio Corporation INIC-3609
```
Here the vendor ID is `13fd` and the product ID is `0840`.
**Step 2: Apply the quirk at runtime.**
```bash
echo "13fd:0840:i" > /sys/module/usb_storage/parameters/quirks
```
Replace `13fd:0840` with your device's actual IDs. The `:i` flag means `US_FL_IGNORE_RESIDUE`.
You can combine multiple flags. Common additions:
- `:i` -- ignore residue (`US_FL_IGNORE_RESIDUE`)
- `:u` -- force BOT mode instead of UAS, for bridges with UAS bugs
**Step 3: Reconnect the drive.** Unplug and replug the USB cable, or bind/unbind the device. The quirk is applied per-module-load, so existing sessions may need the drive reconnected.
### Making It Persistent
Add the quirk to your kernel boot parameters so it survives reboots.
Edit `/etc/default/grub` (GRUB) and add to `GRUB_CMDLINE_LINUX_DEFAULT`:
```
GRUB_CMDLINE_LINUX_DEFAULT="quiet usb_storage.quirks=13fd:0840:i"
```
Then rebuild the GRUB config:
```bash
sudo update-grub
```
For systemd-boot, add to your loader entry or `/etc/kernel/cmdline`:
```
usb_storage.quirks=13fd:0840:i
```
Multiple devices can be separated by commas:
```
usb_storage.quirks=13fd:0840:i,174c:5106:u
```
### Recommended Bridges
If you are buying a USB-SATA adapter or enclosure for optical drive use:
| Bridge | USB IDs | Notes |
|--------|---------|-------|
| ASMedia ASM1153 | `174c:1153` | Reliable. Widely available in SATA-USB 3.0 cables. |
| JMicron JMS578 | `152d:0578` | Good firmware. Supports UASP. |
| Icy Box IB-AC640-C3 | N/A | Uses a known-good bridge internally. Plug-and-play. |
Avoid any enclosure or adapter listing an Initio chipset.
### Best Solution: Direct SATA
Connect your optical drive directly to a motherboard SATA port. This eliminates the USB-SATA bridge entirely and is the most reliable configuration:
- No USB protocol overhead or translation errors.
- No bridge firmware bugs.
- No port resets or re-enumeration.
- Full SATA error recovery handled natively by the kernel's libata driver.
- Sustained read speeds are limited only by the drive, not the USB bus.
If your machine has a free SATA port, use it.
---
## 2. Damaged Disc Handling
### Symptoms
- SCSI MEDIUM ERROR (sense key 0x03) at specific LBAs. `dmesg` shows `sr X:0:0:0: [srY] Unrecoverable read error` or similar.
- Read speed drops to near zero when approaching a damaged area.
- The drive makes audible retrying noises (laser repositioning, spindle speed changes).
- On USB-connected drives: the bridge crashes (see section 1 above) when the drive returns the error.
### How freemkv Handles This
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
### The Drive Taint Issue (LG BU40N)
Some drives, notably the LG BU40N, exhibit a "taint" behavior after encountering MEDIUM ERRORs:
1. The drive hits a damaged sector and returns a MEDIUM ERROR.
2. From that point forward, **all subsequent reads fail** -- even reads to sectors that were previously successful.
3. The only recovery is to physically unplug and reconnect the drive (or power-cycle it).
This is not a freemkv bug. It is a drive firmware behavior triggered by the interaction between the drive's internal error recovery and the USB-SATA bridge's handling of the error response. The drive firmware enters a degraded state that it does not recover from without a power cycle.
Workarounds:
- **Use a direct SATA connection.** This eliminates the bridge interaction that triggers the taint.
- **Use a different bridge.** The ASM1153 and JMS578 are less likely to trigger this behavior.
- **Accept the partial ISO.** freemkv's skip-forward recovery will zero-fill the unreadable blocks and continue. The resulting ISO may be playable with minor glitches in the affected areas.
- **Physical replug between retry passes.** If running multi-pass patch, replug the drive between passes to clear the taint state.
freemkv deliberately does not attempt inline SCSI resets or eject cycles to recover from this state, because those operations were found to make the problem worse on affected hardware (see the design rationale in [`docs/rip-recovery.md`](docs/rip-recovery.md)).
---
## 3. Drive Not Detected
### Check Hardware Visibility
```bash
lsusb
```
Verify the drive appears in the USB device list. If it does not show up, the drive is not visible to the host at all -- check cables, power, and USB port.
```bash
ls /dev/sg*
```
On Linux, optical drives appear as `/dev/sg*` devices (the SCSI Generic interface). freemkv uses `/dev/sg*`, not `/dev/sr*`. If `lsusb` shows the device but no `/dev/sg*` entry exists, the `sg` kernel module may not be loaded:
```bash
sudo modprobe sg
```
### Check Kernel Messages
```bash
dmesg | grep -i usb | tail -30
dmesg | grep -i sg | tail -10
```
Look for:
- USB enumeration errors or failed port resets.
- `sg_add` messages confirming the sg device was registered.
- Permission denied or access errors.
### Permission Issues
On most Linux distributions, `/dev/sg*` devices are owned by `root:disk` or `root:cdrom` with restricted permissions. Running freemkv as an unprivileged user will fail with permission errors.
Options:
- Add your user to the appropriate group:
```bash
sudo usermod -aG disk $USER
```
Then log out and back in for the change to take effect. On some distributions the group is `cdrom` or `optical` instead of `disk`.
- Run with elevated privileges:
```bash
sudo freemkv ...
```
- Install a udev rule for persistent per-device permissions. Create `/etc/udev/rules.d/99-sg-optical.rules`:
```
SUBSYSTEM=="scsi_generic", ATTRS{type}=="5", MODE="0666"
```
Then reload udev rules:
```bash
sudo udevadm control --reload-rules && sudo udevadm trigger
```
### Spin-Up Delay
Optical drives take 30-60 seconds to spin up and become ready after hot-plug or disc insertion. During this window, SCSI commands may return NOT READY or timeout.
freemkv's `Drive::wait_ready()` handles this automatically by polling with TEST UNIT READY until the drive responds. If you are writing your own code using the library, always call `wait_ready()` before `init()`:
```rust
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?; // blocks until disc is ready, up to 30s
drive.init()?;
```
If the drive was just plugged in, wait a full minute before concluding it is not detected.
---
## 4. How to Identify Your USB-SATA Bridge
If you are experiencing the issues described in section 1, you need to know which bridge chipset your adapter or enclosure uses.
### Step 1: Find the Device
```bash
lsusb
```
Look for entries matching your drive or enclosure. Bridges may appear under their own manufacturer name or as a generic SATA device. Common examples:
```
Bus 002 Device 005: ID 13fd:0840 Initio Corporation
Bus 002 Device 006: ID 174c:1153 ASMedia Technology Inc. ASM1153
Bus 002 Device 007: ID 152d:0578 JMicron Technology Corp. JMS578
```
### Step 2: Match the IDs
| Vendor | Product ID | Chipset | Status |
|--------|-----------|---------|--------|
| `13fd` | `3609` | Initio INIC-3609 | Affected. Apply quirk. |
| `13fd` | `3940` | Initio INIC-3619 | Affected. Apply quirk. |
| `13fd` | `0840` | Initio INIC-3069 | Affected. Apply quirk. |
| `174c` | `5106` | ASMedia ASM1051 | Affected (early firmware). Apply quirk. |
| `174c` | `1153` | ASMedia ASM1153 | Good. No quirk needed. |
| `152d` | `0561` | JMicron JMB36x | Affected (some firmware). Apply quirk if issues occur. |
| `152d` | `0578` | JMicron JMS578 | Good. No quirk needed. |
### Step 3: Check dmesg for the Bridge Name
```bash
dmesg | grep -i "usb-storage\|uas\|initio\|asmedia\|jmicron"
```
This often reveals the bridge chipset even when `lsusb` shows a generic name.
### Step 4: If the Enclosure Is Sealed
Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not advertise the bridge chipset on the packaging. In this case:
1. Check `lsusb` while the enclosure is connected.
2. Search the vendor:product ID online -- there are community-maintained lists of which chipsets popular enclosures use.
3. If you cannot determine the chipset and are experiencing bridge crashes, assume it is an Initio and apply the quirk with its IDs.
4. The definitive test: connect the bare drive to a motherboard SATA port. If the problems disappear, the bridge was the cause.
---
## 5. General Debugging Checklist
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
6. **SCSI device:** which `/dev/sg*` the drive was on, and whether it changed after the failure.
7. **The disc:** title, format (BD/DVD/UHD), condition.
Include all of the above in bug reports. SCSI transport errors that resolve with the `US_FL_IGNORE_RESIDUE` quirk or by switching to direct SATA are bridge firmware bugs, not freemkv bugs.

87
benches/sgio_read.rs
View file

@ -1,87 +0,0 @@
// Mimics ISO dump exactly — read + write + progress
use libfreemkv::Drive;
use std::io::Write;
use std::path::Path;
use std::time::Instant;
fn main() {
let device = std::env::args()
.skip(1)
.find(|a| !a.starts_with('-'))
.unwrap_or_else(|| match libfreemkv::find_drive() {
Some(d) => d.device_path().to_string(),
None => {
eprintln!("No drives found");
std::process::exit(1);
}
});
let mut drive = Drive::open(Path::new(&device)).unwrap_or_else(|e| {
eprintln!("Cannot open {}: {}", device, e);
std::process::exit(1);
});
eprintln!("wait_ready...");
let _ = drive.wait_ready();
eprintln!("read_capacity...");
let cap = drive.read_capacity().unwrap();
eprintln!("capacity: {} sectors", cap);
let batch = libfreemkv::disc::detect_max_batch_sectors(drive.device_path());
let mut buf = vec![0u8; batch as usize * 2048];
// Open /dev/null writer like ISO dump does
let file = std::fs::File::create("/dev/null").unwrap();
let mut writer = std::io::BufWriter::with_capacity(4 * 1024 * 1024, file);
eprintln!(
"Reading 1000 batches ({:.1} MB) with write + progress...",
1000.0 * batch as f64 * 2048.0 / 1_048_576.0
);
let start = Instant::now();
let mut ok = 0u32;
let mut fail = 0u32;
let mut bytes: u64 = 0;
// Recovery flag: true matches pre-0.11.13 bench behavior — full SCSI
// ECC retry loop on errors (slower, what the rip path used before the
// adaptive batch sizer landed). Flip to `false` for the fast-fail path
// that current rips use; benches are configurable via this constant.
const READ_WITH_RECOVERY: bool = true;
for i in 0..1000u32 {
let lba = i * batch as u32;
match drive.read(lba, batch, &mut buf, READ_WITH_RECOVERY) {
Ok(_) => {
writer.write_all(&buf).unwrap();
ok += 1;
}
Err(e) => {
fail += 1;
if fail <= 5 {
eprintln!(" FAIL LBA {}: {}", lba, e);
}
buf.fill(0);
writer.write_all(&buf).unwrap();
}
}
bytes += buf.len() as u64;
if i % 50 == 0 && i > 0 {
let elapsed = start.elapsed().as_secs_f64();
let mb = bytes as f64 / 1_048_576.0;
eprint!("\r {:.1} MB | {:.1} MB/s ", mb, mb / elapsed);
}
}
let elapsed = start.elapsed().as_secs_f64();
let mb = ok as f64 * batch as f64 * 2048.0 / 1_048_576.0;
eprintln!(
"\n{} ok, {} fail, {:.1} MB in {:.1}s = {:.1} MB/s",
ok,
fail,
mb,
elapsed,
mb / elapsed
);
}

29
build.rs
View file

@ -3,34 +3,5 @@ fn main() {
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
let lib = format!("{out_dir}/libmacos_scsi.a");
std::process::Command::new("cc")
.args([
"-c",
"src/scsi/macos_shim.c",
"-o",
&obj,
"-framework",
"IOKit",
"-framework",
"CoreFoundation",
"-Wall",
"-O2",
])
.status()
.expect("failed to compile macos_shim.c");
std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to create static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
println!("cargo:rerun-if-changed=src/scsi/macos_shim.c");
}
}

4
docs/README.md
View file

@ -11,13 +11,11 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
| Document | What it covers |
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [Drive Access](drive-access.md) | DriveSession, SCSI transport, profiles, unlock, why raw mode is needed |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
| [CLPI Clip Info](clpi.md) | EP map (coarse + fine entries), timestamp-to-sector mapping, extent calculation |
| [API Design](api-design.md) | Stream API design, PES pipeline, input/output resolution |
## Reading Order

8
docs/aacs.md
View file

@ -189,14 +189,12 @@ In practice, AACS 2.0 UHD discs work through the backward-compatible AACS 1.0 ha
AACS decryption is transparent to the application. The `Disc::scan()` method handles everything automatically:
```rust
use libfreemkv::{Drive, Disc};
use libfreemkv::{DriveSession, Disc};
use libfreemkv::disc::ScanOptions;
use std::path::Path;
let mut drive = Drive::open(Path::new("/dev/sg4")).unwrap();
drive.wait_ready().unwrap();
drive.init().unwrap();
let disc = Disc::scan(&mut drive, &ScanOptions::default()).unwrap();
let mut session = DriveSession::open(Path::new("/dev/sr0")).unwrap();
let disc = Disc::scan(&mut session, &ScanOptions::default()).unwrap();
// Check encryption state
if disc.encrypted {

320
docs/api-design.md
View file

@ -12,97 +12,81 @@
```rust
// Open drive — explicit steps, app prints between them
let mut drive = Drive::open(path)?;
drive.wait_ready()?;
drive.init()?;
drive.probe_disc()?;
let mut session = DriveSession::open(path)?;
session.wait_ready()?;
session.init()?;
session.probe_disc()?;
// Scan disc
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
let disc = Disc::scan(&mut session, &ScanOptions::default())?;
// Browse
disc.titles // Vec<DiscTitle>
disc.titles // Vec<Title>
disc.format // BD / UHD / DVD
disc.capacity_gb()
// Rip with events
disc.rip(&mut session, 0, output, |event| {
match event.kind {
EventKind::BytesRead { bytes, total } => ...,
EventKind::ReadError { sector, error } => ...,
EventKind::Retry { attempt } => ...,
EventKind::SpeedChange { speed_kbs } => ...,
EventKind::Complete { bytes, errors } => ...,
}
})?;
// Rip without events
disc.rip(&mut session, 0, output, event::ignore)?;
```
## PES Pipeline (primary API)
## Stream Chains
The PES pipeline is the main way to move content. All streams produce/consume
PES frames. The pipeline just reads frames and writes frames.
Each stream wraps the next. Builder pattern, no `.build()`.
### Raw m2ts
```rust
// URL-based — any source to any destination
let opts = InputOptions::default();
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
output.write(&frame)?;
}
output.finish()?;
disc.rip(&mut session, 0, File::create("movie.m2ts")?, event::ignore)?;
```
The `FrameSource` and `FrameSink` traits — direction is type-checked, so
calling `read()` on a write-only sink (or `write()` on a read-only source)
is a compile error rather than a runtime fault:
### MKV
```rust
pub trait FrameSource: Send {
fn read(&mut self) -> Result<Option<PesFrame>, Error>;
fn info(&self) -> &DiscTitle;
fn codec_private(&self, track: usize) -> Option<Vec<u8>> { None }
fn headers_ready(&self) -> bool { true }
}
let output = MkvStream::new(File::create("movie.mkv")?)
.title(&disc.titles[0])
.max_buffer(10 * 1024 * 1024);
pub trait FrameSink: Send {
fn write(&mut self, frame: &PesFrame) -> Result<(), Error>;
fn finish(self: Box<Self>) -> Result<(), Error>;
fn info(&self) -> &DiscTitle;
}
disc.rip(&mut session, 0, output, |e| { ... })?;
```
## Streams
All streams implement `FrameSource` (read) and/or `FrameSink` (write); the
directional split prevents runtime "wrong-direction" errors. URL-based
resolvers open any stream by string.
| Stream | Input | Output | URL | Transport |
|--------|-------|--------|-----|-----------|
| DiscStream | Yes | -- | `disc://` `disc:///dev/sg4` | Optical drive via SCSI |
| IsoStream | Yes | Yes | `iso://path.iso` | Blu-ray ISO image |
| MkvStream | Yes | Yes | `mkv://path` | Matroska container |
| M2tsStream | Yes | Yes | `m2ts://path` | BD-TS with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | `network://host:port` | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | `stdio://` | Raw byte pipe |
| NullStream | -- | Yes | `null://` | Discard sink (byte counter) |
All URLs require a `scheme://path` format. Bare paths are rejected.
### MKV with progress (CLI)
```rust
// PES pipeline (frame-level) — input() returns Box<dyn FrameSource>,
// output() returns Box<dyn FrameSink>.
let input = libfreemkv::input("disc:///dev/sg4", &opts)?; // DiscStream
let input = libfreemkv::input("iso://Dune.iso", &opts)?; // IsoStream
let output = libfreemkv::output("mkv://Dune.mkv", &title)?; // MkvOutputStream
let output = libfreemkv::output("m2ts://Dune.m2ts", &title)?; // M2tsOutputStream
let output = libfreemkv::output("network://10.0.0.1:9000", &title)?; // NetworkOutputStream
let output = libfreemkv::output("null://", &title)?; // NullOutputStream
let output = ProgressStream::new(
MkvStream::new(File::create("movie.mkv")?)
.title(&disc.titles[0])
.max_buffer(10 * 1024 * 1024),
total_bytes,
|pct, speed| eprint!("\r {}% {:.1} MB/s", pct, speed),
);
disc.rip(&mut session, 0, output, |e| { ... })?;
```
### FMKV Metadata Header
M2tsStream and NetworkStream embed a JSON metadata header before the BD-TS data:
### Future: transcode
```rust
let output = ProgressStream::new(
TranscodeStream::new(
MkvStream::new(File::create("movie.mkv")?)
.title(&disc.titles[0])
.max_buffer(50 * 1024 * 1024),
)
.codec(H265)
.quality(22),
total_bytes,
|pct, speed| eprint!("\r {}% {:.1} MB/s", pct, speed),
);
disc.rip(&mut session, 0, output, |e| { ... })?;
```
[8B magic "FMKV\0\0\0\0"][4B JSON length][JSON metadata][padding to 192B boundary][BD-TS data...]
```
The header carries title name, duration, codec_privates, and full stream layout
(PIDs, codecs, languages, labels). This allows the receiving end to set up
demuxing and track metadata without scanning the TS.
## Events
@ -114,47 +98,114 @@ pub struct Event {
}
pub enum EventKind {
// Init / scan
DriveOpened { device: String },
DriveReady,
InitComplete { success: bool },
ProbeComplete { success: bool },
ScanComplete { titles: usize },
// Read pipeline
BytesRead { bytes: u64, total: u64 },
ReadError { sector: u64, error: Error },
Retry { attempt: u32 },
SpeedChange { speed_kbs: u16 },
ExtentStart { index: usize, start_sector: u64, sector_count: u64 },
SectorSkipped { sector: u64 },
BatchSizeChanged { new_size: u16, reason: BatchSizeReason },
Complete { bytes: u64, errors: u32 },
// Kept for forward-compat; not emitted in 0.13.6+
Retry { attempt: u32 },
SectorRecovered { sector: u64 },
}
```
Emission notes:
Events report what happened. App decides what to do. GUI shows a dialog. CLI prints a line. Server logs to file.
- `BytesRead { bytes, total }` is emitted from `DiscStream::fill_extents`
after each successful sector read. `bytes` is the cumulative running
total; `total` is the precomputed extent sum (0 if unknown).
- `SpeedChange` is emitted from the public `Drive::set_speed` API path.
It is no longer emitted from a recovery hot loop (recovery loop removed
in 0.13.6).
- `BatchSizeChanged` fires from the `DiscStream` adaptive sizer on shrink
(read failed at a larger size) and on probe-up (clean-read streak hit
the threshold). Consumers use it to display a "recovering" state
distinct from "ripping normally".
- `Retry` and `SectorRecovered` are NOT emitted in 0.13.6+. They were
tied to the inline `Drive::read` recovery phases that were removed; the
variants are kept for forward compatibility so consumers' match arms
don't need conditional compilation.
## Error Codes
Events report what happened. App decides what to do. GUI shows a dialog. CLI
prints a line. Server logs to file.
Lib errors are codes, not messages. Like HTTP status codes.
```rust
pub enum Error {
// Drive
DriveNotFound,
DriveOpenFailed,
DriveNotReady,
// Unlock
UnlockFailed,
NoProfile,
// AACS
AacsNoKeys,
AacsCertVerifyFailed,
AacsAgidAllocFailed,
AacsHandshakeFailed,
AacsVidMacFailed,
// Disc
DiscReadError { sector: u64 },
MplsParseError,
ClpiParseError,
UdfFileNotFound { path: String },
// Mux
LookaheadOverflow,
MuxWriteError,
// SCSI
ScsiError { sense: u8 },
}
```
App maps codes to localized strings. Lib never contains display text.
## Streams
All streams implement the `IOStream` trait (Read + Write). URL-based resolver opens any stream by string.
| Stream | Input | Output | URL | Transport |
|--------|-------|--------|-----|-----------|
| DiscStream | Yes | -- | `disc://` `disc:///dev/sg4` | Optical drive via SCSI |
| IsoStream | Yes | -- | `iso://path.iso` | Blu-ray ISO image |
| MkvStream | Yes | Yes | `mkv://path` | Matroska container |
| M2tsStream | Yes | Yes | `m2ts://path` | BD-TS with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | `network://host:port` | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | `stdio://` | Raw byte pipe |
| NullStream | -- | Yes | `null://` | Discard sink (byte counter) |
All URLs require a `scheme://path` format. Bare paths are rejected.
```rust
// URL-based opening
let input = open_input("disc://", &opts)?; // DiscStream (auto-detect)
let input = open_input("disc:///dev/sg4", &opts)?; // DiscStream (specific device)
let input = open_input("iso://Dune.iso", &opts)?; // IsoStream
let input = open_input("m2ts:///tmp/Dune.m2ts", &opts)?; // M2tsStream
let input = open_input("mkv://Dune.mkv", &opts)?; // MkvStream
let input = open_input("network://0.0.0.0:9000", &opts)?; // NetworkStream (listen)
let input = open_input("stdio://", &opts)?; // StdioStream (stdin)
let output = open_output("mkv://Dune.mkv", &meta)?; // MkvStream
let output = open_output("m2ts://Dune.m2ts", &meta)?; // M2tsStream
let output = open_output("network://10.0.0.1:9000", &meta)?;// NetworkStream (connect)
let output = open_output("stdio://", &meta)?; // StdioStream (stdout)
let output = open_output("null://", &meta)?; // NullStream
// Direct construction (for advanced use)
let mkv = MkvStream::new(writer).meta(&title).max_buffer(10 * 1024 * 1024);
let m2ts = M2tsStream::new(writer).meta(&title);
let net = NetworkStream::connect("10.0.0.1:9000")?.meta(&title);
let null = NullStream::new().meta(&title);
```
### FMKV Metadata Header
M2tsStream and NetworkStream embed a JSON metadata header before the BD-TS data:
```
[8B magic "FMKV\0\0\0\0"][4B JSON length][JSON metadata][padding to 192B boundary][BD-TS data...]
```
The header carries title name, duration, and full stream layout (PIDs, codecs, languages, labels). This allows the receiving end to set up demuxing and track metadata without scanning the TS.
### MkvStream Internals
LookaheadBuffer (default 5MB, configurable):
1. Phase 1: buffer incoming data, scan for codec setup (SPS/PPS)
2. Found it? Write MKV header, flush buffer, switch to streaming
3. Buffer full? Error — app handles it
4. Phase 2: parse TS → frames → MKV clusters, direct to output
Reading: extracts MKV frames, wraps back into BD-TS PES packets.
## File Layout
@ -163,71 +214,38 @@ libfreemkv/src/
├── lib.rs Public exports
├── error.rs Error codes (no English)
├── event.rs Event types for callbacks
├── halt.rs Halt cancellation token (Arc<AtomicBool> wrapper)
├── io/ Pipeline + WritebackFile primitives
│ ├── mod.rs Re-exports WritebackFile, Pipeline, Sink, Flow
│ ├── pipeline.rs Generic Pipeline<I, R> + Sink trait
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
│ ├── capture.rs Drive profile capture for contribution
│ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
│ ├── mod.rs Disc struct, scan, titles, formats
│ ├── sweep.rs Disc::sweep (Pass 1 forward sweep)
│ ├── patch.rs Disc::patch (Pass N retry over mapfile)
│ ├── mapfile.rs ddrescue-format mapfile
│ └── read_error.rs ReadCtx / ReadAction state machine
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── drive.rs DriveSession (open, init, read)
├── disc.rs Disc (scan, rip, titles)
├── scsi/ SCSI transport (Linux, macOS)
├── platform/ Drive unlock (MT1959 A/B)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
├── css/ DVD CSS cipher
├── decrypt.rs Unified decrypt dispatcher (AACS/CSS/None)
├── pes.rs PES frame types, FrameSource / FrameSink traits
├── sector/ Sector I/O (was sector.rs in 0.17)
│ ├── mod.rs SectorSource, SectorSink traits
│ ├── file.rs FileSectorSource, FileSectorSink (ISO-backed)
│ └── decrypting.rs DecryptingSectorSource decorator
├── udf.rs UDF 2.50 filesystem parser
├── mpls.rs MPLS playlist parser
├── clpi.rs CLPI clip info parser
├── ifo.rs DVD IFO parser
├── labels/ BD-J label extraction (5 format parsers)
├── keydb.rs KEYDB download, parse, save
├── identity.rs DriveId from INQUIRY
├── profile.rs Bundled drive profiles
├── speed.rs DriveSpeed enum
├── aacs/ AACS decryption
├── udf.rs UDF filesystem parser
├── mpls.rs Playlist parser
├── clpi.rs Clip info parser
├── mux/
│ ├── mod.rs Public mux exports
│ ├── resolve.rs URL parser + input/output (Box<dyn FrameSource/Sink>)
│ ├── meta.rs FMKV header format
│ ├── disc.rs DiscStream (optical drive → PES)
│ ├── iso.rs IsoStream (ISO image read)
│ ├── isowriter.rs ISO image writer (UDF, AVDP, multi-extent)
│ ├── mod.rs IOStream trait, public exports
│ ├── resolve.rs URL parser + open_input/open_output
│ ├── meta.rs M2tsMeta (FMKV header format)
│ ├── disc.rs DiscStream (optical drive)
│ ├── mkvstream.rs MkvStream (bidirectional Matroska)
│ ├── mkvout.rs MkvOutputStream (PES → MKV)
│ ├── m2ts.rs M2tsStream (BD-TS)
│ ├── pesout.rs PES output streams (M2ts, Network, Stdio, Null)
│ ├── m2ts.rs M2tsStream (BD-TS + FMKV header)
│ ├── network.rs NetworkStream (TCP + FMKV header)
│ ├── stdio.rs StdioStream (stdin/stdout pipe)
│ ├── iso.rs IsoStream (Blu-ray ISO image)
│ ├── null.rs NullStream (discard + byte counter)
│ ├── lookahead.rs LookaheadBuffer (codec header scanning)
│ ├── ts.rs BD-TS demuxer + PAT/PMT scanner
│ ├── tsreader.rs TS reader utilities
│ ├── tsmux.rs TS muxer (PES → BD-TS packets)
│ ├── ps.rs MPEG-2 PS demuxer (DVD)
│ ├── ebml.rs EBML read/write primitives
│ ├── mkv.rs MKV muxer (tracks, clusters, cues)
│ └── codec/ Frame parsers (H.264, HEVC, MPEG-2, VC-1, AC3, EAC3, DTS, TrueHD, LPCM, PGS)
│ └── codec/ Frame parsers (H.264, HEVC, VC-1, AC3, DTS, TrueHD, PGS, LPCM)
└── ...
freemkv/src/
├── main.rs CLI dispatcher (URL routing)
├── pipe.rs PES pipeline — source → dest copy
├── disc_info.rs Disc/file info display
├── pipe.rs Generic source → dest copy
├── rip.rs Rip with progress display
├── remux.rs Remux with progress display
├── disc_info.rs Disc info display
├── info.rs Drive info + profile submission
├── strings.rs i18n string table
├── output.rs Verbosity-filtered output

109
docs/architecture.md
View file

@ -13,9 +13,9 @@ AACS keys are derived internally, and all SCSI communication is handled in-proce
1. **CLI is dumb.** All drive communication, disc parsing, AACS decryption, and
format handling live in the library. CLI binaries are thin wrappers that call
`Drive::open()` and `Disc::scan()`.
`DriveSession::open()` and `Disc::scan()`.
2. **No external files.** Bundled drive profiles are compiled into the binary via
2. **No external files.** 206 drive profiles are compiled into the binary via
`include_str!`. No configuration directory, no runtime file lookups for drive
support.
@ -23,16 +23,12 @@ AACS keys are derived internally, and all SCSI communication is handled in-proce
available. Callers read cleartext sectors without knowing whether the disc
was encrypted.
4. **Structured errors, no English.** Every error has a numeric code (E1000-E8000).
4. **Structured errors, no English.** Every error has a numeric code (E1000-E7000).
The library never formats user-facing messages -- applications do that.
5. **Library-agnostic.** No concept of "supported" vs "unsupported" drives at a
policy level. If a profile exists, the library uses it.
6. **Streams are dumb pipes.** Streams read/write PES frames. They don't know
about encryption, transport format, or source type. Decrypt is a stream-internal
concern; the pipeline just moves frames.
---
## Module Map
@ -41,43 +37,26 @@ AACS keys are derived internally, and all SCSI communication is handled in-proce
libfreemkv (lib.rs)
│
├── Drive Access
│ ├── drive Drive — open, identify, init, unlock, single-shot read
│ ├── scsi ScsiTransport trait + platform backends (sg async, IOKit, SPTI)
│ ├── drive DriveSession — open, identify, unlock, read
│ ├── scsi ScsiTransport trait + SG_IO implementation
│ ├── platform/ Platform trait — per-chipset command handlers
│ │ └── mt1959 MediaTek MT1959 driver (LG, ASUS, HP)
│ │ └── mt1959 MediaTek MT1959 driver (LG, ASUS, hp)
│ ├── profile DriveProfile loading, matching, bundled JSON
│ ├── identity DriveId from INQUIRY + GET_CONFIG 010C
│ ├── speed DriveSpeed enum, SET CD SPEED CDB builder
│ └── event Event system for drive status callbacks
│ └── speed DriveSpeed enum, SET CD SPEED CDB builder
│
├── Disc Scanning
│ ├── disc Disc::scan() — titles, streams, extents, AACS setup
│ ├── udf UDF 2.50 filesystem reader (metadata partitions)
│ ├── mpls MPLS playlist parser — clips, streams, STN table
│ ├── clpi CLPI clip info parser — EP map, sector extents
│ ├── ifo DVD IFO parser — title sets, PGC chains, cell addresses
│ └── labels/ BD-J label extraction (5 formats: Paramount, Criterion, Pixelogic, CTRM, Deluxe)
│ └── jar BD-J JAR label extraction (audio/subtitle names)
│
├── Encryption
│ ├── aacs/ AACS handshake, KEYDB, VUK lookup, MKB, unit decryption
│ ├── css DVD CSS cipher — table-driven, no external keys needed
│ └── decrypt decrypt_sectors() — unified AACS/CSS/None dispatcher
│ ├── aacs KEYDB parsing, VUK lookup, MKB processing, unit decryption
│ └── aacs_handshake ECDH bus authentication, Volume ID, Read Data Key
│
├── Streaming
│ ├── mux/ Stream implementations (Disc, ISO, MKV, M2TS, Network, Stdio, Null)
│ ├── pes PES frame types; FrameSource / FrameSink direction-typed traits
│ └── sector/ SectorSource / SectorSink traits, FileSector{Source,Sink}, DecryptingSectorSource
│
├── I/O Primitives
│ ├── halt Halt cancellation token (one Arc<AtomicBool>, cloneable)
│ └── io/ Pipeline<I, R> + Sink trait + WritebackFile (bounded-cache writer)
│
├── Support
│ ├── keydb KEYDB.cfg download, parse, verify, save
│ ├── error Error enum with numeric codes E1000-E8000
│ └── profile Bundled drive profiles
│
└── lib.rs Public API re-exports
└── error Error enum with numeric codes E1000-E7000
```
---
@ -85,35 +64,27 @@ libfreemkv (lib.rs)
## Drive Access Flow
```
Drive::open(Path::new("/dev/sg4"))
DriveSession::open("/dev/sr0")
│
├─ scsi::open() Open /dev/sg4 (async write/poll/read)
├─ scsi::open() Open /dev/sr0 via SG_IO
├─ DriveId::from_drive() INQUIRY + GET_CONFIG 010C
├─ profile::find_by_drive_id() Match against bundled profiles
├─ profile::find_by_drive_id() Match against 206 bundled profiles
├─ Platform::new() Instantiate chipset driver (Mt1959)
└─ Drive ready for init/unlock/read
└─ Platform::unlock() Activate raw disc access mode
```
After open:
- `init()` -- unlock + firmware upload + speed calibration
- `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
- `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1
(`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
the stop-wedge postmortem (2026-04-25).
After open, the session provides:
- `read_sectors(lba, count, buf)` -- raw sector reads (through platform driver)
- `read_disc(lba, count, buf)` -- standard READ(10) for filesystem data
- `scsi_execute(cdb, dir, buf, timeout)` -- arbitrary SCSI commands
- `status()`, `calibrate()`, `read_config()`, `read_register()`
---
## Disc Scanning Flow
```
Disc::scan(&mut drive, &ScanOptions)
Disc::scan(&mut session, &ScanOptions)
│
├─ READ CAPACITY Get disc size in sectors
├─ udf::read_filesystem() Parse UDF 2.50 (AVDP → VDS → metadata → FSD → root)
@ -121,24 +92,14 @@ Disc::scan(&mut drive, &ScanOptions)
│ ├─ mpls::parse() Extract play items, STN streams
│ └─ For each clip:
│ └─ clpi::parse() EP map → sector extents for the clip's time range
├─ labels::detect() Parse BD-J JARs for stream labels
├─ Detect AACS Check for /AACS directory on disc
└─ Disc::setup_aacs() Handshake + KEYDB → VUK → unit keys (if encrypted)
```
For DVD:
```
Disc::scan_dvd(&mut drive, &ScanOptions)
│
├─ ifo::parse() Parse VIDEO_TS.IFO — title sets, PGC chains
├─ CSS detection Check disc structure flag
└─ CSS key cracking Table-driven, no KEYDB needed
```
The result is a `Disc` with:
- `titles: Vec<DiscTitle>` -- sorted by duration, each with streams, sector extents, codec_privates
- `decrypt_keys()` -- DecryptKeys for content decryption
- `encrypted: bool` -- whether the disc uses AACS/CSS
- `titles: Vec<Title>` -- sorted by duration, each with streams and sector extents
- `aacs: Option<AacsState>` -- decryption keys if available
- `encrypted: bool` -- whether the disc uses AACS
---
@ -153,14 +114,13 @@ Four key resolution paths, tried in order:
| 3 | Processing Keys + MKB → Media Key → VUK | Medium |
| 4 | Device Keys + MKB subset-difference tree → VUK | Slow |
The AACS handshake (`aacs/handshake`) performs ECDH key agreement over the
The AACS handshake (`aacs_handshake`) performs ECDH key agreement over the
AACS 1.0 160-bit elliptic curve to obtain:
- **Volume ID** -- needed for VUK derivation (paths 2-4)
- **Read Data Key** -- needed for AACS 2.0 (UHD) bus decryption
Content decryption uses AES-128-CBC on 6144-byte aligned units. The
`ContentReader` handles this transparently. Streams that read sectors
(DiscStream, IsoStream) decrypt internally — the pipeline sees clean bytes.
`ContentReader` handles this transparently.
---
@ -178,7 +138,6 @@ is baked into the library.
| E5xxx | I/O errors | `IoError` (wraps `std::io::Error`) |
| E6xxx | Disc format errors | `DiscError` (UDF, MPLS, CLPI parse failures) |
| E7xxx | AACS errors | `AacsError` (key resolution, handshake, decryption) |
| E8xxx | KEYDB errors | `KeydbError` (download, parse, save) |
---
@ -186,9 +145,9 @@ is baked into the library.
| Platform | Transport | Status |
|----------|-----------|--------|
| Linux | async sg write/poll/read on `/dev/sg*` | Supported |
| macOS | IOKit SCSITask | Supported |
| Windows | SPTI (`IOCTL_SCSI_PASS_THROUGH_DIRECT`) | Supported |
| Linux | SG_IO ioctl on `/dev/sr*` | Implemented |
| macOS | IOKit SCSI passthrough | Planned |
| Windows | SPTI (`IOCTL_SCSI_PASS_THROUGH_DIRECT`) | Planned |
The `ScsiTransport` trait abstracts the platform. Adding a new platform requires
implementing `execute()` for that OS and wiring it into `scsi::open()`.
@ -199,11 +158,11 @@ implementing `execute()` for that OS and wiring it into `scsi::open()`.
| Chipset | Drives | Status |
|---------|--------|--------|
| MediaTek MT1959 | LG, ASUS, HP | Supported (bundled profiles) |
| MediaTek MT1959 | LG, ASUS, hp | Implemented (206 profiles) |
| Renesas RS8xxx/RS9xxx | Pioneer, some HL-DT-ST | Planned |
The `Platform` trait abstracts chipset-specific commands. Each chipset implements
handlers (unlock, config, register, calibrate, keepalive, status, probe,
10 handlers (unlock, config, register, calibrate, keepalive, status, probe,
read_sectors, timing). All handlers are accessed via SCSI READ BUFFER with
chipset-specific mode and buffer ID bytes.
@ -215,5 +174,7 @@ chipset-specific mode and buffer ID bytes.
cargo build --release
```
Produces a Rust library crate. The `libc` dependency is unix-only (gated).
All three platforms build and pass CI.
Linux builds produce a static library and two binaries (`freemkv-info`,
`freemkv-test`). The `libc` dependency is Linux-only. On non-Linux platforms,
the library compiles but `scsi::open()` returns a platform-not-supported error
until the IOKit/SPTI backends are implemented.

103
docs/disc-to-rip.md
View file

@ -9,18 +9,12 @@ This is the starting point for understanding the library.
Insert disc
│
▼
1. Open drive (drive/mod.rs)
1. Open drive (drive.rs)
│ INQUIRY → identify drive
│ Match bundled profile → chipset, unlock parameters
│
▼
2. Init drive (drive/mod.rs → platform/mt1959)
│ Firmware upload (if needed, 10s recovery wait)
│ Unlock → vendor-specific command activates raw read mode
│ Speed calibration → probe_disc()
│
▼
3. AACS handshake (aacs/handshake.rs) — optional
2. AACS handshake (aacs_handshake.rs) — optional, separate transport
│ Allocate AGID
│ Exchange certificates + nonces (ECDH)
│ Derive bus key
@ -28,6 +22,11 @@ Insert disc
│ (fails gracefully if drive doesn't support AACS for this disc)
│
▼
3. Unlock drive (drive.rs → platform/mt1959.rs)
│ Vendor-specific command activates raw read mode
│ Required — drive firmware blocks all reads without it
│
▼
4. Read UDF filesystem (udf.rs)
│ Sector 256: AVDP → find Volume Descriptor Sequence
│ VDS: Partition Descriptor (physical start) + Logical Volume (metadata start)
@ -36,100 +35,80 @@ Insert disc
│ → docs/udf.md
│
▼
5. Read AACS files from disc (aacs/mod.rs)
5. Read AACS files from disc (aacs.rs)
│ AACS/Unit_Key_RO.inf → SHA1 = disc hash
│ AACS/Content000.cer → AACS version (1.0 or 2.0), bus encryption flag
│ MKB via SCSI → for key derivation fallback
│
▼
6. Resolve encryption keys (decrypt.rs → resolve_encryption)
│ BD AACS:
│ Path 1: disc hash → KEYDB.cfg → VUK (fast, 99% of discs)
│ Path 2: KEYDB media key + Volume ID → VUK
│ Path 3: MKB + processing keys → media key → VUK
│ Path 4: MKB + device keys → subset-difference tree → VUK
│ VUK → decrypt unit keys from Unit_Key_RO.inf
│ DVD CSS:
│ Table-driven cipher — no KEYDB needed
6. Resolve AACS keys (aacs.rs → resolve_keys)
│ Path 1: disc hash → KEYDB.cfg → VUK (fast, 99% of discs)
│ Path 2: KEYDB media key + Volume ID → VUK
│ Path 3: MKB + processing keys → media key → VUK
│ Path 4: MKB + device keys → subset-difference tree → VUK
│ VUK → decrypt unit keys from Unit_Key_RO.inf
│ → docs/aacs.md
│
▼
7. Parse playlists (mpls.rs) — BD/UHD only
7. Parse playlists (mpls.rs)
│ BDMV/PLAYLIST/*.mpls → titles with play items
│ Each play item: clip ID, in/out timestamps
│ STN table: video, audio, subtitle streams with codec + language
│ → docs/mpls.md
│
▼
8. Parse clip info (clpi.rs) — BD/UHD only
8. Parse clip info (clpi.rs)
│ BDMV/CLIPINF/*.clpi → EP map (timestamp → sector mapping)
│ Coarse + fine entries → full PTS and SPN
│ SPN → byte offset → sector extents for reading
│ → docs/clpi.md
│
▼
9. Parse BD-J labels (labels/) — optional
9. Parse BD-J labels (jar.rs) — optional
│ BDMV/JAR/*.jar → Java class constant pool strings
│ 5 format parsers: Paramount, Criterion, Pixelogic, CTRM, Deluxe
│ Audio track labels: "English Descriptive Audio", "French 5.1", etc.
│
▼
10. Stream content (mux/disc.rs → DiscStream)
│ Read sectors → decrypt → TS demux → PES frames
│ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — Disc::patch re-runs against the mapfile.
10. Read + decrypt content (disc.rs → ContentReader)
│ For each aligned unit (6144 bytes = 3 sectors):
│ Read 3 sectors from disc
│ If AACS 2.0: bus decrypt (read_data_key, per-sector AES-CBC)
│ If encrypted: unit decrypt (per-unit key derivation + AES-CBC)
│ Output decrypted content
│
▼
PES frames → output stream (MKV, M2TS, network, etc.)
Decrypted m2ts stream → ready for muxing/backup
```
## API Summary
```rust
// Open + init drive
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?;
drive.init()?;
drive.probe_disc()?;
// Steps 1 + 3 (open + unlock)
let mut session = DriveSession::open(Path::new("/dev/sr0"))?;
// Scan disc (UDF + playlists + AACS — all automatic)
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
// Steps 2 + 4-9 (AACS + scan)
let disc = Disc::scan(&mut session, &ScanOptions::with_keydb("keydb.cfg"))?;
// Stream pipeline — PES frames from any source to any output.
// 0.18: input() returns Box<dyn FrameSource>, output() returns Box<dyn FrameSink>;
// direction is type-checked, so calling .write() on an input is a compile error.
let opts = InputOptions::default();
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
output.write(&frame)?;
// Step 10 (read + decrypt)
let mut reader = disc.open_title(&mut session, 0)?;
while let Some(unit) = reader.read_unit()? {
output.write_all(&unit)?;
}
output.finish()?;
```
Three lines. Everything else is internal.
## Module Reference
| Module | Doc | Purpose |
|--------|-----|---------|
| drive/ | [drive-access.md](drive-access.md) | Open, identify, init, unlock, single-shot read |
| scsi/ | [drive-access.md](drive-access.md) | Platform SCSI transport (Linux, macOS, Windows) |
| drive.rs | [drive-access.md](drive-access.md) | Open, identify, unlock, read |
| scsi.rs | [drive-access.md](drive-access.md) | Platform SCSI transport |
| udf.rs | [udf.md](udf.md) | UDF 2.50 filesystem |
| mpls.rs | [mpls.md](mpls.md) | MPLS playlists + STN streams |
| clpi.rs | [clpi.md](clpi.md) | CLPI clip info + EP map |
| ifo.rs | -- | DVD IFO parser |
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
| labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
| sector/ | -- | SectorSource / SectorSink + DecryptingSectorSource decorator |
| io/ | -- | Pipeline<I, R> + Sink trait + WritebackFile |
| halt.rs | -- | Halt cancellation token |
| keydb.rs | -- | KEYDB download, parse, save |
| error.rs | -- | Error codes (E1xxx-E8xxx) |
| event.rs | -- | Drive event system |
| aacs.rs | [aacs.md](aacs.md) | Key resolution + content decrypt |
| aacs_handshake.rs | [aacs.md](aacs.md) | SCSI bus authentication |
| disc.rs | -- | High-level scan + read API |
| jar.rs | -- | BD-J audio track labels |
| error.rs | -- | Error codes (E1xxx-E7xxx) |

189
docs/drive-access.md
View file

@ -5,72 +5,51 @@ optical drives.
---
## Drive
## DriveSession
`Drive` is the primary API. It owns the SCSI transport, the matched
`DriveSession` is the primary API. It owns the SCSI transport, the matched
drive profile, and the chipset-specific platform driver.
### Opening a Drive
```rust
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
// Full open: identify → match profile → unlock
let mut session = DriveSession::open(Path::new("/dev/sr0"))?;
// No-unlock open: identify → match profile only
let mut session = DriveSession::open_no_unlock(Path::new("/dev/sr0"))?;
// Explicit profile (skip auto-detection)
let mut session = DriveSession::open_with_profile(Path::new("/dev/sr0"), profile)?;
```
`open()` performs: open device → send INQUIRY → match profile → instantiate
platform driver. The drive is ready for `wait_ready()` and `init()`.
**`open()`** performs the full sequence: open device, send INQUIRY, match
profile, instantiate platform driver, and unlock. Unlock failures are silently
ignored (unencrypted discs do not need it). After `open()`, both raw sector
reads and standard READ(10) work immediately.
### Drive Operations
**`open_no_unlock()`** skips the unlock step. This is required when AACS bus
authentication must happen before unlock. The handshake uses standard SCSI
commands that work without raw mode. After authentication completes, the caller
can invoke `session.unlock()` manually.
**`open_with_profile()`** bypasses profile auto-detection. Useful for testing
or when a custom profile is loaded from an external source.
### Session Operations
| Method | Description |
|--------|-------------|
| `wait_ready()` | Wait for disc insertion (30s timeout, TUR polling) |
| `init()` | Firmware upload + unlock + speed calibration |
| `probe_disc()` | Probe disc surface for optimal speeds |
| `read(lba, count, buf, recovery)` | Read sectors. Single-shot — no inline retries or reset. |
| `reset()` | Eject-cycle escape hatch. Caller-invoked only; not on the read path. |
| `lock_tray()` | Prevent tray ejection during rip |
| `unlock_tray()` | Allow tray ejection (also runs on Drop) |
| `eject()` | Eject disc tray |
| `drive_status()` | Query physical state (disc present, tray open, etc.) |
| `has_profile()` | Whether a bundled profile matched |
| `close()` | Consume Drive, cleanup (also runs via Drop) |
### init() Sequence
`init()` orchestrates the full drive unlock:
1. Platform driver `run_init()` — sends vendor-specific SCSI commands
2. If firmware upload needed: upload, wait 10s for drive reset, retry
3. Speed calibration after unlock
4. Max 3 attempts before giving up
### read() — single-shot
`Drive::read(lba, count, buf, recovery)` is the single read method. It issues
exactly one READ(10) CDB and returns the result. The `recovery` parameter only
selects the per-CDB timeout:
| `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------|
| `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 30 s | `Disc::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`:
- **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read
streak.
Inline recovery (5× gentle retry → close + reset + reopen → 5× more) was
removed in 0.13.6. See the stop-wedge postmortem (2026-04-25)
for rationale: the inline reset wedged drive firmware on the LG BU40N (Initio
USB-SATA bridge) without ever recovering a sector. See
[`rip-recovery.md`](rip-recovery.md) for the full three-layer model.
| `unlock()` | Activate raw disc access mode via platform driver |
| `is_unlocked()` | Check if raw mode is active |
| `calibrate()` | Build speed lookup table for the current disc |
| `read_sectors(lba, count, buf)` | Raw sector read (requires unlock + calibrate) |
| `read_disc(lba, count, buf)` | Standard READ(10) with 5s timeout |
| `status()` | Query drive status and feature flags |
| `read_config()` | Read drive configuration block (1888 bytes) |
| `read_register(index)` | Read 16-byte hardware register |
| `probe(sub_cmd, addr, len)` | Generic READ BUFFER with caller parameters |
| `scsi_execute(cdb, dir, buf, timeout)` | Send an arbitrary SCSI CDB |
---
@ -79,7 +58,7 @@ USB-SATA bridge) without ever recovering a sector. See
### Trait
```rust
pub trait ScsiTransport: Send {
pub trait ScsiTransport {
fn execute(
&mut self,
cdb: &[u8],
@ -93,43 +72,17 @@ pub trait ScsiTransport: Send {
All drive communication goes through this trait. The library never opens file
descriptors or calls ioctls outside of a `ScsiTransport` implementation.
### Platform Backends
### Linux: SG_IO
| Platform | Implementation | Device |
|----------|---------------|--------|
| Linux | `SgIoTransport` — async `write`/`poll`/`read` on `/dev/sg*` | `/dev/sg*` |
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
The `SgIoTransport` implementation:
The Linux backend uses the sg driver's asynchronous interface: `write()` submits
the command, `poll()` waits with an enforceable wall-clock timeout, `read()`
retrieves the result. If `poll()` times out, the fd is abandoned (closed in a
background thread) and a fresh fd opened — the kernel's USB error recovery
cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
1. Opens the device path with `O_RDWR | O_NONBLOCK`.
2. Constructs an `sg_io_hdr` struct with the CDB, data buffer, and timeout.
3. Calls `ioctl(fd, SG_IO, &hdr)`.
4. Returns `ScsiResult` with status, bytes transferred, and sense data.
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
The shim handles:
1. `shim_open_exclusive(bsd_name)` — unmounts the target device via `diskutil`,
then walks the IOKit registry to find the `IOBDServices` matching the
requested BSD name (IOBDServices → IOBDBlockStorageDriver → IOMedia → "BSD Name"),
then creates MMCDeviceInterface → SCSITaskDeviceInterface → ObtainExclusiveAccess.
2. `shim_list_drives()` — registry-based enumeration with zero SCSI, zero exclusive
access, zero unmounts. Reads IOBDServices "Device Characteristics" for
vendor/model/firmware and child IOMedia "BSD Name" for the device path.
3. `shim_execute()` / `shim_close()` — raw CDB dispatch and cleanup.
On non-zero SCSI status, the transport parses sense key from the sense buffer
and returns `Error::ScsiError`.
`SgIoTransport::reset` (Linux) does pure userspace state cleanup: an open +
close pair to make the kernel cancel any SG_IO commands queued against a
previous fd, a 2 s sleep to let the kernel finish that cancellation, then a
fresh fd to send ALLOW MEDIUM REMOVAL to clear any stale tray lock. It does
NOT issue `SG_SCSI_RESET` or escalate via STOP+START UNIT. Both were tried
in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA bridge); both failed
to recover wedged drives and made the wedge worse. The macOS reset (which
had been a no-op) was removed entirely in 0.13.6, and the top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
removed at the same time (no callers).
On non-zero SCSI status, the transport parses sense key, ASC, and ASCQ from the
sense buffer and returns `Error::ScsiError`.
### CDB Builders
@ -166,8 +119,8 @@ date for drives where Feature 010C is unavailable.
## Drive Profiles
Profiles are JSON objects compiled into the binary (`profiles.json`).
Each profile contains:
Profiles are JSON objects compiled into the binary (`profiles.json`,
206 entries). Each profile contains:
| Field | Purpose |
|-------|---------|
@ -195,7 +148,7 @@ let profiles = profile::load_all(Path::new("/path/to/profiles.json"))?;
### MediaTek MT1959
Covers all LG, ASUS, and HP optical drives. Two sub-variants share identical
Covers all LG, ASUS, and hp optical drives. Two sub-variants share identical
logic with different SCSI parameters:
| Variant | READ BUFFER mode | Buffer ID |
@ -203,7 +156,7 @@ logic with different SCSI parameters:
| MT1959-A | 0x01 | 0x44 |
| MT1959-B | 0x02 | 0x77 |
The Platform trait maps to command handlers:
The Platform trait maps to 10 command handlers:
| Handler | Function | Description |
|---------|----------|-------------|
@ -230,32 +183,64 @@ Optical drive firmware restricts what applications can read from disc. Without
unlock:
- **READ(10) works for unencrypted filesystem data.** UDF structures, MPLS
playlists, and CLPI clip info are readable without unlock. Standard READ(10)
works on any drive.
playlists, and CLPI clip info are readable without unlock. The `read_disc()`
method uses standard READ(10) and works on any drive.
- **READ(10) fails for encrypted content sectors.** The drive firmware returns
SCSI errors (sense key 0x05, illegal request) when an application attempts to
read sectors containing encrypted m2ts content without prior AACS
authentication via the bus key.
- **The kernel sr driver blocks block-device reads.** On Linux, the kernel's
SCSI CD-ROM driver (`sr`) refuses to expose encrypted disc content through
`/dev/sr0` as a block device. Even if you open the block device directly,
reads to encrypted regions fail.
- **Raw mode bypasses firmware restrictions.** After unlock, the drive accepts
READ(10) with the raw read flag (CDB byte 1 = 0x08) for all sectors,
regardless of encryption status.
regardless of encryption status. This is how raw sector ripping works.
### AACS Before Unlock
### open() vs open_no_unlock()
AACS bus authentication uses standard MMC REPORT KEY / SEND KEY commands.
On some drives these must execute before unlock. The `Disc::scan()` handles
this internally — it manages the handshake/unlock ordering automatically.
These must execute before unlock because:
1. The AACS handshake establishes a bus key via ECDH.
2. The bus key encrypts the Volume ID and Read Data Key responses.
3. The Volume ID is needed to derive the Volume Unique Key (VUK).
4. The VUK is needed to decrypt unit keys from `Unit_Key_RO.inf`.
If `open()` unlocks first, some drives reject the subsequent AACS commands.
The correct sequence for encrypted discs is:
```rust
// 1. Open without unlock
let mut session = DriveSession::open_no_unlock(device)?;
// 2. AACS handshake (uses standard SCSI, no unlock needed)
let auth = aacs_handshake::aacs_authenticate(&mut session, &key, &cert)?;
let vid = aacs_handshake::read_volume_id(&mut session, &mut auth)?;
// 3. Now unlock for raw reads
session.unlock()?;
session.calibrate()?;
// 4. Read and decrypt content
session.read_sectors(lba, count, &mut buf)?;
```
In practice, `Disc::scan()` handles this internally. The default `open()` call
unlocks immediately and is correct for most use cases -- the scan re-opens a
second session with `open_no_unlock()` for the AACS handshake when needed.
---
## Speed Control
After `probe_disc()`, the platform driver maintains a speed lookup table
built by probing the disc surface. On each `read()` call, the driver:
After `calibrate()`, the platform driver maintains a 64-entry speed lookup table
built by probing the disc surface. On each `read_sectors()` call, the driver:
1. Looks up the optimal speed for the target LBA.
1. Looks up the optimal speed for the target LBA in the table.
2. Issues SET CD SPEED (0xBB) if the speed differs from current.
3. Performs the READ(10).

240
docs/mkv-muxer-plan.md Normal file
View file

@ -0,0 +1,240 @@
# MKV Native Muxer — Architecture
## Goal
Replace raw m2ts output with in-pipeline MKV muxing.
Disc → TS demux → MKV mux → .mkv file. One pass, no temp files.
## Data Flow
```
ContentReader::read_batch() returns &[u8] of raw BD transport stream
↓
TsDemuxer::feed(batch) parses 192-byte BD-TS packets, extracts PES
↓
PES reassembly per PID builds complete PES packets with PTS/DTS
↓
ElementaryStreamParser per track finds frame boundaries, extracts codec headers
↓
MkvMuxer::write_frame(track, pts, data) writes EBML clusters + blocks
↓
.mkv file on disk
```
## Current Integration Point
```rust
// rip.rs line ~287
match reader.read_batch() {
Ok(Some(batch)) => {
writer.write_all(batch)?; // ← replace with muxer.feed(batch)
}
}
```
Becomes:
```rust
match reader.read_batch() {
Ok(Some(batch)) => {
muxer.feed(batch)?;
}
}
```
## Components
### 1. BD Transport Stream Demuxer (`ts.rs`)
BD uses 192-byte packets (not standard 188):
```
[0-3] TP_extra_header: 2-bit copy_permission + 30-bit arrival_time_stamp
[4] Sync byte: 0x47
[5] TEI + PUSI + priority + PID[12:8]
[6] PID[7:0]
[7] Scrambling + adaptation + continuity_counter
[8..] Adaptation field (if present) + payload
```
API:
```rust
pub struct TsDemuxer {
pes_assemblers: HashMap<u16, PesAssembler>, // PID → assembler
}
impl TsDemuxer {
pub fn new(pids: &[u16]) -> Self;
pub fn feed(&mut self, data: &[u8]) -> Vec<PesPacket>;
}
pub struct PesPacket {
pub pid: u16,
pub pts: Option<i64>, // 90kHz ticks
pub dts: Option<i64>, // 90kHz ticks
pub data: Vec<u8>, // elementary stream data
}
```
### 2. Elementary Stream Parsers (`codec/`)
Each codec parser finds frame boundaries and extracts initialization data.
**H.264 (`codec/h264.rs`):**
- Parse NAL units (start code 00 00 01 or 00 00 00 01)
- Extract SPS + PPS for codecPrivate
- Frame boundary = Access Unit Delimiter (NAL type 9) or SPS
**HEVC (`codec/hevc.rs`):**
- Parse NAL units
- Extract VPS + SPS + PPS for codecPrivate
- Frame boundary = VCL NAL with first_slice_segment_in_pic_flag
**AC3/EAC3 (`codec/ac3.rs`):**
- Syncword 0x0B77
- Parse frame size from header
- No codecPrivate needed (or minimal)
**DTS (`codec/dts.rs`):**
- Syncword 0x7FFE8001
- Parse frame size
- No codecPrivate needed
**TrueHD (`codec/truehd.rs`):**
- Major sync: 0xF8726FBA
- Access unit = major sync + minor syncs
- AC3 core embedded in first substream
**LPCM (`codec/lpcm.rs`):**
- Fixed frame sizes based on sample rate + channels
- Header describes format
**PGS (`codec/pgs.rs`):**
- Segment types: PCS, WDS, PDS, ODS, END
- Each segment is a complete unit
- No codecPrivate needed
### 3. MKV/EBML Muxer (`mkv.rs`)
Matroska uses EBML (Extensible Binary Meta Language).
**EBML primitives:**
- Variable-length element ID (1-4 bytes)
- Variable-length size (1-8 bytes)
- Data: uint, int, float, string, UTF-8, binary, date
**MKV structure:**
```
EBML Header
Segment
├── SeekHead (index of top-level elements)
├── Info (title, duration, muxing app)
├── Tracks (one entry per stream)
│ ├── TrackEntry (video)
│ │ ├── CodecID: "V_MPEG4/ISO/AVC" or "V_MPEGH/ISO/HEVC"
│ │ ├── CodecPrivate: SPS+PPS (H.264) or VPS+SPS+PPS (HEVC)
│ │ └── Video: PixelWidth, PixelHeight, DisplayWidth, DisplayHeight
│ ├── TrackEntry (audio)
│ │ ├── CodecID: "A_AC3" or "A_TRUEHD" or "A_DTS"
│ │ └── Audio: SamplingFrequency, Channels, BitDepth
│ └── TrackEntry (subtitle)
│ └── CodecID: "S_HDMV/PGS"
├── Chapters (optional, from MPLS chapter marks)
├── Cluster (every ~5 seconds)
│ ├── Timestamp (cluster base time)
│ ├── SimpleBlock (track, relative_ts, data)
│ ├── SimpleBlock ...
│ └── ...
├── Cluster ...
├── Cues (seek index, written at end)
└── Tags (metadata)
```
**API:**
```rust
pub struct MkvMuxer<W: Write + Seek> {
writer: W,
tracks: Vec<MkvTrack>,
cluster_start: Option<i64>,
cue_points: Vec<CuePoint>,
}
impl<W: Write + Seek> MkvMuxer<W> {
pub fn new(writer: W, tracks: &[MkvTrack]) -> Result<Self>;
pub fn write_frame(&mut self, track_idx: usize, pts_ns: i64, keyframe: bool, data: &[u8]) -> Result<()>;
pub fn finish(self) -> Result<()>; // writes Cues + fixes SeekHead
}
```
### 4. Pipeline Glue (`mux.rs`)
Ties everything together:
```rust
pub struct MuxPipeline<W: Write + Seek> {
demuxer: TsDemuxer,
parsers: HashMap<u16, Box<dyn CodecParser>>,
muxer: MkvMuxer<W>,
pid_to_track: HashMap<u16, usize>,
}
impl<W: Write + Seek> MuxPipeline<W> {
pub fn new(writer: W, streams: &[Stream]) -> Result<Self>;
pub fn feed(&mut self, ts_data: &[u8]) -> Result<()>;
pub fn finish(self) -> Result<()>;
}
```
## File Layout
```
libfreemkv/src/
├── mux/
│ ├── mod.rs MuxPipeline (glue)
│ ├── ts.rs BD-TS demuxer (192-byte packets)
│ ├── ebml.rs EBML primitives (write variable-length ints)
│ ├── mkv.rs MKV muxer (Segment, Tracks, Clusters)
│ └── codec/
│ ├── mod.rs CodecParser trait
│ ├── h264.rs H.264 NAL parser
│ ├── hevc.rs HEVC NAL parser
│ ├── ac3.rs AC3/EAC3 frame parser
│ ├── dts.rs DTS frame parser
│ ├── truehd.rs TrueHD/Atmos parser
│ ├── lpcm.rs LPCM frame parser
│ └── pgs.rs PGS subtitle parser
```
## MKV Codec IDs
| Our Codec | MKV CodecID | codecPrivate |
|-----------|------------|--------------|
| H264 | V_MPEG4/ISO/AVC | AVCDecoderConfigurationRecord (SPS+PPS) |
| Hevc | V_MPEGH/ISO/HEVC | HEVCDecoderConfigurationRecord (VPS+SPS+PPS) |
| Vc1 | V_MS/VFW/FOURCC | BITMAPINFOHEADER |
| Mpeg2 | V_MPEG2 | sequence_header |
| Ac3 | A_AC3 | none |
| Ac3Plus | A_EAC3 | none |
| TrueHd | A_TRUEHD | none |
| DtsHdMa | A_DTS | none (core + extension) |
| Dts | A_DTS | none |
| Lpcm | A_PCM/INT/BIG | none |
| Pgs | S_HDMV/PGS | none |
## Timestamps
BD uses 90kHz PTS/DTS. MKV uses nanoseconds.
Conversion: `ns = pts * 1_000_000_000 / 90_000` = `pts * 100_000 / 9`
MKV TimestampScale default = 1,000,000 (1ms precision).
For BD content, 1ms is sufficient.
## Build Order
1. `ebml.rs` — EBML write primitives (smallest, no dependencies)
2. `ts.rs` — BD-TS demuxer (parse 192-byte packets, PES assembly)
3. `codec/ac3.rs` — simplest codec parser (fixed syncword)
4. `mkv.rs` — MKV muxer (header, tracks, clusters, blocks)
5. `mux.rs` — pipeline glue
6. Test with AC3-only stream (simplest case)
7. `codec/h264.rs` — video parser (NAL units, SPS/PPS)
8. Full BD rip test (video + audio + subs)
9. Remaining codecs (HEVC, DTS, TrueHD, PGS, LPCM, VC-1)

201
docs/rip-recovery.md
View file

@ -1,201 +0,0 @@
# Rip recovery — three-layer architecture
`libfreemkv` supports a multi-stage rip model for damaged or protection-bearing
discs: a fast forward sweep that tolerates read failures, in-loop request-size
adaptation that survives transient drive trouble without bailing, and targeted
retry passes against a persistent bad-range map. The stream pipeline
(`DiscStream` + `input`/`output`) operates against the resulting ISO image, so
the mux stage never touches the drive.
Recovery is layered cleanly. Each layer has one responsibility and does not
reach into the others.
| Layer | Where it lives | What it does |
|-------|---------------|--------------|
| 1 — Bad-range retry | `Disc::patch` (one pass over the mapfile per call) | Re-reads non-`+` ranges with the long timeout. Idempotent; caller invokes N times. |
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
The library exposes flat verbs; the caller drives the multipass loop. Autorip
runs `Disc::sweep` once, then loops `Disc::patch` until either the mapfile is
clean or the configured retry budget is exhausted, then hands the ISO off to
the mux pipeline. The `freemkv` CLI does the same shape with a
terminal-output progress sink. Layer 3 runs inside any consumer of
`DiscStream` (direct PES pipeline, ISO playback, etc.) without caller
involvement.
Three primitives compose the disc-side flow:
| Primitive | What it does |
|---------------------------|-----------------------------------------------------------------------|
| `Disc::sweep` | disc → ISO, one forward pass. Writes a sidecar `.mapfile`. Opt-in skip-on-error. |
| `Disc::patch` | Re-reads bad ranges from the drive. One pass per call; caller invokes N times. |
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
## Data model
### Mapfile
Format: [ddrescue](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)-compatible
plain text, greppable, tool-interoperable. Flushed to disk on every `record()`
so a crashed rip loses at most one block.
```
# Rescue Logfile. Created by libfreemkv v0.13.6
# Current pos / status / pass / pass_time
0x000000000 ? 1 0
# pos size status
0x000000000 0x12a35d000 +
0x12a35d000 0x000003000 -
0x12a360000 0x009c4a000 +
0x12d00a000 0x000064000 *
```
Status characters match ddrescue:
| Char | Meaning |
|------|----------------------------------------------------|
| `?` | Not yet attempted |
| `*` | Fast-pass failed; needs edge-trim |
| `/` | Trimmed; interior needs sector scrape |
| `-` | Unreadable this session |
| `+` | Finished (good) |
Position and size are hex byte offsets into the ISO.
### `SweepOptions` and `PatchOptions`
The library no longer dispatches between sweep and patch internally — the
caller picks the verb explicitly per pass. The two option structs are flat
and have no overlap:
```rust
SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true, // damage-jump + zero-fill on read failure
progress: Some(&reporter),
halt: Some(flag),
}
PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true, // walk bad ranges high → low LBA
wedged_threshold: 50,
progress: Some(&reporter),
halt: Some(flag),
}
```
Caller-orchestrated dispatch (the policy `Disc::copy` used to embed):
- No mapfile → `sweep` (fresh Pass 1).
- Mapfile with `?` ranges → `sweep` with `resume: true`.
- Mapfile covers full disc, only `*` / `/` / `-` ranges → `patch`.
- Mapfile clean → done; no further pass needed.
Each consumer (autorip, `freemkv` CLI) implements the loop in roughly five
lines of `Mapfile::stats()` checks.
## Algorithm
### Pass 1 — fast sweep (`Disc::sweep`)
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
2. On success: write data to ISO, mark `+`, advance.
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
→ **damage-jump**: skip ahead by `256×batch×multiplier` sectors (8 MB base for
UHD). Double the multiplier on each jump (8→16→32→64 MB...). Zero-fill the gap as `*`.
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
7. Only transport failures (USB bridge crash) abort the pass.
Pass 1 completes when every byte has been visited (either `+` or `*`).
### Pass 2+ — patch (`Disc::patch`)
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
(walks ranges from highest LBA to lowest, within each range from end to start).
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
does its own ECC recovery inside that window.
2. On success: write the good bytes into the ISO, mark `+`.
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
4. On failure with marginal sense: mark `-`, continue.
5. Update the mapfile after every block — crash-safe resume.
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
When a consumer reads a `DiscStream` directly (no ISO intermediate),
`fill_extents` runs an adaptive sizer in front of `Drive::read`:
1. Try the current preferred batch size (e.g. 32 sectors, one BD ECC block).
2. On failure: halve the batch and retry at the same LBA. Emit
`EventKind::BatchSizeChanged { reason: Shrunk }`.
3. On a clean-read streak: probe back up toward the preferred size. Emit
`EventKind::BatchSizeChanged { reason: Probed }`.
4. If a single-sector read fails: skip (zero-fill, emit
`EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return
`Err(DiscRead)`.
This is layer 3. It exists so a transient single-sector glitch in a 32-sector
batch can be isolated and read individually without the caller needing to
implement retry logic.
## Design choices
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset,
no eject cycle. The `recovery` flag controls only the per-CDB timeout
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
was removed in 0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale:
the inline reset on the LG BU40N (Initio USB-SATA bridge)
wedged drive firmware below the bridge without ever recovering a sector,
and the gentle-retry phase produced long stretches of 0 KB/s with no
recoveries to show for it. Recovery responsibility is now layered: layer 1
handles ranges, layer 3 handles request size, neither touches the
wedge-prone reset path.
**No `MODE SELECT` to disable drive retries.** Research showed neither ddrescue
nor MakeMKV does this. Drive firmware has access to raw analog signal, laser
power control, and drive-specific ECC tuning that userspace can't replicate —
disabling it throws away recovery headroom on marginal sectors. We fail fast
via short SG_IO timeouts in pass 1 and let the firmware work the long timeout
in pass 2 / patch.
**No SCSI reset from any retry path.** `SgIoTransport::reset` (Linux) is
trimmed to a kernel SG_IO state flush plus ALLOW MEDIUM REMOVAL — the
`SG_SCSI_RESET` ioctl and STOP/START UNIT escalation were removed in 0.13.6.
The macOS reset (which had been a no-op) was removed entirely. The top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
also removed (no callers). The remaining `Drive::reset()` is only invoked
explicitly by callers that need an eject-cycle escape hatch — it is never
reached from a read path.
**ISO intermediate, even for single-pass.** Pass 1 always writes an ISO. The
mux stage reads the ISO via `IsoSectorReader`. For single-pass (no retries),
this adds ~2-3 min (local disk mux) but gains resumability across crashes,
re-muxability without re-ripping, and a persistent forensic artifact. Callers
who need pure speed can bypass and use `DiscStream::new(Box::new(drive), …)`
directly — the lib doesn't forbid it, and layer 3 (adaptive batch halving)
still applies there.
**Mapfile in ddrescue format.** Plain text so users can `less` it, `diff` it,
or feed it to ddrescue's own tooling. Crash-safe (flush-per-record). Entries
coalesce on adjacent same-status ranges so files stay small.
**Patches target `-`, `*`, `/`, and `?` alike.** The status state machine is
ddrescue's but `patch` collapses the distinction — it just tries every
non-finished range with the long timeout. Future work can specialize (trim vs.
scrape vs. retry with direction reversal) if there's measured benefit.
## References
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/sweep.rs`](../src/disc/sweep.rs) (`Disc::sweep`), [`src/disc/patch.rs`](../src/disc/patch.rs) (`Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).

8
docs/udf.md
View file

@ -120,14 +120,6 @@ Each directory read involves two sector reads: one for the ICB, then one or more
`read_file()` reads a file by navigating the directory tree, reading the file's ICB to get its data extent, then reading the data sector by sector from the **physical partition** (partition_start + LBA, not metadata_start).
## Buffered Sector Reads
USB optical drives have ~500ms round-trip latency per SCSI command. Since `read_filesystem()` and `read_file()` issue one SCSI READ per sector, a full disc scan can require hundreds of commands -- taking 10+ minutes on USB.
`Disc::scan()` wraps the drive in a `BufferedSectorReader` before reading. On a single-sector read, the buffer prefetches a batch of sectors (sized from the kernel's `max_hw_sectors_kb` for the device) and caches them. Subsequent reads to nearby LBAs return from cache with zero SCSI overhead. After parsing the UDF directory structure, the entire metadata partition is pre-read into the cache, so all ICB lookups during title scanning and encryption resolution are instant.
The buffer is transparent -- `read_filesystem()`, `read_file()`, and all downstream code still call `read_sectors(lba, 1, buf)` as before. The batching happens inside the `SectorSource` implementation.
### UDF Filename Encoding
UDF filenames use a compression ID as the first byte:

Some files were not shown because too many files have changed in this diff Show more