matthew 41d4c6b464 io/writeback: medium-agnostic by construction, no DONTNEED, single detection point
Three coupled changes that together meet the "stable max throughput
on any medium" bar:

1. Remove `is_nfs` from `skip_wait`. WAIT_AFTER runs on every medium
   now. The `bounded_syscall` 30s safety net (already in place) covers
   the wedged-FS case generically — no need to predict NFS-hangs at
   compile time. Re-applies the 0.21.12 fix that landed the flat
   25 MB/s on NFS in the first place.

2. Drop the `posix_fadvise(DONTNEED)` calls after WAIT_AFTER. Bounded
   *dirty* pages (the actual invariant) does not require evicting
   *clean* pages. DONTNEED was forcing read-modify-write on any
   in-window seek-then-write — exactly the pattern matroska
   cluster-size backpatches produce. Empirical 2026-05-15: write side
   of mux fed NFS at 43 MB/s while the file grew at 25 MB/s, an 18
   MB/s overhead almost certainly composed of those RMW cycles. The
   kernel reclaims clean pages under LRU when memory is actually
   needed — we don't have to ask.

3. Replace `detect_nfs(fd) -> bool` with `detect_storage_class(fd)
   -> (StorageClass, chunk_bytes_seed)`. Medium detection happens
   *once* at construction and produces *one* output: the initial
   `chunk_bytes` seed for the autotuner. NFS gets 64 MiB (commit ack
   ~10-30 ms; needs bigger chunks to amortize); other media use the
   caller's hint. No hot-path branches on medium. The autotuner drives
   all subsequent decisions from measured WAIT_AFTER p95 latency,
   identically on every (OS, FS) combination.

Module-level doc rewritten to match: no more "NFS escape hatch", no
"is_nfs" framing. The whole writeback module is now medium-agnostic
except for one labelled detection point.
2026-05-16 11:17:30 -07:00
2026-05-13 19:15:48 -07:00
2026-05-16 10:12:34 -07:00
2026-04-16 17:51:45 +00:00

Crates.io docs.rs License: AGPL-3.0

libfreemkv

Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Bundled drive profiles — no external files needed.

12+ MB/s sustained read speeds on BD. Full init: unlock, firmware upload, speed calibration — all from pure Rust.

Multi-lingual by design — the library outputs structured data and numeric error codes, never English text. Build any UI or localization on top.

API Documentation · Technical Docs

Part of the freemkv project.

Install

[dependencies]
libfreemkv = "0.18"

Quick Start

use libfreemkv::{Drive, Disc, ScanOptions};
use std::path::Path;

// Open drive — profiles are bundled, auto-identified
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?;              // wait for disc
drive.init()?;                     // unlock + firmware upload
drive.probe_disc()?;               // probe disc surface for optimal speeds

// Scan disc — UDF, playlists, streams, AACS (all automatic)
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;

for title in &disc.titles {
    println!("{}{} streams", title.duration_display(), title.streams.len());
}

// Stream pipeline — read PES frames from any source, write to any output
let opts = libfreemkv::InputOptions::default();
let mut input = libfreemkv::input("iso://Disc.iso", &opts)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
    output.write(&frame)?;
}
output.finish()?;

Multi-pass recovery rip

For damaged discs the library exposes two flat verbs — Disc::sweep for the forward Pass 1 and Disc::patch for retrying bad ranges. The library never loops; the multipass policy is the caller's job. See docs/rip-recovery.md and the design notes in freemkv-private/memory/0_18_redesign.md.

use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;

let iso = Path::new("disc.iso");

// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
    decrypt: true,
    resume: false,
    batch_sectors: None,
    skip_on_error: true,
    progress: None,
    halt: None,
})?;

// Pass 2..N: retry every non-finished range. Idempotent.
loop {
    let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
    let stats = map.stats();
    if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }

    let outcome = disc.patch(&mut drive, iso, &PatchOptions {
        decrypt: true,
        block_sectors: None,
        full_recovery: true,
        reverse: true,
        wedged_threshold: 50,
        progress: None,
        halt: None,
    })?;
    if outcome.bytes_recovered_this_pass == 0 { break; }
}

// Mux from the ISO via the normal stream pipeline (no drive involvement).

What It Does

  • Drive access — open, identify, unlock, firmware upload, speed calibration, eject
  • 12+ MB/s reads — auto-detects kernel transfer limits, sustained full speed
  • Disc scanning — UDF 2.50 filesystem, MPLS playlists, CLPI clip info
  • Stream labels — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
  • AACS decryption — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
  • KEYDB updates — download, verify, save from any HTTP URL (zero deps, raw TCP)
  • Content reading — adaptive batch reads with automatic decryption
  • Stream I/O — unified stream pipeline for reading and writing any format

Streams

Stream Input Output Transport
DiscStream Yes -- Optical drive via SCSI
IsoStream Yes -- Blu-ray ISO image file (read via stream pipeline; written via Disc::sweep())
MkvStream Yes Yes Matroska container
M2tsStream Yes Yes BD transport stream with FMKV metadata header
NetworkStream Yes (listen) Yes (connect) TCP with FMKV metadata header
StdioStream Yes (stdin) Yes (stdout) Raw byte pipe
NullStream -- Yes Discard sink (byte counter for benchmarks)

Streams implement FrameSource (read) and/or FrameSink (write); direction is type-checked. input() / output() resolve URL strings to PES stream instances. All URLs use the scheme://path format — bare paths are rejected.

AACS decryption requires a KEYDB.cfg file. If available at ~/.config/aacs/KEYDB.cfg or passed via ScanOptions, the library handles everything — handshake, key derivation, and per-sector decryption — without the application needing to know anything about encryption.

Architecture

Drive                  — open, identify, init, unlock, single-shot read
  ├── ScsiTransport    — SG_IO (Linux), IOKit (macOS), SPTI (Windows)
  ├── DriveProfile     — per-drive unlock parameters (bundled)
  └── PlatformDriver   — MediaTek (supported), Renesas (planned)

Disc                   — scan titles, streams, AACS/CSS state
  ├── UDF reader       — Blu-ray UDF 2.50 with metadata partitions
  ├── MPLS parser      — playlists → titles + clips + streams
  ├── CLPI parser      — clip info → EP map → sector extents
  ├── IFO parser       — DVD title sets, PGC chains, cell addresses
  ├── Labels           — 5 BD-J format parsers (detect + parse)
  ├── AACS             — key resolution + content decryption
  ├── CSS              — DVD CSS cipher (table-driven, no keys needed)
  └── KEYDB            — download + verify + save

Streams                — unified PES pipeline
  ├── FrameSource      — read() PES frames (direction-typed)
  ├── FrameSink        — write() PES frames (direction-typed)
  ├── DiscStream       — sectors → decrypt → TS demux → PES
  ├── IsoStream        — ISO file → decrypt → TS demux → PES
  ├── MkvStream        — MKV mux/demux
  ├── M2tsStream       — BD transport stream
  ├── NetworkStream    — TCP with FMKV metadata header
  ├── StdioStream      — stdin/stdout pipe
  └── NullStream       — discard sink

See docs/ for detailed technical documentation on each module.

Error Codes

All errors are structured with numeric codes. No user-facing English text — applications format their own messages.

Range Category
E1xxx Device errors (not found, permission)
E2xxx Profile errors (unsupported drive)
E3xxx Unlock errors (failed, signature)
E4xxx SCSI errors (command failed, timeout)
E5xxx I/O errors
E6xxx Disc format errors
E7xxx AACS errors
E8xxx KEYDB update errors

Platform Support

Platform Status Backend
Linux Supported SG_IO ioctl
macOS Supported IOKit SCSITask
Windows Supported SPTI

Contributing

Run freemkv info disc:// --share with the freemkv CLI to contribute your drive's profile.

License

AGPL-3.0-only

S
Description
Internal mirror of libfreemkv 1.1-beta HEAD for private kdb builds (tag drops flate2; kdb needs HEAD symbols)
Readme MIT
9.7 MiB
Languages
Rust 99.6%
C 0.3%
Shell 0.1%