Matthew Jackson e3676e7cdf Build BlockGroups in memory so the hot path never seeks
Every BlockGroup frame back-patched its element size via ebml::end_master, which
does two stream_position() calls and two real seeks. BufWriter does not override
Seek::stream_position, so each position query is seek(Current(0)) = flush_buf +
lseek — and each of those flushed the 4 MiB BufWriter while every position-moving
seek reset WritebackPipeline::last_flush_pos. The buffer never got to do its job.

An MPEG-2 title takes this path for EVERY frame (the parser stamps a per-frame
duration, so I, P and B all become BlockGroups) — roughly 350,000 per feature.

A BlockGroup's size is knowable before writing, so there is no need to back-patch
at all. New seek-free twins start_master_buf / end_master_buf patch a placeholder
by buffer INDEX instead of file offset, and build_block_group assembles the whole
element into a persistent buffer that write_block_group and its MVC sibling take,
fill, write once, and hand back — including on the error path — so the allocation
is made once rather than per frame.

Measured with a counting writer that, like BufWriter, does not override
stream_position, over 200 frames:

              seek calls   position-moving
  plain   before 912              451
  plain   after  112               51
  MVC     before 2516            1253
  MVC     after  116               53

Per-frame cost is now zero; the residual is the header, the per-cluster
back-patch and Cues. For 350k BlockGroups that is 1.4M seek calls removed on the
plain path, 4.2M on an MVC title.

end_master is deliberately NOT changed for its other callers. The Cluster master
genuinely streams — frames are appended to an open cluster over time, so its body
cannot be buffered without holding a whole cluster in memory — and the rest
(EBML header, Tracks, Info, Chapters, Cues) run once, not per frame.

Byte-identity is the safety property, and it is structural: end_master always
patches a FIXED-width 8-byte VINT, and the buffered pair writes and patches
exactly that same placeholder, so the encodings cannot differ. Verified by
capture-then-compare over 200 frames (17 keyframes / 183 non-keyframes, multiple
clusters, BlockDuration present and absent, both reference branches) — output
byte-for-byte identical.

Three tests now pin it permanently: buffered vs seeking output byte-for-byte for
empty/tiny/multi-byte bodies, the same for NESTED masters (the MVC path nests
BlockAdditions > BlockMore, where an index-arithmetic slip would surface), and
end_master_buf erroring rather than panicking on a position outside the buffer.
All three verified red against a deliberately divergent placeholder width.

A note on verifying this kind of change: comparing emitted MKV across two
worktrees at different commits shows a spurious 14-byte difference, because
MuxingApp/WritingApp embed the build's git SHA twice. Compare at the same base.
2026-07-29 19:33:31 -07:00
2026-04-16 17:51:45 +00:00

License: MIT

libfreemkv

Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Drive-level unlocking is handled internally; consumers work with disc access and decryption only.

DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require a keydb.cfg (default ~/.config/freemkv/keydb.cfg) supplying disc-specific volume unique keys; no AACS key material is compiled in.

12+ MB/s sustained read speeds on BD. Drive prep (init()) handles unlocking internally via the freemkv-unlock crate — clients never see it; when no drive unlock applies, the library rips via the host-certificate AACS handshake.

Multi-lingual by design — the library outputs structured data and numeric error codes, never English text. Build any UI or localization on top.

Source & API · Technical Docs

Part of the freemkv project.

Install

Consumed by git tag (not published to crates.io):

[dependencies]
libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "vX.Y.Z" }

Quick Start

use libfreemkv::{Drive, Disc, ScanOptions};
use std::path::Path;

// Open drive — identified via INQUIRY
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?;              // wait for disc
drive.init()?;                     // unlock + prep (handled internally)
drive.probe_disc()?;               // probe disc surface for optimal speeds

// Scan disc — UDF, playlists, streams, AACS (all automatic)
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;

for title in &disc.titles {
    println!("{}{} streams", title.duration_display(), title.streams.len());
}

// Stream pipeline — read PES frames from any source, write to any output
let opts = libfreemkv::InputOptions::default();
let mut input = libfreemkv::input("iso://Disc.iso", &opts)?;
let title = input.info().clone();
let mut output = libfreemkv::output("mkv://Movie.mkv", &title)?;
while let Ok(Some(frame)) = input.read() {
    output.write(&frame)?;
}
output.finish()?;

Multi-pass recovery rip

Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the ddrescue mapfile, damage classification and the multipass loop now live in the freemkv-engine crate as freemkv_engine::recovery::{copy, sweep, patch}.

libfreemkv keeps the layers underneath: the raw single-shot read (Drive::read) and the SCSI-fact translation (SenseFamily) that the engine's strategy is built on. The dependency runs engine → libfreemkv, so this crate cannot call into it; front-ends get recovery from the engine directly. See docs/rip-recovery.md for what stayed here.

What It Does

  • Drive access — open, identify, internal unlock + prep, speed control, eject
  • 12+ MB/s reads — auto-detects kernel transfer limits, sustained full speed
  • Disc scanning — UDF 2.50 filesystem, MPLS playlists, CLPI clip info
  • Stream labels — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
  • AACS decryption — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
  • KEYDB updates — download, verify, save from any HTTP URL (zero deps, raw TCP)
  • Content reading — adaptive batch reads with automatic decryption
  • Stream I/O — unified stream pipeline for reading and writing any format

Streams

Stream Input Output Transport
DiscStream Yes -- Optical drive via SCSI
IsoStream Yes -- Blu-ray ISO image file (read via stream pipeline; written by freemkv_engine::recovery)
MkvStream Yes Yes Matroska container
M2tsStream Yes Yes BD transport stream with FMKV metadata header
NetworkStream Yes (listen) Yes (connect) TCP with FMKV metadata header
StdioStream Yes (stdin) Yes (stdout) Raw byte pipe
NullStream -- Yes Discard sink (byte counter for benchmarks)

Streams implement a single unified pes::Stream trait (re-exported as PesStream) exposing read() and write() on one type. input() / output() resolve URL strings to PES stream instances. All URLs use the scheme://path format — bare paths are rejected.

Keys

DVDs (CSS) decrypt out of the box, with no external key file needed.

Blu-rays and UHD (AACS) require a keydb.cfg at ~/.config/freemkv/keydb.cfg (or passed via ScanOptions). No AACS key material is compiled into the binary.

Architecture

Drive                  — open, identify, init, single-shot read
  ├── ScsiTransport    — SG_IO (Linux), IOKit (macOS), SPTI (Windows)
  └── unlock_bridge    — private seam to the freemkv-unlock crate
                         (firmware / AACS cert / CSS bus-auth unlockers)

Disc                   — scan titles, streams, AACS/CSS state
  ├── UDF reader       — Blu-ray UDF 2.50 with metadata partitions
  ├── MPLS parser      — playlists → titles + clips + streams
  ├── CLPI parser      — clip info → EP map → sector extents
  ├── IFO parser       — DVD title sets, PGC chains, cell addresses
  ├── Labels           — 5 BD-J format parsers (detect + parse)
  ├── AACS             — key resolution + content decryption
  ├── CSS              — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
  └── KEYDB            — download + verify + save

Streams                — unified PES pipeline
  ├── PesStream        — pes::Stream: one trait, read()/write() PES frames
  ├── DiscStream       — sectors → decrypt → TS demux → PES
  ├── IsoStream        — ISO file → decrypt → TS demux → PES
  ├── MkvStream        — MKV mux/demux
  ├── M2tsStream       — BD transport stream
  ├── NetworkStream    — TCP with FMKV metadata header
  ├── StdioStream      — stdin/stdout pipe
  └── NullStream       — discard sink

See docs/ for detailed technical documentation on each module.

Error Codes

All errors are structured with numeric codes. No user-facing English text — applications format their own messages.

Range Category
E1xxx Device errors (not found, permission)
E2xxx Profile errors (unsupported drive)
E3xxx Unlock errors (failed, signature)
E4xxx SCSI errors (command failed, timeout)
E5xxx I/O errors
E6xxx Disc format errors
E7xxx AACS errors
E8xxx KEYDB update errors
E9xxx Stream / mux errors (URL, PES, ISO, pipeline, demux)

Platform Support

Platform Status Backend
Linux Supported SG_IO ioctl
macOS Supported IOKit SCSITask
Windows Supported SPTI

Contributing

Run freemkv info disc:// --share with the freemkv CLI to capture your drive's identity for contribution. Drive-unlock profiles are maintained in the freemkv-unlock repository.

License

MIT

S
Description
Internal mirror of libfreemkv 1.1-beta HEAD for private kdb builds (tag drops flate2; kdb needs HEAD symbols)
Readme MIT
13 MiB
Languages
Rust 99.6%
C 0.3%
Shell 0.1%