The 1.6.1 DVD fix removed the USE of the loose predicate but left the
predicate sitting there with the better name. Anyone asking "is this sector
scrambled?" finds is_scrambled before is_scrambled_pack, and reintroduces the
defect that destroyed 1912 bytes of a real VIDEO_TS.IFO — the sector carrying
TT_SRPT — so the disc enumerated 38 titles and an image decrypted from it
enumerated 10, silently, at exit 0.
Byte 0x14 only means "scrambling control" inside an MPEG-2 pack. In an IFO,
UDF or ISO 9660 sector it is whatever that format stores there. The new name
says what the function actually tests and nothing more, so the honest question
has the obvious name and the dangerous one has to be asked for deliberately.
Its doc comment also claimed a caller, decrypt::decrypt_sectors, that does not
exist — so the name was an invitation and the documentation was an argument
for accepting it. It has no production callers at all; it stays public because
an integration test asserts the flag extraction directly.
css::is_scrambled reads bits 4-5 of byte 0x14 and nothing else. That is a
sound test once a caller has committed to a title's VOB data, where every
sector is an MPEG-2 PS pack and byte 0x14 always means what it says.
descramble_region is not such a caller: it is handed arbitrary regions of a
disc, so it also sees IFO, UDF and ISO 9660 sectors — raw structures where
byte 0x14 is whatever that format happens to store there.
Measured on a real disc: the second sector of VIDEO_TS.IFO holds 0x15 at
offset 0x14 while starting 00 26 00 00, which is not a pack. The flag test
read it as scrambled, descrambled it, and destroyed 1912 of its 2048 bytes.
That sector carries TT_SRPT, so the title table went with it — the disc
enumerated 38 titles and an image decrypted from it enumerated 10, silently,
at exit 0.
is_scrambled_pack already existed with the right predicate. Use it here. It
costs nothing: a genuinely scrambled VOB sector always carries the pack start
code, and no IFO sector does.
Verified end to end — the decrypted image's `info` output is now identical to
the source disc's, 38 titles both, differing only in the CSS: Encrypted line.
The fixtures moved with it. Four of them built a sector by setting byte 0x14
alone, which no real scrambled sector looks like; they now build packs.
The `fvi://` arm of `output()` passed the destination `.fvi` path as
`FviSink::create`'s `source_path`, so every index named itself as its
own source. `SourceInfo::default()` supplied the rest, making
`source.medium` always "file" and `source.title` always 0 — three
header members wrong, where FVI_FORMAT.md §6.2 defines `source` as
describing the input.
Beyond the wrong data, it made the output unreproducible: two machines
indexing identical bytes emitted different files purely from where
they wrote them, and a local filesystem path leaked into a shareable
file.
`output()` cannot see the source, so thread the provenance down from
the driver, which can: `mux_stream` derives a `SourceInfo` per
`MuxInput` arm and passes it through `drive_mux` to `output()`. Per the
one-method-per-action rule this is a signature change, not an
`output_with_source()` variant; the parameter is `Option<&SourceInfo>`
so a caller with no provenance declares none rather than back-filling
the destination. `SourceInfo`/`Medium` become public API.
What each arm can honestly reach:
- Session: everything — device path, the caller's title index, the
title's playlist, the scanned volume id.
- Url: the source URL, its scheme's medium, `title_index`, and the
playlist off the opened stream's scanned title.
- Iso: the image path and playlist. The title index is not in
`MuxInput::Iso` (it carries a scanned `DiscTitle`, which has no
index), so it stays 0.
- Live: medium and playlist. The reader is an opaque
`Box<dyn SectorSource>` with no path, and again no title index.
Unreachable members are left empty rather than guessed — the sink
already omits the empty ones.
I broke DVD ripping earlier today and the real-media acceptance gate
caught it on its first full run. Greenland.iso failed with E7013
"Decryption failed"; reverting only this change made it rip clean in 8
seconds. That is a regression I introduced, not a pre-existing defect.
WHAT I GOT WRONG.
Round 9's crypto lens reported that descramble_region "descrambles with
a key it just proved wrong" when the crib check rejects the cached key
and the re-crack also fails. I agreed, and made it Error::DecryptFailed
to match the AACS path, on the reasoning that CSS has no external key
source so a failed crack on a readable sector should never happen.
The premise was wrong. `attack_crib` is a HEURISTIC, not a proof: it
finds a periodic run in the unscrambled header and predicts the run
continues past 0x80. When that prediction does not hold, the crib
reports a mismatch even for a CORRECT key — and the re-crack then fails
BECAUSE the crib was never valid. So crib mismatch plus crack failure is
the signature of a crib false positive, not of a stale key. The cached
key is not proven wrong; it remains the best available evidence, and on
a real DVD it is very probably right. Real discs hit this constantly.
The deeper error was treating "no key" as one thing across schemes. An
AACS unit key either opens a unit or it does not — the Verify-Media-Key
relation decides it, and a wrong key is provable. A CSS title key is
recovered from the data itself by an attack whose success varies sector
by sector, so "the crack failed here" says something about THIS SECTOR's
plaintext, not about the key. Unifying the policy was right for the
schemes that can prove a key wrong. CSS cannot, and I folded it in
anyway.
decrypt_span keeps its shape and the cross-scheme test keeps its two
AACS arms, with CSS now explicitly excluded and the reason stated.
Three tests asserted the wrong behaviour and are corrected, including
one I rewrote earlier today to pin exactly this. Every one of them
passed the whole time the code was broken — because none of them had
ever seen a real disc.
The lesson is the one I kept stating and then did not act on: 3,013 unit
tests, ~400 mutants killed and nine audit rounds did not catch this, and
one acceptance run did. Synthetic media cannot reproduce what a real
disc does.
descramble_region descrambled with the key a sector's own crib had just
proven stale, whenever the re-crack from that sector also failed. The
clear header is not scrambled, so it survives intact: the sector still
opens with a valid pack start and passes every structural check the PS
demuxer applies. Only the payload is corrupted — exactly where nothing
looks. Ok(0) dropped, exit 0.
CSS has no external key source. The title key comes only from cracking
the data, so on a READABLE sector "no key" is not a missing input, it is
recovery failing on bytes we can see. That should never happen, and when
it does the answer is not to emit something.
Now Error::DecryptFailed — the same verdict the AACS path already gives
for a unit no held key opens. Both alternatives to failing are bad data
reported as success: descrambled with a rejected key it is garbage
behind a valid header, and passed through untouched it is ciphertext
where plaintext is meant to be.
WHY IT WAS POSSIBLE, which matters more than the fix:
There is no single place that owns "what do we do when there is no key".
decrypt_sectors_impl looks like the central dispatch, but its AACS arm
is a `return Err` stub — AACS decrypts entirely through
decrypt_sectors_mapped, a separate top-level path. So CSS decided its
own policy inside css/, AACS decided in decrypt.rs and mux/resolve.rs,
and nothing held them to the same answer. The asymmetry was not an
oversight; it was structurally permitted.
How a disc decrypts is one process — resolve a key for this data, apply
it, refuse if it cannot be proven. Only the resolve-and-apply step is
scheme-specific. Filed as a task: the policy belongs in one orchestrator
with the schemes supplying only what genuinely differs.
Two tests changed rather than added, both of which pinned the old
behaviour: the unit test asserted the sector was descrambled, and the
integration test asserted the scramble flag was cleared, which is what
descrambling-with-any-key does. Neither established that the result was
CORRECT — the fourth bad-test shape.
Thirteen agents triaging src/labels and src/disc died on a saturated
machine, leaving 5,836 insertions across 28 files uncommitted in a
worktree. Recovered by 3-way apply onto twelve commits of drift; zero
conflicts. The diff was archived to freemkv-private first, because a
worktree is not a backup and this one had already nearly been lost.
One production change, and it is the right one: mpls_universal::parse
read every playlist off the disc AND converted the entries to labels in
a single function, so the conversion — stream-type mapping, dedup key,
the dense global counters — could only be reached through a synthetic
UDF image. Extracted to build_labels(&[Playlist]), which unit tests can
drive from already-parsed values. Behaviour-preserving: same iteration
order, same skip-on-error.
Two collisions resolved by hand:
A second mod pass_progress_tests, written independently against the
same survivors as the one committed in c610285. Kept mine — it covers
the distinct-counters case and the Progress blanket impl, which theirs
does not — but theirs had three clamp tests mine lacked: good_pct,
bad_pct and pending_pct also clamp an overshoot, and I had only tested
that for work_pct. Merged those in as one test and proved each of the
three clamps load-bearing by removing them individually.
An unused_parens warning in a new fixture.
Method note, recorded because it cost real time: git apply --3way
STAGES its result, so `git diff` reads empty and the tree looks
untouched. I nearly concluded the patch had silently failed. Worse, the
first attempt piped through `head -20`, so `echo exit=$?` reported
head's status rather than git's — the same mistake this audit has
already documented once. Check the real exit status, and check
--cached, not just the working tree.
The worst of the five was a regression suite that never touched the code
it guarded: nine batch-count tests called `safe_batch_count` and
`buggy_batch_count`, both defined in the test file itself. The u16
truncation they exist to prevent could be reintroduced in
sector/prefetched.rs with every one of them green. They now drive the real
producer through the public API, and reinstating the truncation fails five
of the nine. Worth recording that the symptom has changed since the
original fix: the unit-alignment clamp below floors a zero batch at three
sectors, so the bug is now a twenty-fold throughput cliff rather than the
stall it once was.
The MP4 reserve test's only numeric case was dominated by the floor and
the buffer, so BYTES_PER_SAMPLE could be zeroed without failing it. It now
has a case where the per-sample term dominates. The zero-count guard in
FileSectorSource was likewise unfalsifiable — seek-past-EOF and a
zero-length read both succeed — so the test now observes the file cursor.
The AACS media-key ambiguity guard had no test at all; the pool scan is
extracted so the verifier can be injected, because a genuine two-key
collision needs one ciphertext decrypting under two AES-128 keys to
plaintexts sharing a 64-bit magic, which is a 2^64 search and not a
fixture.
macOS implemented the documented cheap, side-effect-free presence probe by
building a full exclusive transport — which force-unmounts the disc. Linux
and Windows issue one TEST UNIT READY with no unmount; macOS was the
outlier. It now walks the IOKit registry for the media object instead.
The C shim's registry reads assumed CoreFoundation types the registry does
not guarantee, so a driver publishing a CFNumber where a CFString was
expected aborted the process from inside public API. Types are checked and
a wrong type treated as absent. The unbounded waitpid on the unmount child
is now a polled deadline, and the last-resort match gained the NULL check
its two siblings already had.
The empty-CDB guard existed only on Linux while a shared helper's comment
claimed all three backends had it. Moved into the helper, so the comment
is now true and macOS and Windows are covered.
One finding was REJECTED with evidence rather than fixed. The TrueHD
buffer-cap test was indeed bogus, but MAX_TRUEHD_BUF turns out to be
unreachable by any input: the parser only retains data when the buffer is
shorter than the declared AU, and that declaration is twelve bits, so the
worst case is 8189 bytes against a 256 KiB cap. An exhaustive sweep over
all 65536 AU headers confirmed it. The fixture now sits at the reachable
ceiling and asserts that instead. The cap itself is left in place as
defence, unreachable by construction, matching how the AC-3 resync guard
was handled earlier in this audit.
Two behaviour changes worth naming: Linux's empty-CDB error becomes
InvalidCdbLength rather than a transport failure, and an unknown device
now reports absent media rather than a not-found error, because the
registry cannot tell an empty drive from a missing one. The latter is a
conflation of the kind this audit has fixed three times; it is recorded
for the next round rather than left silent.
The sweep/patch recovery strategy, mapfile, retry-decision state machine,
section-recover, and damage classification move out of libfreemkv into the
new freemkv-engine crate. libfreemkv keeps the raw single-shot read and
SCSI-fact translation (SenseFamily stays in scsi).
- Delete disc/{sweep,patch,mapfile,read_error,section_recover}.rs, the
Disc::copy/sweep/patch methods, the Copy/Sweep/Patch option+result types,
classify_damage/DamageSeverity, progress_snapshot_from_mapfile, and the
three recovery integration tests.
- Trim public surface the recovery deletion orphaned: delete the dead
READ_PIPELINE_DEPTH const, the write-side SectorSink/FileSectorSink (no
consumer), and the DriveSpeed enum (its one live use — set max drive
speed — becomes Drive::SPEED_MAX_KBPS). Make mapfile_path_for,
decrypt_sectors_mapped pub(crate); gate NoopEvents to test.
- Version 1.6.0.
Round-2 follow-ups to 6d6e60f (inline base-map resolve on the live
single-pass Session/Live mux arms).
Fix 1 (halt threading) — the inline resolve chain sampled ciphertext off
the LIVE drive with no cancel token, so an operator /api/stop during key
resolution was not honored (the FMTS probe can issue hundreds of reads,
each able to stall to the 60s SCSI recovery timeout — violating the
"don't hammer a struggling live drive" rule). Add an optional
`halt: Option<&Halt>` to `resolve_mux_key_map`, `resolve_fmts_key_map`,
`resolve_inline_base_map`, and `Disc::resolve_content_key_map`, and poll
it at each loop boundary (FMTS anchor + per-index probe loops, multi-CPS
extent loop) — returning Err(Halted) promptly. Live/Session arms pass the
driver's halt; sweep/patch pass their own token (via Halt::from_arc);
file-backed probe/ISO callers pass None. Tested with a pre-cancelled halt
(Err Halted, no extent sampling) and a None-halt no-abort case;
mutation-verified (dropping the extent-loop check → Ok, not Err).
Fix 2 (Session-arm coverage) — the MuxInput::Session arm ran the same
resolve→install→decrypt sequence as Live but had NO end-to-end test
(DiscSession only exposed open(), which needs live hardware). Add a
#[cfg(test)] DiscSession::from_parts_for_test (injected reader + scanned
disc, no Drive), an end-to-end AACS decrypt test through the Session arm
(mutation-verified: dropping with_key_map → mux aborts), and a
missing-reader clean-error (not panic) test.
Fix 3 (cleanups) — io_error_code: remove the unreachable typed-Error
downcast branch (From<Error> for io::Error stringifies; no path builds an
io::Error holding a typed Error), keeping the stringify parse is_halt /
is_skippable_title_stub rely on. Add a resolve_keys_for test covering the
largest-title sampling branch. Document the patch wedge-exit coverage gap
(TODO) in passn_handler_ab.rs.
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just
0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements;
only 16-63 are reserved. a reference decoder the spec per-AMODE channel table confirms 10-15 are
decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal
multichannel core frames as undecodable, silencing recoverable audio. Add a
regression test (literal 0..16 range) that fails if the bound reverts to 10.
- keysource: only memoize a NEGATIVE (empty) key-fetch result when every source
genuinely ran and none held the key — never when a source Err'd (network down,
unreachable). A transient outage was being cached as a permanent "no key" for
the fingerprint, permanently dropping a unit that could be recovered once the
source came back. Thread an `errored` flag out of the drivers and gate the
cache insert on it. Tests cover both the recover-after-outage case and that a
genuine absence is still memoized.
- pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS
display sets through the full demux -> parse -> observe -> apply path, both a
forced verdict landing and a non-forced verdict clearing a vendor flag.
- mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD).
- scan_iso test: add independent fixture expectations (volume id) so the parity
test is no longer purely tautological against a re-run of the same composition.
Introduce libfreemkv::scan_iso(path, opts) -> (Disc, Box<dyn SectorSource>),
the file-backed counterpart to DiscSession::scan. It is the single place
that opens a FileSectorSource, reads its capacity, and runs Disc::scan_image,
returning the scanned Disc plus a reusable reader over the same image so
consumers stop hand-rolling that triple.
Add an integration test that materialises a minimal synthetic UDF image to a
real file, asserts scan_iso matches the manual open+scan_image composition,
and confirms the returned reader is still usable (capacity + sector read).
Also covers open-failure and scan-failure error propagation.
Every AACS decrypt now goes through the resolved key map (decrypt_sectors_
mapped): the map keys each content unit up front and a missing key fails at
resolve time. The old trial-decrypt path — try each held key per unit, keep
the first-tried plaintext on a miss — is gone; decrypt_sectors_impl's AACS
arm now fails loud (reaching it means a reader was built without its map,
which would silently apply a wrong key). CSS (self-descramble) and the clear
no-op path are unchanged.
Disc::sweep and Disc::patch resolve a whole-disc key map up front for a
decrypting pass (the fetch secures any missing CPS-unit key, fail-loud) and
decrypt via the map — clear nav/filesystem sectors are in no range and pass
through, so the separate content-range gate and the reactive per-unit
key-fetch recovery are no longer needed. extract_tree keys every unit with
the base Unit Key through the map (its encrypted-flag gate skips clear
files). Multipass sweeps stay --raw.
Removes the obsolete non-mapped-AACS trial/gate/recovery tests (the mapped
path and resolve fail-loud are tested directly).
Every DVD read path — the file-backed mux highway (build_iso_pipeline) and
the live-drive single-pass DiscStream — now resolves the per-VTS CSS title
key through one shared step, css::resolve_dvd_title_key, cracked keylessly in
playback order from the title's own extents. Removes the earlier design that
reused a single scan-time key (meaningless for a per-VTS scheme) and muxed a
detection-miss disc's scrambled sectors as garbage.
- Disc::scan no longer cracks a key up front; it does only the CSS bus-auth
read-unlock, hoisted before the UDF prefetch so scrambled small/menu VOBs
no longer cost a rejected read each (CSS-DVD scan ~25s -> ~6s).
- An uncrackable title hard-fails (E7023) instead of passing ciphertext as
plaintext; --raw skips the crack entirely; a Stop mid-crack surfaces Halted.
- DiscStream::new is now fallible and threads raw + halt.
- Fix a stale codec-parser doc claim (TrueHD/FLAC/MP2/AAC do gate via DropTally).
One method per action (CLAUDE.md), not a foo_with_X pair — create_at was the real
constructor and create was a None wrapper. Callers pass Option<&Path> directly.
The keyserver protocol now returns all 32 index keys as an array for a
forensic content sample (and a single-element array for plain content).
resolve_fmts_key_map sends one forensic batch and maps array element i to
segment index i+1, replacing the per-index blind-probe collection loop
that repeatedly hit the key service. Segment/index parsing and the
aligned-unit content classification are reworked to support this:
- rename variant_select -> index_select (per-index, not per-variant)
- content classification moves to is_clean(buf, ContentFormat) so the
unit selector emits only units the key service accepts
- segment.rs: parse IndividualSegment.tbl index tags + SPN ranges,
build contiguous LBA key ranges from the resolved 32-key array
- decrypt/decorator plumbing for the resolved per-index keys
Fail loud (FmtsKeyMissing) when the forensic query returns < 32 keys or
any segment index stays unresolved.
decrypt:
- decrypt_sectors is now a pure decrypt (apply key, leave plaintext, report
unverified bytes); TS-structure is a separate primitive (is_clean_ts/ps) used
only for key selection and read-verify. The mux passes decrypted bytes through
(the demuxer drops non-conforming packets), ending the NULL-TS conceal loop and
the per-unit key-server refetch storm. Key-proof floor replaces the 75%
supermajority.
recovery:
- Removed the post-read decrypt-verify gate (verify.rs) that mis-aligned the
disc-absolute unit grid against clip-anchored AACS units and false-failed good
clips (e.g. Dunkirk's orphan-CPS clip). Bad sectors are marked by physical read
result; decryptability is proven at scan + mux time.
HD DVD (first-class AACS):
- Role-based candidate-list file sourcing so an HD DVD's /ANY!/ files
(MKBROM.AACS, VTKF000.AACS, CONTENT_CERT.AACS) are found with no disc-type
branch. parse_vtkf parses VTKF000.AACS into the same UnitKeyFile as a BD
Unit_Key_RO.inf, so the shared VUK unwrap applies unchanged. set_unit_base
clip-anchoring. Two decrypt-axis assumptions remain UNVERIFIED-HDDVD-DECRYPT
(no encrypted disc to test).
mux:
- MVC (Blu-ray 3D) track signals unified into one MVCDecoderConfigurationRecord;
release-safe track_vint (3-byte VINT) and pid_index (i32) guards.
hardening:
- Container-aware is_clean / encryption detection; bytes_bad_in_title fail-safe
on a corrupt mapfile; CSS crack gated on DiscFormat::Dvd (HD DVD excluded);
non-vacuous CSS tests; patch NOT_READY/HARDWARE/ILLEGAL_REQUEST/ABORTED
sense-path tests.
The facade removal (5ff0464) left tests/ calling aacs::disc_hash,
aacs::decrypt_unit, aacs::AacsVersion, etc. at the old flat paths,
so the branch did not compile its integration tests. Repoint each
to its real module (inf/derive/content/mkb). No logic change.
The tier-2 marginal specialists now additionally probe permanently-bad
residual sectors before leaving them NonTrimmed, so the alternating-bad
profile's finite read count rose past the old tier-0/1 bound. Guard
still catches runaway; behavior asserts unchanged.
Pass-N recovery is now a chain of time-bounded recovery handlers instead
of one monolithic per-range loop that could grind the front of a bad
range for up to 30 min, wedge the drive, and abort the whole pass.
A bad range is a SubRanges set; recovery is an ordered list of
SectionHandlers (Linear{reverse,fast} covering back/forward x fast/slow,
and Bisect). A coordinator runs each handler with a hard per-handler
deadline: a handler recovers what it can (removing it from the still-bad
set) and hands the rest to the next handler; whatever is still bad after
the chain becomes NonTrimmed residue and we move on to the next range.
Guarantees, now structural rather than bolted-on:
- never hangs: every handler is deadline-bounded; the loop always drains
to recovered-or-residue.
- always moves on: a range that cannot be finished leaves residue and
advances; only a genuine transport fault or user halt ends the pass.
- extensible: a new recovery idea is one SectionHandler impl added to the
chain; a proven-ineffective one is removed. The engine never changes.
Removes ~1.9k lines of the old inner loop (watchdogs, skip escalation,
NOT_READY grind, wedge counters) and their tests. fast_capture is now
inert (the chain supersedes it); breadth-first ordering becomes a future
scheduler concern. New module: disc/section_recover.rs (8 fixture tests,
injectable clock — bounded/never-hang proven without touching a drive).
Two A/B tests updated to the chain's strictly-better recovery counts.
A PatchOptions.fast_capture pass reads each bad range ONCE at the full batch and
leaves every FAILED block NonTrimmed for a later pass — no bisect, no per-sector
grind, no retry. This lets a first retry pass grab the readable blocks (the
sweep's good skip-ahead overshoot) of EVERY section quickly, before any single
section's slow per-sector recovery — instead of grinding section 1 to exhaustion
before even touching section 2. A later pass (fast_capture = false) does the
granular bisect/retry on what's left.
Load-bearing invariant (fixture test): NO data is dropped. A failed block becomes
NonTrimmed (pending, retried by a granular pass), NEVER Unreadable. The test pins
that the readable half of a range recovers, the bad half stays NonTrimmed (not
Unreadable), and the bad block is marked in ONE batch read with zero bisection.
Disc::copy's internal patch keeps fast_capture=false (single-call full recovery).
Live drive probing (cold-single vs in-stream batch vs centered window,
one clean uncached read per sector) showed recovery RATE is identical
across approach: 3/5 every mode. The drive's per-sector ECC is
media-bound and partly stochastic, NOT approach-bound. So scatter
(seek-away recalibration + far-anchor re-read) does not improve recovery
and is dead weight — removed (function, read_good_sectors, SCATTER_*
consts, and its tests).
Add SubRanges: the still-bad (pos,len) sub-ranges of a section with
remove()/split, the work-list foundation for the upcoming per-section
recovery decomposition (recover_section -> stream/bisect/retry phase
helpers). Unit-tested; unused until the decomposition lands.
25 patch + 8 AB profile tests green; precommit clean on Rust 1.86.
The 'reset, read good data, come back for one sector' technique. A
genuinely-damaged sector makes the drive grind its full recovery timeout
and still fail; re-reading consecutive bad LBAs at identical conditions
re-fails AND is the rapid-failure cadence that drops the BU40N into a
firmware fast-fail wedge (CLAUDE.md hard-rule #2).
scatter_recover: on a stuck single sector, make up to SCATTER_MAX_ATTEMPTS
fresh tries, each preceded by read_good_sectors() — a seek to a far
known-good anchor + a sustained read that re-seats the head/servo and
breaks the failure cadence. The fresh re-read uses the FAST timeout
(recovery=false) so a recalibrated marginal sector reads quick and a
truly-dead one fails fast instead of burning 60s per attempt. The seek +
good read IS the settle (matches ddrescue/MakeMKV) — no idle sleep.
Gated to genuine single-sector MEDIUM_ERROR (0x03): transport faults
still abort the pass, NOT_READY keeps its own retry path, wedge-family
senses are left for cooldown/eject. A scatter recovery is recorded
exactly like a normal read success (write-then-mark-Finished); nothing
marks a range good without a successful read + write.
Logs anchor_ms/reread_ms per attempt for live tuning. 5 fixture tests
(recover-marginal, give-up-dead, skip-non-medium, skip-batch, AACS
widen); 2 AB read-count guards raised for scatter's bounded extra reads.
Precommit green on Rust 1.86.
Post-read verify gate (new src/disc/verify.rs): UnitVerifier buffers/aligns the disc-absolute read stream into clip-file 6144-byte units, then makes one decryptability() decision per unit (CPI gate -> held keys -> key_fetch -> strict TS). POST_READ_VERIFY const kill-switch; fail-safe contract (only ever downgrades units it is confident are undecryptable; every doubt skips). Hooked into Disc::sweep (producer observes ciphertext -> WorkItem::MarkBad after the Good, FIFO-ordered) and Disc::patch (post-loop reverify_iso reads recovered units whole from the patched ISO). extract::clip_layouts enumerates AACS clips for the gate.
Standards-correct AACS verify: aacs::unit_is_clean_ts is a strict port of libaacs _verify_ts (all 32 TS syncs, not a majority vote); decrypt_unit accepts a key only on it; the majority verify_ts is removed. Deleted the Disc::verify_clips post-pass bolt-on (its primitive is absorbed by the read-path gate).
libaacs/DVD audit fixes: content-cert bus_encryption flag now read from bit 7 (was bit 0 - defeated the bus-key fail-loud gate); cc_id read from offset 14; title_cps_unit range-validated + 1->0 index-converted per libaacs. Corrected attack_crib ("functionally-equivalent" not "exact" port) and read_disc_key (READ DVD STRUCTURE 0xAD, not REPORT KEY) doc comments.
Also includes accumulated uncommitted work: key-fetch seam and TrueHD/DTS audio fix.
Port + adapt the freemkv native per-picture video index (FVI) from the
old feat/fvi-sink branch onto rc6's codec-agnostic PictureInfo model.
This is a surgical adaptation, not a merge.
Adaptations (fvi_sink.rs, videomap.rs, tests/fvi_pipeline.rs):
- Retarget from the removed crate::mux::codec::mpeg2::PictureInfo (raw
public fields) to rc6's authoritative crate::mux::codec::PictureInfo
in codec/coding.rs, via its accessors.
- type from coding_type() -> CodingType{I,P,B}; emitted for ANY frame
that carries coding (every video codec now fills it), with the
keyframe-flag I/P fallback only when coding is absent.
- Replace the mpeg2-only tff/rff/progressive members with codec-agnostic
members derived through the accessors: field_order (tff/bff/progressive)
and progressive, emitted ONLY when the codec measured the signal
(Option::Some) and omitted otherwise; plus nb_fields.
- Test fixtures rebuilt via PictureInfo::mpeg2(CodingType, Mpeg2Coding{..})
/ coding_type_only(..); added measured_cicp: None to VideoStream
literals for rc6's struct.
Honesty decision (key / random-access):
- The codec-agnostic PictureInfo carries NO GOP-closure (no closed_gop /
gop_start), so key is set from the frame's intra / decode-restart flag
(frame.keyframe == coding.keyframe() for video), NOT a fabricated
clean-RAP claim. The old gop member is honestly omitted. FVI_FORMAT.md
is updated to document this as a limitation: key is an intra picture /
parser-flagged decode-restart point; MPEG-2 open-GOP clean-RAP precision
(closed_gop) is not currently distinguished. §7.1 rewritten for the
new field_order/progressive/nb_fields members.
Wiring:
- mux/mod.rs: pub(crate) mod fvi_sink; pub(crate) mod videomap
(#[allow(dead_code)] on videomap — the VideoMap accumulator is staged
for side-channel reuse, sink builds records directly); pub use
fvi_sink::FviSink.
- mux/resolve.rs: add the fvi:// output scheme to StreamUrl, parse_url,
scheme(), path_str(), input() (write-only reject) and output()
(constructs FviSink), mirroring the mkv:///demux:// patterns.
Provenance fix surfaced by the end-to-end test:
- pipelined_stream::consume_ps was dropping the PS demuxer's byte-exact
source stamp (source: None) when rebuilding PesPacket, so PS/DVD-path
frames reached the mux/index with no provenance (FVI src null). Carry
ps.source through, matching the TS path; the real-pipeline fvi test now
sees the stamped src sectors.
Gate: cargo +1.86 fmt + clippy --lib -D warnings clean; cargo +1.86 test
--lib (2182 passed) and --test fvi_pipeline (2 passed); precommit.sh
libfreemkv green.
Carry per-picture truth and byte-exact source provenance THROUGH the stream so
the muxer (and the upcoming video index) read MEASURED facts instead of
assuming them. Honest data in, honest data out.
- codec/coding.rs: codec-agnostic PictureInfo (CodingType / FieldOrder + the
accessors field_order/coding_type/nb_fields/progressive/keyframe). Each codec
folds its raw signals in; consumers use only accessors, never branch on codec.
- mpeg2: builds PictureInfo from the picture coding extension and carries it +
SourcePos (source_marks, parallel to pts_marks) on every emitted frame.
- pes / codec::Frame: additive `coding` + `source`, forwarded through the
highway; None for audio/subtitle and the network/stdio deserialize hop.
- mkvstream: DEFER muxer construction until the first coded picture, set the
video track's FieldOrder from the MEASURED value, THEN write the header —
right the first time, no guess, no seek-back. An interlaced track that arrives
with no measured order is LOGGED loudly and left UNDETERMINED, never faked.
- mkv: MkvTrack::video no longer guesses TFF (a bitstream property the scan
cannot know is UNDETERMINED at build). Removed VideoStream::top_field_first
(the dead scan-time guess) crate-wide.
- Tests: parser population (every PictureInfo facet + per-PES source carry) and
mux-stream consumption (measured -> correct; missing -> UNDETERMINED, not
faked). Two obsolete tests updated only after confirming (their own comments)
they existed to enforce the deleted hardcoded-TFF.
- FieldOrder now derives from the bitstream's measured top_field_first
(Some(true)→TFF, Some(false)→BFF) instead of hardcoding TFF for all
interlaced content; falls back to TFF when unmeasured. Adds
VideoStream::top_field_first; DVD/BD scan sets None with a precise
TODO(spec) for parser→title plumbing.
- CICP (matrix/transfer/primaries/range) now prefers measured CICP from
the bitstream (VideoStream::measured_cicp) over the coarse ColorSpace
enum, so the container stops assuming a colour space the stream may
contradict. Enum remains the fallback.
- VobSub S_VOBSUB CodecPrivate now emits a `size: WxH` line ahead of the
palette per the .idx format so players place/scale subs correctly.
- SeekHead: when zero cues are written, the CUES Seek entry is Voided
instead of leaving a dangling pointer to the Cues offset (now Tags/EOF).
- AC-3 Channels back-patch offset is captured from the writer instead of
the hardcoded chan_elem_pos+2 (decoupled from the VINT width choice).
- Hoisted inline CICP codes and the dvcC fourcc to named constants citing
ITU-T H.273 / RFC 9559; fixed the stale FieldOrder comment.
- DefaultDuration vs pulldown: precise TODO(spec) left (needs the same
parser→title channel as top_field_first).
Tests: BFF-from-measured-flag, measured-CICP-overrides-enum, VobSub size:
line present/omitted, zero-cue SeekHead Void. precommit (1.86) green.
Fix three DVD video-attribute bugs surfaced by a PAL disc detected as
NTSC:
- PAL/NTSC: parse video_format from VTS_V_ATR bits 5-4, not bits 1-0
(the old mask read permitted_df, so PAL 576i/25fps was mis-detected
as NTSC 480i/29.97). Named consts replace the magic bit positions.
- Anamorphic aspect: write MKV DisplayWidth/Height from the disc's
display_aspect (16:9 720x576 -> 1024x576) instead of square pixels,
so 16:9 DVDs no longer render as 4:3.
- Colour: stamp SD colorimetry (PAL=BT.470BG, NTSC=SMPTE-170M) instead
of BT.709 (HD).
Adds VideoStream.display_aspect (threaded through every muxer) plus
TvSystem/DvdAspect/ColorSpace plumbing, with regression tests. Removes
the deprecated Disc mux set_halt bridge (use with_halt).
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
The library no longer loads a keydb anywhere. The scan path always captures the
disc's AACS inputs (MKB, VID, Unit_Key_RO.inf) and resolves NO key; a caller
resolves a Key from a key source and applies it via Disc::decrypt_with.
- ScanOptions loses keydb_path / unit_key / disable_keydb (and the path search);
it now carries only optional DriveCredentials (host certs) for the live-drive
AACS handshake. do_handshake_cert uses those instead of loading the keydb.
An unlocked / LibreDrive drive takes the OEM Volume-ID path and needs none.
- The mux input() path takes caller-resolved unit_keys instead of a keydb_path,
and applies them via decrypt_with.
- Deleted the now-dead inline resolve_encryption / resolve_encryption_static.
All 700+ lib tests pass.
Rename is_unit_encrypted -> is_aacs_scrambled and decide encryption from the unit's MPEG-TS sync bytes (destroyed by the encrypted body) instead of the TP_extra copy-control (byte 0) or TS scrambling-control (byte 7) flags, which discs do not set reliably. One shared predicate now backs the decrypt gate and out-of-band key validation, so callers agree on what 'encrypted' means. Decryption restores the syncs, so a decrypted unit reads as clear and there is no flag to clear.
is_unit_encrypted read the TP_extra copy-control bits (byte 0), which are a
copy-permission flag, not an encryption flag. On discs whose sampled units are
clear navigation packets (PAT/PMT) those bits can be set while the unit is not
scrambled, so a correct Unit Key was used to 'decrypt' already-plaintext data,
produced garbage, and the key was wrongly treated as failing.
Read the actual flag instead: the TS transport_scrambling_control bits (top two
of TS-header byte 3 = byte 7 of the aligned unit, inside the clear seed). AACS
encrypts whole aligned units, so this one packet's TSC reflects the unit.
decrypt_unit now clears the TSC bits of every packet on the way out so the
result is valid unscrambled TS. Tests updated to the TSC flag.
Two coherent additions to the AACS resolver:
KeyProvider abstraction (provider.rs) — key material comes from pluggable
backends; KeyDb implements it (device/processing keys, host certs,
disc-by-hash / disc-by-vid lookup) plus orphan-DK parsing. ResolveContext
takes a provider array. Adds the SD-tree PK walker
(derive_media_key_from_pk_walked) and a `probe` module (km_verifies MK
oracle, mkb_* record parsers) used for offline key verification. Cvalue
record selection prefers 0x05, falls back to 0x07.
External-UK key source — the second, mutually-exclusive key source for the
keyserver path. ScanOptions/InputOptions gain `unit_key`; when set,
resolve_encryption_static skips keydb entirely and uses the caller-supplied
Unit Key directly (KeySource::ExternalUk). Disc::read_aacs_inputs exposes a
disc's Unit_Key_RO.inf + MKB so a caller can fetch the UK out-of-band; the
library makes no network call itself.
CHANGELOG: redact test-disc title in historical notes.
- Introduce DrmScheme enum (Css/Aacs10/Aacs20/Aacs21) + drm module with
uniform detect/load dispatch across all four protection schemes.
- Land AACS 2.1 Media Key Variant framework in aacs::variants: chain
derivation, MKB record types 0x82/0x83, bit-0x02 SoftKCD and bit-0x04
online-challenge detection. Aacs21 dispatcher arm wired but commented
out pending validation against a Variant-scheme disc.
- Replace aacs2: bool with AacsVersion enum across ContentCertificate,
UnitKeyFile, ResolvedKeys. resolve_keys splits into _v1/_v2/_v21.
- Delete the libredrive raw-read VID shortcut from do_handshake; the
drive enforces the AGID requirement regardless of firmware-upload
state, so the shortcut spuriously dispatched E7017 instead of
surfacing the real downstream walls.
- MkvTrack::audio emits A_DTS/MA, A_DTS/HR, A_DTS per the DTS family
instead of mislabelling everything as A_DTS. Plex transcoder and
strict hardware decoders reject DTS-HD MA payload under a plain
A_DTS track.
- PgsParser is now stateful: pairs display PCS with the following
empty PCS to compute a duration. Frame::duration_ns + PesFrame::duration_ns
carry it through; MkvMuxer::write_frame gains a final Option<u64>
parameter that emits BlockGroup + BlockDuration when set. Fixes
subtitle bitmaps lingering past their intended end-time.
WO-2 (delete SectorReader trait):
- The 0.18 trait split into SectorSource (read-only) and SectorSink
(write-only) is final; the legacy SectorReader alias was a bridge.
- Renames every internal &mut dyn SectorReader (~25 sites) to
&mut dyn SectorSource. The trait method capacity() becomes
capacity_sectors() with a default of 0 (preserves SectorReader's
default-0 behavior).
- Deletes the SectorReader trait, its blanket-to-Source bridge, and
the FileSectorReader type alias. Adds explicit forwarding impls
for Box<dyn SectorSource> and &mut dyn SectorSource so generic
decorators like DecryptingSectorSource<S: SectorSource> compose.
WO-3a (extract Disc::patch):
- Moves Disc::patch (1230 lines) and bytes_bad_in_title from
disc/mod.rs into disc/patch.rs as a split inherent impl. Zero
behavior change — pure mechanical relocation. disc/mod.rs drops
from 3,945 to 2,714 LOC.
WO-6 (partial):
- Deletes src/labels/png_filenames.rs — was a 72-LOC stub with
detect() returning false, never wired into the PARSERS registry.
project docs doc drift fixes (audited 2026-05-13):
- JUMP_BASE_SECTORS: 256→1024 (64 MB base for UHD, not 8 MB)
- PASSN_DAMAGE_THRESHOLD_PCT: 12→6
- PASSN_SKIP_SECTORS_BASE: 64→32
- MAX_RANGE_SECS=180: replaced by proportional range_sectors × 25,
capped at RANGE_BUDGET_CAP_SECS=1800.
The 0.18 trait split into FrameSource (read-only) and FrameSink
(write-only) was an over-engineered API. Consumers don't think
"frame source backed by MKV" — they think "open MKV for reading".
The split paid a real API-complexity cost (two trait names, two
re-exports, dual impls per bidirectional type, deprecation bridge)
for one marginal property: compile-time direction-safety at the
trait-object boundary. The runtime error path on a wrong-direction
call (StreamReadOnly / StreamWriteOnly) is unambiguous and rare in
practice.
Deletions:
- pes::Stream is no longer #[deprecated]
- pes::FrameSource trait + its blanket-from-Stream bridge
- pes::FrameSink trait + the trampoline impls on every concrete type
- The compile-time-direction-safety test scaffolding
- Crate-root FrameSource / FrameSink re-exports
Additions:
- Stream is now Send-bounded (Stream: Send supertrait). Every
concrete impl was already Send-compliant — Box<dyn Read + Send>
and Box<dyn Write + Send> were already in place on the trait
objects MkvStream / M2tsStream / etc hold internally. Promoting
Send into the trait makes Box<dyn Stream> Send too, which lets
autorip drop its SendStream unsafe newtype.
The public API is now: one Stream trait, one concrete type per
format, two constructors (open/create or input/output). Bidirectional
types route through internal Mode { Read | Write } discriminants.
Net: -347 lines libfreemkv, -38 lines autorip, -5 lines freemkv.
scan_with() collapsed every failure path from resolve_encryption() into
None via .ok(), so callers couldn't tell the difference between "no
KEYDB found", "KEYDB failed to parse", "disc hash not in KEYDB and
fallback derivation failed", "AACS files unreadable on disc", and a
handshake that rejected every host cert. autorip's UI was stuck
printing "no decryption keys found (check KEYDB)" for all of them,
which is a particularly bad message when the user has actually loaded
a KEYDB and the real failure is something else.
Changes:
- New pub field Disc.aacs_error: Option<Error>. Populated by scan_with
whenever encrypted && aacs.is_none(). Sentinel KeydbLoad path
"<no keydb in search paths>" distinguishes the no-keydb case from
a real load failure without adding a new Error variant (which would
be a breaking change for downstream exhaustive matches).
- tracing::warn in scan_with at scan_aacs_resolve_failed and
scan_aacs_no_keydb, with error_code and keydb path for grepping.
- tracing in do_handshake: keydb load failure, host-cert exhaustion
(with cert count and last error code), VID read failure post-auth,
and a debug-level success log. Lets us see whether handshake even
got off the ground for a given disc.
Test fixtures updated to set aacs_error: None.
Pass N now reads at 32 sectors per attempt and drops to 1 only on
batch-read failure to probe each sector individually. After 16
consecutive clean single-sector reads it climbs back to 32. Net
effect: NonTrimmed regions walk ~32x faster in clean stretches
without sacrificing per-sector recovery quality — the drop-to-1
retry from the same cursor position guarantees every sector in a
failed batch is individually attempted.
Design contract:
- A batch-read failure (count > 1) is NOT a recorded failure: no
NonTrimmed mark, no consecutive_failures bump, no damage_window
push, cursor stays put. We just drop current_batch to 1 and the
loop re-attempts the same position at single-sector granularity.
- A single-sector failure (count == 1) follows the existing path:
NonTrimmed mark, consecutive_failures++, damage_window.push(false),
post-failure pause, wedge probes.
- Backtrack always at count=1: this path fills a gap that the main
loop's damage-window skip jumped over. Using batched reads there
would lump good sectors into NonTrimmed marks when the gap
contains even one bad sector.
State machine adds:
- `initial_batch` (from opts.block_sectors, default 32 in patch_internal)
- `current_batch` (mutable, starts at initial_batch, drops to 1 on
batch failure)
- `consecutive_singles_ok` (counter, resets on upscale + failure)
- `ADAPTIVE_UPSCALE_THRESHOLD = 16` (matches sweep's pattern for
"16 consecutive good = back to fast mode")
Tests:
- pass_n_size_aware_skip.rs PatternedSectorReader now fills each
sector with its OWN LBA byte (not the starting LBA's byte). This
matches real drive behavior — the pre-0.18.13 fixture's
"fill whole batch with one byte" was a shortcut that only worked
when patch read 1 sector at a time. Existing recovery-quality
assertions all still pass under adaptive batching.
User spec: "try 32, pass, great, fail -> do 1 sector"
Matthew Jackson