9c6b7baf839cc3ef4175933d7d7f064c0e2dbca3
335
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Matthew Jackson
|
f434b9cf2c |
Drop six unused crates, and align the rest with the workspace
Two problems, both invisible until the whole graph is looked at together. DEAD: num-bigint, sha2, num-traits, num-integer, cmac and cbc are declared here and referenced nowhere -- not in src, tests or benches. They were being compiled, audited and offered version bumps forever for no reason. Removing beats bumping. cbc nearly survived the sweep: a substring search for "cbc" matches 44 occurrences of ycbcr_to_rgb in the DVD subtitle decoder, so it looked used. Only a word-boundary search exposed it. SKEW: this crate was the outlier on every shared dependency -- aes 0.8, rand 0.8, base64 0.22.1 and zip 2 against 0.9 / 0.10 / 0.23 / 8 elsewhere. Cargo cannot unify across a major version, so it compiled BOTH: 32 duplicated crates in the freemkv binary's graph, including two complete AES implementations (aes 0.8 + 0.9, cipher 0.4 + 0.5), two digest stacks and two getrandom. Two crypto stacks in one product is worth removing on its own. The aes bump is an API rename -- BlockCipher-prefixed traits, Array for GenericArray -- and the obvious translation uses Array::from_slice, which the new version deprecates and clippy's -D warnings would reject. These use the From<[T; N]> conversion the crate points at instead. 3441 tests pass in debug and release. The AACS crypto here is covered by known-answer tests, so a byte-order or sizing mistake in that rename could not have passed. |
||
|
Matthew Jackson
|
dd749132d5 |
Audit round: an ADTS frame that cannot hold its own CRC, and three drifts
An ADTS header that declares a CRC follows must be at least nine bytes — seven of header plus the two the CRC occupies — because the declared frame length counts them. The structural gate compared against a flat seven and never read the bit that says whether a CRC is there at all, so a frame whose own header describes something impossible was accepted and handed to the muxer as decodable. The pipeline's spawn doc named Sweep, and the thread Sweep would have created, as callers to look for. Neither has been in this crate since the recovery passes moved out. The same paragraph already records fixing this once, for a different departed caller — it simply drifted again a sentence later, so it now says to name callers that live here or name none. Whether a disc is structurally AACS-encrypted was spelled out by hand in both the fast identify and the full scan. They agreed today; nothing made them agree tomorrow, and disagreeing would mean the same disc reported encrypted by one path and clear by the other. There is one definition now, and the comment that pointed at it by line number points at its name. The AC-3 parser built a fresh buffer on every packet — of the order of a hundred thousand times per title — to work around a borrow it cannot avoid. The copy stays; the allocation does not. The buffer is now lent out and handed back, and a test pins that, because reverting it would be invisible in behaviour. Left alone deliberately: send/send_with_halt and finish/finish_with_halt look like one action under two names, and are not. After the consumer fails, one must still accept items and the other must refuse them; that difference is what stops a producer reading an entire disc for a write that died on its first frame. Collapsing them was tried here and the existing test caught it. Both now say so where the choice is made. |
||
|
Matthew Jackson
|
b9ca75f471 |
Identify a frame clip by provenance, not by inferring it from timestamps
Four audit rounds each fixed one rule in SeamPlan::place and broke another, because the question the rules were answering has no answer. Inside a seamless-branching overlap clip k OUT comes AFTER clip k+1 IN — 57.8s of overlap on the real fixture table — so a single timestamp is legitimately inside two clips, and a clip file is not trimmed to its marks, so it also carries material from before its own IN. No rule over timestamps can say which clip a frame came from, and each attempt was right for one disc layout and silently wrong for another: 65s of rewind, 17 minutes stranded, 28 minutes dropped, 55s refused. Frames already carry the byte offset they were read from (PesFrame::source, stamped by the TS demuxer). Clip now carries the byte span its stream occupies in the title feed, recorded while the extents are gathered. So the clip is a LOOKUP: the offset falls in exactly one span. There is no decision to get wrong. Every track of a clip lives in the same stream file and therefore shares one span, so video, audio and subtitles agree by construction. Divergence between them — each track guessing separately under its own tolerance — is how audio and video ended up on different clips and drifted apart in the first place. spans_trusted gates the whole path: unless the spans tile the feed contiguously from zero, an offset means nothing and provenance is ignored in favour of the mark heuristics, which is the 1.6.0 behaviour. A broken map degrades instead of confidently selecting a wrong clip for every frame. A clip referenced twice reuses its first span (the bytes are read once) and is still trusted. Sources that stamp no provenance — a mkv:// remux, the deserialize hop — take the heuristics, which is what they have always used and where they have always been right, because they have no overlapping clips to be ambiguous about. 36 timeline tests, six of them new and covering: the overlap case marks cannot see, all tracks agreeing, out-of-marks material dropped AND counted, a holed span map, a discontiguous one, a repeated clip, and no provenance at all. |
||
|
Matthew Jackson
|
0e8c31a9c3 |
Round 3: fix three defects introduced by the round-2 fixes
Auditing my own fixes found all three. None were in the original code. The VTS crack sort was byte-wise case-SENSITIVE while the filters that select those files (vts_group_of / is_title_vob) are case-insensitive. On a case-sensitive volume a set holding vts_01_1.vob beside VTS_01_2.VOB sorted part 2 first, because V (0x56) precedes v (0x76) — reintroducing exactly the budget-exhaustion the ordering exists to prevent. Now sorted on the same uppercase normalisation the filters apply. Refusing a name that round-trips to empty aborted the WHOLE plan. A sidecar folder named with a single emoji made a backup un-rippable that 1.6.0 handled fine, and reported it as a collision with a file that does not exist. It is now skipped with a warning: the entry is unaddressable either way, but one irrelevant file should not cost the user their rip. The mtime check now applies only to files whose CONTENT the plan read — the IFOs, whose bytes 0xC0/0xC4 place every VOB. Everything else is planned from size alone, which is already checked, so comparing mtime there bought nothing and risked a real false positive: disc backups commonly live on exFAT/FAT32, which stores local time, so a long rip spanning a DST transition would see a whole-hour shift on an untouched multi-gigabyte VOB and abort hours in. |
||
|
Matthew Jackson
|
bc5e3453b4 |
Order a VTS crack by filename, not by directory order
Round 1 removed a largest-first sort from resolve_vts_key because largest-first is the 1.5.1 garbage bug. Auditing that fix showed it was only half right: `planned` comes from walking the UDF directory, which yields File Identifier Descriptors in on-disc authoring order with nothing sorting them. So deleting the sort did not restore playback order, it left the order undefined — whatever the disc happened to list first. DVD-Video numbers a title sets VOBs in playback order by spec (VTS_xx_1.VOB .. VTS_xx_9.VOB, single digit), so ascending filename IS playback order and is deterministic regardless of how the directory is laid out. Order decides correctness here: crack_key shares one sector budget across the whole extent list, a CSS DVDs biggest cell opens with a long clear run, and CSS recovers the title key from scrambled data itself. Starting in the wrong place can exhaust the budget without ever meeting scrambled data, whereupon the caller falls back to the disc-wide key and the whole VTS is descrambled wrongly — corrupt PES behind an intact header, written out at exit 0. |
||
|
Matthew Jackson
|
6d9791affc |
Audit round 1: playback order, silent title drops, image durability
Four fixes from the first audit round. Every finding was verified against a pinned tree and read directly before being accepted. resolve_vts_key sorted a VTS title-VOB extents largest-first. That is the 1.5.1 garbage bug, and it grew back in a new code path: the comment claimed it "matched the scan heuristic", but that heuristic WAS the bug and had already been fixed in decrypt_keys_for_title, which documents the rule (PLAYBACK ORDER, never largest-cell-first) and pins it with a regression test. A CSS DVDs biggest cell opens with a long clear run and crack_key shares one sector budget across the extent list, so starting there can exhaust it without ever MEETING scrambled data — and CSS recovers the key from scrambled data itself. The crack then returns None, the caller falls back to the disc-wide key, and every VOB in that VTS is descrambled wrongly: corrupt PES behind an intact header, written out as a complete extract at exit 0. parse_pgcit dropped titles silently in THREE places — an unparseable PGC, an out-of-range PGC index, and a truncated entry table. The finder caught one; the other two turned up on reading the function. parse_vmg already counts and warns per skipped title SET for exactly this reason, and this was the last place a disc could quietly report fewer titles than it has. write_image called flush() and returned Ok. flush() only pushes bytes into the page cache and promises nothing about durability, so a 6-90 GB image could be reported complete while still unwritten — a crash or an unmounted volume then leaves a truncated file the caller was told was finished. Now into_inner (so a buffered-write error surfaces instead of being dropped by BufWriter::drop) followed by sync_all. timeline used abs() on a saturating_sub result. Every other comparison in that module is saturating because the timestamps come off a disc and are not trusted; abs() panics on i64::MIN, which saturating_sub can produce. |
||
|
Matthew Jackson
|
93ad1f4854 |
Give Windows a free-space gate, and stop two tests timing the scheduler
Three release-profile failures on macOS and Windows, all found by the qa gate on its first run. Release-profile tests on those platforms had never run before it existed, so none of these were regressions — they had simply never been visible. available_space returned None on every non-unix target. That did not merely skip a test, it skipped the GATE: a Windows user extracting a disc to a full volume got a confusing failure part-way through instead of a clear refusal up front, and Windows is where the GUI ships. GetDiskFreeSpaceExW is declared directly against kernel32, matching how scsi::windows already reaches Win32 rather than pulling in a binding crate for one call. It asks for FreeBytesAvailableToCaller, which accounts for per-user quotas — the same question f_bavail answers on unix. The other two asserted on wall-clock timing with no margin: - sleep_until_halted_wakes_mid_sleep bounded the wait at 350 ms and measured 377 ms on a loaded runner. That bound measures the scheduler, not the wake. What the test is for is distinguishing "woke because the flag flipped" from "woke because the 10 s timeout expired", and 2 s does that just as well. - abandon_loses_to_a_close_already_committed released at 600 ms against two 300 ms grace windows plus a 250 ms poll cadence, so the windows could expire first and the caller abandoned — a race, not a defect. The intervals are scaled up so jitter is small relative to them; the ordering under test is unchanged, only the margin. |
||
|
Matthew Jackson
|
a018e1adc4 |
Require a pack start code before descrambling a sector
css::is_scrambled reads bits 4-5 of byte 0x14 and nothing else. That is a sound test once a caller has committed to a title's VOB data, where every sector is an MPEG-2 PS pack and byte 0x14 always means what it says. descramble_region is not such a caller: it is handed arbitrary regions of a disc, so it also sees IFO, UDF and ISO 9660 sectors — raw structures where byte 0x14 is whatever that format happens to store there. Measured on a real disc: the second sector of VIDEO_TS.IFO holds 0x15 at offset 0x14 while starting 00 26 00 00, which is not a pack. The flag test read it as scrambled, descrambled it, and destroyed 1912 of its 2048 bytes. That sector carries TT_SRPT, so the title table went with it — the disc enumerated 38 titles and an image decrypted from it enumerated 10, silently, at exit 0. is_scrambled_pack already existed with the right predicate. Use it here. It costs nothing: a genuinely scrambled VOB sector always carries the pack start code, and no IFO sector does. Verified end to end — the decrypted image's `info` output is now identical to the source disc's, 38 titles both, differing only in the CSS: Encrypted line. The fixtures moved with it. Four of them built a sector by setting byte 0x14 alone, which no real scrambled sector looks like; they now build packs. |
||
|
Matthew Jackson
|
4959b48386 |
Bind borrowed stream labels by the stream they name, not by a slot
A vendor label's `stream_number` is a slot in the one stream table its
config blob describes. The labels merged in from the playlists — to
cover streams the vendor named nothing for — carried a different number
entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's
slot numbering at all. Two coordinate systems, one field name. The
merge matched them by equality and the binder then counted streams
against the result.
Measured over the 44-image corpus: 22 discs merge such labels; of the
566 places one lands on a stream, 443 (78%) are a stream it does not
describe — the label states the PID it read itself from and it is a
different one. 142 of those are stopped by the language check
|
||
|
Matthew Jackson
|
0d4aab99df |
Stop naming specific commercial discs in the AACS and codec comments
The same scrub as the previous commit, over the files it did not reach: the variant-MKB layout notes, the 2.1 segment index observations, the PPS-revert regressions and the playlist-twin tiebreak. Measurements keep their numbers — "a v70 `0x2d` body = 46_100*2 + 16" is the useful part, and the title it came from never was. |
||
|
Matthew Jackson
|
17bb13b077 |
Do not call a track forced off one display set of a sample
A forced verdict is an absence claim, and a sampled run sees a fraction of a track. Measured on real discs: tracks exist that flag about a quarter of their display sets and leave the rest unflagged, so a sample that catches one flagged set and nothing else promotes a full dialogue track to forced -- which players then force on screen. A sampled run now needs two display sets; a run that read every extent end to end has no unread gap and may still promote off one. |
||
|
Matthew Jackson
|
abffa4235a |
Halve the seeks: eight 32 MiB windows, not sixteen 16 MiB ones
Measured on a real title: the same 256 MiB budget cut into sixteen windows per extent took 41.8s against the head-first read's 6.6s, because every jump collapsed the source's read batching into three-sector calls. Expected observations depend on total bytes read, not on how finely they are cut, and measured subtitle density (a display set every ~30-50 MB of clip) makes a 32 MiB window about even money on its own. |
||
|
Matthew Jackson
|
3ed8630535 |
Take a lone sample window from the middle of its extent
A title cut into 50-odd clips gives each extent one window's worth of budget. At the extent's head, every clip is sampled at the same relative position and the first clip's window lands on the opening of the feature -- the one stretch with no subtitles in it. |
||
|
Matthew Jackson
|
5c8b4dc7c5 | Note the bounded display-count over-count on a stalled read retry | ||
|
Matthew Jackson
|
841aa1a1c6 |
Drain a sampled run's tail and merge repeated extent memos
A window's last PES stays open in the demuxer until the next PUSI, which under sampling is in a different window or nowhere — so the last display set of every window was discarded, worst where the sample is thinnest. Flush the demuxer at the end of each window. A playlist that lists the same clip twice now merges the second read into that extent's memo (facts OR'd, display count MAXed, coverage the larger of the two) instead of overwriting it. |
||
|
Matthew Jackson
|
14049bb477 | Document the forced-probe redesign in the changelog | ||
|
Matthew Jackson
|
8ffce6b621 |
Sample PGS across the title instead of reading its head
The content-based forced-subtitle probe spent its whole 256 MiB budget
on the first sectors of a title. A feature's subtitles begin minutes in,
so the probe read the opening logos, hit the budget, observed no display
set at all and contributed nothing to any verdict — the vendor label was
always the only input.
The forced predicate is asymmetric: one non-forced display set disproves
forced permanently, while proving forced needs the whole track, and
genuine forced tracks are tiny where full tracks are huge. So the same
budget is now SPREAD over each extent in ~16 MiB windows placed on the
AACS unit grid, sized in proportion to the extent, ending at the extent's
end. Cost is unchanged; placement is not.
Also:
* Per-track early exit. A track that is disproven (and whose label
needs no correcting) stops asking for budget; an extent that owes
evidence only for such tracks is skipped outright, and evidence
already in the cache is never demuxed a second time.
* Content may now DEMOTE a wrong vendor forced flag, in the probe and
in the muxer, behind one shared guard: absence of forced_on_flag
only means something if some other track demonstrably uses it, and
the track must have the shape of a full dialogue track rather than
of a forced-narrative one. On a disc where no track sets the flag,
nothing is demotable.
* A sampled or budget-cut extent's evidence is memoised with the
COVERAGE behind it. It used to be filed under the extent's full key
and replayed to playlists that would have read far more of the clip,
turning a prefix into an absence claim about the whole extent.
|
||
|
Matthew Jackson
|
b2a274782a |
Report a key source that could not answer as its own failure, not as "no key"
The online key service returned HTTP 502 for about seven hours. Every rip in
that window ended with
key: online > no entry > NO KEY
Error: E7022 No key source has a decryption key for this disc (id: 422EB...)
which reads as "this disc is not in the key database". Operators went hunting
for a VUK that was never missing; the correct action was to wait.
E7022 is a claim about the WORLD: every source answered, and none holds a key
for this disc. A source that could not be reached made no such claim -- nothing
at all was learned. `resolve_and_apply_traced` collapsed the two anyway, with
its own comment naming the incident and pointing at the fix.
Three codes for the three different operator actions, each a variant with a
number and no English (the library ships none):
E7028 KeyServiceUnavailable unreachable / DNS / timeout / 5xx -> wait
E7029 KeyServiceUnauthorized 401 or 403 -> fix token
E7030 KeyServiceRateLimited 429 -> back off
None carries a payload: the key-service URL and its resolved address are
operator-confidential and must not ride out in a Display an operator pastes
into a bug report.
`resolve_and_apply_traced` now keeps `Err` and `Ok(empty)` apart. A source that
answered and holds nothing still records `KeyNode::NoEntry` -- that is the one
true "I looked, it is not there". A source that FAILED records an empty path,
and its reason is stamped onto `Disc::aacs_error`, the channel
`ensure_decryptable_keys` already reads for the E7017-vs-E7022 split. The gate
gained three arms alongside `AacsVidUnavailable` and raises the source's own
code.
`KeyOutcome` deliberately gains no variant. It is matched exhaustively by every
front-end's trace renderer (freemkv's `pipe::render_resolution_trace`,
autorip's `keysource::render_resolution_trace`), and this fix must not become a
breaking change across four repos to say something the error code already says
precisely. Dropping the false `NoEntry` node is enough for the trace line.
The three codes join `is_disc_level_no_key`: a service that is down, refusing
the token or throttling is down for every title, so a rip loop must stop rather
than issue N doomed requests -- and on 429, dig the hole deeper.
`FetchOutcome::errored`, documented as unreachable-in-production, now fires for
real: the negative-result cache stops memoising a transient outage.
Red before green: `key_source_failure_is_not_reported_as_a_missing_disc_key`
drives KeySource -> resolve -> aacs_error -> ensure_decryptable twice over the
same disc and asserts the verdicts differ. With the old conflation restored it
fails on the trace node, and with that assertion removed it fails
`left: 7022, right: 7028` at the gate.
|
||
|
Matthew Jackson
|
ff18d4c3c8 |
Close the MEDIUM mutation gaps across transport, labels and codecs
The remaining triage items after tonight's HIGH fixes: 1,290 lines, almost all tests. Covers disc/mod.rs's DVD scan path (with real minimal VMG/VTS IFO fixtures rather than mocks), drive/mod.rs, labels/class_reader.rs and labels/mod.rs — the two biggest untriaged survivor clusters in the crate — plus hevc.rs and ps.rs. One production change, and it is an extraction rather than a behaviour change: MacScsiTransport::open mapped the shim's negative failure sentinels to typed errors inline, where nothing could reach it without a real IOKit FFI call. It is now map_shim_open_error, so the mapping can be pinned. It matters because collapsing -5 into the DeviceNotFound catch-all turns "another process holds the drive" into "no such drive", and an operator chasing the wrong problem is worse than a blunt error. Gate green on the pinned toolchain including the secrets scanner. |
||
|
Matthew Jackson
|
fb321f51eb |
Reject a short READ CAPACITY reply, and count only entry marks as chapters
Two cases of the same shape: one policy implemented twice, with only one copy hardened. Disc::read_capacity decoded buf[0..4] from READ CAPACITY (10) without checking that the transport actually delivered four bytes, even though its comment claims to mirror decode_read_capacity — which has exactly that check, and documents why. A drive answering GOOD with an empty data phase leaves the buffer zeroed, so last_lba decodes to 0 and the probe reports a one-sector disc instead of an error. It now calls the shared decoder rather than re-deriving it. collect_chapter_summary filtered chapters on mark_type <= 1, counting the reserved type 0. PlaylistMark's own doc says filters must test == 1, and disc/bluray.rs did; the labels path did not, inflating the public chapter_count and letting a playlist whose only marks are reserved pass the chapter_count == 0 skip. Both sites now share PlaylistMark::is_chapter_mark so the copies cannot drift again. Both fixes were confirmed red before green. |
||
|
Matthew Jackson
|
71686f1407 |
Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv, the crate the other seven build against and the one held up as the reference workflow, was the last one still linting the library only — so its ~3,000 tests, by far the largest body of test code in the project, had never been linted at all. Turning the flag on surfaced 74 findings. Most were mechanical and applied with clippy --fix. The rest, by hand: - Four discarded Results in decrypt.rs. css::descramble_region returns a Result and four CSS tests threw it away, so a descramble that FAILED would have surfaced as a confusing buffer-comparison mismatch instead of the actual error. They expect() now. - A dead `kp` field on the PlantedWalk fixture. The test deliberately asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4 rather than against a stored value — its doc comment says so — which makes the field not just unused but a trap: the obvious "fix" of asserting against it would quietly weaken the test to comparing the fixture with itself. Removed. - Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut, three vec!s that only ever needed arrays, a filter_map whose every arm was Some, and a Vec::new()+push chain. - Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering in the generated docs. - A five-[u8; 16]-tuple return type named FourLevelParts. Three lints are allowed at the specific sites, with reasons, because they are wrong for this domain: the underscores in the bitstream-header literals mark BITFIELD boundaries, not digit groups, so regrouping them uniformly would satisfy the lint by destroying the only thing they encode; and in three table-validation loops the loop variable is the domain value under test (a DTS SFREQ code, an AMODE value, a palette entry number), which is what the assertion messages name. |
||
|
Matthew Jackson
|
2efe1425d6 |
refactor(decrypt): one orchestrator owns the no-key decision
There were TWO top-level decrypt paths: decrypt_sectors_impl for CSS and clear media, whose AACS arm was a bare `return Err` stub, and a wholly separate decrypt_sectors_mapped for AACS. Each scheme therefore decided its own answer to "there is no key for these bytes", and nothing held them to the same one. They drifted, in opposite directions, within a single release: css::descramble_region descrambled with a key the sector's own crib had just proven stale — garbage behind an intact clear header, reported Ok. the mapped path returned early for any LBA outside every range, before ever asking whether those bytes were ciphertext, so an unkeyable encrypted unit passed through and extract counted it as good. Both were fixed individually earlier today. This removes the shape that allowed them. decrypt_span is now the single orchestrator: it owns the loop, the refusal, and the loss count, and each scheme supplies only what genuinely differs. apply_aacs_map is a scheme step that reports what it could not open; it no longer decides what that means. The public wrappers (decrypt_sectors, _in_content, _mapped) are unchanged in signature and all funnel through it. Adding a scheme now means adding an arm here, which means answering the refusal question. That is the point. The new test asserts ONE verdict across all three schemes — AACS with no map, AACS with an encrypted unit outside every range, and CSS whose re-crack failed — plus that clear media is NOT a refusal. A per-scheme test cannot hold this: each would keep passing while the two disagreed. Flipping the AACS arm back to pass-through reds it. Also removes the last of the tracing-capture scaffolding. Serialising those captures crate-wide did not fix the 1-in-10 flake, and asserting the predicates directly made the helper, both capture subscribers and an unrelated dead OrderSink unused. Deleted rather than left behind. |
||
|
Matthew Jackson
|
b86f7aef17 |
fix(session): delete two dead accessors, make into_drive fallible
drive() and drive_mut() had ZERO callers — not in libfreemkv, freemkv, autorip, bdemu, keysources or kdb. Deleted rather than converted: dead public API that panics is not an API worth preserving the shape of. into_drive() had two callers and now returns Result. The empty-slot state is reachable through ordinary public use — stage_drive_as_reader moves the drive into the reader slot, and calling into_drive twice moves it out — so the panic was not guarding a caller error. identify() was converted for exactly this reason in this same release; the fix went to one of four public sinks and the other three were left. I deferred this on the assumption the blast radius was large. It was three call sites. Checking beats assuming. Also fixes a REAL FLAKE in the gate, which is worth more than the above. resolve_vid_only_bus_key_gate_reports_true_has_volume_id... failed about one full-suite run in ten while passing every time in isolation. It installed a capturing tracing subscriber to read back the has_volume_id field of a warn. That cannot be made reliable: dispatcher::set_default is THREAD-LOCAL while tracing's callsite-interest cache is GLOBAL. The original author knew, and called rebuild_interest_cache() — necessary but not sufficient. I first serialised every capture in the crate behind one lock (harness::with_captured_tracing, which also removed the same hand-rolled dance from three other sites). Still 1-in-10, because the cache can be re-evaluated against the process-default dispatch rather than the thread-local one. So the predicate is now a named function, handshake_has_volume_id, and the test asserts the VALUE. A boolean does not need a subscriber to check. The gate's hard-error behaviour keeps its own test. Measured: 14 consecutive full-suite runs, 2994 passed, 0 failed. A flaky gate is worse than a missing one — every green after it means less, and this one had been eroding trust in the whole suite. |
||
|
Matthew Jackson
|
9f25a4c454 |
fix(mp4): refuse a video track with no resolved dimensions
Resolution::pixels() returned (0, 0) for Unknown, and the MP4 sink wrote it verbatim into tkhd (ISO/IEC 14496-12 8.3.2) and VisualSampleEntry (12.1.3). Both fields are MANDATORY there, so unlike Matroska — which omits the optional PixelWidth/PixelHeight elements — MP4 has nothing to leave out. The result was a structurally complete file that passes every container check, declares a 0x0 video track, cannot be rendered, and is written with no error anywhere. WHY IT WAS POSSIBLE, which is the part worth keeping: pixels() previously fabricated 1920x1080 for Unknown. That was wrong but playable, so this sink never needed a guard and the absence of one was invisible. Changing the sentinel to (0, 0) moved the defect instead of removing it — a zero PAIR still reads as a usable value, so the sink stored it and serialised it. The accessor's doc comment then ENUMERATED the callers it believed were safe: "the Matroska sink omits the optional elements, the VobSub writer omits its size: line, and no caller divides by either dimension." Two of those three are true. MP4 was not on the list because MP4 has no guard at all, and a prose list cannot enforce itself. mkv.rs's own comment even states the principle — "the check belongs in the one accessor rather than in each caller that remembered to write it" — and labels/mod.rs still carried its own duplicate Unknown test long after the accessor took that job over. So: pixels() now returns Option. Not because Option is tidier, but because every caller genuinely needs a DIFFERENT answer and the compiler is the only thing that reliably makes them choose one. Matroska and the metadata sinks take unwrap_or((0, 0)) with the reason stated at each site; the VobSub path degrades to a palette-only .idx; MP4 fails with E_MP4_UNKNOWN_RESOLUTION (9055). Six call sites, not the five my first grep showed — I piped it through `head` and acted on a truncated list. The compiler caught the sixth. That is the same mistake as trusting a lens that reported silence. |
||
|
Matthew Jackson
|
5360f8d309 |
test: salvage the orphaned labels/disc triage, and extract build_labels
Thirteen agents triaging src/labels and src/disc died on a saturated
machine, leaving 5,836 insertions across 28 files uncommitted in a
worktree. Recovered by 3-way apply onto twelve commits of drift; zero
conflicts. The diff was archived to freemkv-private first, because a
worktree is not a backup and this one had already nearly been lost.
One production change, and it is the right one: mpls_universal::parse
read every playlist off the disc AND converted the entries to labels in
a single function, so the conversion — stream-type mapping, dedup key,
the dense global counters — could only be reached through a synthetic
UDF image. Extracted to build_labels(&[Playlist]), which unit tests can
drive from already-parsed values. Behaviour-preserving: same iteration
order, same skip-on-error.
Two collisions resolved by hand:
A second mod pass_progress_tests, written independently against the
same survivors as the one committed in
|
||
|
Matthew Jackson
|
46eb88c51f |
Feed the CSS crack the canonical extent order, and stop Resolution faking 1080p
Seven defects in the code the test suite executes least — 913 lines of disc/mod.rs alone are run by no test at all, which is why this round scoped from coverage rather than from what previous rounds said they had read. Disc::scan_image kept its own copy of the crack's extent ordering and fed crack_key_outcome largest-cell-first. That is the fifth instance in this audit of a local reimplementation drifting from the canonical one, and the cost here is a key that does not descramble the feature: picking by sector count bypasses the capacity gate and can select a different VTS entirely. The copy is gone — which title comes from the canonical order the scan already applied, and the extents are handed over in playback order, exactly as decrypt_keys_for_title does. Its doc records why the duplicate existed so it cannot grow back. Resolution::pixels returned 1920x1080 for Unknown. That is the FOURTH instance of one trap and the other three were in this same file, two of them fixed hours earlier — without sweeping for siblings, which is the whole reason this one survived. It now returns (0, 0), and the sweep was done properly this time: every remaining Unknown arm across the crate is honest, and the two ColorSpace sites that look like fabrication are emitting H.273 code point 2, which is the spec's own "unspecified". Two callers carried local Unknown-to-zero workarounds — precisely the cost of making callers responsible for a lie — and one is now redundant. BD-ROM Part 3 code 0xA2 is the lossy secondary DTS stream, not lossless Master Audio. A test asserted the wrong mapping as intended behaviour, so correcting the code failed it; the test is deleted with a note pointing at its replacement. That is a NEW failure mode for this audit: not a test that cannot fail, but one that locks the defect in. There is no DtsExpress variant to map to, so it takes the lossy DTS-HD member and the approximation is documented. Also: DiscSession::identify could panic through drive_mut once the public API allows an absent drive — two siblings were converted in an earlier round and this one was missed; an extent end that added without saturating where the rest of the crate saturates; a diag reason string restating the comparator's sort keys and drifting from them, now derived from them; and a short read that advanced the offset by the full request, silently skipping the gap. That last one existed twice, in two reads with the same shape, now merged so they cannot drift apart. The short-read policy is a judgement call I could not derive from a spec: no skip_errors is a hard error, with skip_errors zero-fills and charges the loss. It deliberately does not retry mid-unit, because resuming inside an AACS aligned unit would trade a silent gap for a silent decrypt desync — the worse of the two. |
||
|
Matthew Jackson
|
dea968f32b |
Stop AudioChannels and SampleRate fabricating a value for Unknown
Three copies of the same two mappings existed. The canonical accessors returned 6 channels and 48000 Hz for Unknown; a third copy in diag.rs returned 0. The honest one was the copy. A plausible wrong answer is worse than an obvious one. Six channels at 48 kHz is indistinguishable from a real 5.1 track, so every caller became responsible for remembering to check the variant first — and this crate walked into exactly that: the json:// sink reported a confident 5.1 for audio whose neighbouring fields said "unknown". That was fixed at the call site earlier in this audit; this fixes it at the source. The accessors now return 0, which is what both in-crate call sites already coerced Unknown to by hand, so their guards are gone and the behaviour is unchanged. Zero is also obviously wrong if it ever reaches output, where six is not. The diag.rs duplicates are deleted rather than corrected — a fourth copy would have drifted too. Their only caller was a trace line in the same file, now on the canonical accessors. Their tests moved across and gained the Unknown case, which is the point: restoring either fabricated value fails both. Found by the round-7 correctness agent while fixing the json:// sink; it flagged the third copy as out of its scope rather than touching it. |
||
|
Matthew Jackson
|
dc5b67ed46 |
Stop reporting an uncrackable CSS disc as N empty titles
Same shape as the mkv:// conflation fixed earlier in this round, found by looking for it deliberately. E7023 carried two conditions with opposite correct responses: one title on a multi-VTS DVD failing its own re-crack, where skipping it and finishing the rest is right, and the main feature's crack failing outright, which is disc-wide and dooms every title identically. Because both raised the same code and that code is in is_skippable_title_stub, an uncrackable disc walked all N titles printing "title skipped, it was empty" and exited 0. The disc-wide condition gets E7027 CssNoDiscKey, mirroring the AACS-side E7022 NoDiscKey it is the analogue of, and joins is_disc_level_no_key. The per-title raise keeps E7023 and stays skippable. Because the engine's classifier already tests is_disc_level_no_key before the skippable branch, this reaches the right outcome downstream with no change there: such a disc now stops on the first title and reports no-key instead of returning success with nothing written. Disc::css_error deliberately still stores CssKeyMissing — autorip matches that variant on the field to pick the CSS rather than AACS message, and what consumers classify on is the gate's returned verdict, which is the only thing that changed. Two neighbouring CSS raises were examined and deliberately left alone: the no-key branch in the same function is genuinely unreachable via ensure_decryptable and documented as defensive, and resolve_dvd_title_key is per-title on both of its call paths. Verified by removing the new code from is_disc_level_no_key, which fails both new tests; each pins both directions so neither can silently flip. Not proven end to end against a real uncrackable disc — none available. |
||
|
Matthew Jackson
|
58bdb42f8e |
Group whole E-AC-3 frame sets, and keep short reads unit-aligned
Two defects in fixes landed the same day, both found by round 4 auditing round 3's work rather than trusting it. **The E-AC-3 grouping ignored substreamid, so the timeline still doubled.** Confirmed independently by the correctness and conformance lenses and verified by hand: substreamid appeared only in test helpers, never in the production path. Per ETSI TS 102 366 (A/52) Annex E a frame set is independent substream 0 — mandatory, always first — with its dependents, then the OPTIONAL additional independent substreams 1..7 with theirs, all covering the same time period. Treating an additional independent substream as a new access unit advanced the clock a second time for the same 32 ms, which is exactly the doubling the grouping fix existed to prevent. No fixture caught it because every fixture used substreamid 0. is_dependent_substream becomes substream_role -> Starts | Extends: strmtyp 1 extends; strmtyp 0/2 with substreamid != 0 now extends (this was the bug); strmtyp 0/2 with substreamid == 0, legacy AC-3, and reserved strmtyp 3 start. Reserved 3 starts regardless of its id bits, because its BSI layout is undefined so those bits cannot be trusted — an unknown frame is neither merged into an unrelated programme nor silently discarded. The frame set stays ONE sample rather than being split into a separate track for the associated service, and the reasoning is in the module doc: a substream numbered 1..7 with no substream 0 is not conforming, so extracting one would mean renumbering ids and rebuilding frame sets — a transcode, not a remux. Programme selection is the player's job. A stream joined mid-frame-set (first sync is substreamid 3) is skipped with a debug and resyncs at the next id-0, mirroring the orphan-dependent rule: its mandatory id-0 substream was never seen, so it is neither decodable alone nor timeable. MAX_AC3_BUF 128 KiB -> 1 MiB, because an AU is now a whole frame set: worst case 8 independent x 9 substreams x 8192 B = 576 KiB, which the old cap could have dropped mid-hold. **The forced probe's two round-3 fixes cancelled each other.** CHUNK_SECTORS = 1023 exists (with a const assert) so every read starts on a 3-sector AACS aligned-unit boundary; the short-read fix advanced by actual bytes, making the advance a non-multiple of 3. Every later read was then misaligned, DecryptingSectorSource refused it before reading, the stop became ReadFailed, and no verdict was asserted — so content-based forced detection silently fell back to the vendor label on exactly the encrypted discs the 1023 change was written for. A partially-satisfied read now advances only by whole aligned units and re-reads the <=2 residue sectors from the next boundary, feeding only the aligned prefix so nothing is double-fed and no partial unit reaches the parsers. A read that fully satisfies its request still advances by all of it. When less than one aligned unit comes back the bytes are fed and the same LBA is retried twice before stopping, so a starved source cannot spin — verified by raising the retry limit and watching the test hang. Verified red independently here: reverting substream_role to strmtyp-only fails eac3_additional_independent_substream_stays_in_the_frame_set (6 access units where 3 are correct — the doubling, literally) and the mid-frame-set resync test. Reported, not fixed: dec3_box still hardcodes num_ind_sub - 1 = 0 and num_dep_sub = 0, so it under-declares any stream carrying additional independent or dependent substreams now that frame sets arrive whole. DolbyConfig has no fields for either; a real fix needs the parser to surface observed substream counts. That file is another lens's this round. Unverified: no real multi-programme DD+ stream exists here, so defect 1 rests on synthetic Annex-E fixtures. The retail DD+ check (No Time to Die, all substreamid 0) confirms single-programme discs are unaffected. |
||
|
Matthew Jackson
|
5f8dc392c0 |
Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the minimum, this goes to current stable and fixes what that costs. The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54 clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy collapsible_if lint in between. Stopping at the minimum would have been the most expensive choice available. Roughly 47 lints across the eight repos, the large majority auto-fixed: libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6, freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced by enumerate, and a loop whose first let-else became a while-let. Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only appear once the crate DECLARES 1.89 or later, because let-chains become available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87 reports clean and is meaningless — gate with the real precommit script, which is also the only thing that covers build scripts. The pin still sits below the Mac default, so it keeps doing its job: catching lint drift locally before CI sees it. |
||
|
Matthew Jackson
|
a32373ff40 |
Fix fifteen defects across perf, resource, panics and key hygiene
All 21 findings held up under verification; 15 fixed here, 6 deferred to files another agent held this round, 0 rejected. **A defect in my own round-2 probe fix.** CHUNK_SECTORS was 1024, and 1024 % 3 == 1 — verified — so every chunk after the first was misaligned against the 6144-byte AACS aligned unit and would be REJECTED by DecryptingSectorSource's alignment gate. On an encrypted disc the forced-subtitle probe I added last round would have read almost nothing past its first chunk. Now 1023 sectors (341 aligned units) with a const assertion that fails the build if it stops dividing, plus set_unit_base per extent so the source's gate is anchored where the extent actually starts. **The same probe skipped sectors on a short read**, advancing by the REQUESTED count rather than the bytes actually returned, so a partial read silently left a gap in the middle of the evidence. It now advances by n/SECTOR_BYTES and clamps n to the buffer. **Its cache key omitted the PGS PID set**, so a playlist declaring an extra subtitle PID got another playlist's verdict for a track that had never been probed. And the key was the whole extent list, so partial clip sharing missed entirely. Both fixed by keying (start_lba, sector_count, pid) — and per-extent keying was shown SOUND rather than assumed: ForcedTracker is two monotone booleans, so per-extent evidence composes by field-wise OR, order- and grouping-independently. Making that honest required per-extent demux state, so an extent's evidence comes only from its own bytes, and memoising only extents whose read reached a designed stop. **A reachable panic in the timeline.** mkvstream::parse_block accepts a TimestampScale up to i64::MAX, so a video frame can set high_ns = i64::MAX and the next passive frame panicked adding the backstep. In release it wrapped negative instead, firing the straggler clamp for essentially every passive frame — audio and subtitles rewritten onto the wrong point of the output timeline. All four sites saturate. **A public constructor divided by zero**: PrefetchedSectorSource::new_with_events with unit_align == 0. Now InvalidInput, matching its batch_sectors sibling. **Two Debug impls printed key material.** DiscInputs (volume_id, mkb, unit_key_ro, samples) and UnitKeyFile both derived Debug. Nothing logs them today — fixed as prevention, because the next tracing::debug! someone adds is the leak. A doc claim that DiscInputs "contains no secrets" was false and is corrected. **An env-var multiply could overflow** in file_sector_source; now bounded at 64 GiB like its writeback sibling, with the parse split out so the bound is testable without touching process env. **The mp4 demuxer allowed one sample per file byte** — ~64x RAM amplification. Now file_len/16, since only vide/soun tracks are indexed and the shortest legal AC-3 frame is 128 bytes. **Two pipeline concurrency defects**: a consumer apply() error was invisible to the producer, and abandon/finalise had a TOCTOU where a caller could report an unfinalised output. Both fixed with compare-exchange state rather than a bool. **Two per-frame copies removed**, both MEASURED rather than reasoned: the AU assembler now hands its allocation to the frame (same pointer, unchanged capacity, proven by asserting the pointer) and tsmux reuses one Annex-B buffer across frames. Both keep capacity deliberately — a naive split_off would have cost more than it saved. **A comment pointed at the wrong file** for a mirrored constant; the mirror is now compiler-enforced with a const assertion converting 90 kHz ticks to ns, so drift fails the build. Deferred to another agent's files, all confirmed: detect_rate's fractional-twin snap, the mp4 reserve's u32 truncation, round_up_grain's overflow, the quadratic base-key gap fill, and MkvStream's frame cap counting frames rather than bytes. Every fix verified red by reverting it. Also noted for later: DecodeSampleSet still derives Debug over multi-MB of on-disc ciphertext. |
||
|
Matthew Jackson
|
13897e14f0 |
Resolve the forensic key map once per disc, not once per playlist
resolve_content_key_map loops every title into resolve_mux_key_map, which called resolve_fmts_key_map FIRST — before the CpsUnitCache — and on an FMTS disc returned immediately. So every playlist re-derived facts that belong to the DISC: a full UDF walk plus /AACS/IndividualSegment.tbl, and on an FMTS disc the anchor probe, the 32-index phase probe, and a fetch.fmts_indexes round trip. On a 60-playlist disc, measured on a synthetic fixture: 840 -> 14 metadata reads, 2,400 -> 40 probe reads, and 60 -> 1 key-service calls. Worst case before was up to 256 probe reads and 32 key-service calls per title. The 60 redundant key-service round trips are a strong candidate for the keyserver storm seen in the field. Two memos behind a pub(crate) DiscKeyCache. The table memo (UDF walk + tbl parse) is disc-invariant outright — nothing in that path mentions the title — and runs on EVERY disc, so a plain BD benefits too. Only the deterministic negatives are memoised as "not FMTS"; a DiscRead fault propagates uncached so a later title retries. A blind once-per-disc hoist of the PROBES was rejected as unsafe, and this is the load-bearing reasoning: the title enters through clip_byte_to_lba, which decides which segments are addressable and which LBA every probed clip byte reads from, so two titles with different extent lists probe different physical bytes. A hoist would serve title B an answer derived from title A's media and could silently turn a per-title FmtsKeyMissing into a success. The extent list is the ONLY per-title input, so keying on it is exactly sufficient — matching the ForcedProbeCache precedent. Result-identity was proved, not assumed: NEITHER probe reads the key pool. Verified here independently — probe_fmts_index_keys takes no keys parameter at all; index keys come from `fetch`, and the anchor's reply feeds the phase probe. So the pool's growth across titles, the one thing that does change between calls, cannot move a memoised value, and the result is order-independent. A test resolves three titles through a shared memo and through fresh memos and asserts both the per-title ranges and the final key pool (keys, slots, order) are identical. Not memoised, deliberately: fail-loud FmtsKeyMissing, and any run where an index hit a read fault — that is a property of a transient drive fault, not of the extents, and caching it would spread one bad read across 59 playlists. A fully-memoised title now does zero I/O, which made the old in-loop halt polls unreachable for it, so a check_halt on entry was added with a test that cancels after warming the memos. Also corrects my own overstatement from last round: the CpsUnitCache doc now says plainly that on an FMTS disc it removes NO reads, because this function returns before the extent loop ever runs. Five mutations, all verified red. Pre-existing bug flagged but not fixed: filter_addressable_segments only checks that a segment's START byte maps to some LBA in the title, so a play-all playlist can pass the filter while mapping segment bytes into the wrong clip, whose anchor then returns empty and aborts the sweep. |
||
|
Matthew Jackson
|
38aa895038 |
Memoise multi-CPS key-map sampling per extent, not per title
resolve_content_key_map calls resolve_mux_key_map once per title, and on a
multi-CPS disc that path issues 8 random single-unit reads per extent. A disc's
playlists overwhelmingly reference the same few clips — main feature, play-all,
per-chapter and seamless-branch variants — so the same physical extents were
re-sampled from the drive once per playlist. On a 60-playlist / 15-clip disc
that is ~2,400 non-sequential 6144-byte reads, roughly 8 minutes of pure seeking
at 200 ms per seek, before the mux starts. Now ~600 reads.
Keyed per EXTENT — (format, start_lba, sector_count) — rather than per title's
whole extent list, which is finer-grained than the forced-subtitle probe's cache
and strictly better here: a play-all playlist sharing 4 of 5 extents with the
main feature still hits on those 4.
Why a cached pool index is provably identical to a recomputed one, verified
rather than assumed:
* `pick` iterates the pool IN ORDER and returns the FIRST index whose key
decrypts a sample to clean.
* The pool is APPEND-ONLY. Checked across the whole crate: only `push`, with no
insert/remove/clear/retain/sort/dedup/truncate/drain/swap/reverse anywhere.
So appended keys can only land AFTER a matched index, and the first match for
the same samples cannot shift.
* The samples are a pure function of the three values in the key, read from
read-only optical media.
Two outcomes are deliberately NOT cached, which is what makes this safe rather
than merely faster:
* the inherited index (`None if samples.is_empty() => last_idx`) is per-TITLE
state, not a property of the extent — caching it would let one title's
carry-in index leak into another title's clear extent, i.e. a WRONG key;
* the fail-loud DecryptFailed verdict, so a retry after a key source banks the
missing key re-samples instead of inheriting a stale answer.
Halt is still polled before the cache lookup, so cancellation is unchanged.
resolve_mux_key_map keeps its exact signature and delegates with a fresh cache,
so there is no public API change. ContentFormat gains Eq + Hash (additive).
Four tests, and the two mutants that matter both verified red: disabling the
cache short-circuit fails the hit and recompute-equivalence tests, and wrongly
caching the inherited index fails multi_cps_inherited_index_is_not_cached.
|
||
|
Matthew Jackson
|
807eb053ca |
Only assert a forced-subtitle verdict the read actually supports
probe_and_set_forced broke out of its read loop on a read error but still
applied whatever partial observation it had accumulated as an authoritative
verdict, overwriting the vendor-label-derived forced flag. A disc that faults
early could have a correct flag replaced by a guess from a fraction of the data.
The file already got the zero-observation case right — it deliberately leaves
the vendor flag alone rather than "assert not-forced from having seen nothing".
The defect was that a PARTIAL observation cut short by a fault was treated as
complete.
The fix rests on the two verdicts not being symmetric evidence.
settled_not_forced() is POSITIVE evidence — a non-forced display set was
actually seen, and no unread data can retract it. is_forced() is an ABSENCE
claim — display sets were seen and none was non-forced — which is only sound if
the read got far enough for the absence to mean something.
So every loop exit now yields a named StopReason, and absence claims are
asserted only for a designed stop:
* Exhausted / Budget → conclusive. The budget is deliberately conclusive: the
natural exit is "every track settled not-forced", which a genuinely forced
track never satisfies, so the budget is the ONLY path by which a real forced
verdict is ever reached. Treating it as inconclusive would disable forced
detection entirely.
* Halted / ReadFailed → inconclusive. A cancelled probe's cut-off point is as
arbitrary as a faulted one, so its absence claim is worth no more.
Evaluated per track, matching the existing observed() gate: a track that already
saw a non-forced set keeps its sound verdict even on a truncated run, while a
forced-so-far sibling keeps the vendor flag.
An inconclusive run is also NOT memoised. The cache key is the extent list and a
disc's playlists share clips, so caching a truncated run would replay one read
fault onto every playlist referencing those extents and deny any later title the
chance to re-read them.
Four tests; three of them verified red by forcing absence_is_conclusive() back
to always-true (the old semantics), while the budget test correctly stays green
either way — confirming the guard was not over-corrected.
Also moves the function doc comment back onto probe_and_set_forced; the earlier
probe commit left it attached to the ForcedProbeCache type alias.
|
||
|
Matthew Jackson
|
a9dc3d7244 |
Make encrypt_unit report a refused slice, and expand its key once
Two defects in the encrypt_unit promoted to public API last round, both found by round 2 auditing that new code. It returned silently without encrypting when the slice was shorter than ALIGNED_UNIT_LEN. Its own contract requires the caller to set the container's encrypted flag BEFORE calling — the header is the key seed — so a silent no-op leaves a unit advertised as encrypted while still carrying plaintext, with nothing for an authoring caller to check. It now returns bool and is #[must_use], so ignoring the refusal is a compile-time warning; every call site was updated to assert on it. bool rather than Result deliberately: a wrong buffer length is a programming error at a library boundary, not a disc condition, and a new Error variant would mean a new numeric code plus its rendering in another repo. It also drove CBC from the single-block aes_ecb_encrypt, rebuilding the AES key schedule for each of the 383 blocks in a unit — an order of magnitude slower than its inverse, which expands the key once via aes_cbc_decrypt. The missing counterpart aes_cbc_encrypt now exists alongside it, and encrypt_unit calls it, so the two directions are symmetric in structure as well as in result. For an authoring caller encrypting a 90 GB image that removes ~5.6 billion redundant key expansions. New test pins the boundary: ALIGNED_UNIT_LEN - 1 returns false and leaves the buffer byte-identical, ALIGNED_UNIT_LEN succeeds. The existing round-trip and padding-asymmetry tests still pass, so the CBC rewrite is provably the same transform. |
||
|
Matthew Jackson
|
a39045adf1 |
Bound the forced-subtitle probe, make it cancellable, and stop re-reading clips
The probe's only natural exit was "every PGS track has shown a non-forced
display set". A genuinely FORCED track never satisfies that, so on the common
authoring — a forced-narrative track for foreign dialogue — the loop read the
title's entire extent set at 2 MiB per call with no byte cap, no time cap and
no halt check. It was also invoked once per title rather than once per distinct
clip, and a disc's playlists overwhelmingly reference the same few clips (main
feature, play-all, seamless-branch variants), so the same physical extents were
re-read 30-150 times. The two defects multiplied: tens of GB, times the
playlist count, off an optical drive.
Reached via ScanOptions::probe_forced_subtitles, whose only consumer is
`freemkv info -v` (freemkv/src/disc_info.rs). The rip path leaves it off. So
the symptom is `info -v` never returning on an ordinary UHD, not a corrupt rip.
Three changes:
* PROBE_BUDGET_SECTORS caps a probe at 256 MiB. A forced track's display sets
appear throughout the title, so a bounded prefix classifies it; the budget
only decides how long we keep looking for a non-forced set before accepting
the forced verdict.
* ScanOptions::halt is now plumbed in and checked per chunk, so `info -v` is
cancellable. The probe previously took no halt at all.
* A ForcedProbeCache memoises verdicts against the title's exact extent list.
Keying on byte-identical input means a hit cannot change any result — it
only removes the re-read.
Verdict application is factored into apply_verdicts so the cached and freshly
probed paths cannot diverge.
Three tests pin the behaviour, each against a reader that counts sectors and
never ends: the budget stops at exactly PROBE_BUDGET_SECTORS, a cancelled halt
reads zero sectors, and a second title with identical extents costs no further
reads while a different extent list still misses the cache.
Also worth recording: the CLI acceptance harness never exercised `info -v`
against an optical drive — it reads ISOs from local SSD, where a full-extent
read is fast enough to hide both defects.
|
||
|
Matthew Jackson
|
d09ed76e07 |
Promote AACS unit encryption to real API, and assert decrypt byte-exactly
encrypt_unit becomes public library API rather than a #[cfg(test)] helper.
Authoring an encrypted disc image is a legitimate use of this crate, and
the capability was already written four times over: a pub(crate) test-only
copy in aacs/content.rs plus three hand-rolled duplicates in decrypt.rs,
sector/decrypting.rs and disc/extract.rs. All four now call one function,
removing ~110 lines of duplicated cipher code that could drift from
decrypt_unit independently.
It mirrors decrypt_unit's purity contract: crypto only, no encrypted-flag
handling, because where that flag lives is container-specific (CPI bits in
byte 0 for BD-TS, elsewhere for HD-DVD-PS). Callers set the flag BEFORE
encrypting — bytes 0..16 are the key seed left in plaintext, so touching a
header byte afterwards changes the key a decryptor derives. That footgun is
documented at the function and at every call site.
Two tests pin it: an exact round trip through both directions, and the one
place the pair is deliberately asymmetric — decrypt_unit restores
all-zero-on-disc packets to zero, and the test proves an all-zero plaintext
packet enciphers to non-zero bytes so it is never mistaken for padding.
That asymmetry was previously only prose.
Two decrypt tests were also weaker than their own names:
* aacs_clear_trailing_partial_passes_through asserted only is_ok(), so a
mutant corrupting the clear partial while returning Ok passed. It now
snapshots the buffer and asserts byte equality, matching the
none_keys_is_noop pattern already in the file.
* aacs_decorator_decrypts_encrypted_unit_via_map checked only that 0x47
reappeared at the 192-byte stride, leaving corruption in the other 6112
bytes undetected. The plaintext is fully known, so it now asserts
byte-exact recovery against it.
Both were verified red first by mutating the production path.
|
||
|
Matthew Jackson
|
840cb9aef0 |
Drop the comment that promised tests this crate no longer has
The bisect ReadAction regression tests moved to freemkv-engine with the recovery strategy, but their explanatory block stayed behind — seventeen lines describing a `let _ = handle_read_error(..)` bug and asserting "the tests below prove the required ReadAction values are produced". There are no tests below it; handle_read_error is not even resolvable here any more. Anyone auditing whether that bug is still guarded would read this and conclude yes. The block moved to the engine alongside the tests it describes. |
||
|
Matthew Jackson
|
d8e5b97c86 |
1.6.0: remove recovery strategy (moved to freemkv-engine) + trim dead surface
The sweep/patch recovery strategy, mapfile, retry-decision state machine,
section-recover, and damage classification move out of libfreemkv into the
new freemkv-engine crate. libfreemkv keeps the raw single-shot read and
SCSI-fact translation (SenseFamily stays in scsi).
- Delete disc/{sweep,patch,mapfile,read_error,section_recover}.rs, the
Disc::copy/sweep/patch methods, the Copy/Sweep/Patch option+result types,
classify_damage/DamageSeverity, progress_snapshot_from_mapfile, and the
three recovery integration tests.
- Trim public surface the recovery deletion orphaned: delete the dead
READ_PIPELINE_DEPTH const, the write-side SectorSink/FileSectorSink (no
consumer), and the DriveSpeed enum (its one live use — set max drive
speed — becomes Drive::SPEED_MAX_KBPS). Make mapfile_path_for,
decrypt_sectors_mapped pub(crate); gate NoopEvents to test.
- Version 1.6.0.
|
||
|
MattJackson
|
8822c29905 |
disc: add DiscTitle audio_streams/subtitle_streams/video_streams accessors
Typed iterators over the audio/subtitle/video streams, cleaner than matching on the Stream enum for the common iterate-the-tracks case (stream selection, the desktop UI info panel, disc-info). Additive; all lib tests pass on 1.86. |
||
|
MattJackson
|
508a2c3376 |
disc: promote locate_ranges to pub (engine multipass reads it)
Small pub promotion + fmt one-lining. The relocated multipass progress reporting in freemkv-engine needs locate_ranges externally. No behavior change. |
||
|
MattJackson
|
a02bbbdd24 |
scsi: promote SenseFamily to a lib-level SCSI-fact primitive
Moved SenseFamily::from_sense_key + is_wedge_family from disc/read_error.rs into scsi/mod.rs (with its own tests) and re-exported at the crate root. This is pure SCSI sense-code classification -- objective hardware fact, zero recovery-policy opinion -- so it belongs in the library primitives, unlike the retry-DECISION state machine (ReadCtx/PassSummary/ReadAction/ handle_read_error) built on top of it, which is freemkv's specific recovery strategy and is moving to freemkv-engine next. disc/read_error.rs and disc/section_recover.rs now import SenseFamily from crate::scsi instead of defining/re-exporting their own copy. No behavior change. Precommit green on Rust 1.86 (fmt+clippy+test). |
||
|
MattJackson
|
ba8114f29b |
disc: promote resolve_content_key_map + encrypted_content_ranges to pub
The upcoming freemkv-engine crate needs both to build the multipass sweep/patch recovery strategy externally over Disc's public API. Everything else sweep/patch touch on Disc was already pub; these were the only two gaps. No behavior change -- visibility only. |
||
|
Matthew Jackson
|
e0456c72da |
mux: thread halt into live AACS key-map resolution; cover Session arm
Round-2 follow-ups to
|
||
|
Matthew Jackson
|
3bd2fd23b0 |
Fix audit findings: DTS AMODE bound, key-fetch negative memoization, PGS probe coverage
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just 0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements; only 16-63 are reserved. a reference decoder the spec per-AMODE channel table confirms 10-15 are decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal multichannel core frames as undecodable, silencing recoverable audio. Add a regression test (literal 0..16 range) that fails if the bound reverts to 10. - keysource: only memoize a NEGATIVE (empty) key-fetch result when every source genuinely ran and none held the key — never when a source Err'd (network down, unreachable). A transient outage was being cached as a permanent "no key" for the fingerprint, permanently dropping a unit that could be recovered once the source came back. Thread an `errored` flag out of the drivers and gate the cache insert on it. Tests cover both the recover-after-outage case and that a genuine absence is still memoized. - pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS display sets through the full demux -> parse -> observe -> apply path, both a forced verdict landing and a non-forced verdict clearing a vendor flag. - mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD). - scan_iso test: add independent fixture expectations (volume id) so the parity test is no longer purely tautological against a re-run of the same composition. |
||
|
Matthew Jackson
|
b9568242df |
libfreemkv: 10-phase release audit fixes (v1.5.2..HEAD)
Multi-round audit of the decrypt/AACS/mux-codec refactor. Fixes, in descending severity: - mux/mp4/read.rs: bound untrusted-input allocations. `sample_budget` now also capped by file_len (a fixed-size stsz claiming count=u32::MAX can't inflate the Vec<SampleRef> past the file's own size); trak scan capped at MAX_TRACKS matches; find_box() takes only the first match (cap=1) instead of materializing every match. Removes dead find_boxes wrapper. - disc/mod.rs: merge_content_key_ranges now UNIONS same-key overlapping ranges (coverage-preserving) instead of dropping the non-overlapping tail, which silently left encrypted LBAs uncovered -> ciphertext passthrough in the whole-disc sweep/patch map. Different-key overlap (malformed) still dropped to keep the set disjoint. - sector/decrypting.rs: remove dead unit_key_idx field + with_unit_key_idx setter (vestigial from the pre-keymap trial-decrypt design; AACS is map-only now). Fix stale docs. - decrypt.rs / resolve.rs / error.rs / extract.rs: doc/comment drift from the refactor (AacsKeyMap positive-map semantics, resolve_mux_key_map doc reattachment, decrypt_sectors_in_content legacy-alias, E_MP4_INVALID meaning, multi-CPS orphan by-design note). Test coverage (all mutation-verified real): - DTS NeedMore force-flush buffer bound; FLAC/MPEG-audio PTS carry-forward; mp4 mdhd timescale=0 divide-by-zero guard, MAX_TRACKS cap, sample-count file_len bound, MAX_ALLOC_BYTES cap under inflated file_len. - resolve_fmts_key_map: extracted filter_addressable_segments, resolve_tie_phase, fill_base_key_gaps as pure behavior-preserving helpers, each unit-tested (segment filter, phase-tie arms, gap-fill gaplessness over every extent). |
||
|
Matthew Jackson
|
197489fb7c |
Cache AacsKeyMap key indices; extract + test whole-disc range merge
- AacsKeyMap now derives its distinct key-index set once at construction (from_ranges_phased) instead of re-allocating/sorting it on every decrypt batch; key_indices() returns the cached slice. - Extract the whole-disc content-map range merge out of resolve_content_key_map into merge_content_key_ranges and cover it: sort/disjoint, shared-clip dedup, overlap drop, adjacent-kept. |
||
|
Matthew Jackson
|
0acb326079 |
Fix FMTS per-title resolve, extract multi-CPS keying, trailing-partial guard
- resolve_fmts_key_map: filter segments to those addressable within THIS title's extents; a title with no forensic content (menu/extras playlist, or a different clip) returns Ok(None) and takes the base Unit-Key/CPS path instead of hard-failing FmtsKeyMissing. Previously the first non-forensic title aborted the entire whole-disc sweep (resolve_content_key_map iterates every title) and blocked muxing any non-main title. - FMTS phase probe: an even/odd is_clean tie now only fails loud when BOTH halves are 0 (no clean decrypt). A both-clean tie is source-zero padding (is_clean is true for any key on all-zero content) — the key is valid, default Even, never abort the rip on a padding-heavy sample. - extract_tree: multi-CPS discs now build the exact per-CPS content map (resolve_content_key_map) instead of a blanket key-0 map that silently mis-decrypted every secondary-CPS file into garbage. Single-CPS keeps the blanket key-0 map (one key opens every unit, incl. orphan clips). - decrypt_sectors_mapped: a trailing partial unit that is inside a mapped range AND flagged encrypted in its clear seed now fails loud (a CBC fragment split across a boundary can't be decrypted) instead of being emitted as clear. New aacs_unit_seed_encrypted reads the flag on a partial. - Correct the stale decrypt_sectors doc (AACS arm now always errors; AACS decrypts only via decrypt_sectors_mapped). |
||
|
Matthew Jackson
|
88e58bfc95 |
Decrypt is keymap-only: sweep/patch/extract, no AACS trial-decrypt
Every AACS decrypt now goes through the resolved key map (decrypt_sectors_ mapped): the map keys each content unit up front and a missing key fails at resolve time. The old trial-decrypt path — try each held key per unit, keep the first-tried plaintext on a miss — is gone; decrypt_sectors_impl's AACS arm now fails loud (reaching it means a reader was built without its map, which would silently apply a wrong key). CSS (self-descramble) and the clear no-op path are unchanged. Disc::sweep and Disc::patch resolve a whole-disc key map up front for a decrypting pass (the fetch secures any missing CPS-unit key, fail-loud) and decrypt via the map — clear nav/filesystem sectors are in no range and pass through, so the separate content-range gate and the reactive per-unit key-fetch recovery are no longer needed. extract_tree keys every unit with the base Unit Key through the map (its encrypted-flag gate skips clear files). Multipass sweeps stay --raw. Removes the obsolete non-mapped-AACS trial/gate/recovery tests (the mapped path and resolve fail-loud are tested directly). |
||
|
Matthew Jackson
|
1eb6910bdb |
Harden mux + decrypt paths; fail-loud on unresolvable keys
mp4 demuxer (untrusted input): bound every allocation sized from a box field (stsz/stco/stsc counts, stts/ctts run-lengths, per-sample and moov sizes, plus an absolute cap so a sparse file can't inflate file_len); guard the parse_stsd slice and a zero mdhd timescale; cap track count so the per-track PID can't overflow; rewrite read_moov to handle size==0 / size<8 / 64-bit largesize; parse esds/AudioSpecificConfig for AAC; write tkhd duration in the movie timescale. decrypt: resolve_mux_key_map now fails loud on an extent no key can classify instead of inheriting the previous extent's key, so a keymap never silently carries a wrong key; the sweep/patch key-fetch recovery fails loud when a unit is still unresolved after the retry. AACS: reject inverted forensic segments in both range builders; compare the forensic index in u16 space so an out-of-range value can't truncate onto a valid u8 index. RECOVERED_ERROR no longer latches the damage zone, preserving the 30s wedge cooldown for a following hard error. audio: AAC/MP2/MP3/FLAC carry the last PTS across a PES with no timestamp; the DTS-HD extension-sync search is bounded to after the core; the MP4 16.16 sample-rate field saturates. demux_sink records the video reference before the kind filter so audio:// / sub:// keep multi-clip PTS continuity and the DELAY tag. Remove a dead error variant and the AACS-unsupported-video code; codec comments cite the primary format specs; assorted doc/naming fixes and regression tests throughout. |