0d4aab99df68e2de827c0cb99020712497d6501a
348
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Matthew Jackson
|
0d4aab99df |
Stop naming specific commercial discs in the AACS and codec comments
The same scrub as the previous commit, over the files it did not reach: the variant-MKB layout notes, the 2.1 segment index observations, the PPS-revert regressions and the playlist-twin tiebreak. Measurements keep their numbers — "a v70 `0x2d` body = 46_100*2 + 16" is the useful part, and the title it came from never was. |
||
|
Matthew Jackson
|
b93d10082d | Merge branch 'fix/pgs-forced-probe-sampling' into dev | ||
|
Matthew Jackson
|
b68765fe84 |
Stop naming specific commercial discs in comments and tests
Fifteen references across six files named the discs a defect was first seen on. The parser leak found earlier was not an isolated slip — the same habit runs through the mux comments, the changelog and the AACS content verdict, where a title name was standing in for the shape of the problem. Every one is replaced with the property that actually mattered: a multi-clip title, a UHD Dolby Vision profile 7 dual-layer stream, a disc carrying an authored-bad TS packet. The comments are more useful for it — the reader needs to recognise the shape on a disc they have, not the one we happened to have. `SEG_MainFeature` stays: the parser matches on that literal, so it is a format token rather than a title. |
||
|
Matthew Jackson
|
28d5897b86 |
Apply the shape test to a mixed track too
A track that flags some of its own display sets and not others proves the authoring house makes the distinction, so it needs no sibling to corroborate the flag being in use -- but it still has to look like a full dialogue track before a forced label is cleared. A small track with a couple of flagged signs is a forced track, and demoting it is the mistake the shape test exists to prevent. |
||
|
Matthew Jackson
|
8ffce6b621 |
Sample PGS across the title instead of reading its head
The content-based forced-subtitle probe spent its whole 256 MiB budget
on the first sectors of a title. A feature's subtitles begin minutes in,
so the probe read the opening logos, hit the budget, observed no display
set at all and contributed nothing to any verdict — the vendor label was
always the only input.
The forced predicate is asymmetric: one non-forced display set disproves
forced permanently, while proving forced needs the whole track, and
genuine forced tracks are tiny where full tracks are huge. So the same
budget is now SPREAD over each extent in ~16 MiB windows placed on the
AACS unit grid, sized in proportion to the extent, ending at the extent's
end. Cost is unchanged; placement is not.
Also:
* Per-track early exit. A track that is disproven (and whose label
needs no correcting) stops asking for budget; an extent that owes
evidence only for such tracks is skipped outright, and evidence
already in the cache is never demuxed a second time.
* Content may now DEMOTE a wrong vendor forced flag, in the probe and
in the muxer, behind one shared guard: absence of forced_on_flag
only means something if some other track demonstrably uses it, and
the track must have the shape of a full dialogue track rather than
of a forced-narrative one. On a disc where no track sets the flag,
nothing is demotable.
* A sampled or budget-cut extent's evidence is memoised with the
COVERAGE behind it. It used to be filed under the extent's full key
and replayed to playlists that would have read far more of the clip,
turning a prefix into an absence claim about the whole extent.
|
||
|
Matthew Jackson
|
cf7ee69fd5 |
Record the source, not the destination, in the FVI header
The `fvi://` arm of `output()` passed the destination `.fvi` path as `FviSink::create`'s `source_path`, so every index named itself as its own source. `SourceInfo::default()` supplied the rest, making `source.medium` always "file" and `source.title` always 0 — three header members wrong, where FVI_FORMAT.md §6.2 defines `source` as describing the input. Beyond the wrong data, it made the output unreproducible: two machines indexing identical bytes emitted different files purely from where they wrote them, and a local filesystem path leaked into a shareable file. `output()` cannot see the source, so thread the provenance down from the driver, which can: `mux_stream` derives a `SourceInfo` per `MuxInput` arm and passes it through `drive_mux` to `output()`. Per the one-method-per-action rule this is a signature change, not an `output_with_source()` variant; the parameter is `Option<&SourceInfo>` so a caller with no provenance declares none rather than back-filling the destination. `SourceInfo`/`Medium` become public API. What each arm can honestly reach: - Session: everything — device path, the caller's title index, the title's playlist, the scanned volume id. - Url: the source URL, its scheme's medium, `title_index`, and the playlist off the opened stream's scanned title. - Iso: the image path and playlist. The title index is not in `MuxInput::Iso` (it carries a scanned `DiscTitle`, which has no index), so it stays 0. - Live: medium and playlist. The reader is an opaque `Box<dyn SectorSource>` with no path, and again no title index. Unreachable members are left empty rather than guessed — the sink already omits the empty ones. |
||
|
Matthew Jackson
|
e008e71a17 |
Add the missing direct coverage for parse_stss
Every other sample-table parser (stco, stsc, stts, ctts) had a "count lie" test proving the declared count is bounded by what the box actually holds, and stco/stsc had their own arithmetic pinned. parse_stss had neither - no test in this file ever called it with real entries, only with a too-short buffer. Added the same two: two distinct entries read from their own offsets (catching the o = 8 + i*4 arithmetic and the per-entry bounds check), and a declared count of 3 backed by only 2 real entries (catching the same "trust the box, not the count" contract the other four parsers already had). |
||
|
Matthew Jackson
|
c59e1e3342 |
Pin the sample-table parsers' short-buffer safety and byte arithmetic
parse_stco, parse_stsc, parse_stts, parse_ctts and parse_stss all open with the same "if b.len() < 8" guard before reading count = be32(b, 4), but no fixture anywhere in this file ever called any of them with a buffer shorter than 8 bytes - every test builds a complete box. A <-to-== mutant of that guard only rejects a buffer of EXACTLY 8 bytes and lets everything shorter fall through to an out-of-bounds be32 read, and nothing was exercising that fall-through to notice. One test now drives all five through every length from 0 to 8. Also: co64's 8-byte offsets were never actually built by any co64 fixture in this file (only the 32-bit stco path was exercised), so its manual byte-by-byte u64 assembly was unconstrained the same way parse_elst's was. And sample_offsets's sidx only advanced correctly by coincidence in every existing fixture, because none of them placed three or more samples in a single chunk back to back - the only shape where reusing the wrong sample's size becomes observable. Documented the <=8 boundary as equivalent across all five parsers (and the matching start < end in sample_offsets): the per-entry guard immediately below always breaks on the first entry at that exact boundary, so both branches converge on the same empty result. Confirmed by re-running each mutation against the full suite. |
||
|
Matthew Jackson
|
39d9714ae7 |
Pin parse_esds_asc's two independent boundary checks
asc_len == 0 and end > b.len() reject for different reasons - a useless zero-length ASC, and a truncated one - and nothing distinguished the || from an && that would only reject when both are true simultaneously, or the > from a < that would reject the common case of an esds with bytes after the ASC (more child boxes, padding) instead of only a genuine truncation. Documented the | in read_descriptor_len's accumulator as the same shift-then-mask equivalent already recorded in audio.rs's BitReader::read: the shift always vacates exactly the bits the mask fills, so | and ^ can't disagree. |
||
|
Matthew Jackson
|
dd940583c7 |
Pin mdhd_language's length boundary and three unasserted stsd codec paths
mdhd_language's guard was written as "b.len() < off + 2" (reject too short) but nothing distinguished that from "reject anything not exactly off + 2" - a buffer one byte longer than the minimum has to keep working, and a buffer missing the field entirely has to return None rather than read past the end. parse_stsd's mp4a path had no test at all: the AAC codec_private extraction depends on both codec == Aac and body.len() >= 28 being true together, and with no mp4a fixture anywhere in this file neither half of that condition, nor the boundary itself, was constrained. Also added the same header-length boundary check parse_elst already had (< 8 vs <= 8 - proved equivalent this time, since the very next guard on the empty slice catches the <= 8 case too), and one test walking every recognised audio fourcc (ac-3/ec-3/mp4a/the four dtsX variants) so a deleted match arm for any of them fails loudly instead of silently dropping that track. |
||
|
Matthew Jackson
|
4b7e4ddbb3 |
Pin find_boxes_capped's cap boundary and its size-field byte offsets
Nothing asserted the scan actually STOPS at cap rather than one match past it, or that the declared box size is decoded from its own four bytes rather than an adjacent one - every existing fixture used sizes small enough that all but the last size byte are zero, so an index slip reading the wrong byte would read the same zero and go unnoticed. |
||
|
Matthew Jackson
|
5222458411 |
Pin Stream::read's MAX_ALLOC_BYTES boundary and that write always rejects
Stream::read has its own s.size > MAX_ALLOC_BYTES cap, a separate call site from read_moov's over the same policy - checked they agree (both reject strictly greater than the cap, exact cap allowed) and they do, so this is not one of tonight's one-policy-two-copies bugs. But the boundary itself and the one-byte-over case were unasserted, and Mp4Reader::write returning an error (mp4:// is read-only) had no test either. Building Mp4Reader directly in the test (its fields are private but visible within this module) over the existing FakeBigReader avoids a real 256 MiB backing file for the boundary case. |
||
|
Matthew Jackson
|
dd5118ee9c |
Pin per-track handler routing, PID arithmetic and the shared sample budget
Nothing asserted that a hdlr other than vide/soun gets dropped rather than folded into the audio branch, that the per-track PID formulas (0x1011/0x1100 + track_idx) use the right operator and the right operand, that a sample-less track still advances track_idx for the next one, or that the cross-track sample_budget is actually decremented (as opposed to grown or divided) by each track's real count. All four were reachable with a single track_idx == 0, which made every existing fixture blind to +/-/* confusion on these sites - track_idx never moved past 0 in any of them. Also let audio_trak_missing omit stsz, needed to build a sample-less track for the track_idx test. |
||
|
Matthew Jackson
|
a8db2435cd |
Pin elst byte-offset decoding and a zero-timescale boundary in elst_offset_ticks
parse_elst's existing tests used segment_duration/media_time values that were almost all zero or 0xFF bytes, so an index slip in the version-1 byte extraction (reading a neighbouring byte, or one outside the entry entirely) could return the same value by coincidence and the test wouldn't notice. Added a fixture with every byte distinct and nonzero so any wrong offset is caught. elst_offset_ticks's `empty_movie_ticks > 0` guard on the Some(mts) arm looked like a pure optimisation, but shifting its boundary lets an empty_movie_ticks == 0 call fall into the division instead of skipping it - and a zero movie timescale (unreachable through from_reader, which filters it, but not through this function's own contract) makes that division panic. Pinned the boundary directly so the function stays safe on its own terms. Documented nine further mutants as equivalent rather than chasing them: media_edits/odd_rate and the None-arm's empty_movie_ticks check only gate a tracing::warn!, never the returned offset, and parse_elst's b.len() < 8 vs <= 8 boundary computes the same empty Vec either way once available = 0 is worked through. Confirmed by re-running each mutation against the full test suite. |
||
|
Matthew Jackson
|
ff18d4c3c8 |
Close the MEDIUM mutation gaps across transport, labels and codecs
The remaining triage items after tonight's HIGH fixes: 1,290 lines, almost all tests. Covers disc/mod.rs's DVD scan path (with real minimal VMG/VTS IFO fixtures rather than mocks), drive/mod.rs, labels/class_reader.rs and labels/mod.rs — the two biggest untriaged survivor clusters in the crate — plus hevc.rs and ps.rs. One production change, and it is an extraction rather than a behaviour change: MacScsiTransport::open mapped the shim's negative failure sentinels to typed errors inline, where nothing could reach it without a real IOKit FFI call. It is now map_shim_open_error, so the mapping can be pinned. It matters because collapsing -5 into the DeviceNotFound catch-all turns "another process holds the drive" into "no such drive", and an operator chasing the wrong problem is worse than a blunt error. Gate green on the pinned toolchain including the secrets scanner. |
||
|
Matthew Jackson
|
f8ed0b99f4 |
Pin read_moov's box-size boundaries and MAX_ALLOC_BYTES exactly
read_moov's forward-progress guard (box_size < header_len, OR'd with the EOF check) and the MAX_ALLOC_BYTES cap were only exercised on inputs well away from their boundaries, so a mutation testing pass found the exact edges unasserted: a size-8 (header-only) moov, a box that overruns the file by exactly the amount the OR/AND distinction can see, and a payload of precisely MAX_ALLOC_BYTES. Added a shared FakeBigReader (lifted out of an existing test's local struct so a new test can reuse it) to exercise the MAX_ALLOC_BYTES boundary without a multi-hundred-MiB backing file. |
||
|
Matthew Jackson
|
8189da1b0c |
Document why audio.rs's bit-packing | mutants are equivalent
Every mutation-testing survivor in this file is a | with ^ flip inside a bitstream packer: BitReader::read's accumulate step, the push closures in dac3_box/dec3_box/ddts_box, and the multi-field extractions in parse_eac3 and parse_dts. All nine are the same shape: shift an accumulator left by exactly the width of the next field, then OR it in, so the two operands never share a set bit and | and ^ agree on every input. Confirmed by running cargo-mutants against just these nine mutations after the existing test suite (which already exercises each function's field values) - all nine still survive, as expected for a genuinely equivalent mutant. Recorded the reasoning once at BitReader::read so nobody spends time chasing it site by site. |
||
|
Matthew Jackson
|
698ba36ae4 |
Document the pack_language and detect_rate equivalent mutants
Three pack_language mutants and two detect_rate boundary mutants survive mutation testing with no test able to close them, and it's not for lack of trying: they're equivalent by construction. Recording the proofs next to the code so nobody re-chases them: - (b[0] - 0x60) as u16, shifted << 10 then truncated to u16, is congruent mod 65536 to (b[0] + 0x60) as u16 shifted the same way, because 0x60 * 2 * 1024 is an exact multiple of 65536. The same swap on the second letter (shifted only << 5) is NOT equivalent, which is why only the first letter's mutant survives. - The two | with ^ mutations that OR the three packed fields together are equivalent because the fields (a lowercase letter minus 0x60, so 1..=26) always fit in 5 bits and never share a set bit once shifted into their 0/5/10 positions. - detect_rate's tolerance and tie-break comparisons only diverge from their <= mutants on an exact 0.5 fps distance or an exact tie, and a brute-force search over every achievable integer-nanosecond median found no case that lands on either boundary bit-exactly. |
||
|
Matthew Jackson
|
1eb8bdc9c7 | Merge branch 'mux-mp4' into dev | ||
|
Matthew Jackson
|
90a7fe2ff1 |
Assert the MP4 timing arithmetic, and name the faststart slack rule
MP4 track timing was numerically unasserted. Every operator in the PTS-to-ticks, duration, tkhd_dur and ctts chain could be flipped and the whole suite stayed green, because no test decoded an output file and checked a concrete number — the existing tests assert box presence and gross container shape only. That is the crate's worst failure mode: a title muxes "successfully" with silently wrong A/V sync or total duration, and nothing above can tell. The new tests build tracks with known PTS deltas and compare the emitted stts, ctts and tkhd.duration against computed values. Also lifted the faststart slack rule out of the match guard into faststart_fits(). A leftover hole of 1-7 bytes cannot be expressed as any ISO-BMFF box, since a box header is 8 bytes, so finish() must fall back to moov-at-end rather than write a free box that lies about its own size. The condition now has a name and a test instead of being an unexplained `g == 0 || g >= 8` inside a pattern guard. |
||
|
Matthew Jackson
|
1b95d346bb |
Cite the H.264 spec directly, not a reference implementation
The escape-stripper comments named a third-party decoder as the authority for the cumulative-zero rule. This repo is public and does not cite other implementations; the rule is specified in ITU-T H.264 §7.3.1, which is the citation that belongs here anyway. No behaviour change — comments only. The scan-secrets gate caught it. |
||
|
Matthew Jackson
|
48663c6a2f | Merge branch 'mux-codec' into dev | ||
|
Matthew Jackson
|
a05f1d4498 | Merge branch 'mux-mkv' into dev | ||
|
Matthew Jackson
|
3ecb2510e8 |
Assert what the MKV mux and demux actually produce
A mutation run over mux/mkv.rs and mux/mkvstream.rs left 148 survivors.
Reading them turned up no wrong code, but a lot of code whose output
nothing ever looked at. Most of that is on the read side: parse_track
had a dedicated arm for Language, TrackName, FlagForced, Video and
Channels and not one of them was checked, so a re-mux could have lost
the audio language, the subtitle forced flag, every track label, the
resolution and the channel layout with the suite still green. Ten of
the eleven CodecID comparisons were unasserted too — only HEVC was
pinned — so any of them could have been mis-wired and the stream would
have gone to the wrong parser. The round-trip test now writes a real
three-track title through the muxer and reads it back through the
reader, and a separate test walks every registered CodecID.
The BPS statistics tag was the worst of the write side. Its test
asserted `file_bytes.contains("800")`, which a wrong bitrate passes
trivially — 80000 contains "800". Both the tag and the back-patched
Segment duration are now decoded and compared to a computed number, on
a title that declares no duration so the whole max_block_ticks →
seconds → bits chain is exercised. Cue points get the same treatment:
their CueTrack and CueClusterPosition were never read back, on either
the keyframe path or the i16-forced-split path, which are two hand-
written copies of the same three fields.
The rest closes arithmetic that only a bad disc reaches: a zero frame
rate or zero display-aspect denominator (both divisions), a
TimestampScale that does not fit an i64, a cluster timestamp of exactly
i64::MAX, a TrackNumber of 65537 that truncates onto the valid track 1,
and the shortest legal Block at both VINT widths. Two tests separate a
clean end of stream from a device failure: swallowing the second one
truncates the output at a bad sector and reports the rip complete.
Also pinned: only the first video and first audio track may be default
(the de-duplication lives in MkvStream::create and had no test at all),
the activation trigger is the first VIDEO track rather than track 0,
the measured field order reaches the file rather than just the helper
that computes it, and a Blu-ray 3D base/dependent pair builds the merge
instead of shipping two unrelated H.264 tracks.
96 of the 148 mutants verified killed by hand. Of the remainder, most
are equivalent — disjoint-bit `|` that `^` cannot change, delete-arm
mutants whose fallback is the same constant, guards on tracing calls —
and the write_frame branch at 1452 is unreachable: a cluster is always
open by the time it can be entered.
|
||
|
Matthew Jackson
|
048f125879 |
Kill codec mutation survivors and unify H.264's duplicated escape stripper
The mux/codec parsers (startcode, h264, hevc, dts) had 300 surviving mutants between them, and it turned out to be for the reason you'd fear: the exp-Golomb readers and the AU-boundary bitstream scanners had essentially no direct unit coverage, only indirect exercise through full-frame parse() calls that never touched the actual edge cases. Direct fixes to test gaps: - The shared BitReader's read_ue truncation guard (`leading_zeros > 31`) and skip_start_code's 4-byte-vs-3-byte boundary check had no test at their exact boundary. Added tests that hit the boundary precisely; a `>=`/`==`/`<=` typo either rejects a legal 31-leading- zero code or reads one byte past the buffer. - H.264's private SpsReader duplicates the same read_bits/read_ue shapes with no tests of its own at all (only reached through multi-field SPS parsing, several fields deep). Added direct tests. - HEVC's per-AU trailing-zero strip after the last NAL (no start code following) walks `end` down to trim padding; a wrong-direction typo there walks off the end of the buffer instead of terminating - exactly the "loop must make positive progress on malformed input" class. Added a test with a zero-padded trailing NAL. - HEVC's SEI match guards (`sei_mastering.is_none()` / `sei_content_light.is_none()`) implement "first HDR10 value in the title wins" - untested, and a naive test using both-messages-per-AU can't even exercise the guards because the whole-scan early return above them already handles that case. Split into single-message- per-AU tests that actually reach the arms. - parse_mastering_display/parse_content_light_level's length guards were `< N` with no boundary test; one-byte-short input now confirmed to return None instead of indexing out of bounds. - DTS's drain_front collapses duplicate offset-0 PTS markers after rebasing; untested, and the visible effect (front_pts()) can't tell a working collapse from a broken one since it already returns the right marker either way - the actual defect is unbounded growth of pts_marks over a long recording, so the new test asserts the bound directly across repeated drains. - DTS's dts_core_samples/dts_core_sample_rate header-length guard and next_core_boundary's syncword-length guard got exact-boundary tests the same way; also caught a nblks `<<`/`>>` direction bug candidate in the mutant (confirmed the real code is correct, just untested). Real bug found and fixed, not just a test gap: H.264's parse_sps_high_profile_ext re-implemented emulation-prevention byte stripping inline (a window scan: match `00 00 03` at position i, advance 3, else advance 1) instead of calling the existing unescape_ebsp_prefix used by slice-header parsing. On a run of 3+ real zero bytes ahead of an 0x03 - non-conformant, but this is disc bytes, not a spec-clean encoder - the two disagreed: unescape_ebsp_prefix's cumulative zero counter (matching the H.264 reference decode process and libavcodec's RBSP extractor) treats it as an escape and drops the 0x03; the window scan treats it as real payload and keeps it, corrupting the SPS bits read after it. Extracted the shared rule into `unescape_ebsp` (parameterized on output length so both the 16-byte slice-header prefix and the unbounded SPS case can share it) and pointed both call sites at the one implementation. Added a regression test pinning the shared function's behaviour on the input that used to separate them. All new tests hand-verified against the actual mutation (operator flipped or guard replaced by hand, confirmed red, then restored) per the mutation-testing brief, not just written and trusted. |
||
|
Matthew Jackson
|
65dbcb1ca6 |
Close mutation-testing gaps in the TS/PS mux (ts.rs, ps.rs, tsmux.rs)
A 12,330-mutant run left 159 survivors across these three files, all from missing assertions rather than wrong code — every gap here is a test, no production logic changed. Two shapes accounted for most of them: - Buffer-cap constants (MAX_PES_BUFFER_TOTAL, MAX_PS_BUFFER, MAX_BD_PES_PAYLOAD, PES_BUFFER_INIT_CAP) were only ever read by tests through their own symbol, so a mutated `*`/`-` in the constant's definition changes what the symbol itself evaluates to and every self-referential assertion still passes. Pinned each against a literal computed independently in the test. - Several `>`/`==` boundary checks on framing lengths (MPEG-2 pack header, system header, BD-TS adaptation field) were only ever exercised with slack in the buffer, never at the exact byte the check exists for. Added exact-fit cases for the pack header, system header, and psi_payload_base's AF-consumes-everything boundary. Real, higher-value gaps closed along the way: - ts.rs's per-PID discontinuity_flag and the NULL-TS concealment marker both require adaptation_field_length > 0 before trusting the AF flags byte; neither branch had a test proving af_len == 0 (no flags byte at all, ordinary payload underneath) is left alone. - header_remaining (PES header spillover across TS packets) only had single-continuation-packet coverage, which can't distinguish `-=` from `+=`/`*=` because the corrupted value never gets read again. Added a case spanning two continuations. - ps.rs's parse_stream_id_extension (used for HD-DVD 0xFD routing) walks nine optional PES-header/extension fields with a `pos +=` each; only the PTS/DTS pair had ever been exercised. One test now arms every field and checks the walk lands on the right byte. - find_ps_boundary's `sc + 3 >= len` guard had no test at sc == 0 with a bare 3-byte start code, the case a `+` -> `-` mutation turns into a debug-mode subtract-overflow panic on ordinary tail-of-buffer input. - tsmux.rs: an oversized video access unit must go out as a single unbounded-length PES; the `is_video || small-enough` guard that enforces this had no test with a video frame actually over the bounded-PES threshold, so a `||` -> `&&` mutant survived (it would silently split a keyframe across several look-alike-independent PES units). Also pinned the PES-length and PTS big-endian encodes at values above 255 / with bit 29+ set, where a `>>`/`<<` swap first becomes observable. Every test above was verified by hand: applied the exact mutation, confirmed the test fails (or the specific panic fires), then reverted. Left unclosed, all confirmed equivalent by hand-tracing rather than just left alone: - Every `<<8 | byte` PID/length bit-combine (ts.rs pid/PAT/PMT parsing, ps.rs dvd_audio_pid/hddvd_extended_pid/parse_pts): the two halves never share a bit, so `|` and `^` produce identical output for every input - no test can tell them apart. - ts.rs's `af_len > 183` check in process_packet: fully subsumed by the `payload_start >= TS_PACKET_BYTES` check three lines later for every af_len that could trip it. - ts.rs's out-of-range `pid_index` sentinel (-1 vs 1): unreachable, since a TS PID is masked to 13 bits (max 8191) and the table is always sized to at least 8192. - A cluster of "push an empty slice on an exact boundary" mutants in tsmux.rs's write_pes_chain (offset < hdr_len, af_bytes stuffing guards): the guarded write becomes a length-0 write_all, a no-op either way. Not reached this pass, for lack of a clean seam within the time available - ps.rs's extract_packets bounded-PES-length exact-fit checks (lines 278/282/303, the `sc+6>len` / `sc+6+pes_len>len` / force-flush cap arithmetic). The first two need a scenario where "proceed vs. wait one more byte" is observable in the packet list, and the third only shows up at a start-code offset (sc) that survives to the moment the cap check runs - in this code path sc is always 0 once an unbounded PES buffer starts accumulating, since nothing before it ever drains. Didn't find a construction in the time available; flagged rather than papered over with a self-referential assert. |
||
|
Matthew Jackson
|
528a6b7345 |
Walk past empty extents iteratively instead of recursing
fill_extents skipped an exhausted or zero-sector extent by calling itself, which costs a stack frame per skipped extent. Nothing filters sector_count == 0 out of a UDF or MPLS extent list, so a malformed disc declaring a long run of empty extents recursed once per extent before reading a single sector. Rust does not guarantee tail-call elimination, so that overflows the stack — which aborts the process rather than returning an io::Error, taking a long-running service down with it. The skip is now a loop. The regression test runs on a 256 KiB stack, where the recursive version dies and the loop finishes immediately. |
||
|
Matthew Jackson
|
71686f1407 |
Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv, the crate the other seven build against and the one held up as the reference workflow, was the last one still linting the library only — so its ~3,000 tests, by far the largest body of test code in the project, had never been linted at all. Turning the flag on surfaced 74 findings. Most were mechanical and applied with clippy --fix. The rest, by hand: - Four discarded Results in decrypt.rs. css::descramble_region returns a Result and four CSS tests threw it away, so a descramble that FAILED would have surfaced as a confusing buffer-comparison mismatch instead of the actual error. They expect() now. - A dead `kp` field on the PlantedWalk fixture. The test deliberately asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4 rather than against a stored value — its doc comment says so — which makes the field not just unused but a trap: the obvious "fix" of asserting against it would quietly weaken the test to comparing the fixture with itself. Removed. - Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut, three vec!s that only ever needed arrays, a filter_map whose every arm was Some, and a Vec::new()+push chain. - Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering in the generated docs. - A five-[u8; 16]-tuple return type named FourLevelParts. Three lints are allowed at the specific sites, with reasons, because they are wrong for this domain: the underscores in the bitstream-header literals mark BITFIELD boundaries, not digit groups, so regrouping them uniformly would satisfy the lint by destroying the only thing they encode; and in three table-validation loops the loop variable is the domain value under test (a DTS SFREQ code, an AMODE value, a palette entry number), which is what the assertion messages name. |
||
|
Matthew Jackson
|
42591c77fc |
test: constrain the AC-3/E-AC-3/DTS header decode and the boxes it emits
All 59 measured survivors in mp4/audio.rs: 50 killed, 9 proven equivalent, none left unaddressed. No production change — every extraction reads correct against ETSI TS 102 366 (5.3.2, 5.4.2, Annex E.1.3, F.4, F.6.1) and TS 102 114 5.3.1. It was a fixture gap, not a code defect, and a specific one: the existing fixtures gave several fields the SAME value (fscod=0, bsmod=0, lfeon=1, acmod=7) and asserted only derived channel counts. Nothing asserted the emitted dac3/dec3/ddts payload BYTES at all, so the packer's shifts and masks were entirely unconstrained. A wrong mask there does not crash — it writes a box declaring the wrong channel configuration, and a player believes it. Several kills needed fixtures designed to discriminate rather than merely exercise: the reduced-rate branch needed fscod == 3, which no test in the file reached — and `== -> !=` survived on an fscod=0 fixture only because the reduced table happens to return 48000 there too the DTS LFF mask needed a value where XOR and AND differ in MEANING: on the 5.1 fixtures LFF flips 1 -> 2 and BOTH codes mean "LFE present" the acmod mix-level skips needed three different acmods, because `^` is a no-op unless acmod == 4 exactly the ddts bitrate needed 44100/512, which does not divide evenly — at 48000 the rounding is invisible and `/ -> %` on den/2 survives The 9 equivalents are one pattern: OR-ing a shifted high part with a masked low part on disjoint bit lanes, where the mask is on the same line as the OR. Since cargo-mutants applies one mutation at a time, no single mutant can break both. Each was applied and observed green. FILED, not fixed: reserved sample-rate codes are silently guessed as 48 kHz (audio.rs:133, :20/:153) while a reserved DTS AMODE is refused with a comment explaining why. Same silent-wrong-metadata class, opposite answer. Refusing would make such a stream unmuxable, which is a product call. |
||
|
Matthew Jackson
|
5559987325 |
test(mux): replace a false-green discontinuity test with the property it named
a_signalled_discontinuity_survives_a_backstop_discard asserted that a SOURCE-signalled discontinuity on discarded bytes still reaches the AU that follows. It did not test that. Deleting the disc_marks push, or the mark-retirement loop inside discard_gap_before, left it passing. The mechanism: the `discontinuity = true` rode the FIRST over-cap push, which still has the next AU's delimiter at buf[0] — so it force-flushes as an over-long AU rather than discarding, and THAT AU consumes the mark. The assertion's `.find(|x| x.data.contains(&0x22))` then filters it out, and the flag it reads comes entirely from `pending_gap`, set by the second push's backstop. Behaviourally identical to the test 40 lines above it, under a name promising something else. I wrote it this morning, in the same commit that fixed a different test for having a fixture that never reached the code it named, while cataloguing that exact shape. Third instance today of writing the bug I was hunting. The two mechanisms cannot be isolated in one fixture — a fragment that trips the backstop sets pending_gap regardless — so they now get one test each. The replacement drives disc_marks end to end with no backstop involved: a flagged fragment that carries a complete AU and is emitted, not discarded. Nothing else pinned that path. Removing the disc_marks push reds it. Found by the round-9 opus escalation over test quality, dispatched because the sonnet pass over the same 17,000 lines of new test code returned zero findings. |
||
|
Matthew Jackson
|
8c0de5711e |
fix(mux): resync-gate drops reach errors(), including after the gap resolves
ResyncGate::dropped is zeroed the moment a keyframe disarms the gate,
and the only EOF warning fires for gates STILL armed. So a mid-title gap
that resolves left no trace anywhere — and most gaps do resolve. A rip
with several concealed gaps reported 0 errors and 0 lost bytes while
whole GOPs had been discarded, which disc.rs's own test comment calls
the ONLY channel through which loss is reported.
This is the other half of
|
||
|
Matthew Jackson
|
9f25a4c454 |
fix(mp4): refuse a video track with no resolved dimensions
Resolution::pixels() returned (0, 0) for Unknown, and the MP4 sink wrote it verbatim into tkhd (ISO/IEC 14496-12 8.3.2) and VisualSampleEntry (12.1.3). Both fields are MANDATORY there, so unlike Matroska — which omits the optional PixelWidth/PixelHeight elements — MP4 has nothing to leave out. The result was a structurally complete file that passes every container check, declares a 0x0 video track, cannot be rendered, and is written with no error anywhere. WHY IT WAS POSSIBLE, which is the part worth keeping: pixels() previously fabricated 1920x1080 for Unknown. That was wrong but playable, so this sink never needed a guard and the absence of one was invisible. Changing the sentinel to (0, 0) moved the defect instead of removing it — a zero PAIR still reads as a usable value, so the sink stored it and serialised it. The accessor's doc comment then ENUMERATED the callers it believed were safe: "the Matroska sink omits the optional elements, the VobSub writer omits its size: line, and no caller divides by either dimension." Two of those three are true. MP4 was not on the list because MP4 has no guard at all, and a prose list cannot enforce itself. mkv.rs's own comment even states the principle — "the check belongs in the one accessor rather than in each caller that remembered to write it" — and labels/mod.rs still carried its own duplicate Unknown test long after the accessor took that job over. So: pixels() now returns Option. Not because Option is tidier, but because every caller genuinely needs a DIFFERENT answer and the compiler is the only thing that reliably makes them choose one. Matroska and the metadata sinks take unwrap_or((0, 0)) with the reason stated at each site; the VobSub path degrades to a palette-only .idx; MP4 fails with E_MP4_UNKNOWN_RESOLUTION (9055). Six call sites, not the five my first grep showed — I piped it through `head` and acted on a truncated list. The compiler caught the sixth. That is the same mistake as trusting a lens that reported silence. |
||
|
Matthew Jackson
|
944e6a8b09 |
fix: align the Linux fsync error with macOS, and clear three stale docs
Round 9 findings, triaged and verified against the pinned tree.
writeback_file: a bounded-fsync WorkerLost returned bare ErrorKind::Other
on Linux where macOS returns EIO. Round 8 fixed the Linux arm to return
Err at all — the right fix — but stopped short of matching the value, so
a consumer distinguishing timeout / halt / lost-worker had nothing to
branch on for the third case on one platform. Now EIO on both.
Three doc comments described the pre-fix behaviour, one of them for
longer than the bug existed:
linux.rs durable_sync still said "all three fallbacks return Ok(())"
mod.rs sync_all still said Linux silently swallows fsync failures and
callers must not treat Ok(()) as a durability barrier
mod.rs SequentialSink::finish repeated the same caveat
All three now say what the code does: a bounded-fsync failure is an Err
on every platform, so Ok(()) IS a durability barrier. A doc that
describes a fixed bug is worse than no doc — it tells a caller to write
a workaround for something that no longer exists.
au_assembly: discard_gap_before duplicated drop_marks_before's
mark-retirement body verbatim and added one statement. Mine, from
earlier today. It now calls it. Two copies of the same retirement loop
is exactly how the two call sites would drift back together.
clpi: ClpiStream's audio_format / audio_rate / video_format / video_rate
are decoded from untrusted on-disc bytes on every parse and read by
nothing. The identically-named fields consumed in disc/bluray.rs belong
to mpls::StreamEntry, not to this struct — checked, because an earlier
round wrongly called a live function dead. Deleted, along with the seven
test assertions that pinned them; the tests that pin pid, coding_type
and language remain. Also removed a section-header comment orphaned by
the get_extents deletion, describing a fixture that no longer exists.
|
||
|
Matthew Jackson
|
0bc8d7af9c |
test: constrain the AACS key-map gap fill, the PSI walk, and MP4 field offsets
Third pass over src/mux/. 40 survivors killed, no production change. resolve.rs — the deleted-statement cluster is now fully constrained. All 14 deletable statements probed; 9 were already caught, 5 survived: c.sort_unstable() in fill_base_key_gaps. Every existing case handed it cuts already in LBA order, but IndividualSegment.tbl is a record list. Verified on HEAD: deleting the sort passes all 54 resolve tests. The mutant lays a base-key fill straight over a forensic segment. last_idx = idx (FMTS gap fill) and last_idx = hit (multi-CPS cache hit). An extent with nothing to sample must inherit its neighbour's CPS unit; the mutants fall back to the first unit's key. Exactly the shape this file's own comments name — wrong key, no error, lost_bytes == 0. Both check_halt()? polls in probe_fmts_index_keys. These cannot be killed by outcome, since a later poll returns Halted too. The tests count reads instead, which is what the don't-hammer-a-struggling- drive rule actually says: after a Stop the drive is asked for zero content sectors. The four unresolved += 1 arms each got a test, and deleting each fails exactly one — one-to-one, so no fixture passes for the wrong reason. A control test pins that the baseline table resolves, so an expect_err cannot succeed for an unrelated reason. ts.rs::scan_streams was never entered. Six killed, two of which return wrong answers that look right: reading the PAT/PMT CRC as a table entry invents a stream on PID 546 out of CRC bytes, and dropping the ES_info_length skip decodes a descriptor as an entry and loses the one after it. Every existing PMT fixture declares ES_info_length = 0; a real BD PMT carries a registration descriptor on essentially every entry. Also ISO/IEC 13818-1 2.4.4.3 program_number == 0 is the network PID, not a program. mp4/read.rs — 13. Height read as the width beside it; channelcount; the 4-byte base-128 descriptor varint (every existing esds fixture uses a single byte); all three optional ES_Descriptor fields, whose loss is silent (an AAC track just loses its CodecPrivate); first-vs-last media edit, which is A/V desync of the difference; and the version-1 mvhd timescale offset, emitted by any writer whose duration exceeds 32 bits. dts.rs — 7, from a real cargo-mutants run over the file rather than guesswork. Including a buffer that IS the syncword, which is the state a sync split across PES packets lands in the moment its last byte arrives. Equivalents proven by application, not argued: the sample_encrypted_units guard pair is mutually redundant by construction (total*p/9 < total for p <= 8), so either alone is equivalent and both together are not; the PMT section_len guard is dead code where its PAT twin panics; three of the seven EXSS_HEADER_MIN_BYTES arithmetic mutants still sum to 10. |
||
|
Matthew Jackson
|
e99b634635 |
fix(mux): a backstop discard is a discontinuity; a stream-start trim is not
drop_marks_before retired discontinuity marks alongside timing marks at both of its call sites. At stream start that is right. At the MAX_AU_BUFFER backstop it is not, and the two are now separate. The backstop fires when 8 MiB accumulate with no AU start code in them — corrupt or hostile input — and throws the run away. There IS a prior AU in that case, and whatever emits next definitively does not continue it: a decoder handed that picture resolves its references against frames separated from it by megabytes of discarded data. Retiring the flag meant the resync gate (resync.rs, driven from mux/disc.rs) never armed, so the broken picture went out looking sound. Silent corruption is the one class of loss this crate refuses to have. At stream start the opposite holds. Bytes ahead of the first access-unit delimiter are the tail of an AU that began before sync, and there is no prior AU to be discontinuous from. Marking it would arm the gate at the head of every title and drop its opening GOP. That risk is why this was a decision rather than a fix, and splitting the call sites is what avoids paying it. Recorded as a sticky flag, not an offset mark. A mark placed at the new base is retired moments later by the pre-sync trim that follows resync — the gap has to outlive the bytes that caused it. I found that by writing the test first and watching it fail with the mark approach. The discard is a discontinuity whether or not the source signalled one, and a signalled one on discarded bytes still reaches the AU that follows; both directions are tested. Note the first over-cap run is NOT a discard: the next AU's delimiter is still at buf[0], so it force-flushes as an over-long access unit and loses nothing. Only a run with no opener at all reaches the backstop. The tests push twice for that reason — the single-push version passes without the fix. Swapping either call site for the other fails: reverting the backstop reds the two gap tests, and arming the gate at stream start reds the third. |
||
|
Matthew Jackson
|
b2e1982051 |
fix(clpi): resolve out_time past the last EP entry to the end of the clip
ClipInfo::get_extents fell back to `last EP SPN + 1` whenever out_time lay past the last entry-point. EP entries mark I-frames (BD-ROM Part 3, CPI / EP map) and a clip's final GOP lies after the last one, so a PlayItem covering a whole clip — whose OUT_time is the presentation end — always lands in that arm. The extent then stopped one source packet after the last I-frame. Measured on a fixture with 200,000 source packets and the last EP at SPN 131,072: sector_count came back 12,289 where covering the clip needs 18,750. Everything from the last entry point to EOF is outside the returned extent. Scope, stated plainly: get_extents has NO callers anywhere in the ecosystem today — it is #[allow(dead_code)] and documented as reserved for the timestamp-range read path. Nothing ships this loss. It is fixed now because a latent truncation in extent arithmetic is far cheaper to correct before it has callers than after. The SPN at-or-after an out-of-range out_time is the end of the clip, source_packet_count, with .max(last + 1) so a disc that under-declares its own packet count against its own EP map still yields a sane bound. Also 174 mutants killed across clpi, mpls, ifo and ebml — the first time any of these four files has been examined. And ebml's 8-byte VINT back-patch was duplicated verbatim in end_master and end_master_buf with its top four payload octets unreachable through either (they need a 16 MiB..256 TiB buffer); extracted to fixed_width_vint8 and tested across the full 56-bit payload, no behaviour change. 38 of ebml's 46 survivors are one equivalence cluster: every | in write_size / read_id / read_size / read_uint_val ORs into disjoint bit lanes, where ^ is the identical operation. Applied all 38 at once — green — then spot-checked four individually. |
||
|
Matthew Jackson
|
9de88969ca |
test: constrain the DiscStream loss surface and the empty-title guards
Second mutation pass over src/mux/. 26 survivors killed, no production change. Verified on HEAD before landing: each mutation below passes all 1,237 mux tests unmutated-suite. The priority item was the honest-loss-reporting surface. Both DiscStream::errors and DiscStream::lost_bytes could return a constant with nothing failing — a rip that lost sectors would report zero loss to the caller. This project has already shipped one defect of that shape (a total decryption failure reported as an empty title, exit 0). Driven now through two short-read fills so both land on values that are neither 0 nor 1 and differ from each other; no constant and no field swap survives. MkvStream::finish -> Ok(()) also survived. MkvMuxer::finish has the zero-frame MkvInvalid guard and two tests cover it, but the Stream wrapper above it could return Ok unconditionally and bypass the guard entirely — the empty-title defence was one layer thinner than it looked. au_assembly: pinned au_opener_from behaviourally to the normative byte values for all four modes, with negative cases for codes that are explicitly not openers (MPEG-2 slice 0x01..0xAF, user data 0xB2, extension 0xB5, sequence end 0xB7 per 13818-2 Table 6-1; VC-1 0x0A/0x0B/0x0C; H.264 SPS/PPS/IDR-slice). au_assembly and codec/ hold independent copies of these constants; they agree today, and comparing constants would not catch logic drifting apart, so both sides are now pinned to the spec instead of to each other. demux_sink::sanitize: every filename component demux:// writes comes from disc-controlled text, so the path-separator arm is a traversal guard. Deleting it now fails, including an end-to-end case where base = "../evil/Title" must produce exactly one file inside the chosen directory. stts_and_ctts_expand renamed to stts_expands_runs_to_per_sample_deltas_in_order and given runs with distinct deltas AND distinct lengths. Its old name claimed ctts coverage it never had, which is why the composition-time chain went unconstrained for eight rounds; the doc comment now points at the tests that do cover ctts. Correction to the previous pass: codec/truehd.rs flush -> vec![] IS equivalent. Applied it, full mux suite green. TrueHD buffers across PES but parse emits every complete unit immediately, so a residual buffer at EOF is a truncated access unit and is correctly discarded. The vec![Default::default()] variants are genuinely different and are killed. Deliberately not constrained: mkv::set_opening_capture (diagnostics behind a process-global tracing check, flaky under the parallel runner), and the three stdio.rs header paths (StdioStream holds concrete io::Stdin/Stdout and cannot be driven without a production refactor to injectable Read/Write). |
||
|
Matthew Jackson
|
170fd0c064 |
test: constrain MP4 composition timing, MLP substream directory, and codec-private absence
Mutation testing over src/mux/. No production change — 49 survivors killed, all proven red before green. The MP4 composition-time chain was entirely unconstrained: VideoTiming::ctts, build_ctts and parse_ctts could each return a constant and the suite stayed green. Confirmed on HEAD: build_ctts -> vec![] passes all 1,220 mux tests. A demuxed B-frame title presenting in decode order would have shipped. The cause is a test whose name asserts coverage its body does not deliver — stts_and_ctts_expand builds an stts box and never touches ctts, and write_then_read_round_trip asserts sample sizes and keyframe flags but not one PTS. Same shape as the set_speed forwarding finding, different disguise. mlp_num_substreams / mlp_substr_header_size: every TrueHD fixture in the crate uses one substream and no extraword, so both could return a constant and agree with all of them. These position mlp_parity_ok's window over the AU header, so a constant mis-windows the parity check on exactly the multi-substream AUs that carry 7.1 and Atmos. CodecPrivate absent vs empty: mkv.rs writes Some(bytes) verbatim and omits the element on None (RFC 9559 5.1.4.1.24), so a zero-length Some emits a track header asserting the config IS empty. Four parsers could return Some(vec![]) before any frame. Also: mandatory ISO/IEC 14496-12 boxes (tkhd, vmhd, smhd, dinf, mdhd) could each build empty; HEVC num_extra_slice_header_bits (H.265 7.3.2.3) was never non-zero in any fixture, so the slice-type offset skip was unexercised; chapter names from the disc go straight into <ChapterString> and the & escape must run first; a stray 0x47 in a payload must not latch a TS resync. Documented as equivalent rather than killed: CodecParser::flush and the three parser flush bodies that differ from the mutant only by a tracing call, and DropTally::log_summary. |
||
|
Matthew Jackson
|
c63dafcf1a |
Key each FMTS extent from its own CPS unit, and stop a bad ICB tag reading
as an empty directory On an AACS 2.1 disc the non-forensic gap fill hardcoded pool slot 0 as "the" base Unit Key, so every content LBA outside a forensic segment was keyed with CPS unit 1's key even on a disc carrying several CPS units. It does not fail loudly — it produces garbage plaintext. The gap fill now resolves each extent's own base key from its ciphertext, sharing the sampling and slot-picking the multi-CPS path already had rather than adding a second copy, and memoised in the existing per-disc cache. A disc with one base key still short-circuits with zero extra reads, which the existing probe-cost test pins. Forcing the slot back to a constant fails four tests, so the choice is load-bearing rather than incidental. A directory ICB whose descriptor tag is neither File Entry nor Extended File Entry (ECMA-167 4/14.9, 4/14.17) was turned into a successfully-read EMPTY directory, indistinguishable from a genuinely empty one, while the same tag on a file ICB was already a hard error. Fifth instance in this audit of a failure converted into a plausible success value, and the second in this very function — round 5 fixed a read error becoming a file size of zero here. An unrecorded extent (ECMA-167 4/14.14.1.1: allocated but not recorded, logically zeros that still occupy file space) was dropped entirely rather than contributing its length, so every later extent landed at the wrong file offset. Silent corruption, not an error. Extents now carry a recorded flag and the hole emits zeros without touching the media. The Volume Descriptor Sequence was swept at hardcoded sectors 32..64 while the anchor's own Main VDS Extent pointer was parsed into a comment and ignored; ECMA-167 3/10.2.1 defines that extent by the field, not by position, so a conformant volume placing it elsewhere failed to mount. drive_status decoded byte 5 as Media Status without checking the event header's NEA bit or notification class (MMC-6 §6.7), so a reply carrying no media event descriptor decoded as "no disc". The drive is untrusted input here, and this is the works-on-my-drive class. Two more tests were found asserting the defects they sit next to — one requiring unrecorded extents to be dropped, one that four drive fixtures built non-conformant replies the corrected decoder rightly rejects. Both rewritten. Combined with the DTS one in the previous commit that makes three tests this round that locked a bug in as intended behaviour, which is a different and worse failure than the tautological tests found so far: a tautology fails to catch a regression, these actively defend the defect. Not fixed, adjacent: extract_one_file streams extents sequentially and will now READ an unrecorded extent's sectors rather than writing guaranteed zeros. Offsets are right, and pressed media reads as zeros there, but it is not zero-guaranteed the way read_file now is; that needs a recorded flag through PlannedFile. |
||
|
Matthew Jackson
|
46eb88c51f |
Feed the CSS crack the canonical extent order, and stop Resolution faking 1080p
Seven defects in the code the test suite executes least — 913 lines of disc/mod.rs alone are run by no test at all, which is why this round scoped from coverage rather than from what previous rounds said they had read. Disc::scan_image kept its own copy of the crack's extent ordering and fed crack_key_outcome largest-cell-first. That is the fifth instance in this audit of a local reimplementation drifting from the canonical one, and the cost here is a key that does not descramble the feature: picking by sector count bypasses the capacity gate and can select a different VTS entirely. The copy is gone — which title comes from the canonical order the scan already applied, and the extents are handed over in playback order, exactly as decrypt_keys_for_title does. Its doc records why the duplicate existed so it cannot grow back. Resolution::pixels returned 1920x1080 for Unknown. That is the FOURTH instance of one trap and the other three were in this same file, two of them fixed hours earlier — without sweeping for siblings, which is the whole reason this one survived. It now returns (0, 0), and the sweep was done properly this time: every remaining Unknown arm across the crate is honest, and the two ColorSpace sites that look like fabrication are emitting H.273 code point 2, which is the spec's own "unspecified". Two callers carried local Unknown-to-zero workarounds — precisely the cost of making callers responsible for a lie — and one is now redundant. BD-ROM Part 3 code 0xA2 is the lossy secondary DTS stream, not lossless Master Audio. A test asserted the wrong mapping as intended behaviour, so correcting the code failed it; the test is deleted with a note pointing at its replacement. That is a NEW failure mode for this audit: not a test that cannot fail, but one that locks the defect in. There is no DtsExpress variant to map to, so it takes the lossy DTS-HD member and the approximation is documented. Also: DiscSession::identify could panic through drive_mut once the public API allows an absent drive — two siblings were converted in an earlier round and this one was missed; an extent end that added without saturating where the rest of the crate saturates; a diag reason string restating the comparator's sort keys and drifting from them, now derived from them; and a short read that advanced the offset by the full request, silently skipping the gap. That last one existed twice, in two reads with the same shape, now merged so they cannot drift apart. The short-read policy is a judgement call I could not derive from a spec: no skip_errors is a hard error, with skip_errors zero-fills and charges the loss. It deliberately does not retry mid-unit, because resuming inside an AACS aligned unit would trade a silent gap for a silent decrypt desync — the worse of the two. |
||
|
Matthew Jackson
|
dea968f32b |
Stop AudioChannels and SampleRate fabricating a value for Unknown
Three copies of the same two mappings existed. The canonical accessors returned 6 channels and 48000 Hz for Unknown; a third copy in diag.rs returned 0. The honest one was the copy. A plausible wrong answer is worse than an obvious one. Six channels at 48 kHz is indistinguishable from a real 5.1 track, so every caller became responsible for remembering to check the variant first — and this crate walked into exactly that: the json:// sink reported a confident 5.1 for audio whose neighbouring fields said "unknown". That was fixed at the call site earlier in this audit; this fixes it at the source. The accessors now return 0, which is what both in-crate call sites already coerced Unknown to by hand, so their guards are gone and the behaviour is unchanged. Zero is also obviously wrong if it ever reaches output, where six is not. The diag.rs duplicates are deleted rather than corrected — a fourth copy would have drifted too. Their only caller was a trace line in the same file, now on the canonical accessors. Their tests moved across and gained the Unknown case, which is the point: restoring either fabricated value fails both. Found by the round-7 correctness agent while fixing the json:// sink; it flagged the third copy as out of its scope rather than touching it. |
||
|
Matthew Jackson
|
079c9b1327 |
Add a seeded robustness harness for the untrusted-input parsers
Five parsers that take bytes straight off a disc are now swept with generated input asserting one property: they return Ok or Err and never panic. That is this crate's own hard rule, and the class seven rounds of reading is worst at. Written in-crate rather than with cargo-fuzz, which needs a nightly toolchain this project does not use, and without proptest or arbitrary, because one dev-dependency is a deliberate posture and the parsers take plain byte slices. What is given up is coverage-guided mutation, which is the real loss. What is gained is determinism: the same seed replays the same cases anywhere, so a CI failure reproduces locally verbatim. Three generators, and the second is the one that matters. Pure random bytes die at the magic check and exercise the entry guards only; prefixing valid magic is what reaches the parser body; mutating a mostly-zero record is what reaches the offset and count arithmetic a hostile image would lie about. That claim is MEASURED, not asserted. A harness whose cases all bounce off the entry guards is the fuzzing equivalent of a test that cannot fail, so one test counts how many generated cases parse to completion: 15,606 of 60,000, about 26%. If a future change to a guard drops that to zero, the test fails rather than continuing to report a meaningless pass. Two further tests pin that the three generators produce different bytes and that a seed replays identically. 1.2M cases across all five targets found nothing. On this evidence that is a real negative rather than an empty one. The first version of this file was itself broken in the way this audit keeps finding: its two meta-tests set FREEMKV_HARNESS_CASES and raced, because the test harness runs them in parallel and env mutation is unsound there. The budget is a parameter now, and the environment is read once at the call site. Two crate-internal parsers widened from private to pub(crate) so the harness can reach them. No public API change. |
||
|
Matthew Jackson
|
327087c70e |
Make five tests capable of failing, and stop the presence probe unmounting the disc
The worst of the five was a regression suite that never touched the code it guarded: nine batch-count tests called `safe_batch_count` and `buggy_batch_count`, both defined in the test file itself. The u16 truncation they exist to prevent could be reintroduced in sector/prefetched.rs with every one of them green. They now drive the real producer through the public API, and reinstating the truncation fails five of the nine. Worth recording that the symptom has changed since the original fix: the unit-alignment clamp below floors a zero batch at three sectors, so the bug is now a twenty-fold throughput cliff rather than the stall it once was. The MP4 reserve test's only numeric case was dominated by the floor and the buffer, so BYTES_PER_SAMPLE could be zeroed without failing it. It now has a case where the per-sample term dominates. The zero-count guard in FileSectorSource was likewise unfalsifiable — seek-past-EOF and a zero-length read both succeed — so the test now observes the file cursor. The AACS media-key ambiguity guard had no test at all; the pool scan is extracted so the verifier can be injected, because a genuine two-key collision needs one ciphertext decrypting under two AES-128 keys to plaintexts sharing a 64-bit magic, which is a 2^64 search and not a fixture. macOS implemented the documented cheap, side-effect-free presence probe by building a full exclusive transport — which force-unmounts the disc. Linux and Windows issue one TEST UNIT READY with no unmount; macOS was the outlier. It now walks the IOKit registry for the media object instead. The C shim's registry reads assumed CoreFoundation types the registry does not guarantee, so a driver publishing a CFNumber where a CFString was expected aborted the process from inside public API. Types are checked and a wrong type treated as absent. The unbounded waitpid on the unmount child is now a polled deadline, and the last-resort match gained the NULL check its two siblings already had. The empty-CDB guard existed only on Linux while a shared helper's comment claimed all three backends had it. Moved into the helper, so the comment is now true and macOS and Windows are covered. One finding was REJECTED with evidence rather than fixed. The TrueHD buffer-cap test was indeed bogus, but MAX_TRUEHD_BUF turns out to be unreachable by any input: the parser only retains data when the buffer is shorter than the declared AU, and that declaration is twelve bits, so the worst case is 8189 bytes against a 256 KiB cap. An exhaustive sweep over all 65536 AU headers confirmed it. The fixture now sits at the reachable ceiling and asserts that instead. The cap itself is left in place as defence, unreachable by construction, matching how the AC-3 resync guard was handled earlier in this audit. Two behaviour changes worth naming: Linux's empty-CDB error becomes InvalidCdbLength rather than a transport failure, and an unknown device now reports absent media rather than a not-found error, because the registry cannot tell an empty drive from a missing one. The latter is a conflation of the kind this audit has fixed three times; it is recorded for the next round rather than left silent. |
||
|
Matthew Jackson
|
b8fa5e74dc |
Stop the live rip path muxing Blu-ray 3D differently from the ISO path
Five defects, four of them the same shape: a local reimplementation of logic the crate already had, which had drifted from it. Each is now fixed by calling the canonical version rather than by patching the copy. DiscStream::new — the live disc:// path — built every parser through the plain codec lookup and never asked whether a video stream was an MVC dependent view, though resolve::build_demux_state does. The same 3D disc therefore muxed correctly from an ISO and incorrectly ripped live. The open-coded loop is gone; both paths now call build_demux_state. collect_psi_section reimplemented the continuity-counter gap test and disagreed with process_packet in the same file: it tolerated neither a duplicate packet nor an adaptation-field-only packet, which per ISO/IEC 13818-1 §2.4.3.3 does not increment the counter. A spec-legal PMT continuation was read as desync and the title's stream list came back empty. Both callers now share one `cc_is_gap`, and a duplicate packet's payload is no longer appended twice — doing so would have corrupted the section the check exists to protect. The json:// sink called the channel-count and sample-rate accessors unconditionally, and both fabricate a concrete value for Unknown, so it reported a confident 5.1 at 48 kHz for audio whose format was unknown while its own neighbouring string fields said "unknown". The keys are now omitted, matching mkv.rs. This matters more than it did: a sample-rate ladder fixed earlier in this audit means Unknown now reaches consumers that used to receive a wrong-but-concrete value. For an audio:// or sub:// sink the reference video track's output is filtered out, so its first PTS was never recorded and every delay was computed against zero — baking a wrong DELAY into the filename. The reference is now recorded whenever a frame is on the reference track, independent of whether that track has an output, so a normal title gets a correct delay; where no reference is ever observed the tag is omitted rather than guessed. A third copy of the channel/sample-rate mapping exists in src/diag.rs and was left alone as outside the confirmed set. It is the same drift shape and is recorded for the next round. |
||
|
Matthew Jackson
|
3f7d7af472 |
Bound three allocations an untrusted disc can drive without limit
The Program Stream demuxer appended every fed byte and enforced its 4 MiB cap only inside a branch reached once a start code had been found. Input containing no start code anywhere therefore hit no cap at all, and since a whole title is fed through this demuxer, a zero-filled or ciphertext VOB extent buffered the entire title — up to ~90 GB. When the buffer holds no start code, only a two-byte `00 00` prefix can begin a PS unit on the next feed, so that is kept and the rest dropped. The bound is exact rather than a heuristic: a start code can straddle a feed boundary by at most its first two bytes, so no real byte is discarded, and a test feeding `FF FF 00 00` then `01 E0 ...` pins that. The existing test named for this case fed a real start code first, so the cap it exercised was the in-PES one. Renamed to say what it covers. The BD-J label path had a different shape to anything found so far: the cap is on the COMPRESSED size of a disc file while the allocation scales with the decompressed size. A `.class` gated only by a path prefix inflates to the 64 MiB ceiling, yielding ~33M retained strings from `ldc` operands or ~67M pushes onto a symbolic stack whose depth was unbounded despite the Code attribute's own `max_stack` being parsed and then ignored. Bounded both, the stack by `max_stack` itself (JVMS §4.7.3). The VMG TT_SRPT title count is an untrusted u16 with no de-duplication, so ~800 KB of crafted IFO re-parsed one PGC 65535 times. Capped at 99, the DVD-Video maximum, so no conformant disc is clipped. Every cap carries stated headroom against real media, and each has a test locking that real media still passes. I rewrote three of the new assertions before landing them. They compared the result against the very constant under test — `total <= MAX_TT_SRPT_TITLES` — which passes vacuously the moment someone raises the constant, the most likely future regression and the seventh instance of this tautology shape in this audit. They now assert literals derived from the spec. The TT_SRPT fixture also had to change: with 65535 identical entries the de-duplication collapsed them on its own and the cap was never what bounded the result, so the test passed with the cap removed entirely. Distinct entries defeat dedup and leave the cap as the only guard; de-duplication now has its own fixture. Verified by raising each of the three constants and confirming all three tests fail. |
||
|
Matthew Jackson
|
fdd473d7e9 |
Remove emulation-prevention bytes before reading the H.264 slice header
The bytes after a NAL header are EBSP, not RBSP: ISO/IEC 14496-10 §7.4.1 has the encoder insert 0x03 after any 0x00 0x00, and §7.3.1 removes it before parsing. The measured-picture-type parse read the raw NAL instead, on the stated reasoning that slice_type is too early for an escape to intervene. That holds only up to a point. first_mb_in_slice is ue(v), so a value of 65535 or more needs sixteen leading zero bits and opens the payload with 0x00 0x00, which an encoder must then escape. A UHD frame is ~32,400 macroblocks, so a conforming Blu-ray never reaches it — but 8K does, and the disc is untrusted input. Such a stream decoded slice_type against a byte the encoder had inserted and reported the wrong picture type: a wrong result rather than an error, which is the class this lens exists for. The prefix is un-escaped into a 16-octet buffer rather than the whole NAL: the two ue(v) fields are at most 32 bits each, so nothing longer can be needed, and it keeps a per-frame allocation proportional to the frame off the path. The test pins both directions. It asserts the un-escaped prefix decodes to first_mb_in_slice = 65535 and slice_type = 2, AND that the raw EBSP does NOT — without that second assertion the test would pass whether or not the fix were present, which is the failure mode this audit has now found four times. The bit string was derived independently rather than by hand: my first attempt at the fixture was wrong by one nibble and the test caught it. Also covers the cases that must NOT be unescaped: a 0x03 not preceded by 00 00 is ordinary payload, and 00 00 03 03 keeps its second 0x03 because the escape resets the zero run. |
||
|
Matthew Jackson
|
d444afbdfc |
Turn a release-only slice panic into an error, and stop calling 32 kHz 48 kHz
Four round-6 findings. FileSectorSource::read_sectors guarded its output buffer with a debug_assert, which is compiled out in release — so an undersized buffer panicked with 'range end index out of range' instead of returning an error, out of a public SectorSource impl where the length is caller input. Drive::read_fua already carries this exact guard, with a comment recording the same panic being fixed there, and PrefetchedSectorSource has a regression test for the same case; this impl had been given neither. The new test is red in release for precisely the predicted reason: 'range end index 8192 out of range for slice of length 2049'. parse_track's sample-rate ladder ended in an unconditional S48, so any SamplingFrequency below 44100 was recorded as 48 kHz. A 32000 Hz AC-3 or DTS track is legal and common in broadcast-sourced content, and the wrong rate then propagated into the reconstructed AudioStream. Anything below the lowest mapped rate is now Unknown, which is what the crate's canonical SampleRate::from_hz already returned — the ladder disagreed with it. The ladder itself stays, because the MKV element is a float and wants tolerance rather than exact equality. shim_open_exclusive used the mach port from IOMainPort without checking the return; on failure the port is left untouched and every IOKit call below ran against an uninitialised value. shim_list_drives in the same file does check it. build.rs treated cc and ar as successful if the process merely SPAWNED, so a genuine compile error in the macOS C shim produced no object file and surfaced later as an unexplained link failure against a missing symbol. The shim is macOS-only and is neither linted nor compiled on the other two platforms, so a mistake in it has exactly one chance to be noticed. The last two have no test: one needs IOMainPort to fail, the other needs a deliberately broken C shim, and neither is reachable from the test harness. Both mirror a correct sibling in the same file, which is the evidence available. |
||
|
Matthew Jackson
|
399c3d2769 |
Reject an over-length CDB on every transport, splice H.264 param sets in place
Two fixes from round 5. The Linux and Windows backends truncated a CDB longer than 16 bytes (`cdb.len().min(16)`) where macOS returned InvalidCdbLength. Under SPC-4 a command's length is fixed by its opcode group code, so a shortened CDB is not a shorter form of the same command — it is a DIFFERENT command, and the drive executes it and answers GOOD with data for a request nobody made. A silently wrong result on the layer everything else sits on. Rather than mirror the guard a third time it now lives in scsi::mod as checked_cdb_len, with all three backends routed through it, so it cannot drift per platform again. That also makes it testable everywhere: each platform module is cfg-gated to its own host, so a guard inlined into linux.rs and windows.rs would have had no test coverage on any single machine. The shared helper is the only place the behaviour can be asserted on every platform's CI. The two existing macOS tests were tautological — they replicated the guard's logic inline instead of calling it, so they would have passed with the guard deleted. They now call the real helper. Separately, the H.264 keyframe parameter-set re-assert grew a few-hundred-byte prefix buffer to the full access-unit size, copied the whole frame into it, and dropped the presized buffer: one extra whole-frame allocation and copy per keyframe. A UHD title is ~200,000 frames of 150-400 KB with a keyframe every second or two, so that is thousands of avoidable multi-hundred-KB copies per title, each large enough to go through mmap. It now splices into the reserved headroom in place. This mirrors the identical fix already made in hevc.rs, which the H.264 path had drifted from. Byte-for-byte equivalence is pinned by a test whose expected literals were captured from the pre-change implementation, and which I confirmed still passes when the old build-and-copy code is restored. The no-reallocation claim is measured rather than argued: a counter over 30 bare keyframes, which reports 30 of 30 against the old path and 0 with the splice. The reallocation test initially passed even with PARAM_REASSERT_HEADROOM set to zero, because a small parameter set fits in the presize's incidental slack — it proved the fixture did not reallocate, not that the headroom prevented it. Its SPS is now large enough that the constant is load-bearing, so zeroing it fails the test. Not verified: no runtime behaviour on Linux or Windows: no drive, no ioctl. Both files were confirmed to compile for their own targets. |
||
|
Matthew Jackson
|
5c6a6d0785 |
Round 5: reject a degenerate fixed lace, bound the pending buffer by bytes
Five fixes. Three are real defects with regression tests; two are bounds that were expressible but not expressed. A fixed-size lace (RFC 9559 §10.3.4) whose body is empty declared n frames and carried none. The divisibility check passed, because 0 % n is 0, and `chunks` yields nothing on an empty slice whatever width it is given — so the clamp that existed to avoid chunks(0) returned zero frames where the Lacing Head said n. The whole lace vanished with no error raised and the caller saw a clean short block. A zero-size frame cannot be a valid frame, so it is now malformed. A disc read failure while fetching a directory entry's ICB became a file size of zero rather than an error. Zero is indistinguishable from a genuinely empty file, so an unreadable ICB on a damaged disc silently changed which titles a caller saw as present — read_directory already fails hard on its entry-budget guard, so propagating is also what the surrounding code does. read_file_size still returns Ok(0) for an ICB whose tag is neither File Entry nor Extended File Entry, which is a real zero and not a failure. The pending-frame buffer was capped at 4096 frames, which does not bound memory: frames are arbitrarily large and a UHD video frame runs to a few hundred KB, so the existing cap permitted over a gigabyte. Now bounded by bytes as well, at 64 MiB. round_up_grain overflowed for inputs within one grain of u64::MAX — div_ceil then multiply — and the wrapped product is small, turning the largest possible estimate into a negligible reserve. It saturates, and the reserve is clamped to what a `free` box's 32-bit size field can actually hold, since writing a larger one truncated the size and left mdat beyond a box claiming to be far shorter. No real title comes close; a 90 GB UHD title estimates a few MiB. The AC-3 resync guard now advances the PTS cadence like both of its sibling branches, so the three paths out of that block cannot disagree. This one is defensive and has NO test: reaching it needs input that both parses frames and leaves a megabyte of residue, and the parser's own carry rules drop pre-sync junk and cap a partial frame at 8192 bytes, so no such input was found. Stated here rather than covered by a test that would pass either way. Two findings from this round were rejected on inspection. A reported panic in the .mpls suffix check does not exist: the `.get(..)` on the line above returns None off a char boundary and `filter` never runs its closure, so the byte index is unreachable. A test written for it passed against the unfixed code, which is what surfaced the error. |
||
|
Matthew Jackson
|
f5e169efb3 |
Stop reporting a corrupt mkv:// source as an empty title
E6008 meant two unrelated things: "this title produced no muxable frames", which is a benign stub worth skipping, and "the source file is malformed", which is not. Because a single code carried both, is_skippable_title_stub answered yes to the second one — so feeding a truncated or corrupt mkv:// input made the engine classify it SkippableStub, print a notice saying the title was empty, and exit 0. Silent data loss reported as success. Split into E9053 MkvSourceInvalid for the read path (25 raise sites across mkvstream.rs and ebml.rs's read primitives) and E9054 MkvUnencodable for the four write-side sites, which are the encoder refusing to emit a body at or above the 56-bit VINT limit — an output limit with no input involved, so calling it a corrupt source would be wrong in the other direction. E6008 keeps only the zero-frame guard it was documented to mean. Kept one code for the whole read path rather than one per raise site: nothing a consumer does differs between a bad VINT, a non-UTF-8 string element, a truncated body and a child overrunning its parent. E9052 is the model for when a carve-out earns its keep — laced blocks name one specific RFC 9559 §10.3 feature with its own diagnosis. Also fixed meta_sink.rs raising MkvInvalid for a serde_json encode failure in the json:// sink, where no MKV is involved at all; it now matches the identical guard in mux/meta.rs. Reverting the split at the single code() arm reproduces the old classification: 22 tests fail, including both new assertions. The opposite direction is pinned too — dropping E6008 from the predicate fails the genuine-stub test, which drives the real muxer end to end. |