a8db2435cdbe5e31c773bc1b5f4ee39b413c33c7
35
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Matthew Jackson
|
a8db2435cd |
Pin elst byte-offset decoding and a zero-timescale boundary in elst_offset_ticks
parse_elst's existing tests used segment_duration/media_time values that were almost all zero or 0xFF bytes, so an index slip in the version-1 byte extraction (reading a neighbouring byte, or one outside the entry entirely) could return the same value by coincidence and the test wouldn't notice. Added a fixture with every byte distinct and nonzero so any wrong offset is caught. elst_offset_ticks's `empty_movie_ticks > 0` guard on the Some(mts) arm looked like a pure optimisation, but shifting its boundary lets an empty_movie_ticks == 0 call fall into the division instead of skipping it - and a zero movie timescale (unreachable through from_reader, which filters it, but not through this function's own contract) makes that division panic. Pinned the boundary directly so the function stays safe on its own terms. Documented nine further mutants as equivalent rather than chasing them: media_edits/odd_rate and the None-arm's empty_movie_ticks check only gate a tracing::warn!, never the returned offset, and parse_elst's b.len() < 8 vs <= 8 boundary computes the same empty Vec either way once available = 0 is worked through. Confirmed by re-running each mutation against the full test suite. |
||
|
Matthew Jackson
|
f8ed0b99f4 |
Pin read_moov's box-size boundaries and MAX_ALLOC_BYTES exactly
read_moov's forward-progress guard (box_size < header_len, OR'd with the EOF check) and the MAX_ALLOC_BYTES cap were only exercised on inputs well away from their boundaries, so a mutation testing pass found the exact edges unasserted: a size-8 (header-only) moov, a box that overruns the file by exactly the amount the OR/AND distinction can see, and a payload of precisely MAX_ALLOC_BYTES. Added a shared FakeBigReader (lifted out of an existing test's local struct so a new test can reuse it) to exercise the MAX_ALLOC_BYTES boundary without a multi-hundred-MiB backing file. |
||
|
Matthew Jackson
|
8189da1b0c |
Document why audio.rs's bit-packing | mutants are equivalent
Every mutation-testing survivor in this file is a | with ^ flip inside a bitstream packer: BitReader::read's accumulate step, the push closures in dac3_box/dec3_box/ddts_box, and the multi-field extractions in parse_eac3 and parse_dts. All nine are the same shape: shift an accumulator left by exactly the width of the next field, then OR it in, so the two operands never share a set bit and | and ^ agree on every input. Confirmed by running cargo-mutants against just these nine mutations after the existing test suite (which already exercises each function's field values) - all nine still survive, as expected for a genuinely equivalent mutant. Recorded the reasoning once at BitReader::read so nobody spends time chasing it site by site. |
||
|
Matthew Jackson
|
698ba36ae4 |
Document the pack_language and detect_rate equivalent mutants
Three pack_language mutants and two detect_rate boundary mutants survive mutation testing with no test able to close them, and it's not for lack of trying: they're equivalent by construction. Recording the proofs next to the code so nobody re-chases them: - (b[0] - 0x60) as u16, shifted << 10 then truncated to u16, is congruent mod 65536 to (b[0] + 0x60) as u16 shifted the same way, because 0x60 * 2 * 1024 is an exact multiple of 65536. The same swap on the second letter (shifted only << 5) is NOT equivalent, which is why only the first letter's mutant survives. - The two | with ^ mutations that OR the three packed fields together are equivalent because the fields (a lowercase letter minus 0x60, so 1..=26) always fit in 5 bits and never share a set bit once shifted into their 0/5/10 positions. - detect_rate's tolerance and tie-break comparisons only diverge from their <= mutants on an exact 0.5 fps distance or an exact tie, and a brute-force search over every achievable integer-nanosecond median found no case that lands on either boundary bit-exactly. |
||
|
Matthew Jackson
|
90a7fe2ff1 |
Assert the MP4 timing arithmetic, and name the faststart slack rule
MP4 track timing was numerically unasserted. Every operator in the PTS-to-ticks, duration, tkhd_dur and ctts chain could be flipped and the whole suite stayed green, because no test decoded an output file and checked a concrete number — the existing tests assert box presence and gross container shape only. That is the crate's worst failure mode: a title muxes "successfully" with silently wrong A/V sync or total duration, and nothing above can tell. The new tests build tracks with known PTS deltas and compare the emitted stts, ctts and tkhd.duration against computed values. Also lifted the faststart slack rule out of the match guard into faststart_fits(). A leftover hole of 1-7 bytes cannot be expressed as any ISO-BMFF box, since a box header is 8 bytes, so finish() must fall back to moov-at-end rather than write a free box that lies about its own size. The condition now has a name and a test instead of being an unexplained `g == 0 || g >= 8` inside a pattern guard. |
||
|
Matthew Jackson
|
71686f1407 |
Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv, the crate the other seven build against and the one held up as the reference workflow, was the last one still linting the library only — so its ~3,000 tests, by far the largest body of test code in the project, had never been linted at all. Turning the flag on surfaced 74 findings. Most were mechanical and applied with clippy --fix. The rest, by hand: - Four discarded Results in decrypt.rs. css::descramble_region returns a Result and four CSS tests threw it away, so a descramble that FAILED would have surfaced as a confusing buffer-comparison mismatch instead of the actual error. They expect() now. - A dead `kp` field on the PlantedWalk fixture. The test deliberately asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4 rather than against a stored value — its doc comment says so — which makes the field not just unused but a trap: the obvious "fix" of asserting against it would quietly weaken the test to comparing the fixture with itself. Removed. - Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut, three vec!s that only ever needed arrays, a filter_map whose every arm was Some, and a Vec::new()+push chain. - Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering in the generated docs. - A five-[u8; 16]-tuple return type named FourLevelParts. Three lints are allowed at the specific sites, with reasons, because they are wrong for this domain: the underscores in the bitstream-header literals mark BITFIELD boundaries, not digit groups, so regrouping them uniformly would satisfy the lint by destroying the only thing they encode; and in three table-validation loops the loop variable is the domain value under test (a DTS SFREQ code, an AMODE value, a palette entry number), which is what the assertion messages name. |
||
|
Matthew Jackson
|
42591c77fc |
test: constrain the AC-3/E-AC-3/DTS header decode and the boxes it emits
All 59 measured survivors in mp4/audio.rs: 50 killed, 9 proven equivalent, none left unaddressed. No production change — every extraction reads correct against ETSI TS 102 366 (5.3.2, 5.4.2, Annex E.1.3, F.4, F.6.1) and TS 102 114 5.3.1. It was a fixture gap, not a code defect, and a specific one: the existing fixtures gave several fields the SAME value (fscod=0, bsmod=0, lfeon=1, acmod=7) and asserted only derived channel counts. Nothing asserted the emitted dac3/dec3/ddts payload BYTES at all, so the packer's shifts and masks were entirely unconstrained. A wrong mask there does not crash — it writes a box declaring the wrong channel configuration, and a player believes it. Several kills needed fixtures designed to discriminate rather than merely exercise: the reduced-rate branch needed fscod == 3, which no test in the file reached — and `== -> !=` survived on an fscod=0 fixture only because the reduced table happens to return 48000 there too the DTS LFF mask needed a value where XOR and AND differ in MEANING: on the 5.1 fixtures LFF flips 1 -> 2 and BOTH codes mean "LFE present" the acmod mix-level skips needed three different acmods, because `^` is a no-op unless acmod == 4 exactly the ddts bitrate needed 44100/512, which does not divide evenly — at 48000 the rounding is invisible and `/ -> %` on den/2 survives The 9 equivalents are one pattern: OR-ing a shifted high part with a masked low part on disjoint bit lanes, where the mask is on the same line as the OR. Since cargo-mutants applies one mutation at a time, no single mutant can break both. Each was applied and observed green. FILED, not fixed: reserved sample-rate codes are silently guessed as 48 kHz (audio.rs:133, :20/:153) while a reserved DTS AMODE is refused with a comment explaining why. Same silent-wrong-metadata class, opposite answer. Refusing would make such a stream unmuxable, which is a product call. |
||
|
Matthew Jackson
|
9f25a4c454 |
fix(mp4): refuse a video track with no resolved dimensions
Resolution::pixels() returned (0, 0) for Unknown, and the MP4 sink wrote it verbatim into tkhd (ISO/IEC 14496-12 8.3.2) and VisualSampleEntry (12.1.3). Both fields are MANDATORY there, so unlike Matroska — which omits the optional PixelWidth/PixelHeight elements — MP4 has nothing to leave out. The result was a structurally complete file that passes every container check, declares a 0x0 video track, cannot be rendered, and is written with no error anywhere. WHY IT WAS POSSIBLE, which is the part worth keeping: pixels() previously fabricated 1920x1080 for Unknown. That was wrong but playable, so this sink never needed a guard and the absence of one was invisible. Changing the sentinel to (0, 0) moved the defect instead of removing it — a zero PAIR still reads as a usable value, so the sink stored it and serialised it. The accessor's doc comment then ENUMERATED the callers it believed were safe: "the Matroska sink omits the optional elements, the VobSub writer omits its size: line, and no caller divides by either dimension." Two of those three are true. MP4 was not on the list because MP4 has no guard at all, and a prose list cannot enforce itself. mkv.rs's own comment even states the principle — "the check belongs in the one accessor rather than in each caller that remembered to write it" — and labels/mod.rs still carried its own duplicate Unknown test long after the accessor took that job over. So: pixels() now returns Option. Not because Option is tidier, but because every caller genuinely needs a DIFFERENT answer and the compiler is the only thing that reliably makes them choose one. Matroska and the metadata sinks take unwrap_or((0, 0)) with the reason stated at each site; the VobSub path degrades to a palette-only .idx; MP4 fails with E_MP4_UNKNOWN_RESOLUTION (9055). Six call sites, not the five my first grep showed — I piped it through `head` and acted on a truncated list. The compiler caught the sixth. That is the same mistake as trusting a lens that reported silence. |
||
|
Matthew Jackson
|
0bc8d7af9c |
test: constrain the AACS key-map gap fill, the PSI walk, and MP4 field offsets
Third pass over src/mux/. 40 survivors killed, no production change. resolve.rs — the deleted-statement cluster is now fully constrained. All 14 deletable statements probed; 9 were already caught, 5 survived: c.sort_unstable() in fill_base_key_gaps. Every existing case handed it cuts already in LBA order, but IndividualSegment.tbl is a record list. Verified on HEAD: deleting the sort passes all 54 resolve tests. The mutant lays a base-key fill straight over a forensic segment. last_idx = idx (FMTS gap fill) and last_idx = hit (multi-CPS cache hit). An extent with nothing to sample must inherit its neighbour's CPS unit; the mutants fall back to the first unit's key. Exactly the shape this file's own comments name — wrong key, no error, lost_bytes == 0. Both check_halt()? polls in probe_fmts_index_keys. These cannot be killed by outcome, since a later poll returns Halted too. The tests count reads instead, which is what the don't-hammer-a-struggling- drive rule actually says: after a Stop the drive is asked for zero content sectors. The four unresolved += 1 arms each got a test, and deleting each fails exactly one — one-to-one, so no fixture passes for the wrong reason. A control test pins that the baseline table resolves, so an expect_err cannot succeed for an unrelated reason. ts.rs::scan_streams was never entered. Six killed, two of which return wrong answers that look right: reading the PAT/PMT CRC as a table entry invents a stream on PID 546 out of CRC bytes, and dropping the ES_info_length skip decodes a descriptor as an entry and loses the one after it. Every existing PMT fixture declares ES_info_length = 0; a real BD PMT carries a registration descriptor on essentially every entry. Also ISO/IEC 13818-1 2.4.4.3 program_number == 0 is the network PID, not a program. mp4/read.rs — 13. Height read as the width beside it; channelcount; the 4-byte base-128 descriptor varint (every existing esds fixture uses a single byte); all three optional ES_Descriptor fields, whose loss is silent (an AAC track just loses its CodecPrivate); first-vs-last media edit, which is A/V desync of the difference; and the version-1 mvhd timescale offset, emitted by any writer whose duration exceeds 32 bits. dts.rs — 7, from a real cargo-mutants run over the file rather than guesswork. Including a buffer that IS the syncword, which is the state a sync split across PES packets lands in the moment its last byte arrives. Equivalents proven by application, not argued: the sample_encrypted_units guard pair is mutually redundant by construction (total*p/9 < total for p <= 8), so either alone is equivalent and both together are not; the PMT section_len guard is dead code where its PAT twin panics; three of the seven EXSS_HEADER_MIN_BYTES arithmetic mutants still sum to 10. |
||
|
Matthew Jackson
|
9de88969ca |
test: constrain the DiscStream loss surface and the empty-title guards
Second mutation pass over src/mux/. 26 survivors killed, no production change. Verified on HEAD before landing: each mutation below passes all 1,237 mux tests unmutated-suite. The priority item was the honest-loss-reporting surface. Both DiscStream::errors and DiscStream::lost_bytes could return a constant with nothing failing — a rip that lost sectors would report zero loss to the caller. This project has already shipped one defect of that shape (a total decryption failure reported as an empty title, exit 0). Driven now through two short-read fills so both land on values that are neither 0 nor 1 and differ from each other; no constant and no field swap survives. MkvStream::finish -> Ok(()) also survived. MkvMuxer::finish has the zero-frame MkvInvalid guard and two tests cover it, but the Stream wrapper above it could return Ok unconditionally and bypass the guard entirely — the empty-title defence was one layer thinner than it looked. au_assembly: pinned au_opener_from behaviourally to the normative byte values for all four modes, with negative cases for codes that are explicitly not openers (MPEG-2 slice 0x01..0xAF, user data 0xB2, extension 0xB5, sequence end 0xB7 per 13818-2 Table 6-1; VC-1 0x0A/0x0B/0x0C; H.264 SPS/PPS/IDR-slice). au_assembly and codec/ hold independent copies of these constants; they agree today, and comparing constants would not catch logic drifting apart, so both sides are now pinned to the spec instead of to each other. demux_sink::sanitize: every filename component demux:// writes comes from disc-controlled text, so the path-separator arm is a traversal guard. Deleting it now fails, including an end-to-end case where base = "../evil/Title" must produce exactly one file inside the chosen directory. stts_and_ctts_expand renamed to stts_expands_runs_to_per_sample_deltas_in_order and given runs with distinct deltas AND distinct lengths. Its old name claimed ctts coverage it never had, which is why the composition-time chain went unconstrained for eight rounds; the doc comment now points at the tests that do cover ctts. Correction to the previous pass: codec/truehd.rs flush -> vec![] IS equivalent. Applied it, full mux suite green. TrueHD buffers across PES but parse emits every complete unit immediately, so a residual buffer at EOF is a truncated access unit and is correctly discarded. The vec![Default::default()] variants are genuinely different and are killed. Deliberately not constrained: mkv::set_opening_capture (diagnostics behind a process-global tracing check, flaky under the parallel runner), and the three stdio.rs header paths (StdioStream holds concrete io::Stdin/Stdout and cannot be driven without a production refactor to injectable Read/Write). |
||
|
Matthew Jackson
|
170fd0c064 |
test: constrain MP4 composition timing, MLP substream directory, and codec-private absence
Mutation testing over src/mux/. No production change — 49 survivors killed, all proven red before green. The MP4 composition-time chain was entirely unconstrained: VideoTiming::ctts, build_ctts and parse_ctts could each return a constant and the suite stayed green. Confirmed on HEAD: build_ctts -> vec![] passes all 1,220 mux tests. A demuxed B-frame title presenting in decode order would have shipped. The cause is a test whose name asserts coverage its body does not deliver — stts_and_ctts_expand builds an stts box and never touches ctts, and write_then_read_round_trip asserts sample sizes and keyframe flags but not one PTS. Same shape as the set_speed forwarding finding, different disguise. mlp_num_substreams / mlp_substr_header_size: every TrueHD fixture in the crate uses one substream and no extraword, so both could return a constant and agree with all of them. These position mlp_parity_ok's window over the AU header, so a constant mis-windows the parity check on exactly the multi-substream AUs that carry 7.1 and Atmos. CodecPrivate absent vs empty: mkv.rs writes Some(bytes) verbatim and omits the element on None (RFC 9559 5.1.4.1.24), so a zero-length Some emits a track header asserting the config IS empty. Four parsers could return Some(vec![]) before any frame. Also: mandatory ISO/IEC 14496-12 boxes (tkhd, vmhd, smhd, dinf, mdhd) could each build empty; HEVC num_extra_slice_header_bits (H.265 7.3.2.3) was never non-zero in any fixture, so the slice-type offset skip was unexercised; chapter names from the disc go straight into <ChapterString> and the & escape must run first; a stray 0x47 in a payload must not latch a TS resync. Documented as equivalent rather than killed: CodecParser::flush and the three parser flush bodies that differ from the mutant only by a tracing call, and DropTally::log_summary. |
||
|
Matthew Jackson
|
327087c70e |
Make five tests capable of failing, and stop the presence probe unmounting the disc
The worst of the five was a regression suite that never touched the code it guarded: nine batch-count tests called `safe_batch_count` and `buggy_batch_count`, both defined in the test file itself. The u16 truncation they exist to prevent could be reintroduced in sector/prefetched.rs with every one of them green. They now drive the real producer through the public API, and reinstating the truncation fails five of the nine. Worth recording that the symptom has changed since the original fix: the unit-alignment clamp below floors a zero batch at three sectors, so the bug is now a twenty-fold throughput cliff rather than the stall it once was. The MP4 reserve test's only numeric case was dominated by the floor and the buffer, so BYTES_PER_SAMPLE could be zeroed without failing it. It now has a case where the per-sample term dominates. The zero-count guard in FileSectorSource was likewise unfalsifiable — seek-past-EOF and a zero-length read both succeed — so the test now observes the file cursor. The AACS media-key ambiguity guard had no test at all; the pool scan is extracted so the verifier can be injected, because a genuine two-key collision needs one ciphertext decrypting under two AES-128 keys to plaintexts sharing a 64-bit magic, which is a 2^64 search and not a fixture. macOS implemented the documented cheap, side-effect-free presence probe by building a full exclusive transport — which force-unmounts the disc. Linux and Windows issue one TEST UNIT READY with no unmount; macOS was the outlier. It now walks the IOKit registry for the media object instead. The C shim's registry reads assumed CoreFoundation types the registry does not guarantee, so a driver publishing a CFNumber where a CFString was expected aborted the process from inside public API. Types are checked and a wrong type treated as absent. The unbounded waitpid on the unmount child is now a polled deadline, and the last-resort match gained the NULL check its two siblings already had. The empty-CDB guard existed only on Linux while a shared helper's comment claimed all three backends had it. Moved into the helper, so the comment is now true and macOS and Windows are covered. One finding was REJECTED with evidence rather than fixed. The TrueHD buffer-cap test was indeed bogus, but MAX_TRUEHD_BUF turns out to be unreachable by any input: the parser only retains data when the buffer is shorter than the declared AU, and that declaration is twelve bits, so the worst case is 8189 bytes against a 256 KiB cap. An exhaustive sweep over all 65536 AU headers confirmed it. The fixture now sits at the reachable ceiling and asserts that instead. The cap itself is left in place as defence, unreachable by construction, matching how the AC-3 resync guard was handled earlier in this audit. Two behaviour changes worth naming: Linux's empty-CDB error becomes InvalidCdbLength rather than a transport failure, and an unknown device now reports absent media rather than a not-found error, because the registry cannot tell an empty drive from a missing one. The latter is a conflation of the kind this audit has fixed three times; it is recorded for the next round rather than left silent. |
||
|
Matthew Jackson
|
5c6a6d0785 |
Round 5: reject a degenerate fixed lace, bound the pending buffer by bytes
Five fixes. Three are real defects with regression tests; two are bounds that were expressible but not expressed. A fixed-size lace (RFC 9559 §10.3.4) whose body is empty declared n frames and carried none. The divisibility check passed, because 0 % n is 0, and `chunks` yields nothing on an empty slice whatever width it is given — so the clamp that existed to avoid chunks(0) returned zero frames where the Lacing Head said n. The whole lace vanished with no error raised and the caller saw a clean short block. A zero-size frame cannot be a valid frame, so it is now malformed. A disc read failure while fetching a directory entry's ICB became a file size of zero rather than an error. Zero is indistinguishable from a genuinely empty file, so an unreadable ICB on a damaged disc silently changed which titles a caller saw as present — read_directory already fails hard on its entry-budget guard, so propagating is also what the surrounding code does. read_file_size still returns Ok(0) for an ICB whose tag is neither File Entry nor Extended File Entry, which is a real zero and not a failure. The pending-frame buffer was capped at 4096 frames, which does not bound memory: frames are arbitrarily large and a UHD video frame runs to a few hundred KB, so the existing cap permitted over a gigabyte. Now bounded by bytes as well, at 64 MiB. round_up_grain overflowed for inputs within one grain of u64::MAX — div_ceil then multiply — and the wrapped product is small, turning the largest possible estimate into a negligible reserve. It saturates, and the reserve is clamped to what a `free` box's 32-bit size field can actually hold, since writing a larger one truncated the size and left mdat beyond a box claiming to be far shorter. No real title comes close; a 90 GB UHD title estimates a few MiB. The AC-3 resync guard now advances the PTS cadence like both of its sibling branches, so the three paths out of that block cannot disagree. This one is defensive and has NO test: reaching it needs input that both parses frames and leaves a megabyte of residue, and the parser's own carry rules drop pre-sync junk and cap a partial frame at 8192 bytes, so no such input was found. Stated here rather than covered by a test that would pass either way. Two findings from this round were rejected on inspection. A reported panic in the .mpls suffix check does not exist: the `.get(..)` on the line above returns None off a char boundary and `filter` never runs its closure, so the byte index is unreachable. A test written for it passed against the unfixed code, which is what surfaced the error. |
||
|
Matthew Jackson
|
4fcd28b487 |
Parse MKV lacing, route by real TrackNumber, honour NAL length size and edit lists
Four conformance defects in the read paths, two of them silent corruption. **Lacing was ignored entirely.** RFC 9559 §10.2 defines Xiph, EBML and fixed-size lacing, where one Block carries several frames; the reader took the Block payload verbatim, so a laced Block became a single "frame" consisting of a lacing header followed by concatenated frames — garbage to the codec parser, no error. Audio tracks from other muxers commonly use lacing, so an ordinary foreign MKV was silently mangled. All three modes are now parsed: Xiph 255-run sizes including the trailing-zero rule for exact multiples of 255, EBML unsigned first size plus SIGNED VINT deltas with the 2^((7*n)-1)-1 bias of §10.3.3, and fixed-size even division, with the last frame's size deduced from the remainder. Laced timestamps follow §10.3.5: the first frame takes the Block timestamp and the rest are spaced by the track's DefaultDuration, else BlockDuration/count, else shared with a warn. Parsing was chosen over refusing because refusal would leave freemkv unable to remux common foreign audio at all, and each mode is about fifteen lines. A malformed lacing header now raises a NEW code, E_MKV_LACING_INVALID = 9052, deliberately NOT MkvInvalid — because is_skippable_title_stub classifies MkvInvalid as a skippable nav stub, so reusing it would have recreated the exact conflation that is still open as a separate finding. A test asserts the new code is not skippable. **TrackNumber was assumed to be 1..N in TrackEntry order.** RFC 9559 §5.1.4.1.1 only requires it to be non-zero and unique, so sparse or unordered numbers are legal. Block routing and codec_private both computed track + 1. A real TrackNumber map is now built, recorded only for TrackEntries that yield a stream so dropped track types no longer shift the mapping. Verified red here independently, and the failure mode is worse than mis-routing: with track + 1 restored, a buttons track's payload was attributed to the AUDIO stream — wrong payload into the wrong codec parser. **The NAL length prefix was hardcoded to 4 bytes.** lengthSizeMinusOne lives in avcC byte 4 and hvcC byte 21 (ISO/IEC 14496-15 §5.3.3.1.2, §8.3.3.1.2) and was never read, so a source declaring 1- or 2-byte prefixes had its raw prefixed bytes emitted verbatim with no start codes. All four conversion sites now derive the width from the track's own configuration record. **Edit lists were ignored.** No edts/elst was parsed, so the presentation timeline an edit list defines (ISO/IEC 14496-12 §8.6.5/§8.6.6) was dropped — which is how encoder delay is normally expressed. Leading empty edits and the first media edit's media_time are now applied to both dts and pts, with the movie vs media timescale distinction respected. A list needing more than a constant shift applies the leading edit and warns rather than presenting the result as faithful. 17 tests. I reproduced the lacing mutant independently: returning the body whole kills five of them, including the exact-payload and malformed-header cases. Still open and deliberately untouched: the MkvInvalid / is_skippable_title_stub conflation across ~20 reader raise sites. It is a cross-cutting error.rs change and E_MKV_LACING_INVALID is the template for it. |
||
|
Matthew Jackson
|
013881ac06 |
Restore the MP4 conformance fixes I clobbered while landing another agent's work
detect_rate's nearest-match fix, the colr HLG/BT.470 fix and their four tests were silently reverted. Cause: the r3fix-silent worktree was cut BEFORE the conformance commit landed, and I landed its work by copying whole files into the main tree. mp4/mod.rs was in both agents' file sets, so silent's copy — built on the older base — overwrote the conformance changes wholesale. The gate stayed green throughout, because reverting a fix and its tests together is perfectly consistent. Re-applied the conformance commit's diff for that file with a three-way merge; both agents' changes to mp4/mod.rs now coexist (final_report, UndescribableAudio and the max(track_id) id fix are all still present alongside RATE_TOLERANCE_FPS and the colr resolver delegation). Only mp4/mod.rs was affected. mp4/audio.rs and mkv.rs were in no other agent's file set and were intact. Process lesson, recorded because I would otherwise repeat it: NEVER land a parallel agent's work by copying whole files, when its worktree was cut at an older base than HEAD. Apply its DIFF (git apply -3), or rebase its worktree first. Copying files silently discards anything committed to those files in the interim, and no test can catch it because the tests disappear with the code. |
||
|
Matthew Jackson
|
5f8dc392c0 |
Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the minimum, this goes to current stable and fixes what that costs. The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54 clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy collapsible_if lint in between. Stopping at the minimum would have been the most expensive choice available. Roughly 47 lints across the eight repos, the large majority auto-fixed: libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6, freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced by enumerate, and a loop whose first let-else became a while-let. Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only appear once the crate DECLARES 1.89 or later, because let-chains become available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87 reports clean and is meaningless — gate with the real precommit script, which is also the only thing that covers build scripts. The pin still sits below the Mac default, so it keeps doing its job: catching lint drift locally before CI sees it. |
||
|
Matthew Jackson
|
a32373ff40 |
Fix fifteen defects across perf, resource, panics and key hygiene
All 21 findings held up under verification; 15 fixed here, 6 deferred to files another agent held this round, 0 rejected. **A defect in my own round-2 probe fix.** CHUNK_SECTORS was 1024, and 1024 % 3 == 1 — verified — so every chunk after the first was misaligned against the 6144-byte AACS aligned unit and would be REJECTED by DecryptingSectorSource's alignment gate. On an encrypted disc the forced-subtitle probe I added last round would have read almost nothing past its first chunk. Now 1023 sectors (341 aligned units) with a const assertion that fails the build if it stops dividing, plus set_unit_base per extent so the source's gate is anchored where the extent actually starts. **The same probe skipped sectors on a short read**, advancing by the REQUESTED count rather than the bytes actually returned, so a partial read silently left a gap in the middle of the evidence. It now advances by n/SECTOR_BYTES and clamps n to the buffer. **Its cache key omitted the PGS PID set**, so a playlist declaring an extra subtitle PID got another playlist's verdict for a track that had never been probed. And the key was the whole extent list, so partial clip sharing missed entirely. Both fixed by keying (start_lba, sector_count, pid) — and per-extent keying was shown SOUND rather than assumed: ForcedTracker is two monotone booleans, so per-extent evidence composes by field-wise OR, order- and grouping-independently. Making that honest required per-extent demux state, so an extent's evidence comes only from its own bytes, and memoising only extents whose read reached a designed stop. **A reachable panic in the timeline.** mkvstream::parse_block accepts a TimestampScale up to i64::MAX, so a video frame can set high_ns = i64::MAX and the next passive frame panicked adding the backstep. In release it wrapped negative instead, firing the straggler clamp for essentially every passive frame — audio and subtitles rewritten onto the wrong point of the output timeline. All four sites saturate. **A public constructor divided by zero**: PrefetchedSectorSource::new_with_events with unit_align == 0. Now InvalidInput, matching its batch_sectors sibling. **Two Debug impls printed key material.** DiscInputs (volume_id, mkb, unit_key_ro, samples) and UnitKeyFile both derived Debug. Nothing logs them today — fixed as prevention, because the next tracing::debug! someone adds is the leak. A doc claim that DiscInputs "contains no secrets" was false and is corrected. **An env-var multiply could overflow** in file_sector_source; now bounded at 64 GiB like its writeback sibling, with the parse split out so the bound is testable without touching process env. **The mp4 demuxer allowed one sample per file byte** — ~64x RAM amplification. Now file_len/16, since only vide/soun tracks are indexed and the shortest legal AC-3 frame is 128 bytes. **Two pipeline concurrency defects**: a consumer apply() error was invisible to the producer, and abandon/finalise had a TOCTOU where a caller could report an unfinalised output. Both fixed with compare-exchange state rather than a bool. **Two per-frame copies removed**, both MEASURED rather than reasoned: the AU assembler now hands its allocation to the frame (same pointer, unchanged capacity, proven by asserting the pointer) and tsmux reuses one Annex-B buffer across frames. Both keep capacity deliberately — a naive split_off would have cost more than it saved. **A comment pointed at the wrong file** for a mirrored constant; the mirror is now compiler-enforced with a const assertion converting 90 kHz ticks to ns, so drift fails the build. Deferred to another agent's files, all confirmed: detect_rate's fractional-twin snap, the mp4 reserve's u32 truncation, round_up_grain's overflow, the quadratic base-key gap fill, and MkvStream's frame cap counting frames rather than bytes. Every fix verified red by reverting it. Also noted for later: DecodeSampleSet still derives Debug over multi-MB of on-disc ciphertext. |
||
|
Matthew Jackson
|
3efa6211f3 |
Make six silent mux failures observable
All six confirmed against the code. The governing rule this cluster serves: a
lossy or degraded outcome is never silent, because a corrupt rip the user does
not know about is the worst failure available.
**A 3D MKV re-mux silently lost one eye.** The BlockGroup read path had arms for
BLOCK / BLOCK_DURATION / REFERENCE_BLOCK only, so BLOCK_ADDITIONS fell into the
skip arm — while the writer does emit BlockAdditions > BlockMore > BlockAdditional
for the MVC dependent view. Reconstruction was judged out of scope and the
reasoning is recorded: PesFrame has no side-payload field and the header parser
never reads BlockAdditionMapping, so there is no dependent-view track to route the
AU to. Instead the loss is now LOUD — counted in bytes and events, warned once,
and surfaced through MkvStream's errors()/lost_bytes(), which the driver already
samples into MuxOutcome. One detail in the finding was wrong and is corrected: the
re-mux does NOT still advertise the mvcC mapping, because the header parser
ignores that element, so the output is a plain 2D H.264 track.
**An all-titles rip silently skipped real titles.** The header-buffer-cap
overflow returned Error::MkvInvalid, and is_skippable_title_stub matches exactly
E_MKV_INVALID | E_CSS_KEY_MISSING — verified here — so a 512 MiB-of-frames title
was classified as an empty nav/menu PGC stub and dropped. It now has its own
E9051 / MuxHeaderBufferExceeded { bytes }, outside the skippable set.
**The public pre-mux report contradicted the file.** Mp4Sink::finish() drops an
audio track it cannot describe, which I chose last round over failing an export
whose video is fine — but mp4_fit_report still listed that stream as included, so
the application's plan and the actual output disagreed. Fixed at both levels:
Mp4SkipReason is now non_exhaustive with NoSamples and UndescribableAudio,
Mp4Sink::final_report() describes the FILE rather than the plan, and for the
boxed dyn Stream path a defaulted Stream::undelivered_streams() carries the
information out to MuxOutcome::undelivered_streams with a driver-side warn.
**MP4 track ids could collide.** ids were assigned before the retain that drops
sample-less tracks, while next_id came from the post-retain count, so [1,3]
yielded next_id 3. Now max(track_id) + 1, saturating.
**Stream selection silently skipped its codec_privates prune** when the lists were
not the same length — but codec_privates is consumed POSITIONALLY and trailing
extras are documented as benign, so the length-equality guard was itself the bug.
The prune now runs unconditionally by index.
**The m2ts_mux scaffolding armed params_written on both the absent and the
unparseable codec_private arms** — the same defect already fixed in tsmux.rs.
Split into params_attempted (a latch, since retrying identical bytes cannot help)
and params_emitted, with a warn on each failure arm and an accessor so the
eventual wiring and its test can observe it.
Each fix verified red by mutating back to the prior behaviour: errors() 0 vs 1,
E6008 vs E9051, final_report [0,1] vs [0], next_track_id 3 vs [1,3], and the
selection prune resolving index 1 to the wrong track's record.
API surface deliberately widened: MuxOutcome gains a public field and Mp4Sink
becomes public. Nothing in-repo breaks. Note a behaviour change on the mkv://
input path — a 3D re-mux now reports non-zero loss, so a consumer treating
errors > 0 as disc damage will trip on it. That is intended: the outcome IS
degraded.
|
||
|
Matthew Jackson
|
9ad68dd092 |
Fix four MP4 conformance defects against the standards
**dec3 declared a 0 kbit/s AC-3 substream.** parse_dolby routes bsid < 11 to parse_ac3, which leaves data_rate_kbps = 0 and keeps the AC-3 bsid, yet dolby_sample_entry wrapped that config in ec-3/dec3 for any Codec::Ac3Plus track. ETSI TS 102 366 Annex F.4 assigns ac-3/dac3 to an AC-3 bitstream and F.6 assigns ec-3/dec3 to an Enhanced AC-3 one, so the entry now follows the SYNCFRAME that was actually parsed, not the playlist's codec label. Computing an AC-3 data rate and keeping ec-3 was rejected: it fixes one field while bit_stream_identification and num_dep_sub keep misdescribing the stream. The bsid threshold is hoisted into one constant so parser and entry-chooser cannot drift. Adjacent defect fixed in the same box: data_rate is 13 bits from a u16 source, so push's mask WRAPPED anything above 8191 (9000 became 808); it now saturates. **colr tagged HLG as PQ, and PAL as BT.601.** video_colr carried a second, drifted copy of the ColorSpace-to-CICP map: transfer 16 (PQ) for every BT.2020 stream with no HdrFormat override, and 6 (BT.601) for Bt470bg. Per ITU-T H.273 Table 3, HLG is 18 and BT.470-6 System B/G is 5 — and mkv::cicp_for_video already got both right. video_colr now delegates to that shared resolver, so the duplicated table is gone and cannot drift again. It keeps only its own decision about WHETHER to emit the box, since an absent colr and an all-unspecified colr mean the same thing per ISO/IEC 14496-12. **Exact 24.000 / 30.000 / 60.000 fps was declared 23.976 / 29.97 / 59.94.** detect_rate took the FIRST STD_RATES entry within 0.5 fps, and every 1000/1001 entry precedes its integer twin 0.024 fps away — a 0.1% error across the whole track's mdhd and stts. Fixed as nearest-wins rather than by reordering the table: reordering fixes today's table and re-breaks the moment someone appends a rate, while nearest-wins is order-independent. Verified red here independently by reverting to first-match, which fails exactly the two timing tests. **ddts MultiAssetFlag was set from has_extension.** In the DTSSpecificBox (ETSI TS 102 114) that flag signals more than one audio ASSET. A DTS-HD MA/HRA track is one asset whose extension substream carries the XLL/XBR component, so setting it from "an EXSS sync follows the core" sent a parser looking for a second asset descriptor while StreamConstruction simultaneously said there was no extension — the box contradicting itself. This module parses the core header only and never reads the EXSS asset table, so 0 is the only honest declaration. A StreamConstruction index for core+EXSS was deliberately NOT invented: that field is a table lookup that could not be confirmed against the standard, and a wrong index is worse than an under-declaration. DTS_AMODE_LAYOUT's masks were independently re-derived against ETSI TS 102 114 §5.3.1 and all 16 are CORRECT — only three adjacent comments were wrong (the AMODE 2/3/4 annotations were rotated by one, and AMODE 9's said "5.1 with LFE" when 0x0007 is the 5.0 mask and LFE is OR'd in separately). Comments corrected. Every one of the eight new tests decodes the field back OUT of the emitted bytes — data_rate from the dec3 body's leading 13 bits, MultiAssetFlag from bit 48 of the ddts tail, colr from the nclx payload inside a real stsd, and the frame rate from mdhd.timescale plus stts.sample_delta of a fully muxed MP4 — rather than restating arithmetic. This audit has already caught one of my own tests doing the latter. NOT fixed, root-caused and recorded at the site instead: an A_PCM/INT/BIG track ships with no BitDepth, which the Matroska Codec Specifications make a MUST. The width exists on disc (BD LPCM signals it in the ES header byte 3, DVD in the IFO audio attribute byte 1) but neither source reaches MkvTrack::audio, and the fix needs a new AudioStream member plus a deferred setter in files another agent held this round. Guessing 16 was rejected — it would confidently misdecode every 24-bit disc — as was refusing the track, which would regress the 16-bit majority that currently plays by accident. |
||
|
Matthew Jackson
|
50f37462db |
Remove reference-decoder citations from a public MIT-licensed repo
This crate is MIT licensed. Comments citing another decoder's internal symbols and reproducing its tables verbatim create licence risk that no engineering benefit justifies, so every such citation is replaced with the primary source: ETSI TS 102 114 §5.3.1. Eleven sites across src/mux/codec/dts.rs and src/mux/mp4/audio.rs. The technical substance is unchanged in every case — the deficit-sample-count semantics, the reserved-field skips, the invalid LFF value, and the 16 legal AMODE codes are all spec facts and are now attributed as such. Four CHANGELOG entries that named a validator are reworded; the "no a reference decoder" dependency claim stays, since stating what this project does NOT depend on carries no risk. I initially argued this was a false positive on the grounds that the project's hygiene rules name internal infrastructure and reverse-engineering material, not open-source citations, and that a channel-count table from a standard is fact rather than expression. That reasoning missed the point: the exposure is MIT distributing text derived from GPL/LGPL sources, and that is the maintainer's risk to weigh, not mine. Reversed in full. |
||
|
Matthew Jackson
|
99c5fd3500 |
Reference keyframes per track, size the DTS reserve, correct two claims
The ReferenceBlock offset was computed for ANY video track, but the keyframe tick it measures against was recorded in a single global slot gated to the PRIMARY video track. On a title with two video tracks — an MVC base plus secondary view, or a multi-angle disc — a secondary track's non-keyframe therefore referenced a keyframe on a different track, or 0 (a self-reference) when the primary had not produced one yet. The tick is now recorded per track, so a non-keyframe can only reference a keyframe on its own track. The faststart moov-hole estimate modelled every audio track as (E-)AC-3 at 1536 samples per frame. 1.6.0 added DTS to the writer's carried set, and a DTS core AU is commonly 512 samples — a third of that — so a DTS track's sample table was under-reserved threefold and the mux fell back to moov-at-end, losing faststart on exactly the files 1.6.0 newly supports. mvc_frame_emits_blockgroup_additional_and_reference asserted only that the non-keyframe's ReferenceBlock was Some(_). Its non-MVC sibling, added in the same commit, pins the exact offset; this one now does too, so a mutant emitting a constant or wrong-signed offset no longer passes. The comment above the mp4 sample budget claimed file_len stops a crafted file inflating allocations "past the file's own size". Each indexed sample costs ~52 bytes, so the real ceiling is ~52x file_len (still capped by MAX_SAMPLE_COUNT). The bound is real; the comment overstated how tight it is. |
||
|
Matthew Jackson
|
bcf47cc4ca |
Fix ddts numeric truncation and the decrypt pool's poison asymmetry
ddts CoreSize wrapped to zero on a maximum-size core. core_size is FSIZE + 1 and FSIZE is itself 14 bits, so the maximum is 16384 — one past what the 14-bit CoreSize field holds — and push()'s mask turned that into 0, declaring an empty core frame. Clamped to 16383 instead: one byte short beats telling a decoder there is no core. Proven red first (the field read back as 0). ddts avg/max bitrate under-declared every non-integral frame rate. It computed sample_rate / frame_samples first, so a 512-sample core at 48 kHz truncated 93.75 frames/s to 93. Multiplying before dividing, with round-to-nearest, keeps the precision. set_decrypt_threads skipped the pool swap on a poisoned lock while DECRYPT_THREADS had already been updated, so the new thread count was reported as taking effect while the stale pool kept serving. decrypt_pool() deliberately recovers from poisoning for exactly this reason; the setter now does the same. The pool Arc is immutable once stored, so a prior panic cannot have left it half-written. The CoreSize test decodes the value back out of the emitted box rather than restating the clamp — a first draft asserted the clamp arithmetic against itself, which would have passed against the unfixed writer. |
||
|
Matthew Jackson
|
ec5cd31ae1 |
Stop Mp4Sink losing audio frames and writing an empty sample entry
Mp4Sink::write returned Ok(()) without recording the sample whenever an audio track's frame would not parse into a sample entry. Two consequences, both silent: leading audio frames were lost until one frame parsed, and a track whose frames never parsed disappeared from the output entirely — finish()'s retain() removed the sample-less trak and the run reported success. That contradicts this crate's stated policy that a skipped track is never silently dropped. The drop was never necessary. audio_entry is read in exactly one place, build_trak, reached only from build_moov inside finish() — nothing on the write path consumes it. So write() now records every sample and derives the entry opportunistically from whichever frame parses first. That makes build_trak's `audio_entry.unwrap_or_default()` reachable, which would emit an stsd declaring entry_count=1 around an EMPTY sample entry: a structurally invalid mp4 returned as success. finish() therefore drops any audio track it cannot describe, with a tracing::warn! naming the codec and sample count. Dropping rather than erroring is deliberate. It matches finish()'s existing treatment of sample-less tracks, keeps an export whose video is fine from failing outright, and needs no new error code — a new code would mean a new i18n key across 29 locale files in another repo, which is not this change's scope. The track's bytes stay unreferenced in mdat: wasted space in a valid file, which is the cheaper failure. Test pins both halves — moov describes only the video track, and the unparseable audio bytes still reach mdat rather than being discarded at write time. |
||
|
Matthew Jackson
|
a1304f9e78 |
Stop the mp4 demuxer dropping tracks silently or inventing sample offsets
Three defects in the mp4:// read path, all of the same family: a damaged source was remuxed minus a track, or with fabricated data, and the run reported success. Silent drops. Eight paths dropped a whole track on malformed input with no report of any kind, so an mp4:// source missing its audio looked like a clean run. Each now emits a tracing::warn! naming the track and the missing or inconsistent table (tracing English is permitted in this crate; the numeric error codes are unchanged). The non-A/V handler case is debug!, since skipping a timecode or hint track is normal. Fabricated offsets. sample_offsets ended with a `while offsets.len() < sizes.len()` loop that packed unplaced samples after the last known offset. Those samples have no known location, so the invented offsets made the reader pull frame data from arbitrary file bytes — the exact "emit garbage" outcome the stco/stsc presence guards refuse. It now returns the short list and the caller drops the track. Short stts. `durations.get(i).unwrap_or(0)` gave every sample past the end of a short stts a duration of 0, collapsing the whole tail onto one timestamp. That is the same degenerate timing the `durations.is_empty()` guard was written to refuse, so the guard now refuses both cases. Two shared test fixtures were internally inconsistent and only passed because the reader was lenient: stsz declared 3 samples while stsc placed 1, and the hostile-stsz fixture's stsc/stts covered a single sample. Both are now consistent. The hostile fixture keeps its lying stsz count — that lie is what it tests — but its stsc and stts now cover whatever count survives the file_len bound, so it exercises the allocation bound rather than the inconsistency guards. Two new tests pin the new refusals by mutating the consistent fixture: an stsc that places 1 of 3 samples, and an stts that covers 1 of 3. |
||
|
Matthew Jackson
|
f76688a0dc |
Make the ddts speaker mask agree with its own channel count
The ddts box declares both a channel count and a 16-bit speaker mask, and a decoder may trust either. dts_channel_layout ended in a `_ => 0x0007` catch-all describing five speakers (C + L/R + Ls/Rs), so for AMODE 6, 7, and 10 through 15 the box contradicted the count DTS_AMODE_CH declared alongside it — provoking a downmix or an outright decode error. AMODE 6 (L + R + S) is the reachable case: its S is a single centre-surround, not the Ls/Rs pair, so it is three channels and 0x0012, not four and 0x0006. AMODE 13/14/15 do not occur on retail media, which ships a 5.1 AMODE-9 core plus an extension substream. Replace the catch-all with the full 16-entry ETSI TS 102 114 mask table. Every entry is cross-checked against DTS_AMODE_CH by counting the speakers its bits imply: sixteen independent constraints, all satisfied, and that table is itself pinned to the per-AMODE channel counts in ETSI TS 102 114. AMODE is a 6-bit field, so the reserved 16..=63 are reachable from a malformed stream. The old `unwrap_or(6)` invented a channel count for them that no mask could match; parse_dts now refuses those frames rather than guessing a layout it cannot name. One existing fixture placed the ext-sync pattern at f[4..8], which incidentally set f[7]=0x25 → AMODE 20. That test is about where the pattern sits, not about AMODE, so the frame is now spec-legal (AMODE 9, 48 kHz) with the pattern moved into the payload proper. |
||
|
Matthew Jackson
|
b79ff71b43 |
Fix read-fault misclassification, DTS AMODE channel table, and untestable guards
- resolve_fmts_key_map: distinguish a genuinely-not-FMTS disc from a
transient live-drive read fault. read_filesystem now returns the new
Error::UdfNotFilesystem for a deterministic tag/format mismatch (no AVDP,
no partition descriptor, no FSD); resolve maps only UdfNotFilesystem (fs)
and UdfNotFound (.tbl absent) to Ok(None), and PROPAGATES DiscRead / other
I/O faults so a marginal AACS 2.1 disc fails loud instead of silently
dropping forensic content under a base-Unit-Key-only map.
- DTS_AMODE_CH (mp4/audio.rs): extend 10→16 entries
{1,2,2,2,2,3,3,4,4,5,6,6,6,7,8,8} (the spec per-AMODE channel table / ETSI TS 102 114) so
the spec-legal high AMODEs that now pass the decodability gate declare
their true channelcount (AMODE 13→7, 14/15→8) instead of a truncated 6.
- session.rs resolve_keys "called before scan" guard is now testable:
from_parts_for_test takes Option<Disc>; added a test that a disc-less
session returns a clean DeviceNotReady Err rather than panicking.
- mp4/read.rs: a track with samples but a missing/malformed stts (mandatory
per ISO/IEC 14496-12) is dropped rather than emitting all-zero timestamps,
matching the existing stco/stsc guards; all-tracks-dropped → Mp4Invalid.
- Remove the inert MuxInput::Iso.key_map field (the Iso path re-derives its
map inside build_iso_pipeline); the live path keeps Live.key_map.
All four fixes are mutation-verified.
|
||
|
Matthew Jackson
|
3bd2fd23b0 |
Fix audit findings: DTS AMODE bound, key-fetch negative memoization, PGS probe coverage
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just 0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements; only 16-63 are reserved. a reference decoder the spec per-AMODE channel table confirms 10-15 are decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal multichannel core frames as undecodable, silencing recoverable audio. Add a regression test (literal 0..16 range) that fails if the bound reverts to 10. - keysource: only memoize a NEGATIVE (empty) key-fetch result when every source genuinely ran and none held the key — never when a source Err'd (network down, unreachable). A transient outage was being cached as a permanent "no key" for the fingerprint, permanently dropping a unit that could be recovered once the source came back. Thread an `errored` flag out of the drivers and gate the cache insert on it. Tests cover both the recover-after-outage case and that a genuine absence is still memoized. - pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS display sets through the full demux -> parse -> observe -> apply path, both a forced verdict landing and a non-forced verdict clearing a vendor flag. - mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD). - scan_iso test: add independent fixture expectations (volume id) so the parity test is no longer purely tautological against a re-run of the same composition. |
||
|
Matthew Jackson
|
b9568242df |
libfreemkv: 10-phase release audit fixes (v1.5.2..HEAD)
Multi-round audit of the decrypt/AACS/mux-codec refactor. Fixes, in descending severity: - mux/mp4/read.rs: bound untrusted-input allocations. `sample_budget` now also capped by file_len (a fixed-size stsz claiming count=u32::MAX can't inflate the Vec<SampleRef> past the file's own size); trak scan capped at MAX_TRACKS matches; find_box() takes only the first match (cap=1) instead of materializing every match. Removes dead find_boxes wrapper. - disc/mod.rs: merge_content_key_ranges now UNIONS same-key overlapping ranges (coverage-preserving) instead of dropping the non-overlapping tail, which silently left encrypted LBAs uncovered -> ciphertext passthrough in the whole-disc sweep/patch map. Different-key overlap (malformed) still dropped to keep the set disjoint. - sector/decrypting.rs: remove dead unit_key_idx field + with_unit_key_idx setter (vestigial from the pre-keymap trial-decrypt design; AACS is map-only now). Fix stale docs. - decrypt.rs / resolve.rs / error.rs / extract.rs: doc/comment drift from the refactor (AacsKeyMap positive-map semantics, resolve_mux_key_map doc reattachment, decrypt_sectors_in_content legacy-alias, E_MP4_INVALID meaning, multi-CPS orphan by-design note). Test coverage (all mutation-verified real): - DTS NeedMore force-flush buffer bound; FLAC/MPEG-audio PTS carry-forward; mp4 mdhd timescale=0 divide-by-zero guard, MAX_TRACKS cap, sample-count file_len bound, MAX_ALLOC_BYTES cap under inflated file_len. - resolve_fmts_key_map: extracted filter_addressable_segments, resolve_tie_phase, fill_base_key_gaps as pure behavior-preserving helpers, each unit-tested (segment filter, phase-tie arms, gap-fill gaplessness over every extent). |
||
|
Matthew Jackson
|
1eb6910bdb |
Harden mux + decrypt paths; fail-loud on unresolvable keys
mp4 demuxer (untrusted input): bound every allocation sized from a box field (stsz/stco/stsc counts, stts/ctts run-lengths, per-sample and moov sizes, plus an absolute cap so a sparse file can't inflate file_len); guard the parse_stsd slice and a zero mdhd timescale; cap track count so the per-track PID can't overflow; rewrite read_moov to handle size==0 / size<8 / 64-bit largesize; parse esds/AudioSpecificConfig for AAC; write tkhd duration in the movie timescale. decrypt: resolve_mux_key_map now fails loud on an extent no key can classify instead of inheriting the previous extent's key, so a keymap never silently carries a wrong key; the sweep/patch key-fetch recovery fails loud when a unit is still unresolved after the retry. AACS: reject inverted forensic segments in both range builders; compare the forensic index in u16 space so an out-of-range value can't truncate onto a valid u8 index. RECOVERED_ERROR no longer latches the damage zone, preserving the 30s wedge cooldown for a following hard error. audio: AAC/MP2/MP3/FLAC carry the last PTS across a PES with no timestamp; the DTS-HD extension-sync search is bounded to after the core; the MP4 16.16 sample-rate field saturates. demux_sink records the video reference before the kind filter so audio:// / sub:// keep multi-clip PTS continuity and the DELAY tag. Remove a dead error variant and the AACS-unsupported-video code; codec comments cite the primary format specs; assorted doc/naming fixes and regression tests throughout. |
||
|
Matthew Jackson
|
f7edd4e6a9 |
mux/mp4: DTS audio (dtsc/dtsh + ddts)
Parse the DTS core header (SFREQ/AMODE/RATE/LFF/NBLKS/FSIZE) → a ddts box (sample rate, channel layout mask, core size, computed bitrate, LFE); whole access units (core + DTS-HD extension substreams) pass through, so an HD decoder finds the extension. dtsh when an extension sync follows the core, else dtsc. Fit oracle now carries DTS / DTS-HD MA / DTS-HD HR. Validated on 300 (real DTS-HD MA 7.1): freemkv's mp4 DTS track is byte-identical to ffmpeg -c copy under ffprobe (dts / 48000 / 8ch / 7.1) and decodes clean (exit 0). Channel layout correct. |
||
|
Matthew Jackson
|
a947439171 |
mux/mp4: faststart on by default (reserve-and-fill)
moov now precedes mdat. At create() reserve a moov-sized hole (a free box) between ftyp and mdat: reserve = round_up_4MB(16 B/sample × est_samples) + 4MB buffer, floored at 8MB. Because the hole precedes mdat, sample offsets are fixed from the start — no rewrite, no offset patch. finish() writes moov into the hole and pads the slack with a free box; if the estimate is blown it falls back to moov-at-end. Streams over HTTP without a pre-fetch. Reserve-math + box-order tests added. |
||
|
Matthew Jackson
|
8f55cb78d2 |
mux/mp4: MP4 demuxer — mp4:// as a source
Read side of mp4://: parse moov/trak/stbl (stsd codecs+hvcC/avcC/channel info, stsz/stco/co64+stsc → per-sample offsets, stts+ctts → decode/ composition timing, stss → sync), rebuild a DiscTitle, and emit samples as PesFrames in global decode order. Video NALs are length-prefixed in MP4 — the exact framing the MKV muxer wants — so no reframing. Wired into input() so mp4:// flows to every sink (mkv://, audio://, json://, …). In-memory write→read round-trip test proves symmetry (streams, hvcC, sample sizes survive). Progressive MP4 only; fragmented (moof) is future. |
||
|
Matthew Jackson
|
65e14fe3b7 |
mux/mp4: route through WritebackFile (bounded-cache writeback)
Make Mp4Sink generic over a seekable writer; the CLI output() arm wraps the file in WritebackFile (as mkv:// does) so a UHD-scale mux to slow / NFS staging avoids the dirty-page burst. The mdat backpatch is an ordinary seek WritebackFile already handles (seek_then_patch_roundtrip). Tests switched to in-memory Cursor writers. |
||
|
Matthew Jackson
|
0c9d375548 |
mux/mp4: M2 — audio tracks + fit oracle (multi-track)
Generalize the sink to N tracks: one video + every MP4-mappable audio track. Audio sample entries built from the first frame's bitstream — AC-3 (ac-3/dac3) and E-AC-3 (ec-3/dec3), parsing the (E-)AC-3 BSI for fscod/bsid/bsmod/acmod/lfeon and the data rate. Per-sample audio durations from PTS deltas (no reorder → no ctts, all sync). Fit oracle (mp4_fit_report, exported): video HEVC/H264, audio AC-3/E-AC-3; TrueHD/DTS/LPCM and bitmap subs are excluded with a typed reason so the CLI can report exclusions — never a silent drop. Verified vs ffmpeg -c copy on real discs: AVC+AC3 and HEVC(HDR10)+AC3 both frame-exact on every track (video 2650/4270, audio 5567/3454), duration and colour identical, clean decode. On a pathological 4-clip title ffmpeg's OWN output emits the same DTS-monotonicity warnings (more of them) — source-inherent, not a muxer defect. |
||
|
Matthew Jackson
|
8aff7fe708 |
mux: native progressive MP4 muxer (mp4://) — M1 video track
New mux/mp4: writes ftyp+mdat+moov (moov-at-end), streaming samples into a 64-bit mdat and building the sample tables in memory, patched at finish(). Video track (HEVC/AVC): passthrough length-prefixed NALs (already MP4 framing), full stts/stsz/stsc/co64/stss and signed ctts, CFR-derived decode timeline (pipeline carries presentation PTS only), and a colr box for HDR10 colour signalling. hvc1/avc1 sample entry from the hvcC/avcC codec_private. Fail-loud on codecs with no MP4 mapping. Verified against ffmpeg -c copy on real discs: AVC-SDR (300) and HEVC-HDR10 UHD (Dune) both frame-exact (8159 / 8160 frames), colour-exact (bt2020/smpte2084/bt2020nc), and clean-decoding. Audio + fit oracle land in M2. |