- Cap every inline `//` comment block at 3 lines; distil the rationale
rather than delete it. Enforced by a new comment-guard CI check.
- Fix stale and orphaned rustdoc: broken intra-doc links (renamed or
removed items), doc blocks that had merged onto the wrong item (leaving
the intended item undocumented), and doc-contradicts-code drift.
- Add a doc-links CI check (cargo doc, deny broken intra-doc links) so
both classes of comment rot are caught going forward.
The macOS SCSI transport took exclusive SCSI access plus a one-shot
`diskutil unmountDisk`, but never claimed the disc from DiskArbitration.
diskarbitrationd was therefore free to remount the volume mid-rip (a
Spotlight re-probe is the usual trigger), yanking the device out from
under an in-progress read; the failure surfaced to the user as E1000.
Hold a DADiskClaim plus a mount-approval dissenter scoped to our disc for
the transport lifetime, so the OS cannot remount it until shim_close().
Link the DiskArbitration framework in build.rs.
Refine the First-Play SPRM-taint abstain to fix two defects:
- Over-abstention: a tainted compare that guards a SetGPRM store (not a
control transfer) no longer forces abstention. The predicate only
decides whether the store runs; a following unconditional dispatch is
still resolvable. Taint now gates only JumpTT/Goto (the control
transfers the resolver follows); a store under a tainted guard taints
its destination GPRM (unknown post-state) instead of abstaining.
- Under-abstention: an SPRM laundered through a GPRM (SetGPRM g = SPRM,
then a later GPRM-only compare on g) previously evaded taint and let
the resolver commit to the cold-machine arm. Add per-GPRM taint bits:
a store propagates its source's taint (SPRM or tainted GPRM) and a
concrete/immediate store clears it; eval marks a compare tainted when
it reads a tainted GPRM.
Adds red-before-green tests for both plus taint-clear guards.
1. dvdnav/vmcmd: the register operand of the if_v1/if_v3 compare layouts is the
LOW byte of the reg-or-immediate field (b[5] / b[7]), not the high byte
(b[4] / b[6]). Fix both layouts and their comment; add register-mode KATs
with distinct high/low bytes.
2. mpls: connection_condition is the low nibble of PlayItem byte[10]
(byte[9] is fully reserved), corroborated by the byte-11 stc_id and
byte-12 in_time reads. Relocate every fixture's encoded value and update the
regression test to assert the reserved byte does not leak.
3. labels/deluxe: escape the two disc-derived class-name tracing fields via the
Debug path, matching the other log sites.
4. disc/dvd: add an integration test that drives scan_dvd_titles with a resolved
First-Play nav target so the 1-based promotion join is verified end-to-end.
5. bdnav/vm: add unit tests for the SET/SWAP immediate gate — full register
exchange, and that an immediate-flagged operand is never a write target.
6. dvdnav/nav: honor the documented contract — a First-Play compare that reads a
system parameter is undecidable at scan time, so eval now taints on SPRM reads
and the executor abstains rather than committing to the cold-power-on arm.
GPRM reads never taint.
7. labels/deluxe: narrow four now-internal helpers from pub(crate) to private.
Add an additive coverage job to CI that generates lcov via cargo llvm-cov
and uploads to Codecov (tokenless). Mirrors the test job's pinned 1.94
toolchain and dual freemkv-unlock checkout for the path dep. Informational
only: no gate depends on it. Adds codecov.yml and a README badge.
Audit round-2 (fix-adversary): the round-1 deluxe sweep fed the zip local-file-
header magic, but ZipArchive::new validates the end-of-central-directory at the
buffer's end — which the byte generators (front magic + noise) can never
synthesize — so the archive open always failed and decode_binding was never
reached: false coverage for the very bytecode-decoder panic path it guarded.
Drive decode_binding_class directly on a parsed .class with an empty master-enum
table instead, so the symbolic stack machine is actually exercised.
Continuing the /codeaudit HEAD..v1.6.10 round:
- disc/mod.rs rank_titles: bound the O(n^2 * clips) composite scan to a
sub-second work budget (both title count and per-title clip count are
untrusted disc metadata); above it, skip composite classification and degrade
to the has-video + canonical order. Reorder the per-pair checks so the O(len)
is_subset runs only after the cheap video/size gates. (regression test)
- bdnav/vm.rs SWAP: honour the imm_op1/imm_op2 store gates so a SWAP to an
immediate operand does not corrupt a register.
- harness.rs: fuzz the previously-uncovered new parsers — the Deluxe bytecode
decoder, the Fox dcx.xml scanner, and the combined bdnav index+mobj+VM resolve
path (labels::{deluxe,fox} and bdnav::vm made pub(crate) for in-crate fuzzing).
- dvdnav: pub(crate) the module + submodules (was pub) to match bdnav and stop
leaking the DVD IFO/VM internals as public API; drop an unused re-export.
- labels: escape disc-derived strings in diagnostics (Debug not Display) so a
hostile disc cannot inject terminal control sequences into logs.
From the /codeaudit HEAD..v1.6.10 round (each verified against the libbluray
HDMV VM reference, and design-refuted SOUND on two models):
- bdnav/vm.rs title_obj: resolve First Play via the spec sentinel 0xFFFF (was
num_titles+1). A JumpTitle 0xFFFF now reaches First Play, and an out-of-range
JumpTitle(N+1) correctly abstains instead of mis-jumping there.
- bdnav/vm.rs wr: refuse stores to PSR-tagged registers (matches _store_reg),
so an illegal SET/SWAP to a PSR is a no-op and later compares agree with a
real player instead of diverging.
- disc/profile.rs main_title(): return Option<&TitleProfile> instead of indexing
titles[0], which panicked on a disc that scanned to zero titles.
Regression tests added, each shown red-before-green.
The MSRV split raised the dependents above libfreemkv's own floor — the
bins (freemkv, autorip, bdemu) are 1.98 and freemkv-keysources is 1.97 —
so the consumers job could no longer build them at libfreemkv's 1.94
toolchain (`rustc 1.94.0 is not supported by ... freemkv-keysources
requires rustc 1.97`), leaving dev red since the MSRV change. Pin that one
job to 1.98, which satisfies every dependent's floor. libfreemkv's own
lib/lint/test jobs stay at its 1.94 MSRV.
A nav- or authoring-resolved playlist could promote a video-bearing but
negligible-payload branch above the substantial feature it sits beside:
Spotlight (2015) UHD navigates to a Dolby-Vision seamless-branch playlist
(00001, 0.4 GB across 102 tiny clips) instead of the 91.5 GB single-clip
feature (00002). A player plays the branch; a rip wants the feature.
Require a nav/authoring-promoted title to also carry at least 10% of the
largest video-bearing title's bytes; otherwise selection falls through to
the composite/size ranking, which recovers the real feature. Adds the
Spotlight-shaped regression test.
Verified against the 88-disc hoard: title selection is identical to the
prior release on every disc except the one it fixes (Spider-Man 3 UHD,
00245 wrapper decoy -> 00001 feature).
Also clears three 1.97 clippy lints in the HDMV VM and drops two
reference-implementation filename citations from the bdnav module docs.
Implement the DVD navigation feature resolver (#40): read the disc's own
First-Play program out of VMGI (VIDEO_TS.IFO) and execute it on a bounded
register machine to identify which title a player would land on, then prefer
that title in DVD main-feature selection.
- src/dvdnav/nav.rs: new module.
* Parses the VMGI First-Play PGC command table (FP_PGC pointer at VMGI+0x84,
PGC command-table pointer at PGC+0xE4, pre-command list) and TT_SRPT
(title -> VTS/VTS_TTN, pointer at VMGI+0xC4).
* Executes the pre-command list with the existing vmcmd decoder over 16 GPRMs
+ SPRMs from a cold machine: SetGPRM (mov/add/sub/mul/div/mod/and/or/xor),
compare predicates, Goto, Break/Exit, and JumpTT (title dispatch). Menu /
sub-domain jumps and intra-domain links are treated as undecidable.
* resolve_from_vmg(&[u8]) is pure and total; resolve_main_title(reader, udf)
reads the IFO and returns the resolved (title, vtsn, vts_ttn) or None.
* Every offset/count is bounds-checked; a step budget bounds non-convergence.
- src/disc/dvd.rs: scan_dvd_titles now resolves the nav main title and returns
its playlist_id alongside the titles; test call sites updated for the tuple.
- src/disc/mod.rs: the DVD nav pick feeds sort_titles_by_main_feature as
nav_feature, the same seam the BD path uses, guarded to a video-bearing
title. None -> today's leading-cell heuristic stands (strict improvement).
- src/harness.rs: add resolve_from_vmg to the never-panic sweep.
Tests: 11 nav unit tests (unconditional/conditional/set-then-dispatch convergence,
menu/empty/out-of-range/self-loop/truncation/overflow abstention) built to the
IFO spec using the real First-Play command bytes from the existing decoder KATs
(unconditional JumpTT 1; menu entry). Full suite green on 1.94.
Add a thin, format-agnostic typed VIEW over the scanned Disc model so
downstream reads `profile.titles[i].subtitles[j].forced` (and analogous
audio/video flags) for any disc with no per-format conditionals.
New src/disc/profile.rs:
- DiscProfile / TitleProfile / VideoTrack / AudioTrack / SubtitleTrack
(serde Serialize + Deserialize), re-exported from the disc module.
- DiscProfile::from_disc / From<&Disc>, Disc::profile(), and accessors
(DiscProfile::main_title, TitleProfile::video/audio/subtitles).
- Splits each title's Stream enum into three typed vectors; flattens
normalized fields (language default "und", forced/sdh from the subtitle
qualifier, commentary/descriptive from the audio purpose, label -> name).
- Hoists derived flags so downstream never recomputes: main_title (the
pre-sorted main feature at index 0), is_main, and the default
audio/video track (first non-secondary, keep only the first), subtitle
default always false.
Make the profile types the single serialization source of truth: the
json:// sink now serializes TitleProfile instead of hand-building JSON,
so json:// and any downstream share one schema. Stream arrays are split
by kind and qualifier/purpose decomposed into booleans; json://-only
extras (clips, full chapter_marks list) are spliced back on. meta_sink
tests reconciled to the new schema.
Tests: profile normalization across BdTs + MpegPs titles, typed-vector
split, populated fields, hoisted main/is_main/default flags,
qualifier/purpose mapping, and a serde round-trip.
The Deluxe parser detected com/bydeluxe discs but emitted nothing, so
mpls_universal won at Low. Grounding the decoder on real Universal discs
(Fast Five / F9 / Bourne, studio="uni") surfaced three defects that
kept the full Phase A-D pipeline from ever resolving a stream on any
real disc, all fixed here:
1. Master enums were keyed by the ZIP ENTRY name ("pd.class") but the
binding class references enum constants by JVM INTERNAL name
(getstatic pd.<field>), so Phase C's getstatic count never matched
the identified enums and found zero binding classes. Key master enums
by this_class_name instead.
2. MasterEnumTable resolved getstatic operands by enum VALUE string
("English") but the operand is the obfuscated static-FIELD name
("a"), so every language/purpose ordinal came back unresolved.
Capture the ordered putstatic field names (clinit_enum_field_names)
and key the resolver on them.
3. The Language-enum fingerprint demanded 70+-4 values; Universal's
enum has 65, so Phase A rejected it outright. Give each fingerprint
its own count window (Language widened to span 65 and 70); the
ordered 4+-string prefix stays the real discriminator.
Also: model anewarray/newarray so array construction keeps the symbolic
stack aligned, and skip constructors with an array parameter so the
Universal title-wrapper object (oq.<init>(...,[Lnp;,[Lwb;,...)) is not
mistaken for a subtitle binding. Recover SDH/RNIB from the language
display name (Universal encodes it as a distinct Language value, not a
Purpose). Add config.xml <TitleConfig studio="..."> detection. Raise
confidence to High when the binding table decodes >=1 stream; the
mpls_universal floor still back-fills un-named STN slots.
Fixtures: verbatim pd/lp/tl .class files from FastFive.iso decode to
English/French/Spanish + English Commentary (Dolby Digital) audio and
per-title eng/spa/fra subtitles. Validated end to end: labels-analyze on
FastFive/F9/Bourne now reports parser=deluxe High with real per-stream
languages, names, purposes and codecs instead of mpls_universal Low.
Treat Java bytecode as a documented format we READ (never execute), a
peer of mpls/clpi. The class reader's bounds checking was already sound
(typed Reader with checked u8/u16/u32/u64 + slice, saturating switch
arithmetic, iterator halt-on-truncation); this adds the untrusted-input
entry points the never-panic harness was missing and the malformed-input
regression tests the format warranted.
- harness: add class_reader (magic CAFEBABE) and jar (zip PK header)
sweep targets. The class_reader target also drives the accessors a
label parser exercises (this_class_name, member Code walk, cp_index,
member_ref, load_constant_display) so their indexing is fuzzed, not
just the structural parse. The jar target opens the sweep bytes as a
zip and walks every .class through for_each_class exactly as deluxe/dbp
do.
- class_reader tests: oversized constant_pool_count and interfaces_count
reject with UnexpectedEof (u16 counts keep with_capacity bounded);
out-of-range cp indices return None from every accessor; a Code
attribute whose declared code_length runs past its body is rejected.
scan_with runs the bdnav VM for BDMV discs and feeds the video-gated result
as the new top MAIN_FEATURE_ORDER key (nav-feature > authoring-feature >
standalone > has-video). A nav result is only honoured for a video-bearing
title, so a mis-resolve can never reopen the decoy hole; when the VM abstains
(BD-J, non-convergence) the structural/heuristic order decides exactly as
before. Adds nav selection tests and updates the diag decision reason.
Add src/bdnav: a libbluray-faithful, bounded, never-panic HDMV navigation VM
that runs First-Play with correct power-on PSR/GPR state to resolve the
playlist the disc itself plays as the feature. One generic VM handles the
obfuscated Sony dispatcher and every other HDMV disc with no vendor
special-casing; it abstains (None) on the BD-J boundary, malformed data, or
non-convergence. index/mobj parsers added to the never-panic harness.
Older Fox discs ship a loose plain-XML manifest at
/BDMV/JAR/<id>/dcx.xml (root <dcx>) that names every feature stream's
language, editorial purpose and forced/SDH state in attributes — fully
parseable and previously unowned (2 Fox discs matched no parser, 11 were
mis-owned by the deluxe stub).
New src/labels/fox.rs:
- detect(): loose dcx.xml under /BDMV/JAR/*/ (cheap dir walk), falling
back to a com/foxbd/ jar central-directory prefix so newer Fox discs
are attributed to Fox even though their bytecode form isn't decoded
yet (Phase 2, sketched in-file).
- parse(): scopes extraction to one <playlist name="feature"> element
(the richest, to avoid merging regional 00800/00801 STN tables),
reusing labels/xml.rs. audio type=rnib -> Descriptive, comment* ->
Commentary; subtitle form=sdh -> Sdh, type=embed -> Forced; nested
id="NN" -> 1-based per-type STN slot. High confidence.
- Feature playlist id recorded via tracing::debug for now; dev's
ParseResult has no feature_playlist field (that FeaturePlaylistHint
slot lives on selection-hardening). An in-code note documents wiring
it at reconciliation, matching paramount::parse.
Registered ahead of mpls_universal, alongside the High-confidence vendor
parsers; registry-order lock test updated.
Tests embed the real Life of Pi dcx.xml (/BDMV/JAR/05001), asserting the
00800 feature selection + id, all 11 audio labels (incl. the eng rnib
descriptive track) and 11 subtitle labels (sdh/forced/none), the
no-cross-playlist-merge scope rule, plus negative-detect cases.
The has-video gate alone fixed Spider-Man 3 UHD only by accident: the
00245 decoy = [bumper][00001 feature clip][outro], and streams come from
PlayItem 0 (the bumper) only, so it reported v=0. That is fragile — a
video-bearing composite [feature][outro] still beat the feature on size
(audit R2), and a real feature with a non-video first PlayItem would be
wrongly disqualified (R6).
Add structural clip-set composite detection: a title whose clip set
properly contains another substantial video title's whole clip set is a
play-all/wrapper composite and is demoted below the standalone title it
wraps — independent of stream counts, physical size, and disc capacity
(which is 0/unknown when READ CAPACITY fails, disabling the oversize
gate). Replace the has-video disqualification with the more-permissive
has_probable_video (real video, or size+duration+clips consistent with
video) so a bumper-lead feature is never disqualified while streamless
menus still are. Corroborate the authoring hint by duration (a hint at a
short branch/trailer can't override the feature) and require real video
streams for it. Precedence: authoring -> standalone-over-composite ->
has-video -> canonical (with a reserved top slot for the future BD-nav
key). Unify the session.rs key-sampling/verdict pickers and
image_crack_extents on has_probable_video.
Tests: video-bearing composite (T2), oversize composite with unknown
capacity (T3), short-branch hint not honoured (T4), bumper-lead feature
not disqualified (T6), all-streamless deterministic + single-source pick
(T7), equal-clip twins not composite, video-only title. T2/T3/T6 fail
under the prior has-video-only logic.
Main-feature selection ranked purely on physical size -> duration ->
audio (canonical_title_order), with no requirement that the chosen title
carry a decodable video stream. A playlist-obfuscation decoy can inflate
a streamless playlist past the real feature on size AND duration (Spider-
Man 3 UHD: 00245.mpls, 78.4GB/8350s/v=0 beat 00001.mpls, 76.5GB/8038s/
v=1), so the mux ran a title with zero streams -> zero frames -> E6008.
- Add Disc::main_feature_order layering two gates atop canonical_title_order:
authoring-feature (a title the disc's menu authoring designates AND that
has video) wins outright; then has-video demotes any streamless title
below every title that has video. scan_with re-sorts with it after labels
run so titles[0] is the selected feature. New MAIN_FEATURE_ORDER_KEYS +
DiscTitle::has_video().
- Surface the authoring feature-playlist id: labels::FeaturePlaylistHint,
captured from Paramount playlists.xml <playlist name="Feature" id=..>,
threaded through ParseResult/extract; labels::apply now returns it.
- Unify pickers so key-sampling and the structural verdict (session.rs) and
image_crack_extents prefer the has-video feature, not a streamless decoy.
- diag: decision row logs video= count + main_feature_order reason and a
tag=decision.demoted line per no-video title (closes the field-diagnosis
gap this incident exposed).
Tests: main_feature_order_demotes_streamless_decoy (pre-gate vs post-gate
contrast), _honours_authoring_hint, authoring_hint_never_promotes_a_
streamless_title, feature_playlist_hint_matches_on_id_or_filename; diag
reason test updated. QA green on 1.94 (fmt/clippy -D warnings/test).
Lower the crate MSRV from 1.97 to 1.94 -- the lowest toolchain on which
`cargo build`, `cargo clippy --all-targets -D warnings`, and
`cargo test --tests` all pass clean. Dependencies floor the build at 1.90
(freemkv-unlock 1.90, zip 1.88), but clippy is only warning-clean from
1.94 (older clippy has a `duplicated_attributes` false positive on
`#![cfg(test)]` and flags a now-documented `let_and_return`). Update
`rust-version` and every `dtolnay/rust-toolchain` CI pin to 1.94.0, and
add a CHANGELOG Unreleased entry.
Extend the byte-provenance coverage added in f7d38ca (currently one
TS-path test) with two PS-path tests driving the real `Mpeg2Parser`:
- `ps_stream_stamps_source_provenance`: the PS path forwards `ps.source`
onto `PesPacket`, so demuxed frames carry `Some(SourcePos)`.
- `ps_stream_carries_feed_base_across_reads`: the cumulative `fed_bytes`
base carries ACROSS read buffers -- a frame decoded from the second
sector is stamped `source.byte >= 2048`, not reset per read.
Also record the f7d38ca provenance fix in the CHANGELOG, inline a
`let_and_return` test binding, and trim a redundant sentence from
`PsDemuxer::feed_at`'s doc comment.
MLP/TrueHD carries predictor + restart state across access units, so after a source transport-stream discontinuity (continuity-counter gap) the parser was splicing post-gap audio mid-stream, decoding against stale state and producing a decoder-choking seam (a burst of "restart header sync incorrect" / "Invalid blocksize" errors that could flag the whole track corrupt).
Honor the source's own gap signal: drop forward to the next CRC-valid major-sync AU (a clean re-init point) before resuming, guarded so it never arms before a validated baseline exists (no stream-head whole-track drop). Mirrors the video keyframe resync and the ResyncGate design. Corrects the "audio has no inter-frame refs" comments in truehd/dts/resync. Adds a regression test.
Adds a 'changelog' job that fails when Cargo.toml's version has no matching
'## [X.Y.Z]' heading in CHANGELOG.md, hard on main and PRs into main, warning
on dev/qa — so the unified release can't ship a crate with a blank changelog.
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.
Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.
Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:
- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
authored order (a scratched sector under the .vti — the name came from the
directory, so it is never "absent") into "no order" with no diagnostic. The
sibling clip-extent arms log every read failure with its code; this one now
does too, then falls back to the per-clip heuristic exactly as an unauthored
disc would. Behaviour is otherwise unchanged; the loud line is the point.
- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
every zero-filled hole as good, so a progress-only consumer saw a holed
extraction climb to a clean 100%. Thread the running unreadable total through
and report the real good/unreadable split. The authoritative ExtractResult was
already truthful; only the live channel lied.
- labels: correct a stale comment that described testlog's old lock-based
capture; it now installs one global subscriber and routes to a thread-local
sink.
- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
its sibling stream-id constants and the file's single-source rule.
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).
labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.
dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
Commit b02c402 ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.
Revert all of that plumbing (public signatures restored to their pre-b02c402
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.
Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.
Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.
The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.
Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.
`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.
On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.
On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.
Also in this change:
* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
code was recorded and thrown away: the title shipped base-view 2D off a 3D
disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
only benign failure". Logged, not refused — the base view is a real rip.
* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
nothing for ~30 s while every other drive path returns Halted at the next
command boundary. `spin_cycle` issued both START STOP UNIT commands outside
`checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
blind — ~15 s deaf to Stop, from the recovery path, exactly when the
operator is most likely to press it. Both now use `checked_exec` and
`sleep_until_halted`, which already lived in this file with four tests and
was `#[cfg(test)]`, called from nowhere. It is production code again.
* drive: a READ(10) that returns GOOD status with a residual underrun was
correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
lba/count/status. A residual-underrunning drive was indistinguishable from a
scratched disc — two populations with opposite remedies. It now warns with
transferred vs expected, which is the whole signal.
* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
naming 109 of the 127 declared codes while its doc claimed to pin them all,
and an earlier audit trusted that claim while assigning new ones. The list
is now derived from the declarations by parsing `include_str!("error.rs")`,
so a new constant is covered the moment it is written. A parser self-test
cross-checks the count and three known name/value pairs, so it cannot pass
vacuously.
* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
the logging contract is enforced rather than commented. Three sites carry
long comments insisting they log the error's OWN code; putting a literal
back broke nothing. They are pinned now, along with the two new log lines.
Captures are serialised process-wide: `tracing`'s interest cache is global
while `with_default` is thread-local, and the rebuild on the exiting
capture can land after the entering one's, leaving the cache at "never"
while a capture is live. That produced a real empty-event flake.
* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
every BD/DVD cancellation test calls the scanners directly, so passing
`None` on either branch left the suite green while Stop did nothing.
* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
instead of `Shutdown::Both`, which raced an RST against the server's read
and returned ConnectionReset instead of InvalidInput under load. The port
was already ephemeral; that was never the cause.
Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.
Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.
The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.
The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.
Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.
* `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
`parse_playlist` an `Option`, so neither had a channel to report a
halt. During a cancel each remaining `.mpls` read merely failed and
was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
title list at rc=0. Both now return `Result`; `Ok(None)` from
`parse_playlist` keeps its two benign meanings (unparseable MPLS,
playlist under 30 s) and `Err` means the scan is over.
* `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
cancel into ZERO titles at rc=0 — a DVD reported as carrying no
video at all. `ifo::parse_vmg` compounded it, treating a halted
title set as one of the placeholder TT_SRPT entries it is designed
to tolerate and continuing to the next; both now propagate.
* HD-DVD was already correct and is the model this follows, including
its reasoning: a cancelled enumeration that returned `Ok` would be
cached, displayed and ripped from as if it were the disc.
The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.
Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.
Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.
Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.
A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.
`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.
Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.
AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.
`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.
Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.
Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.
Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.
freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.
UDF and extents
Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
is no longer decoded from FID bytes into a silently empty listing. Carry the
ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
written extent used to reach the read plan as ordinary content and splice
undefined sectors into the rip. file_extents now refuses such a file, and
only when the hole actually occupies byte space — a zero-length one displaces
nothing, and refusing on it dropped whole titles off discs that ripped
correctly. Type-2 sparse extents are kept alongside type-1; they were falling
into a catch-all that exited the descriptor loop and returned a truncated
list as complete. merge_ranges no longer claims a sector neither input
covered. A short skip or an over-long AD chain errors instead of truncating.
HD-DVD and Blu-ray scanning
Bound the XPL nesting depth, title count, clips and chapters per title, and
memoize the clip-name fallback probe — four separate amplification axes, each
of which alone left the worst case unbounded. The clip and title caps are 512,
~10x any retail disc, and a test pins the product of cap and probe budget.
The scan is cancellable: it returned Ok with titles carrying no streams when
halted, presenting a cancelled scan as a successful one. A clip dropped for an
unrecorded extent now says so.
Codecs and muxing
Resume a held E-AC-3 access unit rather than rescanning from its first frame,
and drop it on a discontinuity — a stale hold indexed past the end of the new
buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
und. Correct the DVD palette order. Detect a skip past EOF.
Drive and I/O
Classify dead-bus faults so the wedged-drive path can see them; a catch-all
arm had been flattening the variants before the classifier ran. A prefetch
producer that dies now reports SourceTerminated instead of Ok(0), which the
reader legitimately read as a short read and zero-filled — a whole title
could be fabricated and the pass reported complete.
Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
`resolve_vts_key` called the `Option`-returning `css::crack_key`, which
collapses "no scrambled sector was seen" with "scrambled sectors were
seen and no key came out", and then fell back to the disc-wide key. A
multi-VTS CSS DVD whose second title set resists the Stevenson scan was
descrambled under the FIRST set's key: corrupt PES behind an intact
header, written out with `complete = true` at exit 0.
`CrackOutcome` exists precisely to keep those two apart, and its doc says
callers must surface the second as a hard error. Every sibling path in
the crate already does — `Disc::decrypt_keys_for_title` and the mux path
both map `ScrambledUncracked` to `CssKeyMissing`. This was the one path
that did not. The ordering comment 15 lines above describes this exact
outcome as the bug it exists to prevent; ordering makes the crack far
more likely to succeed, but it cannot make a failed crack safe.
Also, three things nothing could catch:
- The demux output filename took the stream's language raw while the
base beside it was sanitised. A language code is three raw STN bytes
through `from_utf8_lossy`, and `00 00 00` is the ordinary "undefined"
encoding on real discs — a NUL in a path fails `File::create` with
InvalidInput, taking the whole export down before one track file
opened. `sanitize` now maps control characters too; it did not.
- `css::crack_key_scan`'s short-read handling was dead code under test:
every source in the module returned the full request, so reverting
`advance` to the requested count, or dropping the `.max(1)`, left the
suite green. The `.max(1)` is load-bearing — without it a source
returning `Ok(0)` never moves the cursor and never increments the
budget, so the scan spins forever. That mutation now HANGS the test
rather than failing it, which is the honest demonstration.
- `MAX_SUBDIRS`'s const-assert carried `#[cfg(not(test))]` inside a
`#[cfg(test)] mod tests`, so it compiled in no configuration and could
never fire — the dead gate the test above it was written to replace.
Moved to module scope, and verified it now rejects a wrong constant at
compile time.
`forced_sub` was the last unbounded attacker-controlled list in
paramount.rs. `MAX_COM_INDICES` capped the two `*_com1_idx` attributes;
this one had nothing capping it at all, and unlike them it has no value
to filter — a cell is a classification of the position it sits at, so
its bound has to be positional. Extracted as `forced_subs` for the same
reason `com_indices` was extracted: through `labels_from_feature` the
bound is unobservable, because the subtitle loop cannot reach those
cells either, so the assertion could not fail.
The `MAX_COM_INDICES` doc claimed the entry-allocation argument for the
whole constant. It is the VALUE filter that caps the set (values below
the bound, so at most that many distinct entries, however long the
attribute); the `take` caps the WORK. Both are real and they are not
the same bound; the doc now says which is which.
`jar_inventory_dedup_is_not_quadratic` called itself proof by deadline.
It is a hang guard — a return to the linear scan runs for minutes and
would wedge CI rather than fail it — and no assertion in it can tell a
BTreeSet from any other sub-quadratic dedup. Renamed and documented for
what it does. Its margin was measured before keeping it: 0.14s debug /
0.07s release against 10s, ~70x, unlike the 6x that made paramount.rs's
wall-clock test flake.
Both are mine, and both pass with the bound they "cover" deleted.
`a_hostile_commentary_index_list_is_bounded_not_merely_fast` asserted
only label purposes. The out-of-range filler it feeds is unobservable at
the label level and a HashSet collapses the repeats, so removing
MAX_COM_INDICES entirely leaves it green. What it DOES catch is a bound
set too low — verified at 2, where the real indices stop resolving. Named
and documented for that, and it no longer implies it guards enforcement.
`an_index_that_cannot_address_any_cell_is_not_retained` asserted through
the labels, where retention is by definition unobservable: the loop never
queries a cell that high. It now reads the set through `com_indices`,
where the claim is checkable.
Enforcement was and remains proven by
`distinct_unaddressable_indices_are_refused_not_stored`, which was red at
50,000 retained entries and green at zero. Two tests, two properties;
neither pretends to the other's job now.
The gate caught `commentary_index_lookup_is_not_quadratic` failing, then
passing on a re-run. Measured both ways: 1.62s alone, OVER 10s against
its own 10s deadline while the suite's other 3,347 tests ran
concurrently. A 6x margin against a shared CPU is not a margin, and a
test that gets re-run until it passes is not a test.
It was also measuring the wrong thing. Replacing the linear scan with a
HashSet bounded the LOOKUP; the set was still built from every entry the
disc declared. `playlists.xml` is attacker-controlled and has no length
of its own, so a hostile disc could still force an unbounded allocation
before any lookup happened — the parse, not the query, was the exposure.
The code's own comment said "unbounded parsed input" and only fixed half
of it.
`MAX_COM_INDICES` bounds both halves, at the one value that cannot
change behaviour: an index at or beyond `u16::MAX` can never match a
cell, because the labelling loops break at `u16::try_from(i + 1)`. Real
authoring is nowhere near it — the BD STN table admits 32 streams.
The parse moved into a `com_indices` helper so the bound is OBSERVABLE.
Through `labels_from_feature` it is not: a HashSet collapses repeated
values, and an out-of-range index changes no label, so the obvious test
passes with or without the cap — an assertion that cannot fail, which is
what the first draft of this fix shipped. The test now hands in 50,000
DISTINCT unaddressable indices and asserts they are refused. Proven red
with the bound removed (50,000 kept), green with it.
Applies to both `sub_com1_idx` and `aud_com1_idx`.
NOT done here: `labels/mod.rs:1421`'s `jar_inventory_dedup_is_not_quadratic`
is the same wall-clock shape and has the same flakiness. Named so it is
not lost.
Three round-1 audit findings on v1.6.0..HEAD.
Tests: `mux::resolve::input("dir://…")` — the door the CLI rips a folder
through — was never driven with a real folder. The only dir:// input test
uses a missing path, which fails in DirImage::open long before
`image_input`, so dropping the `is_folder` argument (and with it
`session::apply_folder_encryption_verdict`) passed the whole suite. Two
tests now close it, stated as a DIFFERENTIAL between the two doors,
because that is the invariant the shared function exists to hold: a
clear folder that kept its AACS/ directory must scan the same and select
the same extents through `scan_dir` and through `dir://`, and a
scrambled one must be refused through both with E9063. Proven red
against a mutated guard (both fail with E7022, the tree-shape verdict
the probe overrides) and green with it restored.
Docs: css/mod.rs documented `is_scrambled`, a function this line renamed
to `has_scramble_flag_bits`. Eleven stale sites, two of them broken
rustdoc links. Two were not stale names but false statements — they said
the descramble loop keeps the looser raw-flag test, when
`descramble_sector` and `descramble_region` both gate on
`is_scrambled_pack`; that parenthetical is rewritten to say what the
code does and why (the measured VIDEO_TS.IFO case: 38 titles became 10,
silently, at exit 0). The one mention that must stay is the sentence
explaining why the name was rejected.
Constants: io::image_writer and dirimage::encode each re-declared the
2048-byte sector that consts::SECTOR_BYTES already exports; both now
alias it. Note `BATCH_SECTORS = 2048` in image_writer is a different
quantity (sectors per batch, not bytes) and is deliberately left alone.
Bare 2048 literals elsewhere in the crate are out of scope here.
These are eight public repos that ship one product on one version, and
they had drifted: freemkv-engine carried no LICENSE at all (GitHub
reported its licence as none), two crates had no code of conduct, four
had no contributing guide, and none had a security policy — so there was
no private route to report a vulnerability in a disc-decryption tool.
SECURITY.md names GitHub Security Advisories on each repo rather than an
address, so there is nothing to keep in sync and no inbox to go stale.
dev is where work lands and is meant to be pushed to often, so what runs
there should be the cheap answer to "did I break it": lint, tests and the
Linux build. The macOS and Windows jobs now run on qa and main instead of
on every push to dev.
Nothing is deleted and no platform goes unchecked before a release. qa.yml
already covers macOS and Windows independently, and the jobs that live only
here -- the Intel macOS build, the Windows release build -- still run, on
the branches where a cross-platform break is worth blocking on.
They are SKIPPED on dev via `if`, not left unscheduled. A queued job would
be worse than a slow one: release.sh's CI gate refuses while any run for
the commit is still in progress, so a job that never gets a runner blocks
releases silently, with no error anywhere. That is the failure the
real-media note in qa.yml describes, and it is why this is an `if` on the
job rather than a narrower set of trigger branches.
Version bump and changelog for the 1.6.3 sync. All eight crates ship the
same version, so the crates with no functional change this cycle say so
rather than carrying an empty section.
Two problems, both invisible until the whole graph is looked at together.
DEAD: num-bigint, sha2, num-traits, num-integer, cmac and cbc are
declared here and referenced nowhere -- not in src, tests or benches.
They were being compiled, audited and offered version bumps forever for
no reason. Removing beats bumping.
cbc nearly survived the sweep: a substring search for "cbc" matches 44
occurrences of ycbcr_to_rgb in the DVD subtitle decoder, so it looked
used. Only a word-boundary search exposed it.
SKEW: this crate was the outlier on every shared dependency -- aes 0.8,
rand 0.8, base64 0.22.1 and zip 2 against 0.9 / 0.10 / 0.23 / 8
elsewhere. Cargo cannot unify across a major version, so it compiled
BOTH: 32 duplicated crates in the freemkv binary's graph, including two
complete AES implementations (aes 0.8 + 0.9, cipher 0.4 + 0.5), two
digest stacks and two getrandom. Two crypto stacks in one product is
worth removing on its own.
The aes bump is an API rename -- BlockCipher-prefixed traits, Array for
GenericArray -- and the obvious translation uses Array::from_slice,
which the new version deprecates and clippy's -D warnings would reject.
These use the From<[T; N]> conversion the crate points at instead.
3441 tests pass in debug and release. The AACS crypto here is covered by
known-answer tests, so a byte-order or sizing mistake in that rename
could not have passed.
Version updates were never configured here -- only security alerts, which
report but never open a pull request. So the absence of Dependabot PRs was
not "nothing to update", it was "nothing was checking".
Updates target `dev`, never `main`. main is a release pointer that
release.sh moves to each tag, so a bot commit on it would put work there
that no tag contains -- the exact state that aborted the 1.6.2 cascade at
its last step, and pointing a scheduled bot at main would recreate it
weekly.
Minor and patch bumps are grouped into one PR rather than one per crate:
eight repos times a handful of dependencies is a volume nobody reads, and
an unread PR queue is indistinguishable from no updates at all.
The freemkv crates are excluded. They depend on each other by git tag,
re-pinned by release.sh inside the release commit, and Dependabot cannot
see that cascade -- a PR bumping one could pin a version whose tag does
not exist yet.
It fired on `release: published` and committed to main, the branch the
release points at its tag, so it could only ever leave main ahead of the
tag. release.sh now makes the same edit inside the release commit.
In this repo it was a no-op regardless: the pattern it rewrote does not
appear in this README, so it has been running and changing nothing.
A clip's span is measured over the title's full extents when the disc is
scanned, and a frame is placed by the offset it was read from. Those only
agree while the mux reads every byte the scan counted.
On a disc carrying a forensic segment it does not: the read plan omits the
units belonging to another device group, so fewer bytes are fed than the
spans describe, and the shortfall grows through the title. Every frame past
the first segment then looks earlier than it is — placed in a clip it did
not come from, or dropped at a join for failing marks it was never inside.
The spans still tile one another perfectly, so the check that asks whether
they can be trusted cannot see any of it.
The plan is now compared against the extents it was built from. When they
differ the spans are dropped and placement falls back to timestamps, which
is what that path is for and what the surrounding comment already promised
would happen when an offset stops meaning anything. Ordinary discs are
untouched: with no forensic segment the plan IS the extents.
The provenance guard knew one way to lose a frame's source offset —
writing it as absent. Omitting the field entirely does the same thing,
because the frame type fills it in by default, and the guard read straight
past that. A parser rewritten into the second spelling would have kept its
green light while its track silently went back to being placed by guesswork.
It now reports which spelling it found and where.
The title-count clamp was asserted over a fixture too small to hold more
titles than the cap allows, so the walk stopped when the buffer ran out and
the clamp was never what bounded it — the assertion held with the clamp
deleted. The fixture now carries more entries than the cap.
The subdirectory limit was checked by restating the constant's own
definition; the guard itself had never run, and deleting it changed
nothing. The limit is lowered under test so a real folder can exceed it,
and the test now walks one and requires the refusal.
One test also carried two unrelated grounding notes while the test they
described had none, so reading the note above a test told you about a
different one.
A playlist may point at the same .m2ts from several play items — a
seamless split, a looped segment, multiple angles. The file has one set of
bytes, so it has one span, and the byte a frame was read from therefore
identifies the FILE, not which play item's range it falls in. Placement
took the first of them and judged every frame against its marks, so
everything past that range's end was treated as material the playlist
excludes and dropped: half the clip missing from the rip, with the
timeline still charged for its full duration.
The offset narrows a frame to the run; only its timestamp can finish the
job, and each play item carries its own marks. A frame is now matched
against the marks of the entry it actually falls in, falling back to the
first when it falls in none, which leaves genuinely-excluded material
dropped as before.
The test for this construction asserted only that such a playlist is
trusted, and never placed a frame from the second range — so the loss it
described in prose was invisible to it. It now places one from every range
and requires them all to survive.
The deferred-mux replay of buffered frames wrote them without their source
offset, quietly sending the head of every such title down the timestamp
heuristic the rest of this work exists to retire. It passes the offset
through now, and the frame writer that omits it is compiled out of the
library entirely: no production path can discard provenance any more.
Also: the AC-3 buffer-reuse test could not see the regression it named.
Feeding equal-sized packets, a fresh allocation per call yields the same
capacity as a reused one. It now feeds a large packet then a small one,
where only a reused buffer keeps the larger capacity.
An ADTS header that declares a CRC follows must be at least nine bytes —
seven of header plus the two the CRC occupies — because the declared frame
length counts them. The structural gate compared against a flat seven and
never read the bit that says whether a CRC is there at all, so a frame
whose own header describes something impossible was accepted and handed to
the muxer as decodable.
The pipeline's spawn doc named Sweep, and the thread Sweep would have
created, as callers to look for. Neither has been in this crate since the
recovery passes moved out. The same paragraph already records fixing this
once, for a different departed caller — it simply drifted again a sentence
later, so it now says to name callers that live here or name none.
Whether a disc is structurally AACS-encrypted was spelled out by hand in
both the fast identify and the full scan. They agreed today; nothing made
them agree tomorrow, and disagreeing would mean the same disc reported
encrypted by one path and clear by the other. There is one definition now,
and the comment that pointed at it by line number points at its name.
The AC-3 parser built a fresh buffer on every packet — of the order of a
hundred thousand times per title — to work around a borrow it cannot
avoid. The copy stays; the allocation does not. The buffer is now lent out
and handed back, and a test pins that, because reverting it would be
invisible in behaviour.
Left alone deliberately: send/send_with_halt and finish/finish_with_halt
look like one action under two names, and are not. After the consumer
fails, one must still accept items and the other must refuse them; that
difference is what stops a producer reading an entire disc for a write
that died on its first frame. Collapsing them was tried here and the
existing test caught it. Both now say so where the choice is made.
A recovery resumed against an image that has been truncated since the
previous pass — a full disk, an interrupted transfer, a remount — cannot
repair it: the pass only revisits the ranges the recovery data calls bad,
so everything past the cut stays a hole while the counts still describe a
whole disc. There was no way to say that. Reusing the recovery-data error
would have been wrong, because the recovery data is intact; the image is
not.
Registered in the uniqueness list, the code table and the range check, so
a future variant cannot silently reuse 6015 or map to the wrong code.
The `consumers` job checked out all five dependents against this commit and
ran `cargo check --all-targets`. That proves they still COMPILE, which
catches a changed signature and nothing else. The failures worth catching
here keep every signature intact and change behaviour: the library still
builds, the dependent still builds, and the dependent's tests are what go
red. Those never ran.
Run their suites instead. Same checkouts, same patched paths — only the
verb changes.
The demuxer interleaves, so at a boundary the streams do not reset on the
same frame: audio for the next segment can reach the muxer before the video
frame that opens the epoch it belongs to. Only the primary video track may
open one, so those frames rode the just-ended offset and landed a whole
segment in the past. Downstream the strictly-monotonic block nudge then
crushed the entire run onto one instant a tick apart, which is audible.
Such a frame is recognised on its OWN raw PTS, which within an epoch only
advances, so a large backward step is unambiguous — a different signal from
the shared frontier, which is what the old false-positive ratchet keyed on.
It then rides a provisional offset private to its track, computed the same
way the video path computes a real one, and drops it the moment the video
retires an epoch, so the run rejoins with no seam.
The offset is deliberately private: it never writes offset_ns, never
advances the frontier and never retires an epoch, so it cannot move the
video timeline. Letting a passive track open a REAL epoch was tried first
and inflated a 476.776 s title to 656.216 s, because every track meets a
boundary at its own pace and the video path rebased again on top.
Measured on a real DVD title with 8 cell boundaries: audio frames stamped
inside one cadence 120 -> 0, outsized gaps 7 -> 0, subtitle 1 -> 0, and the
video span byte-identical at 476.484. On the HD-DVD title the remaining
boundary gap on its first audio track falls from 0.999 s to 0.275 s.
Acceptance: 76 pass, 0 fail, 0 skip.
A non-video frame can reach the muxer after the video that opened the next
epoch has already been processed. It carries an old-epoch PTS, so adding
the new offset flings it forward by a whole clip. The remap that exists to
catch this compared the frame against the CURRENT frontier and only
accepted it within one backstep, which is the wrong yardstick: by the time
a straggler is seen the frontier has moved on into a new epoch, and how far
below it the frame lands says nothing about where it belongs.
Retiring an epoch now records the frontier it closed at alongside its
offset, and a straggler is judged against the end of the epoch its PTS came
from. Keeping the whole history rather than one previous offset also fixes
the case a single `prev_offset_ns` cannot express at all: a frame from two
or more epochs back.
Measured on a real HD-DVD title whose second audio track put its final
packet at 12834.587 s in a 6434.100 s file. That frame sits 23 s below the
current frontier — outside the old window, so it was refused — but 0.15 s
below the end of the clip it actually came from. It now lands at 6417.216,
one 32 ms cadence step after its neighbour, and the track spans 6416.160
with no outsized gaps. The other two streams are unchanged.
A title that never rebases retires no epoch, so the lookup finds nothing
and the mapping is bit-identical to before.
Four parsers (dvdsub, flac, lpcm, mpegaudio) stamp a source offset on
every frame but had no test that read one back, so a regression to
`source: None` would have been caught only by the brace-balanced audit
in codec/mod.rs — a lint, not a behavioural check. Each now asserts the
emitted frame carries the offset of the packet that supplied its first
byte.
The Blu-ray feed spans had no direct test at all. Add one that walks a
multi-item playlist and requires the spans to tile the feed with no gap
or overlap; it catches a one-sector-per-clip drift, which is exactly the
error class that would misattribute frames near a seam.
`no_provenance_still_places_by_marks` asserted only that placement
returned something, which passes for a frame placed in the wrong clip.
Its probe timestamp lands in an overlap between two clips in the real
mark table, so pinning one clip would assert a coin-flip; instead
require the offset to be one that a clip actually containing that
timestamp would produce.
The TT_SRPT pointer at 0xC4 is a SECTOR offset from the start of
VIDEO_TS.IFO. Reading it as a byte offset lands 1/2048th of the way in and
does not error — it silently returns a different title list, which is how
a disc enumerating 38 titles and an image enumerating 10 looked like a
scan difference rather than a failure.
The table already had tests for its entry layout, dedup and the 99-title
clamp. What it did not have was anything pinning the offset unit itself,
or the grouping of titles by title set, or a truncated table keeping the
rows it can read rather than failing a whole disc.
release.sh dates the heading when it cuts the tag, and refuses a bare
'## [1.6.1]' because it cannot tell an undated heading from one it has
already stamped. Shipping a tag whose public changelog says UNRELEASED is
permanent and cannot be corrected inside that tag, so the check is right
to stop rather than guess.
Every emitted frame must carry the source byte offset of the packet it
came from. That held only for video for as long as it existed, and
nothing asserted it, so nothing caught it: ten parsers built frames with
`source: None` and a multi-clip title could not place audio or subtitles
by byte at all.
Finding them took a brace-balanced scan of the tree by hand — a regex
cannot do it, because a Frame literal contains nested braces and a
non-greedy match stops at the first `}`, which is how five sites survived
the first pass. This is that scan, as a test.
A second test fails if a codec module is added and not listed, since an
unchecked parser is exactly how the gap persists. Modules that emit no
Frame are named explicitly rather than skipped silently.
The guard tripped on itself twice while being written — first on its own
doc comment, then on its own string literals — so it strips line comments
and excludes its own module. A check that matches prose about the defect
rather than the defect is the same mistake in a different place.
The entries had grown into narrative paragraphs; each is now the outcome
first and the mechanism in a sentence or two. The website changelog page
mirrors libfreemkv's, so it inherits this directly.
Two things that shipped in this cycle were not in it at all:
dir:// as a source. It was pulled from 1.6.1 on 2026-08-05 because the
folder reader produced a wrong title list on a real DVD. That defect was
the CSS descramble bug below, fixed since — the acceptance suite now
shows a folder reporting the same 38 titles as its ISO, with matching
streams, languages and runtime, and the CLI has been wired for it all
along. It ships, so it is listed.
The CSS title-table corruption itself: a disc enumerating 38 titles
produced a decrypted image enumerating 10, silently, at exit 0. That is
a data-integrity fix a user needs to know about.
A real rip caught these. The warning added for a track placed from
timestamps under a seam plan fired once, on one subtitle track of a
23-clip title, eighteen minutes in — a display set that arrives as a lone
non-PCS segment is emitted straight through rather than accumulated, and
that path still built its frame with source: None.
Auditing the rest the same way found four more: flac, lpcm, mpegaudio and
the passthrough parser. All are one-PES-one-frame, so the packet's own
facts are the unit's facts.
Every Frame construction in every codec parser now carries a source,
checked by walking balanced braces rather than by eye — the earlier
count was taken with a regex that cannot see a block containing nested
braces, which is how these survived the first pass.
Matthew Jackson