A vendor label's `stream_number` is a slot in the one stream table its
config blob describes. The labels merged in from the playlists — to
cover streams the vendor named nothing for — carried a different number
entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's
slot numbering at all. Two coordinate systems, one field name. The
merge matched them by equality and the binder then counted streams
against the result.
Measured over the 44-image corpus: 22 discs merge such labels; of the
566 places one lands on a stream, 443 (78%) are a stream it does not
describe — the label states the PID it read itself from and it is a
different one. 142 of those are stopped by the language check 94377c7
added; 301 are applied. Those labels carry no editorial payload, so the
direct damage is confined to codec text — but the polluted list is also
what the anchor gate reads, and on 11 disc/stream-type pairs it is what
decides the anchor, which is how it reaches the vendor's forced and SDH
flags. 53 anchor facts are harvested off a merged slot. The clip-info
orphans had the same shape, numbered from `max + 1` of a list they share
no coordinate system with.
A label now either NAMES its stream — `StreamId { clip, pid }`, read out
of the very table `disc::bluray` builds the stream from — or it does
not, and only the ones that do not are ever reached by counting:
* `label_at` returns vendor labels only, so the two numberings can no
longer be confused by construction.
* Playlist and clip-info labels bind by id. Exact, no anchor, no
sequence, no ordinal.
* A named stream outranks a guessed one, so an editorial flag reaches
a stream only where the disc's own numbering puts it there.
* The presence of an id is the provenance the anchor gate was missing.
It reads the vendor's slots alone now, so a slot the vendor never
named no longer breaks the sequence — under-yield is the normal
shape of these blobs — and a title with fewer streams than the list
has slots is no longer eligible to hold it. Ranking prefers the
title that positively confirms most of the list.
* An orphan is in no playlist, hence in no title, so it binds to
nothing rather than to whatever counted its way.
The MPLS floor is one label per physical stream keyed by `(clip, PID)`,
not by `(type, language, codec, pid)`: the same PID in two clips is two
streams, and the old key collapsed them. Its `stream_number` is now the
entry's real slot in its own playlist's table.
41 of 44 images are byte-identical; all three that move lose a label
they should not have had, and no feature title changes on any image. A
dozen featurette playlists stop reporting a feature subtitle's SDH
marking on their own unrelated subtitle; eleven menu and bonus titles
stop advertising the feature's object-audio format on plain stereo; and
on a disc whose every title carries a single audio stream — too short a
table to anchor anything — a regional-variant tag asserted on all
seventeen titles is now asserted on none, in exchange for every title
stating the codec it actually carries, which none of them did.
Six tests had been asserting the invented numbering, including one
pinning the disc-global counter as a deliberate property.
Also fixes a defect in the same family that the corpus work surfaced:
`pgs_forced_probe::apply_verdicts` set `forced` but left `qualifier`,
so a demoted track shipped with a metadata sidecar calling it forced
next to a Matroska header saying it is not. Only a forced claim is
cleared; an SDH marking is not the probe's to touch.
One vendor's playlists.xml carries a per-subtitle-slot cell that looks
like a boolean, and it was parsed as one: value 1 meant forced, anything
else meant not forced. Across every image in the corpus that uses this
format the cell takes four values, and 1 is not the forced one.
Decoding three of those discs and counting every PGS display set:
* 1 marks a FULL dialogue track that additionally contains some
forced-narrative signs. All nine cells bearing it on one disc are
full tracks of 949-1411 display sets; all seven on another are full
tracks of 1602-1651. Neither disc has a small track among them.
* 2 and 3 mark a DEDICATED forced-narrative track, in its own trailing
stream slot, duplicating a language that already holds a full track.
The two 2 slots measured are 15 and 10 display sets with every one
flagged forced; the four 3 slots are 7, 14, 23 and 59 against
1216-2655 on the tracks they duplicate.
So the old reading was wrong in both directions — it flagged full
dialogue tracks forced, which is how one language came to present as two
identical full subtitle tracks with one of them marked forced, and it
threw away the cells naming the real forced tracks.
Content could not have corrected this afterwards. Clearing a wrong
forced label needs a disc whose authoring sets forced_on_flag, and on
the measured disc carrying four genuine forced tracks not one display
set anywhere sets it — there, the vendor cell is the only evidence there
is. The classification is now explicit: only a dedicated forced slot
earns the flag, an unrecognised value never does, and the
contains-forced-signs value is dropped rather than weakened into a
forced label, since a wrong forced flag on a full dialogue track is the
user-visible defect while a missing hint costs nothing.
Four of the crate's own tests had been asserting the boolean reading;
their subject was positional alignment, so they keep it and now use a
real forced value. The other two parsers that emit a forced qualifier
from vendor metadata were audited and are structurally immune — in both,
the forced marker names a slot of its own rather than hanging off a full
track's entry, so the failure has no encoding there — and each is now
pinned by a test saying so.
Sweep of every parser in src/labels/ for the numbering bug fixed in
pixelogic: a blob lists one entry per STN slot, but the parser advances
its per-kind counter only for entries it can use, so every entry it
skips shifts all later labels onto the wrong stream.
Three parsers were affected; the rest key each label off a number the
blob states outright and are immune.
* paramount — `aud` / `sub` are the STN-ordered stream lists, and
`forced_sub` / `*_com1_idx` index those same cells. A cell with an
empty language was skipped without consuming its slot, so every
label behind it bound one stream early while the vendor's own
positional indices still pointed at the raw cell. `stream_number`
is now the cell's 1-based position. The forced flag is the payload
here, so the shift lands `forced` on a full-dialogue track.
* mpls_universal — its counters must agree with the stream list
`disc::bluray` builds from the same STN entries, since that list is
what `apply_labels` counts against. They disagreed twice: a
`coding_type == 0` padding entry was counted here and dropped
there, and a PG coding_type in an audio STN slot (a layout
`mpls::parse_stream_entry` has a dedicated arm for) was counted as
audio here and built as a subtitle there. Both rules now live in
one `label_type_for`.
* deluxe — a binding construction whose Language `getstatic` did not
resolve was skipped outright. It is still an STN slot; it just has
nothing to label. It now advances the counter, with the list it
belongs to taken from its CodingType argument or, failing that,
from what its binding type's resolved siblings showed.
Two paramount tests asserted the renumbering as if it were the spec
(`empty_middle_slot_does_not_inflate_stream_number`,
`audio_stream_numbering_skips_empty_slots`) and are rewritten. Immunity
pins added for ctrm, dbp and criterion so the property cannot rot.
Also scrubs two commercial disc titles from the menu-graphic filename
examples in png_filenames and vocab.
Thirteen agents triaging src/labels and src/disc died on a saturated
machine, leaving 5,836 insertions across 28 files uncommitted in a
worktree. Recovered by 3-way apply onto twelve commits of drift; zero
conflicts. The diff was archived to freemkv-private first, because a
worktree is not a backup and this one had already nearly been lost.
One production change, and it is the right one: mpls_universal::parse
read every playlist off the disc AND converted the entries to labels in
a single function, so the conversion — stream-type mapping, dedup key,
the dense global counters — could only be reached through a synthetic
UDF image. Extracted to build_labels(&[Playlist]), which unit tests can
drive from already-parsed values. Behaviour-preserving: same iteration
order, same skip-on-error.
Two collisions resolved by hand:
A second mod pass_progress_tests, written independently against the
same survivors as the one committed in c610285. Kept mine — it covers
the distinct-counters case and the Progress blanket impl, which theirs
does not — but theirs had three clamp tests mine lacked: good_pct,
bad_pct and pending_pct also clamp an overshoot, and I had only tested
that for work_pct. Merged those in as one test and proved each of the
three clamps load-bearing by removing them individually.
An unused_parens warning in a new fixture.
Method note, recorded because it cost real time: git apply --3way
STAGES its result, so `git diff` reads empty and the tree looks
untouched. I nearly concluded the patch had silently failed. Worse, the
first attempt piped through `head -20`, so `echo exit=$?` reported
head's status rather than git's — the same mistake this audit has
already documented once. Check the real exit status, and check
--cached, not just the working tree.
Ten defects in code no previous round had ever scoped. `src/labels/`
identifies a disc's studio by parsing jar archives and JVM class files off
untrusted media, so every byte here is attacker-controllable — and 813 of
its lines were executed by no test at all.
The worst is a non-terminating loop. A fallback stream-number scan
advanced with `saturating_add`, and the comment says why: a crafted XML
"must not overflow (panic in debug, wrap-to-0 in release)". Once the
counter pins at u16::MAX and that number is taken, the loop cannot exit.
So a fix for an overflow panic produced an unbounded hang, which is
strictly worse — a panic is observable and catchable, and catch_unwind
cannot interrupt a live loop. Reachable from about 8 MB of XML.
Where the same overflow appears in the deluxe decoder the fix is
checked_add and stop, NOT saturation — twice wrong there, because
saturating would peg every stream past the ceiling at one number and
apply_labels binds on (type, number), silently mislabelling tracks. A
correctness bug wearing the costume of success.
Round 7 capped the ldc-string retention per class; nothing capped the
aggregate, so a 64 MiB jar held that budget for every class at once. Same
defect one level up, which is the shape that keeps recurring in this
directory. Four other amplifications are bounded the same way, each with a
stated headroom and a paired test proving real media passes untouched —
the tightest is 5x on a label length, the loosest 2000x on the stream
numbering space, against BD's 32-per-type STN_table limit.
Two are not caps at all: a quadratic membership scan became a set, and an
attacker-derived length added to a cursor without saturation now cannot
wrap. Nothing is excluded by either.
A `#[cfg(test)]` hand-copy of a shipping parser was the ninth bad test
this audit has found, and the first proven by mutation rather than
inspection: deleting the guard from the REAL function left all 26 tests
green, including the one named for that guard. Pointed at the real
function, the same mutation fails.
Separately, all three failure arms of the bounded fsync returned Ok(()) on
both macOS and Linux, so sync_all reported success for a durability
barrier that never ran. Only macOS was in scope; the Linux twin is fixed
here too, because a platform disagreeing with its sibling about whether a
failed sync is an error is the class that already produced an over-length
SCSI CDB macOS rejected and the other two truncated. Note the behaviour
change: a mux whose final sync times out on a wedged mount now fails
rather than exiting 0.
Three of the caps are proven by wall-clock deadline rather than an
operation count, with 18-80x margin on the passing side. On a heavily
oversubscribed machine those could flake.
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest
under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the
minimum, this goes to current stable and fixes what that costs.
The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54
clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy
collapsible_if lint in between. Stopping at the minimum would have been the most
expensive choice available.
Roughly 47 lints across the eight repos, the large majority auto-fixed:
libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6,
freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to
sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced
by enumerate, and a loop whose first let-else became a while-let.
Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only
appear once the crate DECLARES 1.89 or later, because let-chains become
available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87
reports clean and is meaningless — gate with the real precommit script, which is
also the only thing that covers build scripts.
The pin still sits below the Mac default, so it keeps doing its job: catching
lint drift locally before CI sees it.
- DetectFn now takes a SectorSource so a parser can inspect a jar's
central directory in detect() instead of firing on "any BD-J jar".
dbp/deluxe do the real com/<vendor>/ prefix check up front, so each
claims only its own discs (foundational for scaling the registry).
- criterion: treat a stream-map value of 0 as unmapped and synthesize a
real 1-based number, so a 0 can't shadow or collide with a genuine
stream 1 (with regression tests).
- png_filenames: new Low-confidence, last-resort parser reading menu
language from {title}_UHD01_{LANG}_Composite artwork; sits below the
MPLS floor so a real framework parser always wins.
- vocab: add menu_lang() for 639-2/B to 639-2/T menu-token normalization.
Test-hardening release, no runtime changes. Adds spec-grounded unit tests
across the silent-corruption surfaces — UDF/MPLS/CLPI/IFO parsing, BD/DVD
title + extent assembly, AACS/CSS key handling, TS/PS demux + codec parsers,
MKV/EBML container output, the mux pipeline, sector prefetch + decrypt
decorator, drive/SCSI sense decoding, label extraction, and core I/O. Each
test is grounded in the format spec or real on-disc behavior and verified to
fail under a targeted source mutation. No behavior changed.
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
WO-2 (delete SectorReader trait):
- The 0.18 trait split into SectorSource (read-only) and SectorSink
(write-only) is final; the legacy SectorReader alias was a bridge.
- Renames every internal &mut dyn SectorReader (~25 sites) to
&mut dyn SectorSource. The trait method capacity() becomes
capacity_sectors() with a default of 0 (preserves SectorReader's
default-0 behavior).
- Deletes the SectorReader trait, its blanket-to-Source bridge, and
the FileSectorReader type alias. Adds explicit forwarding impls
for Box<dyn SectorSource> and &mut dyn SectorSource so generic
decorators like DecryptingSectorSource<S: SectorSource> compose.
WO-3a (extract Disc::patch):
- Moves Disc::patch (1230 lines) and bytes_bad_in_title from
disc/mod.rs into disc/patch.rs as a split inherent impl. Zero
behavior change — pure mechanical relocation. disc/mod.rs drops
from 3,945 to 2,714 LOC.
WO-6 (partial):
- Deletes src/labels/png_filenames.rs — was a 72-LOC stub with
detect() returning false, never wired into the PARSERS registry.
project docs doc drift fixes (audited 2026-05-13):
- JUMP_BASE_SECTORS: 256→1024 (64 MB base for UHD, not 8 MB)
- PASSN_DAMAGE_THRESHOLD_PCT: 12→6
- PASSN_SKIP_SECTORS_BASE: 64→32
- MAX_RANGE_SECS=180: replaced by proportional range_sectors × 25,
capped at RANGE_BUDGET_CAP_SECS=1800.
Replaces 'first-match-wins by array order' with 'highest-confidence-
wins, array order tiebreaker'. Removes the arbitrariness when more
than one parser can claim a disc (e.g. one with both
bluray_project.bin and playlists.xml).
New types in labels::mod:
pub enum Confidence { Medium, High }
pub struct ParseResult { labels: Vec<StreamLabel>, confidence }
ParseResult::high(labels) / ::medium(labels) constructors
Parser signature change: every parse() now returns
Option<ParseResult> instead of Option<Vec<StreamLabel>>. Updated all
six parsers in lockstep:
paramount: High (fully structured XML)
criterion: High (fully structured XML)
pixelogic: High by default, Medium when an unknown token component
is encountered (the skip-unknown path now propagates the
coverage gap to the caller instead of silently degrading)
ctrm: High (structured key-value)
dbp: High (anchor scan with vocab routing)
deluxe: still returns None pending Phase D — signature aligned
Registry behavior:
extract() iterates all detect-positive parsers, picks highest
Confidence with non-empty labels. Equal confidence falls to array
order (deterministic). Same selection logic in analyze().
LabelAnalysis grew a confidence: Option<Confidence> field so the
diagnostic surface (freemkv-tools labels-analyze) exposes which
confidence tier the selected parser claimed. labels-analyze JSON
and labels-corpus-check structural diff both gained the field.
Precommit (cargo +1.86 fmt + clippy + test) green.
Closes the platform unification: every label parser now routes
purpose/qualifier/codec classification through one source of truth
(vocab.rs) instead of N hand-rolls, and every binary-blob byte
scanner goes through one helper (text::extract_ascii_strings).
pixelogic.rs:
- Drop local extract_strings (~20 lines) — use text::extract_ascii_strings.
- HARDENING: replace with skip-unknown-component +
trace log. Pre-refactor behavior: any single uncatalogued token part
(e.g. a future codec ID, new framework variant) silently dropped the
entire stream record. New behavior: skip just the unknown part,
surface what we know about the stream.
- 8 new unit tests cover basic audio/subtitle paths, commentary,
descriptive, region variant, the new skip-unknown-component
regression, and the non-audio/non-subtitle early-out.
ctrm.rs:
- Replace with
vocab::purpose(&name). Now word-boundary matched — 'Commenter Pro
Track' no longer false-matches Commentary.
- Replace with vocab::qualifier(&name).
Same word-boundary tightening, plus picks up Forced and
DescriptiveService for free.
- Preserved structural commentary signal via
as a fallback when name is silent (e.g. 'audio_commentary_1.name=Track 2').
- 6 new unit tests including the 'Commenter' false-positive regression
and the SDH-only-on-subtitles boundary.
text.rs:
- Drop module-level dead_code allow now that pixelogic uses
extract_ascii_strings.
Net: all 5 framework parsers now on the unified platform. Future work
(deluxe Phase D, paramount/criterion XML hardening) builds on the
same scaffolding.
Precommit green.
- SectorReader trait decouples disc scanning from SCSI
- Disc::scan_image() for ISO and any sector source
- resolve_encryption() handles AACS 1.0/2.0/none in one path
- IsoStream: full UDF/MPLS/CLPI/labels pipeline from ISO files
- StdioStream: stdin/stdout pipe
- Strict scheme:// URL format with validation
- Labels module refactored to SectorReader
- 7 stream types total
Architecture:
- Each BD-J format in own file: paramount.rs, criterion.rs, pixelogic.rs, ctrm.rs
- Standard interface: detect() → bool, parse() → Option<Vec<StreamLabel>>
- PARSERS array in mod.rs — drop in a new parser with one line
- Shared vocab.rs for BD spec codec names only (MLP→TrueHD, AC3→Dolby Digital)
- All other label data passes through raw from disc — no guessing
Changes:
- New: paramount.rs (playlists.xml — Paramount/onQ format)
- Renamed: bluray_project.rs → pixelogic.rs
- Renamed: stream_properties.rs → criterion.rs
- Merged: language_streams.rs + menu_base.rs → ctrm.rs
- Removed: jar module (superseded by labels), dead apply functions
- Added: DriveSession::eject() with PREVENT ALLOW MEDIUM REMOVAL
- Added: DiscRegion enum (Free/BluRay/Dvd)
- Fixed: capture sector ranges now include all files (only skip STREAM/)
- Renamed: StreamLabel.region → variant (not a BD spec field)
Matthew Jackson