cfc12774f25d21eee5a926988cf5e04c5f714a1a
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 MattJackson
|
1565da610a | chore: scrub non-shippable references from tests/comments | ||
|
MattJackson
|
f1926c38dc |
v0.20.1: delete SectorReader, extract Disc::patch, doc/stub cleanup
WO-2 (delete SectorReader trait): - The 0.18 trait split into SectorSource (read-only) and SectorSink (write-only) is final; the legacy SectorReader alias was a bridge. - Renames every internal &mut dyn SectorReader (~25 sites) to &mut dyn SectorSource. The trait method capacity() becomes capacity_sectors() with a default of 0 (preserves SectorReader's default-0 behavior). - Deletes the SectorReader trait, its blanket-to-Source bridge, and the FileSectorReader type alias. Adds explicit forwarding impls for Box<dyn SectorSource> and &mut dyn SectorSource so generic decorators like DecryptingSectorSource<S: SectorSource> compose. WO-3a (extract Disc::patch): - Moves Disc::patch (1230 lines) and bytes_bad_in_title from disc/mod.rs into disc/patch.rs as a split inherent impl. Zero behavior change — pure mechanical relocation. disc/mod.rs drops from 3,945 to 2,714 LOC. WO-6 (partial): - Deletes src/labels/png_filenames.rs — was a 72-LOC stub with detect() returning false, never wired into the PARSERS registry. project docs doc drift fixes (audited 2026-05-13): - JUMP_BASE_SECTORS: 256→1024 (64 MB base for UHD, not 8 MB) - PASSN_DAMAGE_THRESHOLD_PCT: 12→6 - PASSN_SKIP_SECTORS_BASE: 64→32 - MAX_RANGE_SECS=180: replaced by proportional range_sectors × 25, capped at RANGE_BUDGET_CAP_SECS=1800. |
||
|
MattJackson
|
72ab714c1e |
labels/deluxe: Phase D rewrite against ground-truth binding pattern
Replaces the speculative arg-position heuristic with type-presence
detection driven by real disc bytecode. Ground truth captured in
(internal)/research/deluxe-poc/data/ via POC v0.3 binding-
bytecode dumps against disc-01 (Disney) and disc-09 (Warner).
What changed:
1. StackVal::CodingType(String) — new variant. getstatic against
org/bluray/ti/CodingType (the BD-J spec codec enum) now pushes
this, carrying the field name (e.g. DOLBY_LOSSLESS_AUDIO). The
pre-fix code was treating codecs as a Deluxe-internal enum
subclass walk (Phase B), which is the wrong model — codecs are
standard BD-J API references.
2. coding_type_to_codec_hint(field) — new function. Maps
org.bluray.ti.CodingType field names to human-readable codec
strings (DOLBY_LOSSLESS_AUDIO -> "Dolby TrueHD", DOLBY_AC3_AUDIO
-> "Dolby Digital", etc.). Unknown field names pass through
verbatim so future codec values still surface something.
3. find_binding_classes — multi-class variant. Some Deluxe discs
split per-stream tables across two binding classes (audio +
subtitle). Returns top-K candidates by getstatic count, filtered
to >=40% of the top count and capped at 4. Replaces the old
single-class find_binding_class (which was unused after this
change).
4. interpret_streams — rewritten. Args identified by TYPE not
position:
- First EnumRef{kind:"Language"} -> language
- First EnumRef{kind:"Purpose"} -> purpose
- First CodingType(name) -> codec_hint
- First Int(n) -> stream index hint (traced
only; per-type sequential stream_number still wins because BD
spec stream-numbering is anchored on MPLS)
- Construction has CodingType -> Audio stream; otherwise Subtitle
- No Language -> skip (not a stream construction)
This handles BOTH the Disney 5-arg pattern (I, Lbe, Llp, I,
LCodingType) and the Warner 4-arg pattern (I, Law, Lgp,
LCodingType) automatically — same code path because args are
identified by type rather than constructor-signature shape.
5. parse() now walks all binding-class candidates and unions
their constructions before calling interpret_streams. Logs each
candidate at INFO with getstatic_count for diagnosis.
Tests:
- 2 new tests verify the CodingType -> codec_hint mapping for
known + unknown field names.
- Existing interpret_streams tests updated to use the new
signature (dropped CodecTable arg).
- Audio-emission test rewritten to use CodingType arg instead of
the old binding_type substring-match approach.
Confidence is still Medium for now (single-corpus verification);
ready to promote to High once tested against a third Deluxe disc.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
MattJackson
|
3aa1e528c5 |
labels/deluxe: full Phase B/C/D buildout — codec walk, binding decode
Completes the Deluxe parser pipeline. Phase A (master enums) was
already shipping; this commit lands Phases B/C/D so the parser now
emits per-stream StreamLabel records on Deluxe-authored discs.
Phase B (decode_codec_enum): walks the codec enum's subclass
references (one .class per codec ordinal) and extracts the codec
name string from each subclass's constant pool. Heuristic: pick the
first Utf8 entry that's uppercase + underscored + >=4 chars, or one
of the known codec roots (ATMOS/DOLBY/DTS/TRUEHD/MLP/AC3/EAC3/PCM)
when no underscored candidate is found. CodecTable maps ordinal ->
codec string; empty string for ordinals where extraction failed
(logged via tracing, not fatal).
Phase C (find_binding_class): identifies the class that builds the
per-stream label table by counting getstatic operations targeting
any of the master enum classes from Phase A. Class with the highest
count >= 4 wins. Threshold is empirical (real binding classes have
50+ matches; floor of 4 admits small discs while rejecting incidental
single-reference classes).
Phase D (decode_binding + BindingDecoder): symbolic stack machine
that walks the binding class's <clinit> bytecode. Handles:
- constant pushes: iconst_<n>/bipush/sipush/ldc(Integer)
- new <X>: pushes uninit-object marker
- dup: stack copy
- getstatic <Y.Z>: pushes EnumRef when Y is in MasterEnumTable,
else Unknown
- invokespecial X.<init>(...)V: pops args per descriptor; when the
receiver is NewObj(X), emits a Construction { binding_type: X,
args: [...] }
- invokevirtual/invokestatic/invokeinterface: pop args per
descriptor, push return placeholder unless void
- pop/pop2/aastore/putstatic/putfield: standard stack effects
- branches/returns: clear stack (conservative resync — binding
<clinit> is straight-line in practice)
parse_method_arg_count: JVMS field-descriptor parser, handles
primitives, references (L...;), arrays ([...).
interpret_streams: converts Constructions to StreamLabels using
the master enum table + CodecTable. Each construction with a
Language ref becomes a stream. Audio when codec_hint resolves via
binding_type substring match against CodecTable; subtitle otherwise.
Purpose ordinal -> LabelPurpose via the verified Deluxe Purpose enum
order (Normal/Commentary/PiP/Trivia/Descriptive/Score/NoForced/
NoForcedDescriptive). Stream index = sequential per type. Language
goes through vocab::lang for ISO code + variant.
deluxe::parse now returns Some(ParseResult::medium(labels)) when
all four phases produce labels. Medium confidence — the bytecode
mechanism is rigorously tested but the signal-to-StreamLabel
mapping (which arg is which, audio vs subtitle classification) is
heuristic until corpus binding-class bytecode confirms the exact
pattern.
Test coverage: 13 new unit tests in deluxe.rs
parse_method_arg_count: 3 tests (basic types, references, malformed)
BindingDecoder: 4 tests (simple construction, with int pushes,
skips unmatched invokespecial, resolves master-enum ordinal)
interpret_streams: 4 tests (subtitle on no codec, audio on codec
match, purpose routing, skips no-language)
MasterEnumTable: 3 tests (resolve, value, class_name_set)
extract_codec_name: 1 test (uppercase+underscore matching)
class_reader.rs gained a #[cfg(test)] ConstantPool::from_entries
test-only constructor so Phase D tests can build synthetic CP
fixtures without writing raw .class bytes.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
MattJackson
|
7c6b0f82ab |
labels: per-parser confidence + highest-confidence-wins registry
Replaces 'first-match-wins by array order' with 'highest-confidence-
wins, array order tiebreaker'. Removes the arbitrariness when more
than one parser can claim a disc (e.g. one with both
bluray_project.bin and playlists.xml).
New types in labels::mod:
pub enum Confidence { Medium, High }
pub struct ParseResult { labels: Vec<StreamLabel>, confidence }
ParseResult::high(labels) / ::medium(labels) constructors
Parser signature change: every parse() now returns
Option<ParseResult> instead of Option<Vec<StreamLabel>>. Updated all
six parsers in lockstep:
paramount: High (fully structured XML)
criterion: High (fully structured XML)
pixelogic: High by default, Medium when an unknown token component
is encountered (the skip-unknown path now propagates the
coverage gap to the caller instead of silently degrading)
ctrm: High (structured key-value)
dbp: High (anchor scan with vocab routing)
deluxe: still returns None pending Phase D — signature aligned
Registry behavior:
extract() iterates all detect-positive parsers, picks highest
Confidence with non-empty labels. Equal confidence falls to array
order (deterministic). Same selection logic in analyze().
LabelAnalysis grew a confidence: Option<Confidence> field so the
diagnostic surface (freemkv-tools labels-analyze) exposes which
confidence tier the selected parser claimed. labels-analyze JSON
and labels-corpus-check structural diff both gained the field.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
MattJackson
|
4226a53e73 |
labels: fresh-eyes audit — capture variant, dedupe detect, lock registry
Three targeted fixes from a second-pass audit of the labels module. 1. vocab::lang now returns Option<LangInfo> with both code AND a human-readable variant string. Pre-fix: 'Brazilian Portuguese 5.1' became language=por, variant='', dropping the dialect info the disc had explicitly authored. Post-fix: language=por, variant='Brazilian' — matches the convention pixelogic / ctrm / criterion already use for their region variants. dbp now populates StreamLabel::variant from this. Compound table grew a 3-tuple (needle, code, variant); bare matches still return variant=''. 2. dbp and deluxe had duplicated detect() boilerplate (any top-level .jar in /BDMV/JAR/). Both now call jar::has_any_top_level_jar. The trait-level detect contract — see super::PARSERS — can't peek inside a jar without a SectorReader, so loose-detect-plus-real- check-in-parse is the unavoidable pattern for jar-content parsers. Consolidating in jar.rs at least makes the duplication visible. 3. mod.rs comment about parser ordering said 'dbp last'; deluxe is actually now last. Updated to explain the dbp-before-deluxe order is by cost (cp-iteration cheaper than bytecode walking when Phase D lands). Plus a registry-level lock test in mod.rs::registry_tests — asserts the PARSERS array order is exactly [paramount, criterion, pixelogic, ctrm, dbp, deluxe]. This was previously implicit; if someone reorders the array (which changes which parser wins on overlapping signals), unit tests would have stayed green. Now they fail with an explanatory message about why the order matters. Audit findings deferred to follow-ups (each its own commit + design discussion): - Stronger detect contract — current loose-detect-real-check pattern is forced by SectorReader-not-in-detect-signature; could be fixed by changing the trait to take an Option<&mut dyn SectorReader> or similar. - Per-parser confidence scoring — registry currently first-match-wins. A high-confidence parser ought to beat a low-confidence one regardless of array order. - class_reader fuzzing — handles malformed input via Result but no adversarial corpus yet. Precommit (cargo +1.86 fmt + clippy + test) green. |
||
|
MattJackson
|
fdbe469d50 |
labels: add deluxe parser (Phase A — master enum identification)
Closes the 'detected but no parser' gap on Deluxe-authored BD-J discs
(com/bydeluxe/ package signature; ~20% of UHD discs in our corpus per
the 2026-05-10 11-disc capture session).
What ships in this commit:
- detect() — registers the parser in the chain (loose pre-check at the
/BDMV/JAR/ level; real signal in parse via has_path_prefix on
'com/bydeluxe/').
- Phase A master enum identification — walks every .class's <clinit>
ldc sequence and matches against framework-stable fingerprints for:
Language (70 ldcs starting English/French/Spanish/Dutch)
Purpose (8 ldcs starting Normal/Commentary/PiP/Trivia)
VideoFormat (7 ldcs starting HD/HDR10 Plus/HD Dolby)
Region (22 ldcs starting USA_D1/LIC1/LIC2/LIC3 — Disney-only)
Studio (6 ldcs starting Disney/Marvel/Pixar — Disney-only)
All identifications verified out-of-tree on corpus disc-01 (Disney,
The Amateur) and disc-09 (Warner, Dune Part 1) via the standalone
POC.
- Phase B structural skeleton (find_codec_enum) — identifies the
codec enum class by structural signature (>=20 'new' ops, 0 ldcs),
returns the ordered subclass list. Codec string extraction from
subclasses is dead-coded pending the follow-up commit.
What does NOT ship yet:
- Phase D (per-stream binding-class decoder). parse() returns None
intentionally — the master enums alone don't yield StreamLabels
without the streamTable.put(...) bytecode walker. analyze() will
show 'deluxe' in parsers_detected with the enum identification in
tracing logs, so the analyzer reports honestly: 'detected, can't
emit labels yet' rather than silent failure.
Why ship A without D: A is proven on real corpus discs; D's design
needs ground-truth binding bytecode from at least 2 corpus discs
side-by-side to verify the stack-machine pattern. The Phase A
infrastructure (master enum identification + ordinal->name table)
is what D will consume — landing it now unblocks D's design without
holding back the parser registration.
5 unit tests cover the fingerprint matcher logic + a roster lock that
forces explicit consideration when adding/removing fingerprints.
Precommit green.
|