Compare commits

...
21 Commits
Author SHA1 Message Date
MattJackson 26e2d847d5 v0.18.9: bump version 2026-05-10 17:02:24 -07:00
MattJackson 4442fa2df6 disc: wedge AVOIDANCE on Pass 1 — inter-error pause + larger jumps
Complements the wedge-skip backstop (fbdb50c) with proactive
avoidance so we don't HIT the wedge in the first place. User's
take after seeing the Dune Pt 2 rip wedge at 48%: 'we shouldn't be
wedging.'

Empirical observations from the 23:09:12-23:09:55 wedge timeline:

  5 read errors over 43 s, ~8 s apart (drive's own ECC recovery
  takes 5-10 s per failure). Not 'hammering' in any usual sense,
  but cumulative firmware-state buildup over 5 in-cluster errors
  was enough to tip the BU40N into wedge mode at the 5th error.

  Damage cluster spanned ~140 MB (LBAs 19.898M-19.965M). Current
  damage-jump base of 256 sectors × batch=32 = 16 MB first jump,
  doubling to 32 MB, 64 MB... Each jump landed BACK INSIDE the
  140 MB cluster, exposing the drive to MORE in-cluster errors.

Two avoidance levers:

1. Inter-error pause on Pass 1 (PASS_1_FAIL_PAUSE_SECS = 5 s):
   pre-fix Pass 1 ran pause_secs=0 on all errors to 'zoom past'
   damage zones. Successful reads still zoom at zero pause — the
   pause applies only to FAILED reads, giving the drive's firmware
   cool-down between cluster exposures. Cost: ~5 s per scattered
   failure (~30-60 s total on a damage cluster); trivial vs.
   crashing the rip at 48%.

2. Larger damage-jump base (JUMP_BASE_SECTORS = 1024, up from
   256): first jump at batch=32 now covers 64 MB instead of 16 MB,
   second jump 128 MB instead of 32 MB. Two jumps clear 192 MB —
   well past most single-cluster damage patterns. Smaller jumps
   were landing inside the cluster and adding to the wedge counter.

Plus a halt-aware sleep helper (sleep_secs_or_halt) so the new
inter-error pause doesn't degrade halt response time. Halt poll
granularity 100 ms — halt fires within ~100 ms regardless of
remaining pause time. Updated three sleep call sites in disc/mod.rs
(SkipBlock pause, JumpAhead post-pause, Retry pause).

The wedge-SKIP backstop (fbdb50c) stays — combined with this
avoidance work, the flow becomes:
  damage cluster encountered →
    pause 5 s, mark NonTrimmed →
    second failure →
    pause 5 s, mark NonTrimmed →
    ...
    threshold hit →
    damage-jump 64 MB (clears 95% of clusters) →
    if jump lands in another cluster: 128 MB next jump →
    only if drive STILL wedges after all this:
      wedge-skip kicks in (1 GB jump + 30 s cooldown × 16 budget)

Tests:
  pass_1_pauses_briefly_on_skip_for_wedge_avoidance — locks the
    new 5 s pause behavior in place (replaces the old pause=0 test).
  integration test threshold bumped from 5 s to 60 s with comment
    explaining the new bound is 'not infinite' rather than
    'milliseconds-fast'.
  All 433+ tests green on cargo +1.86 fmt + clippy + test.

Precommit green.
2026-05-10 16:55:02 -07:00
MattJackson 23dc55661c labels: apply_labels integration tests + class_reader robustness fuzz tests
Closes the final two audit items from this session.

labels::apply_labels: factored out of apply() so the matching logic
is unit-testable without needing a SectorReader / UdfFs. 11 new
tests in apply_tests cover:
  - codec_hint + variant flow through to AudioStream.label
  - purpose set on audio with no label English text
  - name fallback only when purpose=Normal (CLI owns purpose i18n)
  - subtitle SDH qualifier set; forced flag flipped on Forced
  - per-type 1-based indexing (audio #2 maps to 2nd audio stream,
    not 2nd stream overall)
  - labels for nonexistent streams are no-ops
  - empty labels list leaves streams untouched
  - fill_defaults generates audio + video labels; preserves existing

class_reader: robustness smoke tests. ClassFile::parse must NEVER
panic on adversarial input — only return Err. 9 new tests:
  - empty input
  - short magic (0..4 bytes)
  - wrong magic
  - truncated after magic
  - bad CP tag
  - truncated UTF-8 in CP
  - 200 random byte buffers (deterministic xorshift)
  - 100 magic + random tail (most adversarial — magic check passes,
    everything else garbage)
  - instructions iter on random code (200 buffers)
  - instruction_size on every opcode 0..255 with varied tail buffers
  - modified_utf8 on random byte buffers (500)

The xorshift PRNG keeps the tests deterministic (no rand dep) and
reproducible — failures will be the same buffer every time. This is
the lightweight alternative to a cargo-fuzz setup; if/when we adopt
cargo-fuzz, these tests stay as regression cases.

All 451 tests passing on cargo +1.86 fmt + clippy + test.
2026-05-10 16:38:19 -07:00
MattJackson fbdb50c79f disc: Pass 1 wedge-skip instead of abort-on-first-wedge
Pre-fix: when the drive returned HARDWARE_ERROR or ILLEGAL_REQUEST
during Pass 1 sweep, libfreemkv immediately returned ReadAction::
AbortPass. Autorip surfaced this as a fatal error and stopped the
rip at whatever progress percentage Pass 1 had reached — typically
40-50%. On a disc with one physical-damage cluster, the user would
see Pass 1 die at ~48% with the cryptic message 'E6000: <lba>
0x02/0x04/0x3e' and have no rip output to work with.

Root cause analysis: BU40N firmware transitions into a fast-fail
state when it hits cumulative read failures in a small LBA range —
returns HARDWARE_ERROR for every subsequent read near that LBA, even
sectors that aren't physically damaged. Per project docs 'Bad-sector
handling' rule #2, 'Recovery requires eject+reload OR significant
cool-down.' Aborting on first wedge throws away the rest of the
disc; the right response is to SKIP the wedged region (mark as
NonTrimmed for Pass N), pause for drive cooldown, and continue.

Fix: in handle_read_error, the HARDWARE_ERROR / ILLEGAL_REQUEST arm
now branches on bisect_on_marginal:

  Pass 1 (bisect_on_marginal=false): JumpAhead with WEDGE_JUMP_SECTORS
    (1 GB at 2048 bytes/sector) and WEDGE_PAUSE_SECS (30 s cooldown).
    Tracks wedge_count in ReadCtx; resets on any successful read.
    Truly aborts only after WEDGE_ABORT_THRESHOLD (16) consecutive
    wedges with no good read in between — generous enough to clear
    most physical-damage clusters, bounded enough to not loop forever
    on a permanently bricked drive.

  Pass N (bisect_on_marginal=true): unchanged AbortPass. Pass N's
    job is single-sector recovery; if the drive won't talk near a
    specific LBA, skipping doesn't help. Pass N exits and lets the
    outer layer decide retry/eject/surface.

5 unit tests cover the new policy:
  pass_1_hardware_error_jumps_ahead_not_aborts — JumpAhead emitted
    with correct sectors+pause, wedge_count incremented.
  pass_1_hardware_error_aborts_after_threshold — AbortPass kicks in
    on the WEDGE_ABORT_THRESHOLD-th consecutive wedge.
  pass_1_good_read_resets_wedge_count — on_success clears
    wedge_count; subsequent wedge gets fresh skip budget.
  pass_n_hardware_error_still_aborts — Pass N's AbortPass behavior
    intact.
  pass_1_illegal_request_also_routes_to_wedge_skip — both wedge
    sense families get the skip treatment.

Impact: on the Dune Pt 2 disc that consistently wedged at 48%
(physical damage at LBA ~19.9M), Pass 1 will now jump ahead 1 GB
on the wedge, give the drive 30 s cooldown, and continue scanning
the rest of the disc. The damaged region becomes Pass N's job to
revisit. Worst case if the drive stays wedged: 16 GB of NonTrimmed
disc area before honest AbortPass.

Precommit (cargo +1.86 fmt + clippy + test) green; 430 passing.
2026-05-10 16:37:59 -07:00
MattJackson 7cc74f0087 labels/xml: shared tolerant XML helper, paramount + criterion onto it
Replaces two near-duplicate hand-rolled XML scrapers in paramount.rs
and criterion.rs with a single labels::xml module that's robust to:

- Case-insensitive tag / attribute names ('<Playlist>' matches the
  same as '<playlist>'; 'Name=...' matches 'name=...').
- XML namespace prefixes (matches '<ns:tag>' for tag='tag').
- Arbitrary whitespace inside open tags and around '=' separators
  ('<tag  name = "X">' works).
- Both quote styles for attribute values (" and ').
- Self-closing tag forms ('<tag/>' and '<tag />').
- '>' chars inside quoted attribute values (no premature end-of-tag).

Three functions:
  xml::attr(element, name) -> Option<String>
      Extract attribute value from an open-tag fragment.
  xml::text(xml, tag) -> Option<String>
      Trimmed text content of first <tag>...</tag>.
  xml::find_element(xml, tag, from) -> Option<(start, end)>
      Locate next <tag>...</tag> for iteration; handles self-closing.

22 unit tests cover the robustness properties: case-insensitivity,
namespace stripping, whitespace tolerance, quote styles, self-close
forms, no-substring-false-positive (looking for 'lang' must NOT
match 'lang_id' or 'language'), '>' inside quoted attrs, iteration
across repeated elements.

paramount.rs: drops local extract_attr; find_feature_playlist now
walks xml::find_element('playlist', ...) so it works regardless of
case and self-closing style. Pre-refactor: required exactly
'<playlist ' (single space, exact case) and '/>' for self-close.

criterion.rs: drops local extract_tag; parse_stream_infos and
parse_playback_config iterate via xml::find_element. Same case-
sensitivity + namespace gains. The 'COMMENTARY' / 'SDH' / 'DS'
content-value match is now case-insensitive too (previously a disc
authored with 'commentary' would have been miscategorized as Normal).

Pre-refactor known failure modes (none observed yet, but trivial
to trip on a future disc): vendor switches whitespace around '=',
uses single quotes, capitalizes a tag, prefixes a namespace. All
now handled.

Out of scope by design: XML entity decoding (&amp;, &lt;), CDATA
sections, comments, processing instructions. None observed in BD-J
authored label data. If a future disc trips them, the entity
decoder is a localized addition.

Precommit (cargo +1.86 fmt + clippy + test) green.
2026-05-10 16:21:51 -07:00
MattJackson 3aa1e528c5 labels/deluxe: full Phase B/C/D buildout — codec walk, binding decode
Completes the Deluxe parser pipeline. Phase A (master enums) was
already shipping; this commit lands Phases B/C/D so the parser now
emits per-stream StreamLabel records on Deluxe-authored discs.

Phase B (decode_codec_enum): walks the codec enum's subclass
references (one .class per codec ordinal) and extracts the codec
name string from each subclass's constant pool. Heuristic: pick the
first Utf8 entry that's uppercase + underscored + >=4 chars, or one
of the known codec roots (ATMOS/DOLBY/DTS/TRUEHD/MLP/AC3/EAC3/PCM)
when no underscored candidate is found. CodecTable maps ordinal ->
codec string; empty string for ordinals where extraction failed
(logged via tracing, not fatal).

Phase C (find_binding_class): identifies the class that builds the
per-stream label table by counting getstatic operations targeting
any of the master enum classes from Phase A. Class with the highest
count >= 4 wins. Threshold is empirical (real binding classes have
50+ matches; floor of 4 admits small discs while rejecting incidental
single-reference classes).

Phase D (decode_binding + BindingDecoder): symbolic stack machine
that walks the binding class's <clinit> bytecode. Handles:
  - constant pushes: iconst_<n>/bipush/sipush/ldc(Integer)
  - new <X>: pushes uninit-object marker
  - dup: stack copy
  - getstatic <Y.Z>: pushes EnumRef when Y is in MasterEnumTable,
    else Unknown
  - invokespecial X.<init>(...)V: pops args per descriptor; when the
    receiver is NewObj(X), emits a Construction { binding_type: X,
    args: [...] }
  - invokevirtual/invokestatic/invokeinterface: pop args per
    descriptor, push return placeholder unless void
  - pop/pop2/aastore/putstatic/putfield: standard stack effects
  - branches/returns: clear stack (conservative resync — binding
    <clinit> is straight-line in practice)
  parse_method_arg_count: JVMS field-descriptor parser, handles
  primitives, references (L...;), arrays ([...).

interpret_streams: converts Constructions to StreamLabels using
the master enum table + CodecTable. Each construction with a
Language ref becomes a stream. Audio when codec_hint resolves via
binding_type substring match against CodecTable; subtitle otherwise.
Purpose ordinal -> LabelPurpose via the verified Deluxe Purpose enum
order (Normal/Commentary/PiP/Trivia/Descriptive/Score/NoForced/
NoForcedDescriptive). Stream index = sequential per type. Language
goes through vocab::lang for ISO code + variant.

deluxe::parse now returns Some(ParseResult::medium(labels)) when
all four phases produce labels. Medium confidence — the bytecode
mechanism is rigorously tested but the signal-to-StreamLabel
mapping (which arg is which, audio vs subtitle classification) is
heuristic until corpus binding-class bytecode confirms the exact
pattern.

Test coverage: 13 new unit tests in deluxe.rs
  parse_method_arg_count: 3 tests (basic types, references, malformed)
  BindingDecoder: 4 tests (simple construction, with int pushes,
    skips unmatched invokespecial, resolves master-enum ordinal)
  interpret_streams: 4 tests (subtitle on no codec, audio on codec
    match, purpose routing, skips no-language)
  MasterEnumTable: 3 tests (resolve, value, class_name_set)
  extract_codec_name: 1 test (uppercase+underscore matching)

class_reader.rs gained a #[cfg(test)] ConstantPool::from_entries
test-only constructor so Phase D tests can build synthetic CP
fixtures without writing raw .class bytes.

Precommit (cargo +1.86 fmt + clippy + test) green.
2026-05-10 16:15:39 -07:00
MattJackson 7c6b0f82ab labels: per-parser confidence + highest-confidence-wins registry
Replaces 'first-match-wins by array order' with 'highest-confidence-
wins, array order tiebreaker'. Removes the arbitrariness when more
than one parser can claim a disc (e.g. one with both
bluray_project.bin and playlists.xml).

New types in labels::mod:
  pub enum Confidence { Medium, High }
  pub struct ParseResult { labels: Vec<StreamLabel>, confidence }
  ParseResult::high(labels) / ::medium(labels) constructors

Parser signature change: every parse() now returns
Option<ParseResult> instead of Option<Vec<StreamLabel>>. Updated all
six parsers in lockstep:
  paramount: High (fully structured XML)
  criterion: High (fully structured XML)
  pixelogic: High by default, Medium when an unknown token component
             is encountered (the skip-unknown path now propagates the
             coverage gap to the caller instead of silently degrading)
  ctrm:      High (structured key-value)
  dbp:       High (anchor scan with vocab routing)
  deluxe:    still returns None pending Phase D — signature aligned

Registry behavior:
  extract() iterates all detect-positive parsers, picks highest
  Confidence with non-empty labels. Equal confidence falls to array
  order (deterministic). Same selection logic in analyze().

LabelAnalysis grew a confidence: Option<Confidence> field so the
diagnostic surface (freemkv-tools labels-analyze) exposes which
confidence tier the selected parser claimed. labels-analyze JSON
and labels-corpus-check structural diff both gained the field.

Precommit (cargo +1.86 fmt + clippy + test) green.
2026-05-10 16:01:20 -07:00
MattJackson 4226a53e73 labels: fresh-eyes audit — capture variant, dedupe detect, lock registry
Three targeted fixes from a second-pass audit of the labels module.

1. vocab::lang now returns Option<LangInfo> with both code AND a
   human-readable variant string. Pre-fix: 'Brazilian Portuguese 5.1'
   became language=por, variant='', dropping the dialect info the
   disc had explicitly authored. Post-fix: language=por,
   variant='Brazilian' — matches the convention pixelogic / ctrm /
   criterion already use for their region variants. dbp now
   populates StreamLabel::variant from this. Compound table grew a
   3-tuple (needle, code, variant); bare matches still return
   variant=''.

2. dbp and deluxe had duplicated detect() boilerplate (any top-level
   .jar in /BDMV/JAR/). Both now call jar::has_any_top_level_jar.
   The trait-level detect contract — see super::PARSERS — can't peek
   inside a jar without a SectorReader, so loose-detect-plus-real-
   check-in-parse is the unavoidable pattern for jar-content parsers.
   Consolidating in jar.rs at least makes the duplication visible.

3. mod.rs comment about parser ordering said 'dbp last'; deluxe is
   actually now last. Updated to explain the dbp-before-deluxe order
   is by cost (cp-iteration cheaper than bytecode walking when Phase
   D lands).

Plus a registry-level lock test in mod.rs::registry_tests — asserts
the PARSERS array order is exactly [paramount, criterion, pixelogic,
ctrm, dbp, deluxe]. This was previously implicit; if someone reorders
the array (which changes which parser wins on overlapping signals),
unit tests would have stayed green. Now they fail with an explanatory
message about why the order matters.

Audit findings deferred to follow-ups (each its own commit + design
discussion):
- Stronger detect contract — current loose-detect-real-check pattern
  is forced by SectorReader-not-in-detect-signature; could be fixed
  by changing the trait to take an Option<&mut dyn SectorReader> or
  similar.
- Per-parser confidence scoring — registry currently first-match-wins.
  A high-confidence parser ought to beat a low-confidence one
  regardless of array order.
- class_reader fuzzing — handles malformed input via Result but no
  adversarial corpus yet.

Precommit (cargo +1.86 fmt + clippy + test) green.
2026-05-10 15:50:32 -07:00
MattJackson 92f34a289e labels: refactor pixelogic + ctrm onto shared platform + hardening
Closes the platform unification: every label parser now routes
purpose/qualifier/codec classification through one source of truth
(vocab.rs) instead of N hand-rolls, and every binary-blob byte
scanner goes through one helper (text::extract_ascii_strings).

pixelogic.rs:
- Drop local extract_strings (~20 lines) — use text::extract_ascii_strings.
- HARDENING: replace  with skip-unknown-component +
  trace log. Pre-refactor behavior: any single uncatalogued token part
  (e.g. a future codec ID, new framework variant) silently dropped the
  entire stream record. New behavior: skip just the unknown part,
  surface what we know about the stream.
- 8 new unit tests cover basic audio/subtitle paths, commentary,
  descriptive, region variant, the new skip-unknown-component
  regression, and the non-audio/non-subtitle early-out.

ctrm.rs:
- Replace  with
  vocab::purpose(&name). Now word-boundary matched — 'Commenter Pro
  Track' no longer false-matches Commentary.
- Replace  with vocab::qualifier(&name).
  Same word-boundary tightening, plus picks up Forced and
  DescriptiveService for free.
- Preserved structural commentary signal via
  as a fallback when name is silent (e.g. 'audio_commentary_1.name=Track 2').
- 6 new unit tests including the 'Commenter' false-positive regression
  and the SDH-only-on-subtitles boundary.

text.rs:
- Drop module-level dead_code allow now that pixelogic uses
  extract_ascii_strings.

Net: all 5 framework parsers now on the unified platform. Future work
(deluxe Phase D, paramount/criterion XML hardening) builds on the
same scaffolding.

Precommit green.
2026-05-10 15:33:22 -07:00
MattJackson fdbe469d50 labels: add deluxe parser (Phase A — master enum identification)
Closes the 'detected but no parser' gap on Deluxe-authored BD-J discs
(com/bydeluxe/ package signature; ~20% of UHD discs in our corpus per
the 2026-05-10 11-disc capture session).

What ships in this commit:

- detect() — registers the parser in the chain (loose pre-check at the
  /BDMV/JAR/ level; real signal in parse via has_path_prefix on
  'com/bydeluxe/').
- Phase A master enum identification — walks every .class's <clinit>
  ldc sequence and matches against framework-stable fingerprints for:
    Language    (70 ldcs starting English/French/Spanish/Dutch)
    Purpose     (8  ldcs starting Normal/Commentary/PiP/Trivia)
    VideoFormat (7  ldcs starting HD/HDR10 Plus/HD Dolby)
    Region      (22 ldcs starting USA_D1/LIC1/LIC2/LIC3 — Disney-only)
    Studio      (6  ldcs starting Disney/Marvel/Pixar — Disney-only)
  All identifications verified out-of-tree on corpus disc-01 (Disney,
  The Amateur) and disc-09 (Warner, Dune Part 1) via the standalone
  POC.
- Phase B structural skeleton (find_codec_enum) — identifies the
  codec enum class by structural signature (>=20 'new' ops, 0 ldcs),
  returns the ordered subclass list. Codec string extraction from
  subclasses is dead-coded pending the follow-up commit.

What does NOT ship yet:

- Phase D (per-stream binding-class decoder). parse() returns None
  intentionally — the master enums alone don't yield StreamLabels
  without the streamTable.put(...) bytecode walker. analyze() will
  show 'deluxe' in parsers_detected with the enum identification in
  tracing logs, so the analyzer reports honestly: 'detected, can't
  emit labels yet' rather than silent failure.

Why ship A without D: A is proven on real corpus discs; D's design
needs ground-truth binding bytecode from at least 2 corpus discs
side-by-side to verify the stack-machine pattern. The Phase A
infrastructure (master enum identification + ordinal->name table)
is what D will consume — landing it now unblocks D's design without
holding back the parser registration.

5 unit tests cover the fingerprint matcher logic + a roster lock that
forces explicit consideration when adding/removing fingerprints.

Precommit green.
2026-05-10 15:22:11 -07:00
MattJackson 4ec75a03f2 labels: shared platform (vocab/text/jar) + dbp refactor
Establishes the shared infrastructure layer for label parsers so that
Java-touching parsers (dbp, deluxe) don't reimplement jar walking and
all parsers route language/purpose/qualifier classification through
one source of truth instead of N hand-rolls.

New modules:

  vocab.rs       expanded from 27 -> ~370 lines
                 + lang(text) -> Option<&'static str>      (English/multi-word
                                                            -> ISO 639-2; ~45
                                                            languages, compound
                                                            phrases like
                                                            'Brazilian Portuguese'
                                                            and 'Castilian Spanish')
                 + purpose(text) -> LabelPurpose            (Commentary,
                                                            Descriptive, Score,
                                                            Ime; word-boundary
                                                            matched)
                 + qualifier(text) -> LabelQualifier        (SDH, Forced,
                                                            DescriptiveService)
                 + has_word internal primitive — enforces word-boundary
                   matching so 'Commenter' no longer matches 'commentary' and
                   'engineering' no longer matches 'english'. Existing parsers
                   used .contains() and got lucky on the corpus; vocab now
                   guarantees the boundary in one place. 20+ unit tests.

  text.rs        NEW (~85 lines)
                 + extract_ascii_strings(data, min_len) — promoted from two
                   near-duplicate copies (pixelogic min=4, dbp min=5);
                   threshold passed in. 7 unit tests including
                   trailing-without-terminator + high-bit-byte handling.

  jar.rs         NEW (~120 lines)
                 + for_each_jar(reader, udf, fn)  — walk every top-level
                                                    .jar under /BDMV/JAR/,
                                                    yield to callback.
                 + has_path_prefix(archive, prefix) — cheap 'is this MY
                                                      framework's jar?' check
                                                      via central-dir filenames.
                 + for_each_class(archive, fn)    — parse every .class entry
                                                    through class_reader,
                                                    yield (name, &ClassFile).
                 + try_each_class(archive, fn)    — same with early-return on
                                                    first Some(R) match.

Refactored:

  dbp.rs         v2 on the new platform:
                 - dropped extract_printable raw byte scan
                 - dropped its own English -> ISO 639-2 map
                 - dropped its own parse_attributes hand-roll
                 + iterates CpInfo::Utf8 via class_reader (structurally clean,
                   no false-positive risk from method bytecode bytes)
                 + routes language/purpose/qualifier through vocab
                 All 7 prior dbp tests still pass; +2 new ones cover
                 vocab routing.

dead-code allows on text.rs (extract_ascii_strings) and jar.rs
(try_each_class) come off when pixelogic and deluxe land — they're
staged for next steps.

Precommit green (cargo +1.86 fmt + clippy + test).
2026-05-10 15:16:25 -07:00
MattJackson 99236ffd55 labels: add class_reader, hand-rolled JVM .class file parser
Foundation for label parsers that need structured access to .class
files inside /BDMV/JAR/<x>.jar. Replaces noak (~3KLOC dep) with a
~1000-line std-only reader.

Public API:
- ClassFile::parse(&[u8]) -> Result<ClassFile>
- ConstantPool::{get, utf8, class_name, string, integer, member_ref, iter}
- Member::code(&pool) -> Option<CodeAttribute>
- CodeAttribute::instructions() -> Instructions iterator
- Instruction::{name, operand_u8, operand_u16, cp_index}
- Opcode constants (LDC, AASTORE, NEW, GETSTATIC, INVOKESPECIAL, ...)

Spec coverage:
- Constant pool: all 17 tag types incl. Long/Double 2-slot quirk
- Modified UTF-8 incl. 0xC0 0x80 -> U+0000 special case
- Bytecode iteration with full opcode size table
- Variable-length tableswitch / lookupswitch / wide

12 unit tests cover the opcode table edge cases (padded switch tables,
wide-iinc 6-byte form), modified-UTF-8 decoder, and iterator
stop-on-truncated behaviour.

Module is currently #![allow(dead_code)] — the public API is staged
for labels::deluxe (Phases A-E bytecode walker) and a labels::dbp
refactor onto the constant-pool iterator. Tests exercise the API
in isolation. The allow comes off as those callers land.

Also fixes two pre-existing clippy lints that 1.86's stricter checks
flagged after I touched the labels module:
- src/mux/disc.rs: while-let-loop in test fixture
- tests/pass_n_size_aware_skip.rs: type_complexity in helper signature

Precommit (cargo +1.86 fmt + clippy + test) green.
2026-05-10 15:06:34 -07:00
MattJackson 9e3327a569 v0.18.8: bump version (Pass-1 fast-skip never reached autorip 0.18.7 — Cargo.lock mis-resolved) 2026-05-10 14:01:23 -07:00
MattJackson 0081955686 labels: add dbp parser (Magnolia Pictures BD-J framework)
5th BD-J authoring framework recognized. Discriminator: any top-
level .jar in /BDMV/JAR/ that contains com/dbp/ package paths.
Identified during the 2026-05-10 corpus session via string-mining
disc-07's BD-J jar — perm files reference bd-live.magpictures.com
(Magnolia / Magnet Releasing).

Stream labels live as plain ASCII strings inside compiled .class
files in the form

    LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,...
    HTextField,Subtitle1,English SDH,Fontstrip_Composite,...
    ATextField,Subtitle0,None,Fontstrip_Composite,...

— a quirk of the menu-rendering layer encoding TextField positions
and content as constant strings the Java compiler retained in the
class string pool. The leading single-letter prefix is string-pool
ordering noise; parser anchors on `TextField,`. Subtitle0 is the
disable-subtitles button and is skipped.

The parser:
  - reads each top-level .jar via udf.read_file
  - opens it with the existing zip dependency
  - confirms com/dbp/ presence in the central directory
  - walks .class entries, extracts printable strings, matches the
    `TextField,(Audio|Subtitle)<N>,<label>,...` pattern
  - maps human-readable language names ("English", "Castilian
    Spanish", "Brazilian Portuguese", "Canadian French", ...) to
    ISO 639-2 codes via a parser-local table (per the rules-of-
    engagement memo, each parser knows its own format)
  - preserves the full disc-authored label string in `name` so
    consumers display it raw without the lib guessing further
    structure
  - detects SDH / Forced qualifiers and Commentary / Descriptive
    purposes from substring matches; everything else falls through
    to fill_defaults using BD-spec MPLS data

Verified live on the corpus: disc-07 (Civil War UHD) now matches
parser=dbp with 3 audio + 2 subtitle labels, exactly the count
visible in the disc's authored TextField definitions and what BD
spec MPLS reports.

Limitation: dbp's detect() returns true for ANY top-level .jar in
/BDMV/JAR/ (every BD-J disc has one), since the discriminator
trait function takes only `&UdfFs` and can't read jar contents.
parse() does the real com/dbp/ check — a non-dbp disc gets
parsed-as-dbp, archive_has_dbp returns false, parse() returns
None, and we fall through. Diagnostic noise: parsers_detected
includes "dbp" on non-dbp BD-J discs. Real fix is refactoring the
DetectFn signature to take a SectorReader; deferred.

7 unit tests cover the TextField extraction, language detection
(simple + compound: "Brazilian Portuguese", "Castilian Spanish",
"Canadian French", "Latin American Spanish", "Australian English"
plus the disc-corpus typo "Austrailian English"), SDH/Forced/RNIB
qualifier detection, and Commentary/Descriptive purpose detection.
Don't-guess discipline preserved: unknown languages return ""
(consumer falls back to MPLS spec data via fill_defaults).
2026-05-10 13:15:07 -07:00
MattJackson 3244fdd683 v0.18.7: Pass 1 fast-skip, defer recovery to Pass N
Pass 1 sweep was grinding through damage zones because the marginal-
media handler returned `Bisect` for every failed 32-sector batch —
forcing 32 single-sector reads per bad block at ~5s each on a real
BU40N-vs-Dune-Pt-2 trace. AND the JumpAhead trigger required a 16-
block damage window to fill before firing, so entry into a
contiguous damage zone took ~40 minutes of grinding before the
first jump fired. Architecturally wrong: Pass 1's job is "fast and
accurate, get the most data in the shortest time." Bisection +
recovery is Pass N's purpose-built role.

ReadCtx now carries two new fields:
  - `consecutive_outer_failures: u64` — outer-batch failures since
    last outer success. Bisect inner failures don't count.
  - `bisect_on_marginal: bool` — whether to return Bisect on a
    marginal-media batch failure.
  - `fast_jump_threshold: u64` — outer-failures count that triggers
    JumpAhead before the damage window has filled.

`for_sweep` (Pass 1) sets `bisect_on_marginal=false`,
`fast_jump_threshold=4`, and zeroes the post-failure pause. Failed
batches become SkipBlock → whole block NonTrimmed → advance, no
sleep. After 4 consecutive outer failures: JumpAhead with the
existing escalating multiplier.

`for_patch` (Pass N) sets `bisect_on_marginal=true`,
`fast_jump_threshold=u64::MAX`, keeps the original cooldown pauses.
Pass N's whole reason to exist is to grind on bad ranges with
proper recovery semantics — single-sector reads, 60s recovery
timeout, retry budget, escalating skip — and that's unchanged.

`on_success` resets `consecutive_outer_failures` only when not
bisecting, so a good single-sector read inside Pass N's bisect
doesn't pretend we've escaped the damaged batch.

Tests:
  - `pass_n_marginal_with_batch_gt_1_bisects` — Pass N still bisects.
  - `pass_1_marginal_skips_instead_of_bisecting` — Pass 1 doesn't.
  - `pass_1_jumps_after_4_consecutive_outer_failures` — fast-entry.
  - `pass_n_does_not_fast_jump` — fast-entry is Pass-1-only.
  - `outer_success_resets_consecutive_outer_failures` — counter reset.
  - `bisect_inner_success_does_not_reset_outer_counter` — semantics.
  - `pass_1_does_not_pause_on_skip` — explicit zero-pause contract.
  - `long_failure_streak_extends_pause_on_pass_n` — Pass N still
    extends pauses on long failure streaks (renamed from the old
    sweep-based test).

Integration test `test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed`
updated: it used to assert Pass 1 recovers all sectors via bisect
(bytes_good=total). New contract: Pass 1 marks NonTrimmed; Pass N
recovers. Test now asserts Pass-1-only outcome (bytes_pending=total,
complete=false) consistent with the redesign.

Real-world impact on the user's BU40N + Dune Pt 2 trace from this
session: a damage zone that was on track to take ~40 minutes of
Pass-1 grinding will now jump in ~20 seconds. Pass N still has the
full 7-pass recovery budget to revisit those NonTrimmed ranges.
2026-05-10 12:50:12 -07:00
MattJackson ef5487e5a5 labels: distinguish "parser detected" from "parser succeeded"
Adds `parsers_detected: Vec<&'static str>` to `LabelAnalysis`. Records
every parser whose discriminator matched, regardless of whether its
parse() then returned Some/None.

Why: when `parser=None` we currently can't tell apart:
  (a) no parser recognized this disc — missing parser, candidate for a
      new module
  (b) a parser recognized it but parse() returned None / empty —
      capture truncated, or genuine empty authoring data, or a parser
      bug

Surfaced concretely on the 11-disc capture session 2026-05-10:
disc-04 had `bluray_project.bin` in jar_inventory (pixelogic detect()
returned true) but parse() returned None because file content was
past the 1 GB capture window. Old API: parser=None — looked like
"missing parser." New API: detected=[pixelogic], parser=None — clearly
"capture problem, not a parser gap."

The tracing log line on the no-parser-emitted-labels path now
distinguishes the two cases too.

No behavior change to production rip path. extract() is unchanged;
only the diagnostic analyze() returns the richer result.
2026-05-10 12:36:05 -07:00
MattJackson e7dffa63d2 v0.18.6: bump version (unified release with bdemu/freemkv/autorip) 2026-05-10 10:03:03 -07:00
MattJackson e5989c8270 labels: expose analyze() for corpus regression tooling
Promotes `mod labels` to `pub mod labels` and adds `analyze()` plus
`LabelAnalysis` (both `#[doc(hidden)]`) so an out-of-tree diagnostic
binary (freemkv-tools labels-analyze) can introspect which BD-J parser
matched a given disc, what JAR files the discriminators saw, and what
labels came out — without going through the production `apply()` path
that mutates DiscTitles.

Also adds `tracing::info!(parser = name, "label parser matched")` /
"no label parser matched" inside `extract()`. Dev-only signal: users
get the same seamless behavior; developers can finally tell whether a
disc hit a real parser or fell through to the codec-name fallback in
fill_defaults().

The new `jar_inventory()` helper deduplicates and sorts filenames
under any `/BDMV/JAR/<x>/` subdirectory — same plumbing the existing
`jar_file_exists()` discriminators use, just enumerated rather than
predicate-tested. Used by `analyze()` to surface unrecognized
parser-source files when no parser matches, which is the input to
"do we need a new parser?" triage.

No behavior change to the production label path. `apply()` and
`extract()` remain functionally identical; the new public surface
exists alongside.
2026-05-10 07:42:45 -07:00
MattJackson e1c938b82a ci: drop --locked from libfreemkv workflows
libfreemkv is a library — Cargo.lock is gitignored (standard for
libs). --locked refuses to create a lockfile on a fresh runner,
so it always fails CI. --locked stays in the binary crates
(freemkv, autorip, bdemu) which DO track Cargo.lock and benefit
from the dependency-race hard-fail behaviour.
2026-05-09 20:40:31 -07:00
MattJackson ed801a708b fmt: rustfmt-mandated reflow of canonical_title_order tests
The 0.18.4 commit landed with rustfmt diffs in the new
canonical_order tests because my local validation script piped
'cargo fmt --check' to 'tail -1', masking the diff output and
reporting green when fmt was actually unhappy. CI's lint job
caught it immediately. No code change — pure formatting.
2026-05-09 20:34:40 -07:00
MattJackson d94451954c v0.18.4: cargo --locked everywhere — hard-fail dependency races 2026-05-09 20:30:34 -07:00
22 changed files with 5775 additions and 235 deletions
+4
View File
@@ -14,6 +14,10 @@ jobs:
with:
components: clippy, rustfmt
- run: cargo fmt --check
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
- run: cargo clippy -- -D warnings
test:
+2
View File
@@ -28,6 +28,8 @@ jobs:
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
- run: cargo test
publish:
+18
View File
@@ -1,5 +1,23 @@
# Changelog
## 0.18.4 (2026-05-09)
### Build / CI hardening — no library code changes
- All `cargo` invocations in `.github/workflows/*.yml` now use
`--locked`. Previously `cargo build` / `cargo test` could silently
re-resolve `Cargo.lock` if a lock-pinned version wasn't yet on
crates.io — masking dependency races between same-tag releases of
sibling crates. The 0.18.3 release hit this: autorip's docker image
built `libfreemkv v0.18.2` because libfreemkv 0.18.3 hadn't
published yet at the time autorip's CI ran, and cargo silently fell
back to the previous version. With `--locked`, that race now hard-
fails the build with a clear "package X is not in registry" error
and we retrigger after the upstream lands.
- `cargo publish --locked` in the release workflow guards against
publishing a lockfile-mismatched release.
## 0.18.3 (2026-05-09)
### Behaviour change
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "0.18.3"
version = "0.18.9"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
+56 -15
View File
@@ -1634,9 +1634,7 @@ impl Disc {
match action {
read_error::ReadAction::Retry { pause_secs } => {
if pause_secs > 0 {
std::thread::sleep(std::time::Duration::from_secs(pause_secs));
}
sleep_secs_or_halt(pause_secs, opts.halt.as_ref());
}
read_error::ReadAction::Bisect => {
read_ctx.bisecting = true;
@@ -1713,9 +1711,7 @@ impl Disc {
break 'outer;
}
bytes_done = bytes_done.saturating_add(block_bytes);
if pause_secs > 0 {
std::thread::sleep(std::time::Duration::from_secs(pause_secs));
}
sleep_secs_or_halt(pause_secs, opts.halt.as_ref());
pos += block_bytes;
}
read_error::ReadAction::JumpAhead {
@@ -1770,9 +1766,7 @@ impl Disc {
"damage-jump"
);
pos = jump_pos;
if pause_secs > 0 {
std::thread::sleep(std::time::Duration::from_secs(pause_secs));
}
sleep_secs_or_halt(pause_secs, opts.halt.as_ref());
}
read_error::ReadAction::AbortPass => {
let (status, sense) = extract_scsi_context(&err);
@@ -1974,6 +1968,34 @@ pub struct PatchOutcome {
pub wedged_threshold: u64,
}
/// Sleep `secs` seconds, but break early if `halt` flips to true.
/// Used by Pass 1's wedge-avoidance inter-error pause so halt
/// remains responsive regardless of how long the pause is.
/// Polling granularity 100 ms — bounded latency on halt regardless
/// of pause length.
pub(crate) fn sleep_secs_or_halt(
secs: u64,
halt: Option<&std::sync::Arc<std::sync::atomic::AtomicBool>>,
) {
if secs == 0 {
return;
}
let Some(h) = halt else {
std::thread::sleep(std::time::Duration::from_secs(secs));
return;
};
let total = std::time::Duration::from_secs(secs);
let slice = std::time::Duration::from_millis(100);
let start = std::time::Instant::now();
while start.elapsed() < total {
if h.load(std::sync::atomic::Ordering::Relaxed) {
return;
}
let remaining = total.saturating_sub(start.elapsed());
std::thread::sleep(remaining.min(slice));
}
}
pub fn mapfile_path_for(iso_path: &std::path::Path) -> std::path::PathBuf {
let mut s = iso_path.as_os_str().to_os_string();
s.push(".mapfile");
@@ -3290,7 +3312,12 @@ mod tests {
/// Build a DiscTitle with full control over the fields the title
/// sorter cares about. Used by the canonical-title-order tests.
fn title_with(playlist: &str, duration_secs: f64, size_bytes: u64, n_clips: usize) -> DiscTitle {
fn title_with(
playlist: &str,
duration_secs: f64,
size_bytes: u64,
n_clips: usize,
) -> DiscTitle {
let mut t = title_with_video(Codec::Hevc, Resolution::R2160p);
t.playlist = playlist.into();
t.duration_secs = duration_secs;
@@ -3317,13 +3344,24 @@ mod tests {
const CAPACITY: u64 = 58_500_000_000; // 58.5 GB
let mut titles = vec![
// Title 1 in the raw MPLS order — virtual play-all
title_with("00020.mpls", 4.0 * 3600.0 + 13.0 * 60.0, 92_400_000_000, 253),
title_with(
"00020.mpls",
4.0 * 3600.0 + 13.0 * 60.0,
92_400_000_000,
253,
),
// Title 2 — actual movie
title_with("00800.mpls", 2.0 * 3600.0 + 2.0 * 60.0, 57_200_000_000, 1),
];
titles.sort_by(|a, b| Disc::canonical_title_order(a, b, CAPACITY));
assert_eq!(titles[0].playlist, "00800.mpls", "main feature should land at index 0");
assert_eq!(titles[1].playlist, "00020.mpls", "virtual play-all should be pushed back");
assert_eq!(
titles[0].playlist, "00800.mpls",
"main feature should land at index 0"
);
assert_eq!(
titles[1].playlist, "00020.mpls",
"virtual play-all should be pushed back"
);
}
/// Non-branching disc: longest 1-clip title is the movie. Sort
@@ -3333,12 +3371,15 @@ mod tests {
fn canonical_order_preserves_natural_ranking_on_normal_disc() {
const CAPACITY: u64 = 60_000_000_000;
let mut titles = vec![
title_with("00100.mpls", 600.0, 5_000_000_000, 1), // 10 min menu
title_with("00100.mpls", 600.0, 5_000_000_000, 1), // 10 min menu
title_with("00800.mpls", 7320.0, 55_000_000_000, 1), // 2h02m main feature
title_with("00200.mpls", 1800.0, 2_000_000_000, 1), // 30 min extra
];
titles.sort_by(|a, b| Disc::canonical_title_order(a, b, CAPACITY));
assert_eq!(titles[0].playlist, "00800.mpls", "longest valid title still wins");
assert_eq!(
titles[0].playlist, "00800.mpls",
"longest valid title still wins"
);
assert_eq!(titles[1].playlist, "00200.mpls");
assert_eq!(titles[2].playlist, "00100.mpls");
}
+447 -24
View File
@@ -25,11 +25,25 @@ pub struct ReadCtx {
/// Failed reads since the last success. Resets to 0 on success.
/// Drives long-pause escalation on persistent failure.
pub consecutive_failures: u64,
/// Failed OUTER batch reads since the last outer success — bisect
/// inner-sector failures are NOT counted here. Drives the
/// fast-entry damage-jump on Pass 1 (skip the disc-level grind
/// once we're clearly in a damaged region; Pass N will recover
/// the actual sectors). Reset on outer success.
pub consecutive_outer_failures: u64,
/// Sliding window of recent read outcomes (true=ok, false=fail).
/// Capped at `damage_window_max`. Drives damage-jump decisions.
pub damage_window: Vec<bool>,
pub damage_window_max: usize,
pub damage_threshold_pct: usize,
/// Trigger a damage-jump after this many consecutive outer-batch
/// failures, even when the damage_window isn't full yet. Pass 1
/// uses a small value (4) so we don't spend ~40 minutes grinding
/// to fill a 16-block window before the first jump on a damage
/// zone we entered cleanly. Pass N uses a larger value (or
/// disables this — see `bisect_on_marginal`) because Pass N's
/// whole job IS to grind on the bad ranges.
pub fast_jump_threshold: u64,
/// Multiplier applied to damage-jump distance. Doubles each jump,
/// resets to 1 after `damage_window_max` consecutive good reads.
pub jump_multiplier: u64,
@@ -43,38 +57,73 @@ pub struct ReadCtx {
/// failed batch, so the handler doesn't recursively request another
/// bisect on the inner-sector failures.
pub bisecting: bool,
/// Whether to return `Bisect` on a marginal-media batch failure.
/// Pass 1 sweep sets this false: a failed batch becomes
/// SkipBlock (mark the whole 32-sector ECC block NonTrimmed,
/// advance, let Pass N recover the salvageable sectors with
/// proper recovery semantics). Pass N sets this true: bisection
/// is its core job, and it has the right tools (single-sector
/// reads, 60s recovery timeout, retry budget, escalating skip).
pub bisect_on_marginal: bool,
/// Count of consecutive firmware-wedge responses (HARDWARE_ERROR
/// or ILLEGAL_REQUEST sense keys) since the last successful read.
/// Pass 1 uses this to drive the wedge-skip path: each wedge
/// triggers a 1 GB jump + cooldown pause. Reaching
/// `WEDGE_ABORT_THRESHOLD` consecutive wedges with no good read
/// in between → real AbortPass.
pub wedge_count: u64,
}
impl ReadCtx {
/// Initial context for a Pass 1 sweep with the documented constants.
/// Initial context for a Pass 1 sweep. The job is "fast and
/// accurate, get the most data in the shortest time" — Pass N
/// is the one that grinds on the bad ranges. So bisect-on-
/// marginal is OFF (failed batches become SkipBlock; whole 32-
/// sector blocks marked NonTrimmed for Pass N to revisit), and
/// the damage-jump fast-path triggers after just 4 consecutive
/// outer-batch failures.
pub fn for_sweep(batch: u16) -> Self {
Self {
batch,
consecutive_good: 0,
consecutive_failures: 0,
consecutive_outer_failures: 0,
damage_window: Vec::with_capacity(16),
damage_window_max: 16,
damage_threshold_pct: 12,
fast_jump_threshold: 4,
jump_multiplier: 1,
not_ready_retries: 0,
bridge_degradation_count: 0,
bisecting: false,
bisect_on_marginal: false,
wedge_count: 0,
}
}
/// Initial context for a Pass 2-N patch.
/// Initial context for a Pass 2-N patch. Pass N's whole reason to
/// exist is to recover sectors Pass 1 skipped — bisection on
/// marginal media is part of the job, and the fast-jump
/// threshold is loose so we don't bail too early on a range that
/// has scattered good sectors mixed in.
pub fn for_patch(batch: u16) -> Self {
Self {
batch,
consecutive_good: 0,
consecutive_failures: 0,
consecutive_outer_failures: 0,
damage_window: Vec::with_capacity(16),
damage_window_max: 16,
damage_threshold_pct: 12,
// Pass N is allowed to grind: window-based jump only,
// matching the historical behaviour for patch passes.
fast_jump_threshold: u64::MAX,
jump_multiplier: 1,
not_ready_retries: 0,
bridge_degradation_count: 0,
bisecting: false,
bisect_on_marginal: true,
wedge_count: 0,
}
}
@@ -83,6 +132,16 @@ impl ReadCtx {
self.consecutive_good += 1;
self.consecutive_failures = 0;
self.not_ready_retries = 0;
// Any successful read clears the wedge-skip counter — the
// drive recovered, so further wedges should reset the skip
// budget instead of accumulating toward a real abort.
self.wedge_count = 0;
// Outer-success only: a good single-sector read inside a
// bisect doesn't mean we've left the damaged batch. Only an
// outer-batch success resets the outer-failure counter.
if !self.bisecting {
self.consecutive_outer_failures = 0;
}
self.damage_window.push(true);
if self.damage_window.len() > self.damage_window_max {
self.damage_window.remove(0);
@@ -128,6 +187,63 @@ const NOT_READY_MAX_RETRIES: u32 = 3;
const BRIDGE_DEGRADATION_PAUSE_SECS: u64 = 15;
const BRIDGE_DEGRADATION_MAX_RETRIES: u32 = 5;
/// Pass 1 inter-error pause. ZERO on the clean path (sweep zooms
/// past damage windows) but applied to every failed read so the
/// drive's firmware gets cool-down time between damage-cluster
/// exposures. Wedge-avoidance change 2026-05-10: pre-fix `pause_secs
/// = 0` on Pass 1 errors caused back-to-back exposures that
/// accumulated firmware wedge state.
const PASS_1_FAIL_PAUSE_SECS: u64 = 5;
/// Base of the damage-jump distance formula: `jump_sectors =
/// JUMP_BASE_SECTORS × batch × jump_multiplier`. Bumped 2026-05-10
/// from 256 → 1024 (4×) so the first damage-jump at batch=32 covers
/// 64 MB instead of 16 MB. Empirically the BU40N's damage clusters
/// are 100+ MB wide; 16 MB jumps landed inside the cluster and the
/// re-read added to the firmware wedge counter. 64 MB → 128 MB
/// (after one doubling) clears almost any single-cluster damage in
/// 2 jumps.
const JUMP_BASE_SECTORS: u64 = 1024;
// Firmware-wedge skip policy for Pass 1 sweep
// ===========================================
//
// When the BU40N (or similar drives) hits a physical-damage cluster,
// its firmware can transition into a "wedge" state where it returns
// HARDWARE_ERROR or ILLEGAL_REQUEST for every subsequent read —
// often for many LBAs after the actual bad sector. Per project docs
// "Bad-sector handling" rule #2: "Recovery requires eject+reload OR
// significant cool-down."
//
// Pass 1's pre-fix behavior was to immediately AbortPass on the
// first HARDWARE_ERROR / ILLEGAL_REQUEST, killing the rip at
// whatever percentage it had reached. That's the wrong call when:
// - the damage zone may be small (jumping past it could resume
// normal reads), AND
// - even if the drive stays wedged, finishing the sweep gives us
// an honest mapfile for Pass N to attack later.
//
// New policy: treat wedge sense codes the same way the damage-window
// treats persistent failure — JumpAhead by a large distance with a
// cooldown pause. Allow up to WEDGE_ABORT_THRESHOLD consecutive
// wedges (no successful read in between) before declaring the drive
// truly stuck and surfacing AbortPass to autorip.
/// One-gigabyte jump (1024 MiB) on each wedge. Big enough to clear
/// almost any single-cluster damage zone we've seen.
const WEDGE_JUMP_SECTORS: u64 = 524_288;
/// Cooldown pause after each wedge. Per project docs the drive needs
/// "significant cool-down"; 30 s strikes a balance between giving
/// the drive a chance to recover and not stalling the rip if the
/// drive is permanently stuck.
const WEDGE_PAUSE_SECS: u64 = 30;
/// Bail after this many consecutive wedges with no good read in
/// between. At 1 GB jumps this lets us scan ~16 GB worth of fully
/// wedged area before giving up — generous enough to clear most
/// physical-damage clusters, bounded enough to not loop forever on
/// a permanently bricked drive.
const WEDGE_ABORT_THRESHOLD: u64 = 16;
/// THE single error-handling entry point. Updates `ctx`, returns the
/// action the caller must apply.
///
@@ -137,6 +253,13 @@ const BRIDGE_DEGRADATION_MAX_RETRIES: u32 = 5;
pub fn handle_read_error(err: &Error, ctx: &mut ReadCtx) -> ReadAction {
ctx.consecutive_failures += 1;
ctx.consecutive_good = 0;
// Outer-failure counter — only OUTER batch failures count toward
// the fast-jump trigger. Bisect inner failures are part of
// recovering an already-failed batch and don't represent
// independent damage signal.
if !ctx.bisecting {
ctx.consecutive_outer_failures += 1;
}
tracing::warn!(
target: "freemkv::disc",
@@ -186,11 +309,54 @@ pub fn handle_read_error(err: &Error, ctx: &mut ReadCtx) -> ReadAction {
ctx.not_ready_retries = 0;
}
// 4. Hardware / illegal request: the drive said "no, won't do it".
// Retrying won't change the answer. Surface to the outer layer
// so it can eject + prompt user.
// 4. Hardware error / illegal request — the firmware-wedge family.
// The drive transitioned into a fast-fail state where it
// rejects reads near the LBA. Two policies:
//
// Pass 1 sweep (bisect_on_marginal=false): the wedge is
// *recoverable* by skipping. Jump a large distance ahead
// (1 GB), pause for cooldown, mark the skipped region
// NonTrimmed so Pass N revisits. Allow up to
// WEDGE_ABORT_THRESHOLD consecutive wedges before truly
// giving up. This replaces the pre-fix "abort the entire
// rip on first wedge" behavior that caused 48%-and-die
// failures on discs with one bad cluster.
//
// Pass N patch (bisect_on_marginal=true): Pass N's job IS
// single-sector recovery; a wedge means the drive won't
// give us the specific sectors we asked for. Skipping
// doesn't help here. Abort and let autorip decide whether
// to retry, eject, or surface the failure to the user.
if sense_key == scsi::SENSE_KEY_HARDWARE_ERROR || sense_key == scsi::SENSE_KEY_ILLEGAL_REQUEST {
return ReadAction::AbortPass;
if ctx.bisect_on_marginal {
return ReadAction::AbortPass;
}
// Pass 1 wedge-skip path.
if !ctx.bisecting {
ctx.wedge_count += 1;
}
if ctx.wedge_count >= WEDGE_ABORT_THRESHOLD {
tracing::warn!(
target: "freemkv::disc",
phase = "wedge_abort",
wedge_count = ctx.wedge_count,
threshold = WEDGE_ABORT_THRESHOLD,
"Pass 1 wedge-skip exhausted — drive appears permanently stuck"
);
return ReadAction::AbortPass;
}
tracing::warn!(
target: "freemkv::disc",
phase = "wedge_skip",
wedge_count = ctx.wedge_count,
jump_sectors = WEDGE_JUMP_SECTORS,
pause_secs = WEDGE_PAUSE_SECS,
"Pass 1 wedge detected — jumping ahead and pausing for drive cooldown"
);
return ReadAction::JumpAhead {
sectors: WEDGE_JUMP_SECTORS,
pause_secs: WEDGE_PAUSE_SECS,
};
}
// 5. Marginal media (MEDIUM_ERROR / ABORTED_COMMAND) on a multi-
@@ -198,11 +364,20 @@ pub fn handle_read_error(err: &Error, ctx: &mut ReadCtx) -> ReadAction {
// individually. Bisect into single-sector reads (gentler on the
// bridge too — shorter SCSI transactions). Avoid recursive
// bisect.
//
// Pass 1 sweep sets `bisect_on_marginal=false` to skip this:
// its job is "fast and accurate, get the most data in the
// shortest time." Pass N is purpose-built to recover
// individual sectors with proper recovery semantics, and Pass
// 1 grinding through 32-sector bisects costs ~2.5 min per bad
// block AND fills the damage window slower than it should.
// Whole-block NonTrimmed → SkipBlock → advance → Pass N
// revisits.
let is_marginal = matches!(
sense_key,
scsi::SENSE_KEY_MEDIUM_ERROR | scsi::SENSE_KEY_ABORTED_COMMAND
);
if is_marginal && ctx.batch > 1 && !ctx.bisecting {
if is_marginal && ctx.batch > 1 && !ctx.bisecting && ctx.bisect_on_marginal {
return ReadAction::Bisect;
}
@@ -228,22 +403,77 @@ pub fn handle_read_error(err: &Error, ctx: &mut ReadCtx) -> ReadAction {
bad_count * 100 / ctx.damage_window.len()
};
let pause_secs = if ctx.consecutive_failures >= CONSECUTIVE_FAIL_LONG_PAUSE_THRESHOLD {
// Inter-error pause on Pass 1 — wedge avoidance.
//
// Pre-2026-05-10 Pass 1 ran `pause_secs = 0` on all errors so
// sweep would zoom past damage zones in seconds instead of
// minutes. Empirically on the BU40N this caused firmware-wedge
// events: each read failure leaves residual state in the drive's
// firmware, and back-to-back errors without cooldown accumulate
// toward a wedge threshold. We hit wedge on Dune Pt 2 after 5
// errors in 43 s spread over a 140 MB damage cluster — even
// though wall-clock pacing was slow, the drive had no breathing
// room between exposures.
//
// New policy: Pass 1 keeps `pause = 0` on the CLEAN path
// (successful reads zoom past damage windows fine), but every
// failed read takes a PASS_1_FAIL_PAUSE_SECS pause before the
// next read. Cost: ~5 s extra per scattered failure, ~30-60 s
// total in a damage cluster — trivial compared to the alternative
// of crashing the whole rip at 48%. Long failure streaks still
// escalate via CONSECUTIVE_FAIL_LONG_PAUSE.
let pause_secs = if !ctx.bisect_on_marginal {
if ctx.consecutive_failures >= CONSECUTIVE_FAIL_LONG_PAUSE_THRESHOLD {
CONSECUTIVE_FAIL_LONG_PAUSE_SECS
} else {
PASS_1_FAIL_PAUSE_SECS
}
} else if ctx.consecutive_failures >= CONSECUTIVE_FAIL_LONG_PAUSE_THRESHOLD {
CONSECUTIVE_FAIL_LONG_PAUSE_SECS
} else {
POST_FAILURE_PAUSE_SECS
};
// 7. Damage-jump: too many failures in window → skip ahead by an
// escalating gap. Multiplier capped so we can't accidentally
// skip the entire rest of the disc (observed 2026-05-07: a
// saturated multiplier produced a 56 GB jump). Saturating
// arithmetic on the sector calc as defence in depth.
const MAX_JUMP_MULTIPLIER: u64 = 64; // 64 × 256 × batch sectors
if ctx.damage_window.len() >= ctx.damage_window_max && bad_pct >= ctx.damage_threshold_pct {
// 7. Damage-jump: too many failures → skip ahead by an escalating
// gap. Multiplier capped so we can't accidentally skip the
// entire rest of the disc (observed 2026-05-07: a saturated
// multiplier produced a 56 GB jump). Saturating arithmetic on
// the sector calc as defence in depth.
//
// Jump base bumped 2026-05-10 from 256 to 1024 sectors per
// multiplier unit (= 64 MB first jump at batch=32, up from
// 16 MB). The smaller base routinely landed jumps back inside
// damage clusters of 100+ MB, each landing adding to the
// firmware's wedge counter. 64 MB initial + 128 MB second +
// 256 MB third clears almost any single-cluster damage
// pattern we've seen in 2 jumps.
//
// Two triggers, evaluated in order:
//
// a. **Fast-entry** — `consecutive_outer_failures >= fast_jump_threshold`.
// Fires on Pass 1 (threshold=4) so we don't spend ~40 min
// grinding to fill a 16-block damage window before the
// first jump on a damage zone we entered cleanly. Doesn't
// fire on Pass N (threshold=u64::MAX).
//
// b. **Window-based** — original behaviour: 12% bad in a
// sliding window of 16 outer reads. Pass N's only path,
// and Pass 1's fallback if the failures are scattered
// enough that we don't hit the consecutive threshold.
const MAX_JUMP_MULTIPLIER: u64 = 64;
let fast_trigger = !ctx.bisecting && ctx.consecutive_outer_failures >= ctx.fast_jump_threshold;
let window_trigger =
ctx.damage_window.len() >= ctx.damage_window_max && bad_pct >= ctx.damage_threshold_pct;
if fast_trigger || window_trigger {
let mult = ctx.jump_multiplier.min(MAX_JUMP_MULTIPLIER);
let sectors = 256u64.saturating_mul(ctx.batch as u64).saturating_mul(mult);
let sectors = JUMP_BASE_SECTORS
.saturating_mul(ctx.batch as u64)
.saturating_mul(mult);
ctx.jump_multiplier = (ctx.jump_multiplier.saturating_mul(2)).min(MAX_JUMP_MULTIPLIER);
// Reset the outer-failure counter so a long damaged region
// doesn't keep firing fast-jump every read after the initial
// jump fired. The window-based trigger handles further jumps.
ctx.consecutive_outer_failures = 0;
return ReadAction::JumpAhead {
sectors,
pause_secs: pause_secs + POST_JUMP_EXTRA_PAUSE_SECS,
@@ -285,16 +515,41 @@ mod tests {
}
}
fn illegal_request_err() -> Error {
Error::DiscRead {
sector: 100,
status: Some(2),
sense: Some(ScsiSense {
sense_key: scsi::SENSE_KEY_ILLEGAL_REQUEST,
asc: 0x24,
ascq: 0x00,
}),
}
}
#[test]
fn medium_error_with_batch_gt_1_bisects() {
let mut ctx = ReadCtx::for_sweep(32);
fn pass_n_marginal_with_batch_gt_1_bisects() {
let mut ctx = ReadCtx::for_patch(32);
let action = handle_read_error(&medium_err(), &mut ctx);
assert_eq!(action, ReadAction::Bisect);
}
#[test]
fn pass_1_marginal_skips_instead_of_bisecting() {
// Pass 1's job is "fast and accurate" — leave bisection to
// Pass N. A failed batch becomes SkipBlock (whole 32-sector
// block marked NonTrimmed for Pass N to revisit).
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&medium_err(), &mut ctx);
match action {
ReadAction::SkipBlock { .. } => {}
other => panic!("expected SkipBlock for Pass 1, got {other:?}"),
}
}
#[test]
fn medium_error_with_batch_1_skips() {
let mut ctx = ReadCtx::for_sweep(1);
let mut ctx = ReadCtx::for_patch(1);
let action = handle_read_error(&medium_err(), &mut ctx);
match action {
ReadAction::SkipBlock { pause_secs } => assert!(pause_secs >= 1),
@@ -304,7 +559,7 @@ mod tests {
#[test]
fn medium_error_while_bisecting_does_not_recurse() {
let mut ctx = ReadCtx::for_sweep(32);
let mut ctx = ReadCtx::for_patch(32);
ctx.bisecting = true;
let action = handle_read_error(&medium_err(), &mut ctx);
match action {
@@ -314,19 +569,165 @@ mod tests {
}
#[test]
fn hardware_error_aborts() {
fn pass_1_jumps_after_4_consecutive_outer_failures() {
// The fast-entry trigger: Pass 1 should JumpAhead after 4
// consecutive outer-batch failures, BEFORE the 16-block
// damage window has filled. Otherwise we spend ~40 minutes
// of bisecting/grinding to fill the window before the first
// jump on a damage zone we entered cleanly.
let mut ctx = ReadCtx::for_sweep(32);
// First three should NOT jump (still under threshold of 4).
for _ in 0..3 {
let a = handle_read_error(&medium_err(), &mut ctx);
assert!(
!matches!(a, ReadAction::JumpAhead { .. }),
"should not jump until 4 consecutive outer failures"
);
}
// Fourth should jump.
let a = handle_read_error(&medium_err(), &mut ctx);
assert!(
matches!(a, ReadAction::JumpAhead { .. }),
"expected JumpAhead at 4th consecutive outer failure, got {a:?}"
);
}
#[test]
fn pass_n_does_not_fast_jump() {
// Pass N's whole reason to exist is to grind on bad ranges.
// It should NOT bail after 4 consecutive failures the way
// Pass 1 does — it bisects and skips with proper recovery.
let mut ctx = ReadCtx::for_patch(32);
for _ in 0..4 {
let a = handle_read_error(&medium_err(), &mut ctx);
assert!(
!matches!(a, ReadAction::JumpAhead { .. }),
"Pass N must not fast-jump; got {a:?}"
);
}
}
#[test]
fn outer_success_resets_consecutive_outer_failures() {
let mut ctx = ReadCtx::for_sweep(32);
for _ in 0..3 {
handle_read_error(&medium_err(), &mut ctx);
}
assert_eq!(ctx.consecutive_outer_failures, 3);
// An outer-success (bisecting=false) should reset the counter.
ctx.bisecting = false;
ctx.on_success();
assert_eq!(ctx.consecutive_outer_failures, 0);
}
#[test]
fn bisect_inner_success_does_not_reset_outer_counter() {
let mut ctx = ReadCtx::for_patch(32);
for _ in 0..3 {
handle_read_error(&medium_err(), &mut ctx);
}
assert_eq!(ctx.consecutive_outer_failures, 3);
// A successful inner-sector read during bisect is not the
// same as escaping the bad outer batch.
ctx.bisecting = true;
ctx.on_success();
assert_eq!(
ctx.consecutive_outer_failures, 3,
"bisect inner success must not reset outer-failure counter"
);
}
#[test]
fn pass_1_hardware_error_jumps_ahead_not_aborts() {
// New wedge-skip policy: Pass 1 (bisect_on_marginal=false)
// should JumpAhead with a 1 GB skip + cooldown pause instead
// of immediately aborting. Aborting on first wedge was the
// pre-fix behavior that killed rips at 48% on damaged discs.
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&hardware_err(), &mut ctx);
match action {
ReadAction::JumpAhead {
sectors,
pause_secs,
} => {
assert_eq!(sectors, WEDGE_JUMP_SECTORS);
assert_eq!(pause_secs, WEDGE_PAUSE_SECS);
}
other => panic!("expected JumpAhead, got {other:?}"),
}
assert_eq!(ctx.wedge_count, 1);
}
#[test]
fn pass_1_hardware_error_aborts_after_threshold() {
// After WEDGE_ABORT_THRESHOLD consecutive wedges with no good
// read in between, autorip should see a real AbortPass so it
// can surface "drive is stuck, power-cycle required" to the
// user — rather than looping forever on a permanently bricked
// drive.
let mut ctx = ReadCtx::for_sweep(32);
for i in 0..WEDGE_ABORT_THRESHOLD - 1 {
let action = handle_read_error(&hardware_err(), &mut ctx);
assert!(
matches!(action, ReadAction::JumpAhead { .. }),
"iter {i}: expected JumpAhead, got {action:?}"
);
}
// The Nth wedge crosses the threshold.
let action = handle_read_error(&hardware_err(), &mut ctx);
assert_eq!(action, ReadAction::AbortPass);
}
#[test]
fn long_failure_streak_extends_pause() {
let mut ctx = ReadCtx::for_sweep(1);
fn pass_1_good_read_resets_wedge_count() {
// A single successful read between wedges must clear the
// skip counter — otherwise a disc with a few scattered bad
// zones would eventually run out of skip budget even though
// the drive was recovering between zones.
let mut ctx = ReadCtx::for_sweep(32);
for _ in 0..(WEDGE_ABORT_THRESHOLD - 1) {
handle_read_error(&hardware_err(), &mut ctx);
}
assert_eq!(ctx.wedge_count, WEDGE_ABORT_THRESHOLD - 1);
ctx.on_success();
assert_eq!(ctx.wedge_count, 0);
// After the success, we should still get JumpAhead (not
// AbortPass) on the next wedge.
let action = handle_read_error(&hardware_err(), &mut ctx);
assert!(matches!(action, ReadAction::JumpAhead { .. }));
}
#[test]
fn pass_n_hardware_error_still_aborts() {
// Pass N (bisect_on_marginal=true) keeps the original
// AbortPass behavior — single-sector recovery can't make
// progress through a wedge, so the right answer is to bail
// and let the outer layer decide.
let mut ctx = ReadCtx::for_patch(1);
let action = handle_read_error(&hardware_err(), &mut ctx);
assert_eq!(action, ReadAction::AbortPass);
}
#[test]
fn pass_1_illegal_request_also_routes_to_wedge_skip() {
// ILLEGAL_REQUEST is the other half of the wedge family:
// drive saying "I won't parse your CDB" after entering the
// fast-fail state. Same treatment as HARDWARE_ERROR.
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&illegal_request_err(), &mut ctx);
assert!(matches!(action, ReadAction::JumpAhead { .. }));
}
#[test]
fn long_failure_streak_extends_pause_on_pass_n() {
// Pass N keeps the cooldown behaviour: after many consecutive
// failures, pauses extend to give the drive time to recover.
// Pass 1 explicitly does NOT pause — see
// `pass_1_does_not_pause_on_skip` below.
let mut ctx = ReadCtx::for_patch(1);
for _ in 0..15 {
handle_read_error(&medium_err(), &mut ctx);
}
// After many consecutive failures we should be in the long-pause regime
let final_action = handle_read_error(&medium_err(), &mut ctx);
match final_action {
ReadAction::SkipBlock { pause_secs } => {
@@ -339,6 +740,28 @@ mod tests {
}
}
#[test]
fn pass_1_pauses_briefly_on_skip_for_wedge_avoidance() {
// Pre-2026-05-10 Pass 1 ran pause_secs=0 on all errors (zoom
// past damage zones in seconds). That caused firmware wedges
// on the BU40N — back-to-back errors with no cooldown built
// up firmware state until the drive entered the wedge fast-
// fail mode. New policy: a brief inter-error pause (5 s) on
// Pass 1 to give the drive's firmware time to settle between
// damage-zone exposures. Successful reads remain zero-pause
// — only errors cost time, and only a few seconds per
// scattered failure. Trivial cost compared to crashing the
// whole rip at 48%.
let mut ctx = ReadCtx::for_sweep(32);
let action = handle_read_error(&medium_err(), &mut ctx);
match action {
ReadAction::SkipBlock { pause_secs } => {
assert_eq!(pause_secs, PASS_1_FAIL_PAUSE_SECS);
}
other => panic!("expected SkipBlock, got {other:?}"),
}
}
#[test]
fn damage_window_fills_then_jumps() {
let mut ctx = ReadCtx::for_sweep(1);
File diff suppressed because it is too large Load Diff
+55 -91
View File
@@ -3,7 +3,7 @@
//! Clean structured XML with Content/Qualifier per stream and
//! stream number mapping via playbackconfig.
use super::{LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType};
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
use std::collections::HashMap;
@@ -12,7 +12,7 @@ pub fn detect(udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "streamproperties.xml")
}
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<ParseResult> {
let sp_data = super::read_jar_file(reader, udf, "streamproperties.xml")?;
let sp_text = std::str::from_utf8(&sp_data).ok()?;
@@ -66,7 +66,8 @@ pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLab
if labels.is_empty() {
return None;
}
Some(labels)
// High confidence: streamproperties.xml is fully structured.
Some(ParseResult::high(labels))
}
struct StreamInfo {
@@ -78,105 +79,68 @@ struct StreamInfo {
qualifier: LabelQualifier,
}
fn parse_stream_infos(xml: &str) -> Vec<StreamInfo> {
fn parse_stream_infos(text: &str) -> Vec<StreamInfo> {
let mut infos = Vec::new();
let mut pos = 0;
while pos < xml.len() {
let (tag, stream_type) = if let Some(p) = xml[pos..].find("<AudioStreamInfos>") {
(p + pos, StreamLabelType::Audio)
} else if let Some(p) = xml[pos..].find("<SubtitleStreamInfos>") {
(p + pos, StreamLabelType::Subtitle)
} else {
break;
};
for (tag_name, stream_type) in [
("AudioStreamInfos", StreamLabelType::Audio),
("SubtitleStreamInfos", StreamLabelType::Subtitle),
] {
let mut from = 0;
while let Some((start, end)) = xml::find_element(text, tag_name, from) {
let block = &text[start..end];
let id = xml::text(block, "ID").unwrap_or_default();
let lang_id = xml::text(block, "LangInfoID").unwrap_or_default();
let content = xml::text(block, "Content").unwrap_or_default();
let qualifier_str = xml::text(block, "Qualifier").unwrap_or_default();
let end_tag = match stream_type {
StreamLabelType::Audio => "</AudioStreamInfos>",
StreamLabelType::Subtitle => "</SubtitleStreamInfos>",
};
let (language, variant) = if lang_id.contains('_') {
let parts: Vec<&str> = lang_id.splitn(2, '_').collect();
(parts[0].to_lowercase(), parts[1].to_string())
} else {
(lang_id.to_lowercase(), String::new())
};
let block_end = match xml[tag..].find(end_tag) {
Some(p) => tag + p + end_tag.len(),
None => break,
};
let purpose = if content.eq_ignore_ascii_case("COMMENTARY") {
LabelPurpose::Commentary
} else {
LabelPurpose::Normal
};
let block = &xml[tag..block_end];
let id = extract_tag(block, "ID").unwrap_or_default();
let lang_id = extract_tag(block, "LangInfoID").unwrap_or_default();
let content = extract_tag(block, "Content").unwrap_or_default();
let qualifier_str = extract_tag(block, "Qualifier").unwrap_or_default();
let qualifier = match qualifier_str.to_ascii_uppercase().as_str() {
"SDH" => LabelQualifier::Sdh,
"DS" => LabelQualifier::DescriptiveService,
_ => LabelQualifier::None,
};
let (language, variant) = if lang_id.contains('_') {
let parts: Vec<&str> = lang_id.splitn(2, '_').collect();
(parts[0].to_lowercase(), parts[1].to_string())
} else {
(lang_id.to_lowercase(), String::new())
};
let purpose = match content.as_str() {
"COMMENTARY" => LabelPurpose::Commentary,
_ => LabelPurpose::Normal,
};
let qualifier = match qualifier_str.as_str() {
"SDH" => LabelQualifier::Sdh,
"DS" => LabelQualifier::DescriptiveService,
_ => LabelQualifier::None,
};
infos.push(StreamInfo {
id,
stream_type,
language,
variant,
purpose,
qualifier,
});
pos = block_end;
infos.push(StreamInfo {
id,
stream_type,
language,
variant,
purpose,
qualifier,
});
from = end;
}
}
infos
}
fn parse_playback_config(xml: &str, map: &mut HashMap<String, u16>) {
let mut pos = 0;
while pos < xml.len() {
let tag_start = if let Some(p) = xml[pos..].find("<AudioStreams>") {
Some(p + pos)
} else {
xml[pos..].find("<SubtitlesStreams>").map(|p| p + pos)
};
let tag_start = match tag_start {
Some(p) => p,
None => break,
};
let block_end = xml[tag_start..]
.find("</AudioStreams>")
.or_else(|| xml[tag_start..].find("</SubtitlesStreams>"))
.map(|p| tag_start + p + 20)
.unwrap_or(xml.len());
let block = &xml[tag_start..block_end];
if let (Some(stream_id_str), Some(info_id)) = (
extract_tag(block, "StreamID"),
extract_tag(block, "StreamInfo_ID"),
) {
if let Ok(stream_num) = stream_id_str.parse::<u16>() {
map.insert(info_id, stream_num);
fn parse_playback_config(text: &str, map: &mut HashMap<String, u16>) {
for tag_name in ["AudioStreams", "SubtitlesStreams"] {
let mut from = 0;
while let Some((start, end)) = xml::find_element(text, tag_name, from) {
let block = &text[start..end];
if let (Some(stream_id_str), Some(info_id)) = (
xml::text(block, "StreamID"),
xml::text(block, "StreamInfo_ID"),
) {
if let Ok(stream_num) = stream_id_str.parse::<u16>() {
map.insert(info_id, stream_num);
}
}
from = end;
}
pos = block_end;
}
}
fn extract_tag(xml: &str, tag: &str) -> Option<String> {
let open = format!("<{tag}>");
let close = format!("</{tag}>");
let start = xml.find(&open)? + open.len();
let end = xml[start..].find(&close)? + start;
Some(xml[start..end].trim().to_string())
}
+182 -14
View File
@@ -4,7 +4,7 @@
//! When both exist, language_streams.txt provides structured types while
//! menu_base.prop provides stream number → button name mapping.
use super::{LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType, vocab};
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, vocab};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
use std::collections::HashMap;
@@ -14,7 +14,7 @@ pub fn detect(udf: &UdfFs) -> bool {
|| super::jar_file_exists(udf, "language_streams.txt")
}
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<ParseResult> {
// Try language_streams.txt first (richer structured data)
let ls_labels = parse_language_streams(reader, udf);
@@ -22,12 +22,18 @@ pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLab
let mb_labels = parse_menu_base(reader, udf);
// If we have both, merge: language_streams for structure, menu_base for names
match (ls_labels, mb_labels) {
(Some(ls), Some(mb)) => Some(merge(ls, mb)),
(Some(ls), None) => Some(ls),
(None, Some(mb)) => Some(mb),
(None, None) => None,
let labels = match (ls_labels, mb_labels) {
(Some(ls), Some(mb)) => merge(ls, mb),
(Some(ls), None) => ls,
(None, Some(mb)) => mb,
(None, None) => return None,
};
if labels.is_empty() {
return None;
}
// High confidence: both language_streams.txt and menu_base.prop
// are structured key-value formats with documented types.
Some(ParseResult::high(labels))
}
fn merge(ls: Vec<StreamLabel>, mb: Vec<StreamLabel>) -> Vec<StreamLabel> {
@@ -172,6 +178,164 @@ fn parse_language_streams(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<
Some(labels)
}
#[cfg(test)]
mod tests {
use super::*;
/// Build a minimal menu_base.prop text and run `parse_menu_base`'s
/// inner logic via a temporary closure. This isolates the prop
/// parsing without needing a SectorReader.
fn parse_props(text: &str) -> Vec<StreamLabel> {
// Mirror the inner loop of parse_menu_base exactly. Kept
// separate so the test doesn't need disc fixtures.
use std::collections::HashMap;
let mut entries: HashMap<String, HashMap<String, String>> = HashMap::new();
for line in text.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
let Some(eq_pos) = line.find('=') else {
continue;
};
let full_key = &line[..eq_pos];
let value = &line[eq_pos + 1..];
if let Some(dot_pos) = full_key.rfind('.') {
entries
.entry(full_key[..dot_pos].to_string())
.or_default()
.insert(full_key[dot_pos + 1..].to_string(), value.to_string());
}
}
let mut labels = Vec::new();
for (prefix, props) in &entries {
let is_audio = props
.get("class")
.is_some_and(|c| c.contains("AudioButton"))
|| prefix.starts_with("audio_");
let is_subtitle = props
.get("class")
.is_some_and(|c| c.contains("SubtitleButton"))
|| prefix.starts_with("subtitle_");
let stream_num_str = props
.get("streamNumber")
.or_else(|| props.get("audioStream"))
.or_else(|| props.get("subtitleStream"));
let stream_num: u16 = match stream_num_str.and_then(|s| s.parse().ok()) {
Some(n) if n > 0 => n,
_ => continue,
};
if !is_audio && !is_subtitle {
continue;
}
let name = props.get("name").cloned().unwrap_or_default();
let purpose = match vocab::purpose(&name) {
LabelPurpose::Normal if prefix.contains("comm") => LabelPurpose::Commentary,
p => p,
};
let qualifier = if is_subtitle {
vocab::qualifier(&name)
} else {
LabelQualifier::None
};
let stream_type = if is_audio {
StreamLabelType::Audio
} else {
StreamLabelType::Subtitle
};
let language = props
.get("audioLanguage")
.or_else(|| props.get("subtitleLanguage"))
.cloned()
.unwrap_or_default();
labels.push(StreamLabel {
stream_number: stream_num,
stream_type,
language,
name,
purpose,
qualifier,
codec_hint: String::new(),
variant: String::new(),
});
}
labels.sort_by_key(|l| (l.stream_type as u8, l.stream_number));
labels
}
#[test]
fn commentary_via_name() {
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=2\n\
audio_1.name=Director's Commentary\n\
audio_1.audioLanguage=eng\n",
);
assert_eq!(labels.len(), 1);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
assert_eq!(labels[0].language, "eng");
}
#[test]
fn commentary_via_prefix_when_name_silent() {
let labels = parse_props(
"audio_commentary_1.class=AudioButton\n\
audio_commentary_1.streamNumber=2\n\
audio_commentary_1.name=Track 2\n\
audio_commentary_1.audioLanguage=eng\n",
);
assert_eq!(labels[0].purpose, LabelPurpose::Commentary);
}
#[test]
fn commenter_does_not_false_match_commentary() {
// Regression for the pre-refactor `name.contains("comment")`
// bug: this would wrongly classify a "Commenter Pro" track as
// Commentary. vocab::purpose enforces a word boundary.
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=2\n\
audio_1.name=Commenter Pro Track\n\
audio_1.audioLanguage=eng\n",
);
assert_eq!(labels[0].purpose, LabelPurpose::Normal);
}
#[test]
fn descriptive_via_name() {
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=3\n\
audio_1.name=English Descriptive Audio\n",
);
assert_eq!(labels[0].purpose, LabelPurpose::Descriptive);
}
#[test]
fn sdh_only_on_subtitles() {
// SDH applied to a subtitle stream.
let labels = parse_props(
"subtitle_1.class=SubtitleButton\n\
subtitle_1.streamNumber=4\n\
subtitle_1.name=English SDH\n",
);
assert_eq!(labels[0].qualifier, LabelQualifier::Sdh);
}
#[test]
fn sdh_not_applied_to_audio_stream_even_if_name_contains_sdh() {
// Audio streams should not pick up SDH (it's a subtitle
// concept). Edge case: badly-authored name happens to include
// "SDH" — we don't propagate it to audio metadata.
let labels = parse_props(
"audio_1.class=AudioButton\n\
audio_1.streamNumber=5\n\
audio_1.name=English SDH (track?)\n",
);
assert_eq!(labels[0].qualifier, LabelQualifier::None);
}
}
// ── menu_base.prop parser ──────────────────────────────────────────────────
fn parse_menu_base(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
@@ -231,16 +395,20 @@ fn parse_menu_base(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<Str
}
let name = props.get("name").cloned().unwrap_or_default();
let name_lower = name.to_lowercase();
let purpose = if name_lower.contains("comment") || prefix.contains("comm") {
LabelPurpose::Commentary
} else {
LabelPurpose::Normal
// Purpose: ask vocab first (word-boundary matched — avoids the
// "Commenter" false positive the prior `name.contains("comment")`
// had). Then fall back to the structural prefix check
// (`audio_commentary.foo`-style keys group commentary streams
// regardless of display name).
let purpose = match vocab::purpose(&name) {
LabelPurpose::Normal if prefix.contains("comm") => LabelPurpose::Commentary,
p => p,
};
let qualifier = if is_subtitle && name_lower.contains("sdh") {
LabelQualifier::Sdh
// Qualifier: only apply to subtitles (SDH is a subtitle concept).
let qualifier = if is_subtitle {
vocab::qualifier(&name)
} else {
LabelQualifier::None
};
+261
View File
@@ -0,0 +1,261 @@
//! "dbp" framework — Magnolia Pictures BD-J authoring shop (per
//! `bd-live.magpictures.com` referenced in the disc's
//! `com/dbp/bluray.MenuXlet.perm`). Detected on UHD discs whose
//! `/BDMV/JAR/<x>.jar` (top-level, not in a subdir) contains
//! `com/dbp/` package paths.
//!
//! Stream labels live as plain ASCII strings inside compiled `.class`
//! files in the jar — a quirk of the menu-rendering layer encoding
//! its TextField positions and content as constant strings the
//! Java compiler retained in the class string pool. Format observed
//! in the corpus (Civil War UHD, 2024):
//!
//! ```text
//! LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,...
//! RTextField,Audio2,English Descriptive Audio,Fontstrip_Composite,...
//! HTextField,Subtitle1,English SDH,Fontstrip_Composite,...
//! ATextField,Subtitle0,None,Fontstrip_Composite,...
//! ```
//!
//! The single uppercase letter before `TextField` is string-pool
//! prefix noise — the parser anchors on `TextField,` regardless of
//! what precedes it. `Subtitle0` is the disable-subtitles menu
//! button and is skipped (not a real subtitle stream).
//!
//! ## Implementation
//!
//! v2 (2026-05-10): rewritten on top of [`super::class_reader`] —
//! iterates `CpInfo::Utf8` constant-pool entries instead of raw byte
//! scanning each class file. Equivalent label coverage (the literal
//! `TextField,...` strings live in the CP as Utf8 entries), but
//! structurally cleaner: no false-positive risk from method bytecode
//! or attribute names happening to contain `TextField,`. Language /
//! purpose / qualifier classification moved to [`super::vocab`] so all
//! Java-parser families share one source of truth.
use super::class_reader::CpInfo;
use super::{ParseResult, StreamLabel, StreamLabelType, jar, vocab};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
use std::collections::BTreeMap;
/// dbp detect can't peek inside a jar without a SectorReader (the
/// trait function only takes `&UdfFs`), so we trigger on the cheap
/// signal "any top-level .jar in /BDMV/JAR/." That fires on every
/// BD-J disc, but parse() does the real `com/dbp/` check and
/// returns None on a mismatch — so this parser only ever consumes
/// time on discs that fell through every earlier parser.
pub fn detect(udf: &UdfFs) -> bool {
jar::has_any_top_level_jar(udf)
}
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<ParseResult> {
jar::for_each_jar(reader, udf, |_entry_name, archive| {
if !jar::has_path_prefix(archive, "com/dbp/") {
return None;
}
let labels = scan_jar(archive);
if labels.is_empty() {
None
} else {
// High confidence: TextField,Audio1,... is a stable anchor
// pattern + vocab routes language/purpose/qualifier.
Some(ParseResult::high(labels))
}
})
}
fn scan_jar(archive: &mut jar::Jar) -> Vec<StreamLabel> {
// BTreeMap so we keep the highest-numbered (last-written) label
// for each stream slot deterministic across runs. The same
// TextField,Audio1,... string can appear in multiple classes
// (button-state variants, localization fallbacks). Last write
// wins — they should all agree, but the structure is defensive.
let mut audios: BTreeMap<u16, String> = BTreeMap::new();
let mut subs: BTreeMap<u16, String> = BTreeMap::new();
jar::for_each_class(archive, |_class_name, class| {
for (_idx, cp) in class.constant_pool.iter() {
if let CpInfo::Utf8(s) = cp {
collect_textfield(s, &mut audios, &mut subs);
}
}
});
let mut out = Vec::new();
for (num, label) in audios {
out.push(make_label(num, label, StreamLabelType::Audio));
}
for (num, label) in subs {
out.push(make_label(num, label, StreamLabelType::Subtitle));
}
out
}
fn collect_textfield(
s: &str,
audios: &mut BTreeMap<u16, String>,
subs: &mut BTreeMap<u16, String>,
) {
// Anchor on "TextField," — the prefix character before it varies
// (string-pool ordering inside compiled Java) and is irrelevant.
let Some(idx) = s.find("TextField,") else {
return;
};
let after = &s[idx + "TextField,".len()..];
let mut parts = after.splitn(3, ',');
let kind_n = parts.next().unwrap_or("").trim();
let label = parts.next().unwrap_or("").trim();
if label.is_empty() {
return;
}
if let Some(rest) = kind_n.strip_prefix("Audio") {
if let Ok(n) = rest.parse::<u16>() {
audios.insert(n, label.to_string());
}
} else if let Some(rest) = kind_n.strip_prefix("Subtitle") {
if let Ok(n) = rest.parse::<u16>() {
// Subtitle0 is conventionally the "None / Off" disable
// button, not an actual subtitle stream.
if n > 0 {
subs.insert(n, label.to_string());
}
}
}
}
fn make_label(num: u16, label: String, stream_type: StreamLabelType) -> StreamLabel {
let lang_info = vocab::lang(&label);
let language = lang_info.map(|l| l.code).unwrap_or("").to_string();
let variant = lang_info.map(|l| l.variant).unwrap_or("").to_string();
let qualifier = vocab::qualifier(&label);
let purpose = vocab::purpose(&label);
StreamLabel {
stream_number: num,
stream_type,
language,
name: label,
purpose,
qualifier,
codec_hint: String::new(),
variant,
}
}
#[cfg(test)]
mod tests {
use super::super::{LabelPurpose, LabelQualifier};
use super::*;
#[test]
fn collect_extracts_audio_and_subtitle_indices() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
let lines = [
"LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,296,763,275,25,left",
"RTextField,Audio2,English Descriptive Audio,Fontstrip_Composite,296,803,275,25,left",
"RTextField,Audio3,Spanish 5.1 Dolby Digital,Fontstrip_Composite,296,843,275,25,left",
"ATextField,Subtitle0,None,Fontstrip_Composite,1312,843,275,25,left",
"HTextField,Subtitle1,English SDH,Fontstrip_Composite,1312,763,275,25,left",
"DTextField,Subtitle2,Spanish,Fontstrip_Composite,1312,803,275,25,left",
];
for s in &lines {
collect_textfield(s, &mut audios, &mut subs);
}
assert_eq!(audios.len(), 3);
assert_eq!(audios[&1], "English Dolby Atmos");
assert_eq!(audios[&2], "English Descriptive Audio");
assert_eq!(audios[&3], "Spanish 5.1 Dolby Digital");
// Subtitle0 ("None") is skipped — disable button, not a stream.
assert_eq!(subs.len(), 2);
assert_eq!(subs[&1], "English SDH");
assert_eq!(subs[&2], "Spanish");
}
#[test]
fn collect_ignores_non_textfield_strings() {
let mut audios = BTreeMap::new();
let mut subs = BTreeMap::new();
for s in [
"GraphicButton,SU_Audio",
"AudioMenu",
"CommentaryMenuAlternateScenes",
"PrimaryAudioControl",
] {
collect_textfield(s, &mut audios, &mut subs);
}
assert!(audios.is_empty());
assert!(subs.is_empty());
}
#[test]
fn make_label_routes_via_vocab() {
let l = make_label(1, "English SDH".to_string(), StreamLabelType::Subtitle);
assert_eq!(l.language, "eng");
assert_eq!(l.qualifier, LabelQualifier::Sdh);
assert_eq!(l.purpose, LabelPurpose::Normal);
}
#[test]
fn make_label_descriptive_audio() {
let l = make_label(
2,
"English Descriptive Audio".to_string(),
StreamLabelType::Audio,
);
assert_eq!(l.language, "eng");
assert_eq!(l.purpose, LabelPurpose::Descriptive);
}
#[test]
fn make_label_commentary() {
let l = make_label(
3,
"English Director's Commentary".to_string(),
StreamLabelType::Audio,
);
assert_eq!(l.language, "eng");
assert_eq!(l.purpose, LabelPurpose::Commentary);
}
#[test]
fn make_label_compound_languages_populate_variant() {
let brazilian = make_label(1, "Brazilian Portuguese 5.1".into(), StreamLabelType::Audio);
assert_eq!(brazilian.language, "por");
assert_eq!(brazilian.variant, "Brazilian");
let castilian = make_label(1, "Castilian Spanish".into(), StreamLabelType::Audio);
assert_eq!(castilian.language, "spa");
assert_eq!(castilian.variant, "Castilian");
let canadian = make_label(
1,
"Canadian French Dolby Digital".into(),
StreamLabelType::Audio,
);
assert_eq!(canadian.language, "fra");
assert_eq!(canadian.variant, "Canadian");
}
#[test]
fn make_label_bare_language_has_empty_variant() {
let l = make_label(1, "English Dolby Atmos".into(), StreamLabelType::Audio);
assert_eq!(l.language, "eng");
assert_eq!(l.variant, "");
}
#[test]
fn make_label_unknown_language_is_empty() {
// vocab::lang returns None — make_label converts both fields to "".
let l = make_label(1, "Klingon Dolby Atmos".into(), StreamLabelType::Audio);
assert_eq!(l.language, "");
assert_eq!(l.variant, "");
}
#[test]
fn make_label_rnib_descriptive_service() {
let l = make_label(1, "English RNIB".into(), StreamLabelType::Subtitle);
assert_eq!(l.language, "eng");
assert_eq!(l.qualifier, LabelQualifier::DescriptiveService);
}
}
+1547
View File
File diff suppressed because it is too large Load Diff
+151
View File
@@ -0,0 +1,151 @@
//! BD-J jar utilities — common scaffolding for parsers that read
//! `/BDMV/JAR/*.jar`.
//!
//! Composes with [`class_reader`](super::class_reader) for structured
//! `.class` access. Used by `dbp` (string-pool scan via constant pool)
//! and `deluxe` (bytecode pattern matching) — those parsers express
//! "open every top-level jar, look at every .class inside" without
//! repeating the zip-archive boilerplate.
// `try_each_class` is staged for `labels::deluxe`, which needs the
// early-return form to short-circuit class iteration on a match.
// dead-code allow comes off when deluxe lands.
#![allow(dead_code)]
use super::class_reader::ClassFile;
use crate::sector::SectorReader;
use crate::udf::UdfFs;
use std::io::Cursor;
use zip::ZipArchive;
/// In-memory zip archive: backed by a `Vec<u8>` read from UDF. Owns
/// the buffer; callers pass it to [`has_path_prefix`], [`for_each_class`],
/// etc.
pub type Jar = ZipArchive<Cursor<Vec<u8>>>;
/// True if `/BDMV/JAR/` contains at least one top-level `.jar` file
/// (not under a subdir). Used by `detect()` in parsers whose real
/// signal lives inside a jar — they can't open the jar without a
/// `SectorReader`, so they use this cheap pre-check and do the real
/// `com/<vendor>/` discriminator in `parse()`.
pub fn has_any_top_level_jar(udf: &UdfFs) -> bool {
let Some(jar_dir) = udf.find_dir("/BDMV/JAR") else {
return false;
};
jar_dir
.entries
.iter()
.any(|e| !e.is_dir && e.name.to_lowercase().ends_with(".jar"))
}
/// Open every top-level `*.jar` entry in `/BDMV/JAR/` and yield each
/// `(entry_name, Jar)` to `f`. Returns the first `Some(R)` the callback
/// produces, or `None` if every jar was visited without a hit.
///
/// "Top-level" means entries directly under `/BDMV/JAR/`, not nested
/// under a subdir. (Pixelogic, Criterion, Paramount, etc. put their
/// data files inside `/BDMV/JAR/<x>/`; dbp and Deluxe put their jar
/// directly at `/BDMV/JAR/<name>.jar`.)
///
/// Entries that fail to read from UDF or that aren't valid zips are
/// silently skipped — same defensive shape as the existing dbp parser.
pub fn for_each_jar<R, F>(reader: &mut dyn SectorReader, udf: &UdfFs, mut f: F) -> Option<R>
where
F: FnMut(&str, &mut Jar) -> Option<R>,
{
let jar_dir = udf.find_dir("/BDMV/JAR")?;
for entry in &jar_dir.entries {
if entry.is_dir {
continue;
}
if !entry.name.to_lowercase().ends_with(".jar") {
continue;
}
let path = format!("/BDMV/JAR/{}", entry.name);
let Ok(bytes) = udf.read_file(reader, &path) else {
continue;
};
let Ok(mut archive) = ZipArchive::new(Cursor::new(bytes)) else {
continue;
};
if let Some(r) = f(&entry.name, &mut archive) {
return Some(r);
}
}
None
}
/// True if any entry in this jar's central directory starts with
/// `prefix`. Fast — only reads filenames, never extracts bytes.
///
/// Used by parsers as a cheap "is this MY framework's jar?" check
/// (e.g. `has_path_prefix(archive, "com/dbp/")` for dbp,
/// `has_path_prefix(archive, "com/bydeluxe/")` for Deluxe).
pub fn has_path_prefix(archive: &mut Jar, prefix: &str) -> bool {
for i in 0..archive.len() {
if let Ok(f) = archive.by_index(i) {
if f.name().starts_with(prefix) {
return true;
}
}
}
false
}
/// Iterate every `.class` entry in the jar, parse it with
/// [`class_reader`], and call `f` with `(entry_name, &ClassFile)`.
///
/// Entries that fail to read or parse are silently skipped — this is
/// label-extraction code, robustness matters more than completeness.
/// Callers that need to know which classes failed should use the
/// lower-level [`class_reader`] API directly.
pub fn for_each_class<F>(archive: &mut Jar, mut f: F)
where
F: FnMut(&str, &ClassFile),
{
for i in 0..archive.len() {
let Ok(mut entry) = archive.by_index(i) else {
continue;
};
if !entry.name().ends_with(".class") {
continue;
}
let name = entry.name().to_string();
let mut bytes = Vec::with_capacity(entry.size() as usize);
if std::io::Read::read_to_end(&mut entry, &mut bytes).is_err() {
continue;
}
let Ok(class) = ClassFile::parse(&bytes) else {
continue;
};
f(&name, &class);
}
}
/// Like [`for_each_class`] but allows the callback to short-circuit
/// iteration. Returns the first `Some(R)` the callback produces.
pub fn try_each_class<R, F>(archive: &mut Jar, mut f: F) -> Option<R>
where
F: FnMut(&str, &ClassFile) -> Option<R>,
{
for i in 0..archive.len() {
let Ok(mut entry) = archive.by_index(i) else {
continue;
};
if !entry.name().ends_with(".class") {
continue;
}
let name = entry.name().to_string();
let mut bytes = Vec::with_capacity(entry.size() as usize);
if std::io::Read::read_to_end(&mut entry, &mut bytes).is_err() {
continue;
}
let Ok(class) = ClassFile::parse(&bytes) else {
continue;
};
if let Some(r) = f(&name, &class) {
return Some(r);
}
}
None
}
+626 -9
View File
@@ -7,11 +7,17 @@
//! 3. Implement `pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLabel>>`
//! 4. Add `mod myformat;` below and one line to `PARSERS` array
pub(crate) mod class_reader;
mod criterion;
mod ctrm;
mod dbp;
mod deluxe;
pub(crate) mod jar;
mod paramount;
mod pixelogic;
pub(crate) mod text;
pub mod vocab;
pub(crate) mod xml;
use crate::disc::{DiscTitle, Stream};
use crate::sector::SectorReader;
@@ -68,18 +74,70 @@ pub enum LabelQualifier {
// ── Parser registry ────────────────────────────────────────────────────────
//
// Each entry: (name, detect_fn, parse_fn)
// Order = priority. First match wins. Highest quality output first.
// Each entry: (name, detect_fn, parse_fn). Order = tiebreaker only —
// the registry picks the highest-confidence parse result, falling back
// to array order on confidence ties.
type DetectFn = fn(&UdfFs) -> bool;
type ParseFn = fn(&mut dyn SectorReader, &UdfFs) -> Option<Vec<StreamLabel>>;
type ParseFn = fn(&mut dyn SectorReader, &UdfFs) -> Option<ParseResult>;
/// Per-parser claim of how reliable its output is. Used by the
/// registry to pick between parsers when more than one matches (e.g.
/// a disc that has both `bluray_project.bin` and `playlists.xml`).
///
/// A parser SHOULD return `High` only when its full schema was
/// extracted with no fallback or guessing. `Medium` is for matched-
/// but-degraded outputs (some streams missing fields, fingerprint
/// matched but a sub-table couldn't be decoded, etc.). The registry
/// prefers `High` over `Medium`; ties fall to array order.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum Confidence {
Medium,
High,
}
/// Successful parser result. `None` from `parse()` still means "this
/// isn't my disc" (no labels at all); `Some(ParseResult { labels, .. })`
/// with `labels.is_empty()` is also a "no labels" case but reachable
/// via the analyzer (used by deluxe today to signal "I recognized the
/// framework but Phase D not yet implemented").
#[derive(Debug, Clone)]
pub struct ParseResult {
pub labels: Vec<StreamLabel>,
pub confidence: Confidence,
}
impl ParseResult {
/// Convenience for the common "I parsed N labels with full schema
/// coverage" case.
pub fn high(labels: Vec<StreamLabel>) -> Self {
ParseResult {
labels,
confidence: Confidence::High,
}
}
/// Convenience for "I matched but had to fall back on some fields".
pub fn medium(labels: Vec<StreamLabel>) -> Self {
ParseResult {
labels,
confidence: Confidence::Medium,
}
}
}
const PARSERS: &[(&str, DetectFn, ParseFn)] = &[
("paramount", paramount::detect, paramount::parse),
("criterion", criterion::detect, criterion::parse),
("pixelogic", pixelogic::detect, pixelogic::parse),
("ctrm", ctrm::detect, ctrm::parse),
// ("deluxe", deluxe::detect, deluxe::parse), // TODO: bytecode parser
// dbp and deluxe both detect on "any top-level .jar in /BDMV/JAR/"
// (every BD-J disc trips that) and do the real vendor-prefix check
// in parse(). Order between them is the tiebreaker on equal
// confidence; dbp goes first because its parse path is cheaper
// (constant-pool iteration vs. deluxe's bytecode walking).
("dbp", dbp::detect, dbp::parse),
("deluxe", deluxe::detect, deluxe::parse),
];
/// Search disc for config files, extract labels, apply to streams.
@@ -90,7 +148,18 @@ pub fn apply(reader: &mut dyn SectorReader, udf: &UdfFs, titles: &mut [DiscTitle
if labels.is_empty() {
return;
}
apply_labels(&labels, titles);
}
/// Apply a pre-extracted set of labels to titles' streams. Match
/// labels to streams by (stream_type, 1-based stream_number per type).
/// Audio streams update `purpose` + `label` (codec/variant info; never
/// English purpose text). Subtitle streams update `qualifier` and the
/// `forced` flag.
///
/// Extracted from `apply()` so the matching logic is unit-testable
/// without needing a SectorReader / UdfFs.
pub(crate) fn apply_labels(labels: &[StreamLabel], titles: &mut [DiscTitle]) {
for title in titles.iter_mut() {
let mut audio_idx: u16 = 0;
let mut sub_idx: u16 = 0;
@@ -268,14 +337,151 @@ fn generate_audio_label(
}
fn extract(reader: &mut dyn SectorReader, udf: &UdfFs) -> Vec<StreamLabel> {
for (_name, detect, parse) in PARSERS {
if detect(udf) {
if let Some(labels) = parse(reader, udf) {
return labels;
let mut best: Option<(&'static str, ParseResult)> = None;
for (name, detect, parse) in PARSERS {
if !detect(udf) {
continue;
}
tracing::info!(parser = name, "label parser detected");
let Some(result) = parse(reader, udf) else {
continue;
};
if result.labels.is_empty() {
continue;
}
// Pick highest confidence. Equal confidence → first wins
// (array order tiebreaker).
match &best {
None => best = Some((name, result)),
Some((_, b)) if result.confidence > b.confidence => best = Some((name, result)),
_ => {}
}
}
match best {
Some((name, r)) => {
tracing::info!(
parser = name,
confidence = ?r.confidence,
label_count = r.labels.len(),
"label parser selected",
);
r.labels
}
None => {
tracing::info!("no label parser matched");
Vec::new()
}
}
}
/// Diagnostic introspection — returns the parser that matched, the
/// labels it emitted, and the inventory of files under `/BDMV/JAR/*/`
/// that the discriminators looked at. Intended for `freemkv-tools
/// labels-analyze` and corpus regression tooling, not production code
/// paths. The matching/parsing logic is identical to [`extract`]; only
/// the return shape is richer (includes confidence, all detected
/// parsers, and any parsers that produced empty results).
#[doc(hidden)]
pub fn analyze(reader: &mut dyn SectorReader, udf: &UdfFs) -> LabelAnalysis {
let inventory = jar_inventory(udf);
let mut parsers_detected: Vec<&'static str> = Vec::new();
let mut all_results: Vec<(&'static str, ParseResult)> = Vec::new();
for (name, detect, parse) in PARSERS {
if !detect(udf) {
continue;
}
tracing::info!(parser = name, "label parser detected");
parsers_detected.push(name);
if let Some(r) = parse(reader, udf) {
all_results.push((name, r));
}
}
// Selection logic mirrors `extract`: highest confidence + non-empty,
// array order tiebreaker.
let chosen = all_results
.iter()
.filter(|(_, r)| !r.labels.is_empty())
.max_by(|(_, a), (_, b)| {
// Cmp first by confidence (higher first), then position
// (earlier first). max_by yields the maximum, so we
// invert the index comparison.
a.confidence
.cmp(&b.confidence)
.then(std::cmp::Ordering::Equal)
});
let (parser, confidence, labels) = match chosen {
Some((name, r)) => (Some(*name), Some(r.confidence), r.labels.clone()),
None => (None, None, Vec::new()),
};
if parsers_detected.is_empty() {
tracing::info!("no label parser matched");
} else if parser.is_none() {
tracing::info!(
detected = ?parsers_detected,
"label parsers detected but produced no labels"
);
}
LabelAnalysis {
parser,
parsers_detected,
confidence,
jar_inventory: inventory,
labels,
}
}
/// Result of [`analyze`].
#[doc(hidden)]
#[derive(Debug, Clone)]
pub struct LabelAnalysis {
/// Which parser was SELECTED — the one whose `ParseResult` had
/// the highest confidence among non-empty results (array order
/// tiebreaker). `None` means either no parser recognized the
/// disc, OR every parser that recognized it returned no labels.
/// Use `parsers_detected` to disambiguate.
pub parser: Option<&'static str>,
/// Confidence of the selected parser, `None` if no parser was
/// selected.
pub confidence: Option<Confidence>,
/// Every parser whose discriminator matched, in registry order.
/// Distinguishes "we recognized this disc but couldn't extract
/// labels" from "we don't recognize this disc at all" — the
/// former points at a parser bug or a truncated capture, the
/// latter points at a missing parser.
pub parsers_detected: Vec<&'static str>,
/// Filenames found under any `/BDMV/JAR/*/` subdirectory, deduped
/// and sorted. Helps spot unknown authoring formats when no
/// parser detected.
pub jar_inventory: Vec<String>,
/// Raw labels emitted by the selected parser (empty if `parser`
/// is `None`).
pub labels: Vec<StreamLabel>,
}
/// List filenames found under any `/BDMV/JAR/<x>/` subdirectory of
/// the disc. Deduped, sorted. Returns an empty vec if no JAR dir is
/// present.
fn jar_inventory(udf: &UdfFs) -> Vec<String> {
let Some(jar_dir) = udf.find_dir("/BDMV/JAR") else {
return Vec::new();
};
let mut out: Vec<String> = Vec::new();
for entry in &jar_dir.entries {
if entry.is_dir {
for child in &entry.entries {
if !child.is_dir && !out.contains(&child.name) {
out.push(child.name.clone());
}
}
}
}
Vec::new()
out.sort();
out
}
// ── Shared helpers ─────────────────────────────────────────────────────────
@@ -311,3 +517,414 @@ pub(crate) fn read_jar_file(
let path = find_jar_file(udf, filename)?;
udf.read_file(reader, &path).ok().filter(|d| !d.is_empty())
}
// ── Registry-level tests ────────────────────────────────────────────────────
#[cfg(test)]
mod registry_tests {
use super::*;
/// Lock the parser roster + order. If someone reorders the array
/// or adds/removes a parser, this test forces them to update the
/// expectation explicitly. The order is load-bearing: first
/// matching `parse()` wins, so reordering changes which parser
/// claims a disc on overlapping detect signals.
///
/// dbp + deluxe MUST stay at the end (their detect triggers on
/// "any BD-J disc"; placing them earlier would short-circuit the
/// stricter parsers above them).
#[test]
fn parsers_registry_order_locked() {
let names: Vec<&str> = PARSERS.iter().map(|(n, _, _)| *n).collect();
assert_eq!(
names,
vec![
"paramount",
"criterion",
"pixelogic",
"ctrm",
"dbp",
"deluxe"
],
"PARSERS array order changed — confirm dbp + deluxe stay last \
(loose detect, real check in parse), and stricter parsers \
(paramount/criterion/pixelogic/ctrm — all file-presence \
gated detect) stay first."
);
}
/// Per-parser sanity: every parser has both detect and parse
/// hooked up. Catches accidental nullification (e.g. someone
/// stubbing `parse` to always-None during a refactor).
#[test]
fn parsers_registry_all_entries_populated() {
for (name, detect, parse) in PARSERS {
// Function pointers can't be Null in safe Rust, so the
// assertion is just that the array entry was constructed
// — which the iter above already implies. The test
// exists to fail compile if someone changes the tuple
// shape (e.g. adds a 4th field) without updating callers,
// and as a marker for "these parsers exist."
let _ = (name, detect, parse);
}
assert!(!PARSERS.is_empty(), "PARSERS array must not be empty");
}
}
// ── apply() integration tests ──────────────────────────────────────────────
//
// End-to-end coverage for the apply_labels + fill_defaults pipeline
// without needing a SectorReader / UdfFs. Synthetic DiscTitle +
// StreamLabel inputs, assert on the resulting Stream field values.
#[cfg(test)]
mod apply_tests {
use super::*;
use crate::disc::{
AudioChannels, AudioStream, Codec, ColorSpace, FrameRate, HdrFormat, Resolution,
SampleRate, SubtitleStream, VideoStream,
};
fn audio(pid: u16, codec: Codec, channels: AudioChannels, language: &str) -> Stream {
Stream::Audio(AudioStream {
pid,
codec,
channels,
language: language.into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: String::new(),
})
}
fn subtitle(pid: u16, language: &str) -> Stream {
Stream::Subtitle(SubtitleStream {
pid,
codec: Codec::Pgs,
language: language.into(),
forced: false,
qualifier: LabelQualifier::None,
codec_data: None,
})
}
fn video() -> Stream {
Stream::Video(VideoStream {
pid: 0x1011,
codec: Codec::Hevc,
resolution: Resolution::R2160p,
frame_rate: FrameRate::F23_976,
hdr: HdrFormat::Hdr10,
color_space: ColorSpace::Bt2020,
secondary: false,
label: String::new(),
})
}
fn title_with(streams: Vec<Stream>) -> DiscTitle {
DiscTitle {
playlist: "00800.mpls".into(),
playlist_id: 800,
duration_secs: 7200.0,
size_bytes: 0,
clips: Vec::new(),
streams,
chapters: Vec::new(),
extents: Vec::new(),
content_format: crate::disc::ContentFormat::BdTs,
codec_privates: Vec::new(),
}
}
fn audio_label(num: u16, lang: &str, codec_hint: &str, variant: &str) -> StreamLabel {
StreamLabel {
stream_number: num,
stream_type: StreamLabelType::Audio,
language: lang.into(),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::None,
codec_hint: codec_hint.into(),
variant: variant.into(),
}
}
#[test]
fn apply_attaches_codec_hint_and_variant_to_audio() {
let mut titles = vec![title_with(vec![
video(),
audio(0x1100, Codec::TrueHd, AudioChannels::Surround51, "eng"),
])];
let labels = vec![audio_label(1, "eng", "Dolby Atmos", "")];
apply_labels(&labels, &mut titles);
if let Stream::Audio(a) = &titles[0].streams[1] {
assert_eq!(a.label, "Dolby Atmos");
} else {
panic!("expected audio stream");
}
}
#[test]
fn apply_combines_variant_and_codec_hint() {
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::TrueHd,
AudioChannels::Surround51,
"por",
)])];
let labels = vec![audio_label(1, "por", "Dolby Atmos", "Brazilian")];
apply_labels(&labels, &mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.label, "(Brazilian) Dolby Atmos");
} else {
panic!("expected audio stream");
}
}
#[test]
fn apply_sets_purpose_on_audio_commentary() {
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::Ac3,
AudioChannels::Stereo,
"eng",
)])];
let labels = vec![StreamLabel {
stream_number: 1,
stream_type: StreamLabelType::Audio,
language: "eng".into(),
name: String::new(),
purpose: LabelPurpose::Commentary,
qualifier: LabelQualifier::None,
codec_hint: String::new(),
variant: String::new(),
}];
apply_labels(&labels, &mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.purpose, LabelPurpose::Commentary);
// Label stays empty: no codec/variant; purpose is conveyed
// structurally, NOT as English text.
assert_eq!(a.label, "");
} else {
panic!("expected audio stream");
}
}
#[test]
fn apply_uses_name_fallback_only_for_normal_purpose() {
// Name fallback fires when purpose=Normal and codec/variant are empty.
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::TrueHd,
AudioChannels::Surround71,
"eng",
)])];
let labels = vec![StreamLabel {
stream_number: 1,
stream_type: StreamLabelType::Audio,
language: "eng".into(),
name: "Director's Cut Edition".into(),
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::None,
codec_hint: String::new(),
variant: String::new(),
}];
apply_labels(&labels, &mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.label, "Director's Cut Edition");
} else {
panic!("expected audio stream");
}
}
#[test]
fn apply_name_fallback_suppressed_for_non_normal_purpose() {
// Name fallback must NOT fire when purpose != Normal — the
// CLI is responsible for rendering purpose text.
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::Ac3,
AudioChannels::Stereo,
"eng",
)])];
let labels = vec![StreamLabel {
stream_number: 1,
stream_type: StreamLabelType::Audio,
language: "eng".into(),
name: "Commentary by Director".into(),
purpose: LabelPurpose::Commentary,
qualifier: LabelQualifier::None,
codec_hint: String::new(),
variant: String::new(),
}];
apply_labels(&labels, &mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.label, "", "label must not contain English purpose text");
assert_eq!(a.purpose, LabelPurpose::Commentary);
}
}
#[test]
fn apply_sets_qualifier_on_subtitle_sdh() {
let mut titles = vec![title_with(vec![subtitle(0x1200, "eng")])];
let labels = vec![StreamLabel {
stream_number: 1,
stream_type: StreamLabelType::Subtitle,
language: "eng".into(),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::Sdh,
codec_hint: String::new(),
variant: String::new(),
}];
apply_labels(&labels, &mut titles);
if let Stream::Subtitle(s) = &titles[0].streams[0] {
assert_eq!(s.qualifier, LabelQualifier::Sdh);
// SDH doesn't flip the `forced` flag.
assert!(!s.forced);
} else {
panic!("expected subtitle");
}
}
#[test]
fn apply_flips_forced_flag_on_subtitle_forced_qualifier() {
let mut titles = vec![title_with(vec![subtitle(0x1200, "eng")])];
let labels = vec![StreamLabel {
stream_number: 1,
stream_type: StreamLabelType::Subtitle,
language: "eng".into(),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::Forced,
codec_hint: String::new(),
variant: String::new(),
}];
apply_labels(&labels, &mut titles);
if let Stream::Subtitle(s) = &titles[0].streams[0] {
assert_eq!(s.qualifier, LabelQualifier::Forced);
assert!(s.forced);
}
}
#[test]
fn apply_indexes_streams_by_type_separately() {
// Audio and subtitle each have their own 1-based index; an
// Audio #2 label maps to the 2nd audio stream, not the 2nd
// stream overall (which could be a subtitle).
let mut titles = vec![title_with(vec![
video(),
audio(0x1100, Codec::TrueHd, AudioChannels::Surround51, "eng"),
subtitle(0x1200, "eng"),
audio(0x1101, Codec::Ac3, AudioChannels::Stereo, "fra"),
])];
let labels = vec![
audio_label(1, "eng", "Dolby Atmos", ""),
audio_label(2, "fra", "Dolby Digital", ""),
StreamLabel {
stream_number: 1,
stream_type: StreamLabelType::Subtitle,
language: "eng".into(),
name: String::new(),
purpose: LabelPurpose::Normal,
qualifier: LabelQualifier::Sdh,
codec_hint: String::new(),
variant: String::new(),
},
];
apply_labels(&labels, &mut titles);
// Audio #1
if let Stream::Audio(a) = &titles[0].streams[1] {
assert_eq!(a.label, "Dolby Atmos");
}
// Audio #2 (4th stream overall)
if let Stream::Audio(a) = &titles[0].streams[3] {
assert_eq!(a.label, "Dolby Digital");
}
// Subtitle #1
if let Stream::Subtitle(s) = &titles[0].streams[2] {
assert_eq!(s.qualifier, LabelQualifier::Sdh);
}
}
#[test]
fn apply_ignores_labels_for_nonexistent_streams() {
// A label for stream #99 with no matching stream is a no-op.
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::TrueHd,
AudioChannels::Surround51,
"eng",
)])];
let labels = vec![audio_label(99, "fra", "Dolby Digital", "")];
apply_labels(&labels, &mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.label, "", "label must be untouched");
}
}
#[test]
fn apply_empty_labels_does_not_touch_streams() {
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::TrueHd,
AudioChannels::Surround51,
"eng",
)])];
apply_labels(&[], &mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.label, "");
}
}
// ── fill_defaults() tests ───────────────────────────────────────────────
#[test]
fn fill_defaults_generates_audio_label_when_empty() {
let mut titles = vec![title_with(vec![audio(
0x1100,
Codec::TrueHd,
AudioChannels::Surround71,
"eng",
)])];
fill_defaults(&mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.label, "Dolby TrueHD 7.1");
}
}
#[test]
fn fill_defaults_preserves_existing_audio_label() {
let mut titles = vec![title_with(vec![Stream::Audio(AudioStream {
pid: 0x1100,
codec: Codec::TrueHd,
channels: AudioChannels::Surround71,
language: "eng".into(),
sample_rate: SampleRate::S48,
secondary: false,
purpose: LabelPurpose::Normal,
label: "Pre-set Atmos".into(),
})])];
fill_defaults(&mut titles);
if let Stream::Audio(a) = &titles[0].streams[0] {
assert_eq!(a.label, "Pre-set Atmos");
}
}
#[test]
fn fill_defaults_generates_video_label_with_hdr() {
let mut titles = vec![title_with(vec![video()])];
fill_defaults(&mut titles);
if let Stream::Video(v) = &titles[0].streams[0] {
assert!(v.label.contains("4K"), "expected 4K, got {}", v.label);
assert!(v.label.contains("HDR10"), "expected HDR10, got {}", v.label);
}
}
}
+19 -30
View File
@@ -12,7 +12,7 @@
//! sub_com1_idx="23,24,25" />
//! ```
use super::{LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType};
use super::{LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, xml};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
@@ -20,7 +20,7 @@ pub fn detect(udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "playlists.xml")
}
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<ParseResult> {
let data = super::read_jar_file(reader, udf, "playlists.xml")?;
let text = std::str::from_utf8(&data).ok()?;
@@ -30,8 +30,8 @@ pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLab
let mut labels = Vec::new();
// Parse audio streams
if let Some(aud) = extract_attr(&feature, "aud") {
let com_idx = extract_attr(&feature, "aud_com1_idx").and_then(|s| s.parse::<usize>().ok());
if let Some(aud) = xml::attr(&feature, "aud") {
let com_idx = xml::attr(&feature, "aud_com1_idx").and_then(|s| s.parse::<usize>().ok());
for (i, lang) in aud.split(',').enumerate() {
let lang = lang.trim();
@@ -57,12 +57,12 @@ pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLab
}
// Parse subtitle streams
if let Some(sub) = extract_attr(&feature, "sub") {
let forced: Vec<bool> = extract_attr(&feature, "forced_sub")
if let Some(sub) = xml::attr(&feature, "sub") {
let forced: Vec<bool> = xml::attr(&feature, "forced_sub")
.map(|s| s.split(',').map(|f| f.trim() == "1").collect())
.unwrap_or_default();
let com_indices: Vec<usize> = extract_attr(&feature, "sub_com1_idx")
let com_indices: Vec<usize> = xml::attr(&feature, "sub_com1_idx")
.map(|s| s.split(',').filter_map(|i| i.trim().parse().ok()).collect())
.unwrap_or_default();
@@ -100,32 +100,29 @@ pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLab
if labels.is_empty() {
return None;
}
Some(labels)
// High confidence: paramount's playlists.xml is fully structured
// and we extract every documented field.
Some(ParseResult::high(labels))
}
/// Find the feature playlist element (the one with the most audio tracks).
fn find_feature_playlist(xml: &str) -> Option<String> {
fn find_feature_playlist(text: &str) -> Option<String> {
let mut best: Option<String> = None;
let mut best_aud_count = 0;
let mut from = 0;
let mut pos = 0;
while let Some(start) = xml[pos..].find("<playlist ") {
let abs_start = pos + start;
let end = match xml[abs_start..].find("/>") {
Some(p) => abs_start + p + 2,
None => break,
};
let element = &xml[abs_start..end];
while let Some((start, end)) = xml::find_element(text, "playlist", from) {
let element = &text[start..end];
// Prefer name="Feature" explicitly
if let Some(name) = extract_attr(element, "name") {
// Prefer name="Feature" explicitly.
if let Some(name) = xml::attr(element, "name") {
if name.eq_ignore_ascii_case("Feature") {
return Some(element.to_string());
}
}
// Otherwise pick the one with the most audio streams
if let Some(aud) = extract_attr(element, "aud") {
// Otherwise pick the one with the most audio streams.
if let Some(aud) = xml::attr(element, "aud") {
let count = aud.split(',').count();
if count > best_aud_count {
best_aud_count = count;
@@ -133,15 +130,7 @@ fn find_feature_playlist(xml: &str) -> Option<String> {
}
}
pos = end;
from = end;
}
best
}
/// Extract an XML attribute value from an element string.
fn extract_attr(element: &str, name: &str) -> Option<String> {
let needle = format!("{name}=\"");
let start = element.find(&needle)? + needle.len();
let end = element[start..].find('"')? + start;
Some(element[start..end].to_string())
}
+101 -22
View File
@@ -5,9 +5,13 @@
//!
//! Token format: `{lang}_{codec?}_{purpose?}_{region?}_`
use super::{LabelPurpose, LabelQualifier, StreamLabel, StreamLabelType, vocab};
use super::{
Confidence, LabelPurpose, LabelQualifier, ParseResult, StreamLabel, StreamLabelType, text,
vocab,
};
use crate::sector::SectorReader;
use crate::udf::UdfFs;
use std::sync::atomic::{AtomicBool, Ordering};
/// Known audio codec tokens
const AUDIO_CODECS: &[&str] = &["MLP", "AC3", "DTS", "DDL", "WAV", "AC"];
@@ -20,9 +24,18 @@ pub fn detect(udf: &UdfFs) -> bool {
super::jar_file_exists(udf, "bluray_project.bin")
}
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLabel>> {
pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<ParseResult> {
let data = super::read_jar_file(reader, udf, "bluray_project.bin")?;
let strings = extract_strings(&data);
// min_len=4 matches the prior local extract_strings impl. The token
// grammar is `{lang3}_{codec?}_{purpose?}_{region?}_` so the
// shortest meaningful run is 4 chars (lang + underscore).
let strings = text::extract_ascii_strings(&data, 4);
// Tracked across all parse_token calls in this run: did any stream
// hit an unrecognized token component (skip-unknown path)? If yes
// we downgrade confidence to Medium — the labels are still valid
// but the corpus surfaced something we don't catalogue.
let saw_unknown = AtomicBool::new(false);
let mut labels = Vec::new();
let mut in_feature = false;
@@ -50,7 +63,7 @@ pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLab
continue;
}
if let Some(label) = parse_token(s) {
if let Some(label) = parse_token_inner(s, Some(&saw_unknown)) {
match label.stream_type {
StreamLabelType::Audio => {
audio_num += 1;
@@ -73,10 +86,15 @@ pub fn parse(reader: &mut dyn SectorReader, udf: &UdfFs) -> Option<Vec<StreamLab
if labels.is_empty() {
return None;
}
Some(labels)
let confidence = if saw_unknown.load(Ordering::Relaxed) {
Confidence::Medium
} else {
Confidence::High
};
Some(ParseResult { labels, confidence })
}
fn parse_token(s: &str) -> Option<StreamLabel> {
fn parse_token_inner(s: &str, saw_unknown: Option<&AtomicBool>) -> Option<StreamLabel> {
let clean = s.trim().trim_start_matches('\t').trim_end_matches('_');
let parts: Vec<&str> = clean.split('_').collect();
if parts.len() < 2 {
@@ -127,7 +145,18 @@ fn parse_token(s: &str) -> Option<StreamLabel> {
} else if part.starts_with("PGStream") {
is_subtitle = true;
} else {
return None;
// Unknown token component — skip this single part rather
// than discarding the entire stream record. Pre-refactor
// behavior was `return None` here, which silently dropped
// any stream containing a single uncatalogued token (e.g.
// a new codec ID or framework variant). Better to surface
// what we know than discard a whole stream over one part,
// but flag the parse as Medium-confidence so callers know
// some data was elided.
tracing::debug!(part = %part, "pixelogic: unrecognized token component, skipping");
if let Some(flag) = saw_unknown {
flag.store(true, Ordering::Relaxed);
}
}
}
@@ -153,22 +182,72 @@ fn parse_token(s: &str) -> Option<StreamLabel> {
})
}
fn extract_strings(data: &[u8]) -> Vec<String> {
let mut strings = Vec::new();
let mut current = String::new();
// extract_strings removed — replaced by super::text::extract_ascii_strings(data, 4).
for &b in data {
if (0x20..0x7f).contains(&b) {
current.push(b as char);
} else {
if current.len() > 3 {
strings.push(current.clone());
}
current.clear();
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_token_basic_audio() {
let l = parse_token_inner("eng_MLP_", None).unwrap();
assert_eq!(l.stream_type, StreamLabelType::Audio);
assert_eq!(l.language, "eng");
assert_eq!(l.codec_hint, "TrueHD");
assert_eq!(l.purpose, LabelPurpose::Normal);
}
if current.len() > 3 {
strings.push(current);
#[test]
fn parse_token_basic_subtitle_sdh() {
let l = parse_token_inner("eng_SDH_", None).unwrap();
assert_eq!(l.stream_type, StreamLabelType::Subtitle);
assert_eq!(l.language, "eng");
assert_eq!(l.qualifier, LabelQualifier::Sdh);
}
#[test]
fn parse_token_commentary() {
let l = parse_token_inner("eng_MLP_ACOM_", None).unwrap();
assert_eq!(l.stream_type, StreamLabelType::Audio);
assert_eq!(l.purpose, LabelPurpose::Commentary);
}
#[test]
fn parse_token_descriptive() {
let l = parse_token_inner("eng_AC3_ADES_", None).unwrap();
assert_eq!(l.purpose, LabelPurpose::Descriptive);
}
#[test]
fn parse_token_with_region() {
let l = parse_token_inner("eng_MLP_US_", None).unwrap();
assert_eq!(l.language, "eng");
assert_eq!(l.variant, "US");
}
#[test]
fn parse_token_unknown_component_does_not_kill_stream() {
// Regression: pre-refactor, an unrecognized token part returned
// None for the whole stream, silently dropping it. New
// behavior: skip the unknown part, surface what we know.
let l = parse_token_inner("eng_MLP_FUTUREFLAG_FOR_", None).unwrap();
assert_eq!(l.stream_type, StreamLabelType::Audio);
assert_eq!(l.language, "eng");
assert_eq!(l.codec_hint, "TrueHD");
assert_eq!(l.qualifier, LabelQualifier::Forced);
}
#[test]
fn parse_token_no_audio_or_subtitle_signal_returns_none() {
// A token that has only a language and an unknown part with
// no audio/subtitle classifier should still return None —
// there's no way to file it as a stream.
assert!(parse_token_inner("eng_UNKNOWN_", None).is_none());
}
#[test]
fn parse_token_rejects_non_lang_prefix() {
assert!(parse_token_inner("XX_MLP_", None).is_none());
assert!(parse_token_inner("ENG_MLP_", None).is_none()); // uppercase not accepted as ISO 639-2
}
strings
}
+89
View File
@@ -0,0 +1,89 @@
//! Text-extraction helpers used by parsers that scan binary blobs for
//! embedded label strings.
//!
//! Promoted from two near-duplicate implementations:
//! - `pixelogic::extract_strings` (`bluray_project.bin`, min_len=4)
//! - `dbp::extract_printable` (`.class` files in jars, min_len=5)
//!
//! Single implementation, threshold passed in. Callers that have a
//! more structured parse path (e.g. `class_reader` for .class) should
//! prefer that — this helper is for genuinely unstructured input.
/// Walk `data`, emit every maximal run of printable-ASCII bytes
/// (`0x20..=0x7E`) whose length is at least `min_len`.
///
/// Non-printable bytes (including `\t`, `\n`, NUL) terminate the
/// current run. Output strings are guaranteed valid UTF-8 (they're
/// pure 7-bit ASCII). Strings shorter than `min_len` are dropped.
pub fn extract_ascii_strings(data: &[u8], min_len: usize) -> Vec<String> {
let mut out = Vec::new();
let mut current = String::new();
for &b in data {
if (0x20..=0x7E).contains(&b) {
current.push(b as char);
} else if current.len() >= min_len {
out.push(std::mem::take(&mut current));
} else {
current.clear();
}
}
if current.len() >= min_len {
out.push(current);
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn extracts_simple_runs() {
let got = extract_ascii_strings(b"hello\0world\0", 3);
assert_eq!(got, vec!["hello", "world"]);
}
#[test]
fn applies_minimum_length() {
let got = extract_ascii_strings(b"hi\0ok\0longer\0", 5);
assert_eq!(got, vec!["longer"]);
}
#[test]
fn treats_tab_and_newline_as_separators() {
// \t (0x09) and \n (0x0A) are below 0x20, so they break runs.
let got = extract_ascii_strings(b"alpha\tbeta\ngamma", 3);
assert_eq!(got, vec!["alpha", "beta", "gamma"]);
}
#[test]
fn handles_trailing_run_without_terminator() {
// Run at the very end of the buffer should still be emitted.
let got = extract_ascii_strings(b"prefix\0tail", 3);
assert_eq!(got, vec!["prefix", "tail"]);
}
#[test]
fn rejects_high_bit_bytes() {
// 0x80+ is non-printable per this helper's definition.
let mut buf = b"good".to_vec();
buf.push(0xC3);
buf.push(0xA9);
buf.extend_from_slice(b"more");
let got = extract_ascii_strings(&buf, 3);
assert_eq!(got, vec!["good", "more"]);
}
#[test]
fn empty_input_returns_empty() {
let got = extract_ascii_strings(&[], 1);
assert!(got.is_empty());
}
#[test]
fn min_len_zero_emits_singletons() {
// Pathological but well-defined.
let got = extract_ascii_strings(b"a\0b", 0);
assert_eq!(got, vec!["a", "b"]);
}
}
+432 -9
View File
@@ -1,19 +1,40 @@
//! Shared label vocabulary — values we are 100% confident about.
//! Shared label vocabulary — canonical mappings used by ≥1 label parser.
//!
//! Labels come from BD-J authoring tool files (bluray_project.bin,
//! playlists.xml, menu_base.prop, etc.) — NOT from BD spec fields.
//! This is NOT for MPLS/CLPI/STN data. Those follow the BD spec directly.
//! playlists.xml, menu_base.prop, .class string pools, etc.) — NOT
//! from BD spec fields. This module is the central, regression-tested
//! source of truth for:
//!
//! Rules:
//! - Only map values we are 100% certain about (published codec names).
//! - Unknown codes (csp, eda, cf, etc.) pass through raw from disc.
//! - The app/CLI handles display text, not the lib.
//! - Codec brand name aliases (`MLP` → `TrueHD`).
//! - English / multi-word language name → ISO 639-2 code.
//! - English text → [`LabelPurpose`] (Commentary / Descriptive / etc.).
//! - English text → [`LabelQualifier`] (SDH / Forced / Descriptive Service).
//!
//! Rules of engagement (carried over from
//! `(internal)/memory/feedback_label_data_rules.md`):
//!
//! 1. Only map values we are 100% certain about — published codec
//! names, well-known ISO 639-2 mappings, vendor-documented purpose
//! keywords.
//! 2. Unknown codes / unrecognized phrases pass through raw or return
//! `None`. We never guess.
//! 3. Matching is case-insensitive and word-boundary-aware where
//! relevant (so "Commenter" doesn't match "commentary"). Anchoring
//! on whole tokens is the responsibility of this module — callers
//! pass raw text, we handle it.
//!
//! This module is NOT for BD spec STN codec IDs; those decode in
//! `mpls.rs` separately.
use super::{LabelPurpose, LabelQualifier};
// ── Codec aliases ────────────────────────────────────────────────────────────
/// Map a codec identifier found in label data to its display name.
///
/// These are well-known codec identifiers used across multiple BD-J
/// authoring tools. NOT BD spec STN codec IDs — those are decoded
/// separately in mpls.rs.
/// authoring tools. Unknown codes pass through unchanged so callers
/// can still surface vendor-specific tokens we haven't catalogued.
pub fn codec(code: &str) -> &str {
match code {
"MLP" => "TrueHD",
@@ -25,3 +46,405 @@ pub fn codec(code: &str) -> &str {
_ => code,
}
}
// ── Language: English / multi-word names → ISO 639-2 ─────────────────────────
/// Result of [`lang`] — ISO code + human-readable regional variant.
///
/// `code` is ISO 639-2 (always 3 lowercase letters).
/// `variant` is the regional dialect as a human-readable English word
/// (`"Brazilian"`, `"Castilian"`, `"Canadian"`, `"Simplified"`, ...)
/// or `""` when the input names just a bare language without
/// dialect ("Spanish" → variant=""). The variant matches the
/// convention pixelogic / ctrm / criterion already use for their
/// `StreamLabel::variant` field: a short display token the UI can
/// surface verbatim.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct LangInfo {
pub code: &'static str,
pub variant: &'static str,
}
/// Map a free-form language label fragment to an ISO 639-2 code AND
/// (where applicable) its regional variant.
///
/// Handles both bare English names ("English", "Spanish") and the
/// multi-word vendor variants we've seen in the corpus ("Brazilian
/// Portuguese", "Castilian Spanish", "Canadian French"). Match is
/// case-insensitive; longer compound phrases win over their bare
/// counterparts (so "Brazilian Portuguese" returns
/// `LangInfo { code: "por", variant: "Brazilian" }`, not consumed by
/// the bare "Portuguese" entry).
///
/// Bare-name matches return `variant: ""`.
///
/// Returns `None` for unrecognized input — callers decide whether to
/// fall back to MPLS spec codes, pass through raw, or drop the stream.
/// Never guesses.
///
/// Why the variant: the prior `lang() -> Option<&str>` shape silently
/// dropped regional dialect info. "Brazilian Portuguese 5.1" became
/// `language="por", variant=""` — UI displayed plain "Portuguese"
/// even though the disc had explicitly labeled this stream Brazilian.
/// Capturing the variant here parallels how pixelogic and ctrm
/// populate `StreamLabel::variant` from their own region tables.
pub fn lang(text: &str) -> Option<LangInfo> {
let lower = text.to_lowercase();
// Multi-word compounds first — longest-match wins.
for (needle, code, variant) in COMPOUND_LANGS {
if lower.contains(needle) {
return Some(LangInfo { code, variant });
}
}
// Bare names: word-boundary match (avoid "english" inside "englishman"
// or any other accidental substring).
for (needle, code) in BARE_LANGS {
if has_word(&lower, needle) {
return Some(LangInfo { code, variant: "" });
}
}
None
}
const COMPOUND_LANGS: &[(&str, &str, &str)] = &[
("brazilian portuguese", "por", "Brazilian"),
("euro portuguese", "por", "European"),
("european portuguese", "por", "European"),
("castilian spanish", "spa", "Castilian"),
("latin american spanish", "spa", "Latin American"),
("latin spanish", "spa", "Latin American"),
("canadian french", "fra", "Canadian"),
("parisian french", "fra", "Parisian"),
("australian english", "eng", "Australian"),
("austrailian english", "eng", "Australian"), // disc-corpus typo, keep matching
("british english", "eng", "British"),
("simplified chinese", "zho", "Simplified"),
("traditional chinese", "zho", "Traditional"),
("mandarin chinese", "zho", "Mandarin"),
("cantonese chinese", "zho", "Cantonese"),
];
const BARE_LANGS: &[(&str, &str)] = &[
("english", "eng"),
("spanish", "spa"),
("french", "fra"),
("german", "deu"),
("italian", "ita"),
("japanese", "jpn"),
("chinese", "zho"),
("mandarin", "zho"),
("cantonese", "zho"),
("portuguese", "por"),
("polish", "pol"),
("czech", "ces"),
("hungarian", "hun"),
("dutch", "nld"),
("korean", "kor"),
("arabic", "ara"),
("hindi", "hin"),
("turkish", "tur"),
("thai", "tha"),
("swedish", "swe"),
("norwegian", "nor"),
("danish", "dan"),
("finnish", "fin"),
("hebrew", "heb"),
("russian", "rus"),
("greek", "ell"),
("vietnamese", "vie"),
("indonesian", "ind"),
("malay", "msa"),
("ukrainian", "ukr"),
("romanian", "ron"),
("bulgarian", "bul"),
("croatian", "hrv"),
("serbian", "srp"),
("slovak", "slk"),
("slovenian", "slv"),
("estonian", "est"),
("latvian", "lav"),
("lithuanian", "lit"),
("icelandic", "isl"),
("basque", "eus"),
("catalan", "cat"),
("galician", "glg"),
];
// ── Purpose ──────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelPurpose`].
///
/// Recognized keywords (case-insensitive, word-boundary matched):
/// - "commentary", "director's commentary" → `Commentary`
/// - "descriptive", "description", "audio description", "described" → `Descriptive`
/// - "score", "music only" → `Score`
/// - "ime" (alternate music for closing themes etc.) → `Ime`
/// - anything else → `Normal`
///
/// Word-boundary matching means "Commentary track" matches but
/// "Commenter Pro audio" does not.
pub fn purpose(text: &str) -> LabelPurpose {
let lower = text.to_lowercase();
// Multi-word compounds first — they're more specific.
if lower.contains("audio description") || lower.contains("descriptive service") {
return LabelPurpose::Descriptive;
}
if lower.contains("music only") {
return LabelPurpose::Score;
}
if has_word(&lower, "commentary") {
return LabelPurpose::Commentary;
}
if has_word(&lower, "descriptive")
|| has_word(&lower, "description")
|| has_word(&lower, "described")
{
return LabelPurpose::Descriptive;
}
if has_word(&lower, "score") {
return LabelPurpose::Score;
}
if has_word(&lower, "ime") {
return LabelPurpose::Ime;
}
LabelPurpose::Normal
}
// ── Qualifier ────────────────────────────────────────────────────────────────
/// Classify a free-form English label string into a [`LabelQualifier`].
///
/// Recognized keywords (case-insensitive, word-boundary matched):
/// - "sdh", "captions" → `Sdh`
/// - "forced", "forced narrative" → `Forced`
/// - "rnib", "descriptive service" → `DescriptiveService`
/// - anything else → `None`
///
/// SDH (Subtitles for the Deaf and Hard of hearing) wins over Forced
/// when both keywords are present, because an SDH track is its own
/// stream regardless of whether the player flags it as "forced".
pub fn qualifier(text: &str) -> LabelQualifier {
let lower = text.to_lowercase();
if has_word(&lower, "sdh") || has_word(&lower, "captions") {
return LabelQualifier::Sdh;
}
if lower.contains("descriptive service") || has_word(&lower, "rnib") {
return LabelQualifier::DescriptiveService;
}
if has_word(&lower, "forced") {
return LabelQualifier::Forced;
}
LabelQualifier::None
}
// ── Internal: word-boundary matching ────────────────────────────────────────
/// True if `needle` appears in `haystack` surrounded by non-alphanumeric
/// boundaries (or string ends). `haystack` is assumed lowercase already.
///
/// This is the load-bearing primitive for `lang` / `purpose` /
/// `qualifier`: bare-token matchers MUST use it, otherwise we match
/// "english" inside "englishman" and "sdh" inside "lambdash". The
/// existing parsers used `.contains()` and got lucky on the corpus;
/// vocab guarantees the boundary.
fn has_word(haystack: &str, needle: &str) -> bool {
if needle.is_empty() {
return false;
}
let bytes = haystack.as_bytes();
let nb = needle.as_bytes();
let mut i = 0;
while i + nb.len() <= bytes.len() {
if &bytes[i..i + nb.len()] == nb {
let before = if i == 0 { None } else { Some(bytes[i - 1]) };
let after = bytes.get(i + nb.len()).copied();
let bound = |c: Option<u8>| match c {
None => true,
Some(b) => !b.is_ascii_alphanumeric(),
};
if bound(before) && bound(after) {
return true;
}
}
i += 1;
}
false
}
// ── Tests ────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn codec_known_aliases() {
assert_eq!(codec("MLP"), "TrueHD");
assert_eq!(codec("AC3"), "Dolby Digital");
assert_eq!(codec("AC"), "Dolby Digital");
assert_eq!(codec("DDL"), "Dolby Digital Plus");
assert_eq!(codec("atmos"), "Dolby Atmos");
}
#[test]
fn codec_unknown_passes_through() {
assert_eq!(codec("FX9"), "FX9");
assert_eq!(codec(""), "");
}
fn li(code: &'static str, variant: &'static str) -> LangInfo {
LangInfo { code, variant }
}
#[test]
fn lang_bare_names_have_empty_variant() {
assert_eq!(lang("English"), Some(li("eng", "")));
assert_eq!(lang("english"), Some(li("eng", "")));
assert_eq!(lang("Spanish 5.1 Dolby Digital"), Some(li("spa", "")));
assert_eq!(lang("japanese"), Some(li("jpn", "")));
assert_eq!(lang("Italian"), Some(li("ita", "")));
}
#[test]
fn lang_compounds_carry_variant() {
assert_eq!(
lang("Brazilian Portuguese 5.1"),
Some(li("por", "Brazilian"))
);
assert_eq!(lang("Castilian Spanish"), Some(li("spa", "Castilian")));
assert_eq!(
lang("Canadian French Dolby Digital"),
Some(li("fra", "Canadian"))
);
assert_eq!(
lang("Latin American Spanish"),
Some(li("spa", "Latin American"))
);
assert_eq!(lang("Simplified Chinese"), Some(li("zho", "Simplified")));
assert_eq!(lang("British English"), Some(li("eng", "British")));
}
#[test]
fn lang_compounds_win_over_bare() {
// Brazilian Portuguese must map to (por, Brazilian) via the
// compound rule, not be intercepted by bare "portuguese"
// (which would yield (por, "") and lose the variant).
assert_eq!(lang("Brazilian Portuguese").unwrap().variant, "Brazilian");
assert_eq!(lang("Canadian French").unwrap().variant, "Canadian");
}
#[test]
fn lang_unknown_returns_none() {
assert_eq!(lang("Klingon Dolby Atmos"), None);
assert_eq!(lang(""), None);
assert_eq!(lang("eng"), None); // 3-letter codes are not English names
}
#[test]
fn lang_word_boundary_avoids_substring_false_positive() {
// No false positive — "engineering" must NOT match "english".
assert_eq!(lang("Audio Engineering Demo"), None);
}
#[test]
fn purpose_recognizes_commentary() {
assert_eq!(purpose("English Commentary"), LabelPurpose::Commentary);
assert_eq!(purpose("Director's Commentary"), LabelPurpose::Commentary);
}
#[test]
fn purpose_recognizes_descriptive() {
assert_eq!(
purpose("English Descriptive Audio"),
LabelPurpose::Descriptive
);
assert_eq!(purpose("Audio Description"), LabelPurpose::Descriptive);
assert_eq!(purpose("Described Video"), LabelPurpose::Descriptive);
}
#[test]
fn purpose_descriptive_service_routes_to_descriptive() {
// "Descriptive Service" is qualifier territory but the purpose
// implication is Descriptive — vocab::purpose treats it as such.
assert_eq!(
purpose("English Descriptive Service"),
LabelPurpose::Descriptive
);
}
#[test]
fn purpose_word_boundary_avoids_commenter_false_positive() {
// "Commenter Pro audio" does NOT match "commentary" — the
// existing dbp/ctrm hand-rolls would have. Vocab is stricter.
assert_eq!(purpose("Commenter Pro audio track"), LabelPurpose::Normal);
}
#[test]
fn purpose_recognizes_score() {
assert_eq!(purpose("Music Only"), LabelPurpose::Score);
assert_eq!(purpose("Isolated Score"), LabelPurpose::Score);
}
#[test]
fn purpose_unknown_is_normal() {
assert_eq!(purpose("English Dolby Atmos"), LabelPurpose::Normal);
assert_eq!(purpose(""), LabelPurpose::Normal);
}
#[test]
fn qualifier_recognizes_sdh() {
assert_eq!(qualifier("English SDH"), LabelQualifier::Sdh);
assert_eq!(qualifier("English Captions"), LabelQualifier::Sdh);
}
#[test]
fn qualifier_recognizes_forced() {
assert_eq!(qualifier("English Forced"), LabelQualifier::Forced);
assert_eq!(qualifier("Forced Narrative"), LabelQualifier::Forced);
}
#[test]
fn qualifier_recognizes_descriptive_service() {
assert_eq!(
qualifier("English RNIB"),
LabelQualifier::DescriptiveService
);
assert_eq!(
qualifier("English Descriptive Service"),
LabelQualifier::DescriptiveService
);
}
#[test]
fn qualifier_sdh_wins_over_forced_when_both_present() {
// SDH track is its own stream regardless of forced flag.
assert_eq!(qualifier("English Forced SDH"), LabelQualifier::Sdh);
}
#[test]
fn qualifier_unknown_is_none() {
assert_eq!(qualifier("English"), LabelQualifier::None);
assert_eq!(qualifier(""), LabelQualifier::None);
}
#[test]
fn has_word_basic() {
assert!(has_word("english forced", "english"));
assert!(has_word("english forced", "forced"));
assert!(has_word("english", "english"));
}
#[test]
fn has_word_rejects_substring() {
assert!(!has_word("engineering", "english"));
assert!(!has_word("englishman", "english"));
assert!(!has_word("aenglish", "english"));
}
#[test]
fn has_word_punctuation_boundary() {
// "(SDH)" is a valid boundary — parentheses count as non-alphanum.
assert!(has_word("english (sdh)", "sdh"));
assert!(has_word("commentary,extra,info", "commentary"));
}
}
+452
View File
@@ -0,0 +1,452 @@
//! Tolerant XML scraping helpers — promoted from two near-duplicate
//! hand-rolls in `paramount.rs` (attribute extraction) and `criterion.rs`
//! (tag-text extraction).
//!
//! These are NOT a full XML parser. They handle the subset of XML the
//! BD-J authoring tools we've seen actually emit: ASCII tag/attr
//! names, no entity references inside label strings, optional XML
//! namespaces. Hardening goals over the prior `find("<tag>")` /
//! `find(r#"name=""#)` matchers:
//!
//! 1. **Case-insensitive** tag and attribute names — vendors casing
//! is inconsistent across authoring-tool revisions.
//! 2. **Namespace-aware** — strip an optional `ns:` prefix so
//! `<ns:playlist>` and `<playlist>` both match.
//! 3. **Whitespace-tolerant** — multiple/tab/newline characters
//! around `=` between attribute name and value; whitespace inside
//! the opening tag.
//! 4. **Quote-style tolerant** — both `"value"` and `'value'`.
//! 5. **Self-closing tag handling** — `<tag />` and `<tag/>` both
//! work; [`text`] returns `Some("")` for empty content.
//!
//! Out of scope (intentionally simple): XML entity decoding
//! (`&amp;`, `&lt;`, etc.), CDATA sections, comments, processing
//! instructions, DTD declarations. None of the BD-J authored disc
//! data we've observed exercises any of those — labels are plain
//! ASCII/Latin-1 in attribute values.
/// Extract the value of attribute `name` from one XML element
/// fragment (e.g. `<playlist name="Feature" id="00222" />`).
///
/// Returns the raw attribute text (no entity decoding) or `None` if
/// the attribute isn't present. Empty string for `name=""` is
/// represented as `Some("")`.
pub fn attr(element: &str, name: &str) -> Option<String> {
let bytes = element.as_bytes();
let name_lower = name.to_ascii_lowercase();
let name_bytes = name_lower.as_bytes();
let mut i = 0;
while i + name_bytes.len() < bytes.len() {
// Find the next position where `name=` could start. We need
// a word boundary before the name (whitespace or `<` or `:`).
if i > 0 && is_name_char(bytes[i - 1]) {
i += 1;
continue;
}
if !slice_eq_ignore_case(&bytes[i..i + name_bytes.len()], name_bytes) {
i += 1;
continue;
}
let after_name = i + name_bytes.len();
// The character immediately after the name must not be a
// name-continuation (otherwise we matched a prefix like
// `lang_id` when looking for `lang`).
if after_name < bytes.len() && is_name_char(bytes[after_name]) {
i = after_name;
continue;
}
// Walk past whitespace, then `=`, then more whitespace, then
// the opening quote.
let mut j = after_name;
while j < bytes.len() && is_ws(bytes[j]) {
j += 1;
}
if j >= bytes.len() || bytes[j] != b'=' {
i = j.max(i + 1);
continue;
}
j += 1; // past '='
while j < bytes.len() && is_ws(bytes[j]) {
j += 1;
}
if j >= bytes.len() {
return None;
}
let quote = bytes[j];
if quote != b'"' && quote != b'\'' {
// Unquoted attribute values aren't part of well-formed
// XML (HTML5 allows them, XML doesn't). Skip.
i = j;
continue;
}
let value_start = j + 1;
let close = bytes[value_start..].iter().position(|&b| b == quote)?;
let value = &element[value_start..value_start + close];
return Some(value.to_string());
}
None
}
/// Extract the trimmed text content of the first occurrence of
/// `<tag>...</tag>` in `xml`. Returns `None` if the tag isn't found
/// or its opening tag is malformed.
///
/// Whitespace around the inner text is stripped. Self-closing
/// `<tag />` yields `Some("")`. Nested same-name tags are NOT
/// handled — the first close encountered wins (this matches the
/// prior behavior in criterion.rs).
pub fn text(xml: &str, tag: &str) -> Option<String> {
let (open_end, body_start) = find_open_tag(xml, tag, 0)?;
// Self-closing — already consumed in find_open_tag if `/>`.
if open_end == body_start {
// Means find_open_tag returned the same offset twice for
// self-closing form. (Not currently the case in our impl,
// but defensive.)
return Some(String::new());
}
// For self-closing tags, body_start is past `/>` and we have no
// content. Detect by checking the char at body_start - 1 was `/`.
if body_start >= 2 && &xml[body_start - 2..body_start] == "/>" {
return Some(String::new());
}
// Find the matching close tag. Case-insensitive + namespace-aware.
let close_start = find_close_tag(xml, tag, body_start)?;
Some(xml[body_start..close_start].trim().to_string())
}
/// Locate the next `<tag>` opening AND its closing `</tag>` in
/// `xml`, starting at byte offset `from`. Returns `(element_start,
/// element_end)` — `element_start` is the `<` of the opening tag,
/// `element_end` is one past the `>` of the closing tag. Useful for
/// iterating over repeated elements like `<playlist>` blocks in
/// `paramount`.
///
/// Self-closing elements return the same offset for body_end as the
/// element_end (i.e. `element_end - element_start` includes only the
/// `<tag .../>` text).
pub fn find_element(xml: &str, tag: &str, from: usize) -> Option<(usize, usize)> {
let bytes = xml.as_bytes();
let mut i = from;
while i < bytes.len() {
if bytes[i] != b'<' {
i += 1;
continue;
}
// Try matching tag name at i+1 (after `<`).
let after_lt = i + 1;
if !matches_tag_name_at(bytes, after_lt, tag) {
i += 1;
continue;
}
// Found an open tag at offset i. Walk to find the closing `>`
// of the open tag itself.
let mut j = after_lt;
// Skip past the tag name (and optional namespace prefix).
while j < bytes.len() && (is_name_char(bytes[j]) || bytes[j] == b':') {
j += 1;
}
// Walk attributes — track quoting state.
let mut self_closing = false;
while j < bytes.len() {
match bytes[j] {
b'>' => {
j += 1;
break;
}
b'/' if j + 1 < bytes.len() && bytes[j + 1] == b'>' => {
self_closing = true;
j += 2;
break;
}
b'"' | b'\'' => {
let q = bytes[j];
j += 1;
while j < bytes.len() && bytes[j] != q {
j += 1;
}
if j < bytes.len() {
j += 1;
}
}
_ => j += 1,
}
}
if self_closing {
return Some((i, j));
}
// Find matching close. Doesn't handle nested same-name; OK
// for our authoring-tool subset.
let close_start = find_close_tag(xml, tag, j)?;
let close_end = find_byte(bytes, b'>', close_start)? + 1;
return Some((i, close_end));
}
None
}
// ── Internal helpers ───────────────────────────────────────────────────────
/// True if `bytes[start..]` opens a tag named `tag`, allowing an
/// optional `ns:` namespace prefix. Comparison is case-insensitive.
/// The character after the tag name must not be a name-continuation
/// (so `<player>` doesn't match `<play>`).
fn matches_tag_name_at(bytes: &[u8], start: usize, tag: &str) -> bool {
let tag_lower = tag.to_ascii_lowercase();
let tag_bytes = tag_lower.as_bytes();
// Skip optional `prefix:` (one or more name chars + `:`).
let mut name_start = start;
let mut scan = start;
while scan < bytes.len() && is_name_char(bytes[scan]) {
scan += 1;
}
if scan < bytes.len() && bytes[scan] == b':' {
name_start = scan + 1;
}
if name_start + tag_bytes.len() > bytes.len() {
return false;
}
if !slice_eq_ignore_case(&bytes[name_start..name_start + tag_bytes.len()], tag_bytes) {
return false;
}
// Boundary: char after the tag name must be `>`, `/`, whitespace.
let after = name_start + tag_bytes.len();
if after >= bytes.len() {
return false;
}
matches!(bytes[after], b'>' | b'/' | b' ' | b'\t' | b'\n' | b'\r')
}
/// Find the offset of the next `</tag>` (or `</ns:tag>`) in `xml`
/// starting at `from`. Case-insensitive; returns the offset of the
/// `<`. None if not found.
fn find_close_tag(xml: &str, tag: &str, from: usize) -> Option<usize> {
let bytes = xml.as_bytes();
let mut i = from;
while i + 2 < bytes.len() {
if bytes[i] == b'<' && bytes[i + 1] == b'/' {
// Check tag name (with optional namespace).
if matches_tag_name_at(bytes, i + 2, tag) {
return Some(i);
}
}
i += 1;
}
None
}
/// Find the open tag of `<tag>` in `xml` starting at `from`. Returns
/// `(after_open_lt, after_open_gt)` — the offsets are: just past
/// the `<` of the open tag, and just past the `>` of the open tag.
fn find_open_tag(xml: &str, tag: &str, from: usize) -> Option<(usize, usize)> {
let bytes = xml.as_bytes();
let (elem_start, _) = find_element(xml, tag, from)?;
let after_lt = elem_start + 1;
// Find the `>` that ends the open tag (handling quoted attrs).
let mut j = after_lt;
while j < bytes.len() {
match bytes[j] {
b'>' => return Some((after_lt, j + 1)),
b'/' if j + 1 < bytes.len() && bytes[j + 1] == b'>' => {
return Some((after_lt, j + 2));
}
b'"' | b'\'' => {
let q = bytes[j];
j += 1;
while j < bytes.len() && bytes[j] != q {
j += 1;
}
if j < bytes.len() {
j += 1;
}
}
_ => j += 1,
}
}
None
}
fn find_byte(bytes: &[u8], target: u8, from: usize) -> Option<usize> {
bytes[from..]
.iter()
.position(|&b| b == target)
.map(|p| p + from)
}
/// True if `c` can be part of an XML name token (rough). We accept
/// alphanumerics, `_`, `-`, `.`.
fn is_name_char(c: u8) -> bool {
c.is_ascii_alphanumeric() || c == b'_' || c == b'-' || c == b'.'
}
fn is_ws(c: u8) -> bool {
matches!(c, b' ' | b'\t' | b'\n' | b'\r')
}
fn slice_eq_ignore_case(a: &[u8], b_lower: &[u8]) -> bool {
if a.len() != b_lower.len() {
return false;
}
a.iter()
.zip(b_lower.iter())
.all(|(&x, &y)| x.to_ascii_lowercase() == y)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn attr_basic() {
assert_eq!(
attr(r#"<playlist name="Feature" id="00222" />"#, "name"),
Some("Feature".into())
);
assert_eq!(
attr(r#"<playlist name="Feature" id="00222" />"#, "id"),
Some("00222".into())
);
}
#[test]
fn attr_case_insensitive_name() {
assert_eq!(
attr(r#"<playlist Name="Feature" />"#, "name"),
Some("Feature".into())
);
assert_eq!(
attr(r#"<playlist NAME="Feature" />"#, "Name"),
Some("Feature".into())
);
}
#[test]
fn attr_accepts_single_quotes() {
assert_eq!(
attr(r#"<playlist name='Feature' />"#, "name"),
Some("Feature".into())
);
}
#[test]
fn attr_whitespace_around_equals() {
assert_eq!(
attr(r#"<playlist name = "Feature" />"#, "name"),
Some("Feature".into())
);
assert_eq!(
attr("<playlist name\n=\n\"Feature\" />", "name"),
Some("Feature".into())
);
}
#[test]
fn attr_missing_returns_none() {
assert_eq!(attr(r#"<playlist name="X" />"#, "id"), None);
assert_eq!(attr("", "name"), None);
}
#[test]
fn attr_no_substring_false_positive() {
// Looking for "lang" should NOT match "lang_id" or
// "language" because of the name-char boundary check.
assert_eq!(attr(r#"<x lang_id="fra" language="eng" />"#, "lang"), None);
}
#[test]
fn attr_empty_value() {
assert_eq!(attr(r#"<x name="" id="1" />"#, "name"), Some("".into()));
}
#[test]
fn text_basic() {
assert_eq!(text("<x>hello</x>", "x"), Some("hello".into()));
assert_eq!(
text("<x> hello world </x>", "x"),
Some("hello world".into())
);
}
#[test]
fn text_case_insensitive_tag() {
assert_eq!(text("<X>foo</X>", "x"), Some("foo".into()));
assert_eq!(text("<Foo>bar</foo>", "foo"), Some("bar".into()));
}
#[test]
fn text_namespace_prefix() {
assert_eq!(text("<ns:tag>value</ns:tag>", "tag"), Some("value".into()));
assert_eq!(text("<foo:Bar>v</foo:Bar>", "bar"), Some("v".into()));
}
#[test]
fn text_self_closing() {
assert_eq!(text("<x/>", "x"), Some("".into()));
assert_eq!(text("<x />", "x"), Some("".into()));
assert_eq!(text("<x attr=\"y\" />", "x"), Some("".into()));
}
#[test]
fn text_with_attrs() {
assert_eq!(
text(r#"<x id="1" name="y">hello</x>"#, "x"),
Some("hello".into())
);
}
#[test]
fn text_missing_close_returns_none() {
assert_eq!(text("<x>hello", "x"), None);
}
#[test]
fn text_skips_inner_tags_naively() {
// Limitation noted: nested same-name tags aren't handled.
// Different-name nesting works (we just return everything
// between the open and close).
assert_eq!(
text("<x><y>nested</y></x>", "x"),
Some("<y>nested</y>".into())
);
}
#[test]
fn find_element_basic() {
let xml = r#"<x /> <y attr="1">body</y>"#;
let (s, e) = find_element(xml, "y", 0).unwrap();
assert_eq!(&xml[s..e], r#"<y attr="1">body</y>"#);
}
#[test]
fn find_element_self_closing() {
let xml = r#"<x />"#;
let (s, e) = find_element(xml, "x", 0).unwrap();
assert_eq!(&xml[s..e], "<x />");
}
#[test]
fn find_element_handles_quoted_gt_in_attr() {
// A `>` inside a quoted attribute value should not terminate
// the open tag prematurely.
let xml = r#"<x attr="foo>bar">body</x>"#;
let (s, e) = find_element(xml, "x", 0).unwrap();
assert_eq!(&xml[s..e], r#"<x attr="foo>bar">body</x>"#);
}
#[test]
fn find_element_iteration() {
let xml = "<p>a</p><p>b</p><p>c</p>";
let mut positions = Vec::new();
let mut from = 0;
while let Some((s, e)) = find_element(xml, "p", from) {
positions.push(&xml[s..e]);
from = e;
}
assert_eq!(positions, vec!["<p>a</p>", "<p>b</p>", "<p>c</p>"]);
}
#[test]
fn find_element_with_namespace() {
let xml = r#"<root><ns:item id="1" /></root>"#;
let (s, e) = find_element(xml, "item", 0).unwrap();
assert_eq!(&xml[s..e], r#"<ns:item id="1" />"#);
}
}
+1 -1
View File
@@ -84,7 +84,7 @@ pub(crate) mod identity;
pub(crate) mod ifo;
pub(crate) mod io;
pub mod keydb;
pub(crate) mod labels;
pub mod labels;
pub(crate) mod mpls;
pub mod mux;
pub mod pes;
+2 -5
View File
@@ -657,11 +657,8 @@ mod tests {
// produces no frames, but the call still routes through the blanket
// dispatch into Stream::read.
let mut frames = 0usize;
loop {
match src.read().expect("read") {
Some(_) => frames += 1,
None => break,
}
while src.read().expect("read").is_some() {
frames += 1;
if frames > 1024 {
panic!("unexpected unbounded frame stream from empty title");
}
+27 -13
View File
@@ -459,11 +459,17 @@ fn test_disc_copy_completes_full_disc_with_failing_reader() {
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
// Hard bound — even at 0 ms per read, 1024 sectors with skip-forward
// should complete in well under a second on any host.
// Hard bound — Pass 1 must NOT infinite-loop on a fully-failing
// reader. The threshold accommodates the 2026-05-10 wedge-
// avoidance pause (PASS_1_FAIL_PAUSE_SECS = 5 s on each failed
// batch). With batch=32 and 1024 sectors that's up to ~5 batch
// failures + a few damage-jump pauses before fast-trigger jumps
// us past end-of-disc — well-bounded total, ~20-30 s typical.
// The point of this test is "finishes cleanly, not infinitely",
// not "completes in milliseconds."
assert!(
elapsed < Duration::from_secs(5),
"Pass 1 took {elapsed:?} on a 2 MB synthetic disc — expected < 5 s"
elapsed < Duration::from_secs(60),
"Pass 1 took {elapsed:?} on a 2 MB synthetic disc — expected < 60 s (not infinite)"
);
// Per RIP_DESIGN.md §2.1: Pass 1 must reach end of disc regardless of
@@ -624,19 +630,27 @@ fn test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed() {
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
// Pass 1 reads every batch at bpt=32 (no batch reduction, no skip-ahead).
// BlockSizeFailingReader fails on multi-sector reads but succeeds on single-sector.
// Bridge degradation handling retries failed batches as individual sectors,
// so all data is recovered as Finished. bytes_good == total_bytes is correct.
// Pass 1's job is "fast and accurate, get the most data in the
// shortest time." It no longer bisects on marginal media — that's
// Pass N's purpose-built role. So a BlockSizeFailingReader that
// fails on multi-sector reads and succeeds on single-sector
// results in: every batch fails → SkipBlock → whole 32-sector
// ECC block marked NonTrimmed → Pass N (Disc::patch) revisits and
// recovers via single-sector reads with proper recovery semantics.
//
// Pass 1 alone:
assert_eq!(
result.bytes_good, total_bytes,
"Pass 1 at bpt=32 recovers all sectors via single-sector retry on MEDIUM_ERROR"
result.bytes_good, 0,
"Pass 1 doesn't bisect on marginal media — failed batches become NonTrimmed for Pass N to revisit"
);
assert_eq!(
result.bytes_pending, 0,
"no pending sectors after full recovery"
result.bytes_pending, total_bytes,
"every sector is NonTrimmed (pending) after Pass 1, awaiting Pass N"
);
assert!(
!result.complete,
"complete=false because NonTrimmed regions remain (Pass N's work)"
);
assert!(result.complete, "complete=true when all sectors recovered");
}
// ── 9. PassProgress carries separate unreadable vs pending byte counts ─────
+3 -1
View File
@@ -30,8 +30,10 @@ struct PatternedSectorReader {
trace: Arc<Mutex<Vec<(u32, u16)>>>,
}
type ReadTrace = Arc<Mutex<Vec<(u32, u16)>>>;
impl PatternedSectorReader {
fn new(capacity: u32, bad_lbas: HashSet<u32>) -> (Self, Arc<Mutex<Vec<(u32, u16)>>>) {
fn new(capacity: u32, bad_lbas: HashSet<u32>) -> (Self, ReadTrace) {
let trace = Arc::new(Mutex::new(Vec::new()));
(
Self {