9043a4ba7aaf14880186501234bbdaf63b76e50a
441
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 matthew
|
c2634b4346 | v0.18.17: bump version | ||
|
matthew
|
aae8c6d0a6 | libfreemkv: add Clone derive to MuxAtomics | ||
|
matthew
|
85ed1885e8 | v0.18.16: bump version | ||
|
matthew
|
4c8f39b798 | add debug logging for MKB processing | ||
|
matthew
|
b3eeeb4b91 | optical vs block batch sizing | ||
|
matthew
|
531e999fe0 | v0.18.15: bump version | ||
|
matthew
|
e5d2e78c85 |
disc/scan: surface AACS resolution error on Disc.aacs_error
scan_with() collapsed every failure path from resolve_encryption() into None via .ok(), so callers couldn't tell the difference between "no KEYDB found", "KEYDB failed to parse", "disc hash not in KEYDB and fallback derivation failed", "AACS files unreadable on disc", and a handshake that rejected every host cert. autorip's UI was stuck printing "no decryption keys found (check KEYDB)" for all of them, which is a particularly bad message when the user has actually loaded a KEYDB and the real failure is something else. Changes: - New pub field Disc.aacs_error: Option<Error>. Populated by scan_with whenever encrypted && aacs.is_none(). Sentinel KeydbLoad path "<no keydb in search paths>" distinguishes the no-keydb case from a real load failure without adding a new Error variant (which would be a breaking change for downstream exhaustive matches). - tracing::warn in scan_with at scan_aacs_resolve_failed and scan_aacs_no_keydb, with error_code and keydb path for grepping. - tracing in do_handshake: keydb load failure, host-cert exhaustion (with cert count and last error code), VID read failure post-auth, and a debug-level success log. Lets us see whether handshake even got off the ground for a given disc. Test fixtures updated to set aacs_error: None. |
||
|
matthew
|
aecffdef0b | v0.18.14: bump version | ||
|
matthew
|
ec99008282 |
disc/patch: relabel "possible wedge" heuristic log to avoid confusion
The 'All probes failed — possible wedge condition' log fired during patch probing whenever 10+ consecutive failures hit AND a probe sweep at the local zone returned 0 successes. This was distinct from the read_error.rs 'wedge_transition' log that fires when the SCSI sense family ACTUALLY flips into Hardware/IllegalRequest fast-fail mode. Two logs both saying 'wedge' caused operator confusion during the 2026-05-11 Dune Pt 2 wedge investigation — was the drive wedged, or was it just a zone of fully-bad sectors? They mean different things. Relabel to 'patch_zone_fully_bad' with explicit pointer to read_error.rs for the canonical wedge detection. Same triggering condition; just clearer wording in the log stream. |
||
|
matthew
|
90504d53d6 |
labels/bdmt: drop description fields that are just XML child elements
Disc-04 (Top Gun: Maverick) re-test 2026-05-11 surfaced a real-world
bdmt_eng.xml where <di:description> contained no prose, only nested
<di:thumbnail href="…"/> elements. The previous parser surfaced
the raw XML fragment as the description string ("<di:thumbnail
href=\"tgm_meta_sm.jpg\" />\\r\\n <di:thumbnail
href=\"tgm_meta_lg.jpg\" />"). Worse than no description.
Fix: filter description candidates that begin with `<` after
trimming. Real prose never starts with an angle bracket; XML-only
content always does. Net: title extraction unaffected (it uses its
own element-priority path); description field drops when it would
otherwise carry XML noise.
Two new bdmt tests, 12 of 12 passing.
|
||
|
matthew
|
2ac636eab3 |
labels: append CLPI orphan streams after gap-fill
Three layered sources of stream labels now, in precedence order:
1. **Framework parser** (paramount/criterion/pixelogic/ctrm/dbp/deluxe)
— editorial labels with purpose/qualifier ("English Atmos",
"Director's Commentary", "English SDH"). High or Medium confidence.
2. **MPLS gap-fill** (`fill_gaps_from_mpls`) — every stream the
playlist references gets at least a basic lang+codec label, even
when the framework parser missed it.
3. **CLPI orphan append** (`append_clpi_orphans`) — streams in
/BDMV/CLIPINF/*.clpi ProgramInfo that NO MPLS playlist references.
Empirical (2026-05-11): ~5% of streams across the 11-disc corpus,
most dramatic on disc-02 (HDMV-only) at 40% CLPI-only.
Orphan numbering: each appended orphan gets
`stream_number = max(existing per type) + N` so playlist-reachable
streams keep their original positions and orphans sort cleanly at
the tail.
Orphan dedup: (stream_type, language, codec_hint) tuple — fuzzier
than PID matching (PIDs aren't carried on StreamLabel) but it's the
only signal available downstream of the gap-fill. False positives
(genuine orphan that happens to share lang+codec with an existing
entry) silently drop, which is the conservative failure mode — the
user-facing display would just see a confusing duplicate otherwise.
`mpls_universal::language_display_name` and `::codec_name` promoted
from private fn to pub(crate) so this module can build orphan labels
with consistent naming.
Tests: 2 new in gap_fill_tests — synthetic-input verification of the
dedup tuple logic and the stream_number assignment. 6/6 tests in the
gap-fill module now passing.
|
||
|
matthew
|
3b10b3ab9b | disc/read_error: rustfmt fixup for wedge-prevention commit | ||
|
matthew
|
1a3b154836 |
disc/read_error: wedge PREVENTION — jump on first error + 30s cooldown
Rewrites the Pass 1 wedge handling from "slow skip after the drive has already wedged" to "prevent the wedge transition in the first place." Driven by 2026-05-11 empirical data: the BU40N transitioned into IllegalRequest fast-fail mode at exactly 7 medium errors in 6.5 seconds (~1 read/sec retry cadence). Once there, only physical eject + reload clears it — 30s pauses + 1 GB jumps do not. The fix is the user's mental model from that session: "We can detect bad reads, failed reads, and asking to read again fast after causes a wedge. We need to prevent the wedge in the first place." Two changes to the centralized error handler: 1. **`for_sweep().fast_jump_threshold = 1`** (was 4). Pass 1 now JumpAheads on the FIRST outer-batch failure, not the 4th. The drive never gets back-to-back retries at the same LBA in Pass 1 — every error → jump 64 MB forward + long cooldown. Pass N keeps `fast_jump_threshold = u64::MAX` because retries on already-known- bad LBAs are its whole job. 2. **`ZONE_ENTRY_COOLDOWN_SECS = 30`**. The FIRST error after a clean run (when `consecutive_outer_failures == 1` and we're not bisecting) uses this long pause instead of the standard 5 s FAIL_PAUSE_SECS. Gives the BU40N's firmware / bridge internal retry counters 30 s of breathing room before the next read, preventing the "7 errors in 6.5 s" cascade. Subsequent errors in the same zone use the standard 5 s pause (we've already jumped past the initial damage; further errors mean we landed in another bad cluster). Pass N exempt from the zone-entry cooldown — `bisect_on_marginal= true` skips the long-pause arm. Pass N's per-sector retries on known-bad LBAs would multiply uselessly with 30 s/error. Test updates: 4 tests' expected behavior changed under the new policy. Renamed `pass_1_marginal_skips_instead_of_bisecting` → `pass_1_marginal_jumps_immediately_not_bisecting`. Renamed `pass_1_jumps_after_4_consecutive_outer_failures` → `pass_1_jumps_immediately_on_first_outer_failure`. Updated `both_passes_pause_on_failed_read_for_wedge_avoidance` (now `pass_1_zone_entry_uses_long_cooldown` + `pass_n_pauses_uniformly_on_failed_read`). Cost analysis: - Clean disc (no errors): unchanged. 0% overhead. - Lightly damaged (1-2 zones): +30 s per zone = ~1 min total. Fine. - Heavily damaged (10+ zones): +5+ min total. The trade for never wedging the drive and getting a usable Pass N afterwards. Expected behavior on the next damaged-disc rip: - Pass 1 hits damage at LBA X → jumps 64 MB forward immediately, pauses 30 s - Drive's firmware never accumulates the retry pressure that triggers IllegalRequest fast-fail - bytes_maybe accumulates faster (we skip more), but Pass N picks up the slack with proper per-sector recovery — and Pass N can actually RUN because the drive isn't wedged |
||
|
matthew
|
1fd734e2c5 |
clpi+labels: extract program_info stream table + CLPI vs MPLS audit
Two layered changes, in service of the empirical question "is CLPI
truly redundant with MPLS for label data?":
1. **clpi.rs ProgramInfo parser**. The existing CLPI parser only
walked the EP map (for sector-range lookups). Added a parser for
the ProgramInfo section's per-stream stream_coding_info table:
pid, coding_type, audio_format/rate, video_format/rate, ISO 639-2
language. Spec layout per libbluray clpi_parse.c. Best-effort —
malformed program_info leaves `streams: vec![]`, EP map keeps
working. `ClipInfo` gains a `streams: Vec<ClpiStream>` field.
2. **labels/clpi_audit.rs**. Diagnostic that walks both
`/BDMV/CLIPINF/*.clpi` (via the new program_info parser) and
`/BDMV/PLAYLIST/*.mpls`, builds a (PID → fields) merged view, and
classifies each row:
- `Match`: both sources agree (same coding_type + language)
- `ClpiOnly`: PID in CLPI but no MPLS playlist references it
(orphan stream on disc — reachable via low-level access, not via menu)
- `MplsOnly`: PID in MPLS but no CLPI lists it (would indicate a
parser bug; verified empirically that this NEVER happens)
- `Divergent`: same PID, different coding_type or language between
sources (playlist re-tagged or attribute encoding mismatch)
Surfaced via `labels-analyze` as `clpi_vs_mpls_audit: {matches,
clpi_only, mpls_only, divergent, total_pids}`. Doesn't affect the
label output — pure diagnostic.
Empirical findings on the 11-disc corpus (excl. disc-04 truncated):
- 226 matches / 0 mpls_only / 8 clpi_only / 5 divergent across 239 PIDs
- 6 of 10 non-truncated discs have CLPI-only streams (orphans)
- disc-02 (HDMV-only) is the most dramatic: 40% of its 5 streams are
CLPI-only — MPLS sees 3, CLPI sees 5
- Conclusion: CLPI is NOT truly redundant. ~5% of streams disc-wide
are CLPI-exclusive. Future work: layer CLPI as a tertiary source
below MPLS in the labels pipeline (orphan streams marked with even
lower confidence than MPLS).
|
||
|
matthew
|
e78b4effe9 |
labels: surface MPLS chapter summary in LabelAnalysis
LabelAnalysis gains `chapter_summary: Vec<ChapterSummary>` — one row per .mpls file in /BDMV/PLAYLIST/, with chapter count (PlaylistMark entries with mark_type ≤ 1) and approximate playlist duration in seconds. Sorted by playlist filename. Sourced from the existing crate::mpls parser (no new format work). Useful for identifying the main feature playlist at a glance — it's the one with the longest duration. Verified on disc-11 (Dune Pt 2): 00800.mpls correctly identified as 2h 45m 49s with 18 chapters amid 30+ shorter playlists. Doesn't touch the per-title `disc::DiscTitle::chapters` field which disc::bluray.rs already populates from the same marks during disc init — this is purely the diagnostic surface for labels-analyze. |
||
|
matthew
|
ee9bcf24dd |
labels: gap-fill MPLS streams when framework parser under-yields
When a framework parser (paramount, criterion, pixelogic, ctrm, dbp, deluxe) is chosen but its label list covers only a subset of the stream slots MPLS knows about, merge MPLS-derived entries for the uncovered (stream_type, stream_number) slots. Framework labels keep their richer fields (purpose=Commentary, codec_hint with "Atmos", qualifier=Sdh); MPLS only fills slots the framework left unnamed. Implementation: - `fn fill_gaps_from_mpls` walks the MPLS label list, pushing any entry whose (type, number) tuple isn't already in the framework output. Stable sort by (type, number) groups audios before subtitles in the merged result. - Called from both `extract()` and `analyze()`. Skipped when the chosen parser is itself `mpls_universal` (no gaps possible). - `LabelAnalysis::gap_fill_added` field reports how many slots got filled — useful diagnostic from `labels-analyze`. - `StreamLabelType` gains `Eq + Hash` so the dedup HashSet works. Tested via 4 new unit tests (155 of 155 labels tests passing, was 151). End-to-end on partial-yield corpus discs: - disc-05 (Oppenheimer): pixelogic 4/5 already covered, gap_fill_added=0 - disc-11 (Dune Pt 2): pixelogic 8/11 already covered, gap_fill_added=0 (Real-world gap-fill activations are rare in the current corpus because pixelogic already incorporates MPLS-equivalent data when matching; the merge is defensive for less-thorough frameworks.) |
||
|
matthew
|
d7581d5194 |
labels/mpls_universal: dense stream numbering across playlists
Per-playlist stream_number counters were resetting between MPLS files, so a disc with 2 MPLS files each listing the same 8 audio streams produced labels with stream_number 1..8 then 1..8 again (dedup kept whichever PID was different, leaving the numbering visibly broken — multiple "audio1: eng" rows). Move the counters outside the per-file loop and increment only when an entry survives dedup. Surviving entries now get dense 1..N numbering across the whole disc per stream_type. Verified on corpus disc-02 (HDMV-only): was `audio1, audio1, audio1` for the 3 distinct audio codecs (TrueHD/AC-3/DTS-HD MA), now `audio1, audio2, audio3`. Same fix applies to disc-01 (12 audio streams across multiple MPLS) and disc-09 (14 audio streams). |
||
|
matthew
|
222f77fd09 |
labels: universal MPLS fallback + bdmt disc metadata + png stub
Three new modules in the labels platform, all layered so framework- specific parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe) always take precedence over the universal layer. **mpls_universal.rs** (~600 LOC, 9 tests): consumes the already-parsed `crate::mpls::Playlist::streams` and emits StreamLabel entries with language + codec_hint per stream. Returns `Confidence::Low` (new variant) so framework parsers' Medium/High always win the registry's max-by-confidence tiebreaker; MPLS only gets picked when no framework matched. Closes the "no BD-J disc" case (HDMV-only navigation) that previously produced zero labels — language and base codec are spec-mandated in MPLS STN tables on every Blu-ray ever made. **bdmt.rs** (~350 LOC, 10 tests): reads `/BDMV/META/DL/bdmt_<lang>.xml` files into a new `DiscMetadata` struct (localized title names per ISO 639-2 code, descriptions, optional box-set position). Runs independently of the parser registry — disc-level metadata, not per-stream, so the registry's confidence selection doesn't apply. Surfaced on a new `LabelAnalysis::disc_metadata` field. **png_filenames.rs** (noop stub): pattern documentation + dead-code detect/parse for future reactivation. Deferred because MPLS already delivers per-stream lang/codec/type on every disc; PNG filename language tokens only add studio variant disambiguation (FRC vs FRP, LAS vs CSP) — niche enough to not justify the implementation cost right now. Wiring changes in `mod.rs`: - New `Confidence::Low` variant (PartialOrd places it below Medium/High) - New `ParseResult::low()` constructor - `mpls_universal` appended last to `PARSERS` registry - `LabelAnalysis::disc_metadata: Option<DiscMetadata>` field - `analyze()` runs `bdmt::parse` independently and surfaces result - `pub use bdmt::DiscMetadata` re-export so the labels-analyze tool in freemkv-tools can construct the JSON payload Total: 151 of 151 labels tests passing (was 132 — added 19 new). |
||
|
matthew
|
3e3563a0ff | v0.18.13: bump version | ||
|
matthew
|
2a534b23f5 |
disc/patch: adaptive batching — 32 sectors, drop to 1 on failure
Pass N now reads at 32 sectors per attempt and drops to 1 only on batch-read failure to probe each sector individually. After 16 consecutive clean single-sector reads it climbs back to 32. Net effect: NonTrimmed regions walk ~32x faster in clean stretches without sacrificing per-sector recovery quality — the drop-to-1 retry from the same cursor position guarantees every sector in a failed batch is individually attempted. Design contract: - A batch-read failure (count > 1) is NOT a recorded failure: no NonTrimmed mark, no consecutive_failures bump, no damage_window push, cursor stays put. We just drop current_batch to 1 and the loop re-attempts the same position at single-sector granularity. - A single-sector failure (count == 1) follows the existing path: NonTrimmed mark, consecutive_failures++, damage_window.push(false), post-failure pause, wedge probes. - Backtrack always at count=1: this path fills a gap that the main loop's damage-window skip jumped over. Using batched reads there would lump good sectors into NonTrimmed marks when the gap contains even one bad sector. State machine adds: - `initial_batch` (from opts.block_sectors, default 32 in patch_internal) - `current_batch` (mutable, starts at initial_batch, drops to 1 on batch failure) - `consecutive_singles_ok` (counter, resets on upscale + failure) - `ADAPTIVE_UPSCALE_THRESHOLD = 16` (matches sweep's pattern for "16 consecutive good = back to fast mode") Tests: - pass_n_size_aware_skip.rs PatternedSectorReader now fills each sector with its OWN LBA byte (not the starting LBA's byte). This matches real drive behavior — the pre-0.18.13 fixture's "fill whole batch with one byte" was a shortcut that only worked when patch read 1 sector at a time. Existing recovery-quality assertions all still pass under adaptive batching. User spec: "try 32, pass, great, fail -> do 1 sector" |
||
|
matthew
|
dcb46f54ec | v0.18.12: bump version | ||
|
matthew
|
5ddbf43ab2 |
disc/patch: leave failed reads NonTrimmed mid-multipass, not Unreadable
User design call after watching Pass 2 mark ~20 KB as "Cosmetic" (permanently Unreadable) after just 10 retries within a single pass: "i think it's good or maybe until all passes are done. then it's gone." That contradicts what the multi-pass design promises a user. The project goal in CLAUDE.md is "recover 100% of readable data from any optical disc, automatically." Marking sectors Unreadable after a SINGLE pass's per-range retry budget gives up on sectors that subsequent passes might recover — drive reads are stochastic, the sector that fails 10 times in Pass 2 may succeed on attempt 1 in Pass 3 after temperature / bus state / prior-read patterns shift. The patch.rs doc comment already noted ~36% of patch-marked Unreadable sectors turned out to be readable in re-rip experiments. Three sites in `Disc::patch` were emitting `PatchItem::Unreadable` mid-pass: - backtrack hit damage (line ~2659) - all-retries-exhausted on a single LBA (line ~2846) - redundant second mark after the wedge-suspicion log (line ~2970) All three now emit `PatchItem::NonTrimmed` instead. Failed bytes stay "maybe" (NonTrimmed) so the next pass gets another shot. The per-range skip-limit (10) and per-pass wedge-threshold (50) still bound time-per-pass; they just no longer turn the bytes terminal. The `PatchItem::Unreadable` variant stays in the enum (with #[allow(dead_code)]) because the orchestrator-side end-of-recovery promotion will use it: autorip, after the final retry pass completes, scans the mapfile and promotes still-NonTrimmed → Unreadable. That promotion lands in a follow-up commit on the autorip side — separable from this libfreemkv change. Loss accounting unchanged: `bytes_pending + bytes_unreadable` is the "lost or pending" total that `abort_on_lost_secs` consults (disc/mod.rs:1327). Moving bytes from one bucket to the other mid-pass doesn't affect whether the rip would abort; it only affects display (UI shows "Maybe" vs "Cosmetic") and whether subsequent passes retry the bytes (the actual fix). Test update: `test_pass_progress_separates_unreadable_from_pending` was renamed to `test_pass2_leaves_failed_reads_as_pending_not_unreadable` and rewritten to assert the new invariant — Pass 2 leaves all failed bytes as bytes_pending (no mid-pass Unreadable promotion). Original assertions were checking the pre-design-call behavior. Precommit (cargo +1.86 fmt + clippy + test) green. |
||
|
matthew
|
c4c901f073 |
disc/read_error: unify Pass 1 and Pass N error handling
User's design call after watching the avoidance work prevent a wedge
on the live rip (no wedge events across 6 read errors): "Pass N
and 1 should both be very very similar in recovery. almost identical
just smaller sectors imo in pass n. pause times the same imo as a
failed read is a failed read."
The error-handling code path was already centralized (one
handle_read_error fn, called by both Disc::sweep and Disc::patch).
The TUNING was split — Pass 1 used 5 s inter-error pauses + a
wedge-skip-and-continue policy; Pass N used 1 s pauses + immediate
AbortPass on HARDWARE_ERROR / ILLEGAL_REQUEST. That asymmetry made
Pass N vulnerable to the same wedge that Pass 1's avoidance fixed.
Changes:
1. FAIL_PAUSE_SECS = 5 — single constant, applied uniformly to both
passes. Dropped PASS_1_FAIL_PAUSE_SECS and POST_FAILURE_PAUSE_SECS
in favor of one value. CONSECUTIVE_FAIL_LONG_PAUSE_SECS kept as a
distinct (but currently equal) value for future tuning escalation.
2. HARDWARE_ERROR / ILLEGAL_REQUEST path is now symmetric:
- Pass 1: JumpAhead WEDGE_JUMP_SECTORS (1 GB) + WEDGE_PAUSE_SECS
cooldown, mark skipped region NonTrimmed.
- Pass N: JumpAhead WEDGE_PASS_N_SKIP_SECTORS (64 sectors / 128 KB)
+ WEDGE_PAUSE_SECS cooldown. Pass N's batch=1 means a 1 GB skip
would abandon the entire current NonTrimmed range; small skip
moves past the bricked LBA + buffer, outer patch loop picks up
the next sector.
- Both share WEDGE_ABORT_THRESHOLD — same 16-skip budget before
real AbortPass on a permanently stuck drive.
3. wedge_skip / wedge_abort tracing logs now include `pass=1|N`
so post-mortems can see which pass hit the wedge condition.
Cost analysis:
Pre-reframe worry was "5 s × 5500 NonTrimmed sectors per Pass N
pass × 7 passes = 53 hours." Reality: most NonTrimmed sectors
recover on first or second retry, so most reads are successful and
pay 0 pause. The few that DON'T recover hit the 10-skip budget and
get marked Unreadable — bounded at 10 × 5 s = 50 s per truly-bad
sector. Worst-case Pass N pause overhead on a typical damaged disc
is single-digit minutes, not hours. And it's strictly cheaper than
the alternative (wedge kills the entire multi-pass recovery).
Tests:
- `both_passes_pause_on_failed_read_for_wedge_avoidance` — locks the
unified pause-tuning policy (was pass_1_pauses_briefly).
- `pass_n_hardware_error_also_skips_not_aborts` — was
`pass_n_hardware_error_still_aborts`. New behavior verified:
JumpAhead with WEDGE_PASS_N_SKIP_SECTORS + WEDGE_PAUSE_SECS.
- `pass_n_hardware_error_aborts_after_threshold` — new. Confirms
Pass N respects the same WEDGE_ABORT_THRESHOLD as Pass 1.
- pass_1_does_not_pause_on_skip is gone (it was the old "Pass 1
pause=0" assertion, irrelevant after the avoidance work).
Empirical validation: avoidance was already proven on a live rip
tonight — 6 read errors on a damaged disc, sense_family=Medium
throughout, wedge_count=0, Pass 1 continued cleanly past 40%
where it previously died at 48%. This commit extends the same
discipline to Pass N's recovery loop.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
matthew
|
01bf3a16db |
labels/deluxe: Phase D rewrite against ground-truth binding pattern
Replaces the speculative arg-position heuristic with type-presence
detection driven by real disc bytecode. Ground truth captured in
freemkv-private/research/deluxe-poc/data/ via POC v0.3 binding-
bytecode dumps against disc-01 (Disney) and disc-09 (Warner).
What changed:
1. StackVal::CodingType(String) — new variant. getstatic against
org/bluray/ti/CodingType (the BD-J spec codec enum) now pushes
this, carrying the field name (e.g. DOLBY_LOSSLESS_AUDIO). The
pre-fix code was treating codecs as a Deluxe-internal enum
subclass walk (Phase B), which is the wrong model — codecs are
standard BD-J API references.
2. coding_type_to_codec_hint(field) — new function. Maps
org.bluray.ti.CodingType field names to human-readable codec
strings (DOLBY_LOSSLESS_AUDIO -> "Dolby TrueHD", DOLBY_AC3_AUDIO
-> "Dolby Digital", etc.). Unknown field names pass through
verbatim so future codec values still surface something.
3. find_binding_classes — multi-class variant. Some Deluxe discs
split per-stream tables across two binding classes (audio +
subtitle). Returns top-K candidates by getstatic count, filtered
to >=40% of the top count and capped at 4. Replaces the old
single-class find_binding_class (which was unused after this
change).
4. interpret_streams — rewritten. Args identified by TYPE not
position:
- First EnumRef{kind:"Language"} -> language
- First EnumRef{kind:"Purpose"} -> purpose
- First CodingType(name) -> codec_hint
- First Int(n) -> stream index hint (traced
only; per-type sequential stream_number still wins because BD
spec stream-numbering is anchored on MPLS)
- Construction has CodingType -> Audio stream; otherwise Subtitle
- No Language -> skip (not a stream construction)
This handles BOTH the Disney 5-arg pattern (I, Lbe, Llp, I,
LCodingType) and the Warner 4-arg pattern (I, Law, Lgp,
LCodingType) automatically — same code path because args are
identified by type rather than constructor-signature shape.
5. parse() now walks all binding-class candidates and unions
their constructions before calling interpret_streams. Logs each
candidate at INFO with getstatic_count for diagnosis.
Tests:
- 2 new tests verify the CodingType -> codec_hint mapping for
known + unknown field names.
- Existing interpret_streams tests updated to use the new
signature (dropped CodecTable arg).
- Audio-emission test rewritten to use CodingType arg instead of
the old binding_type substring-match approach.
Confidence is still Medium for now (single-corpus verification);
ready to promote to High once tested against a third Deluxe disc.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
matthew
|
b6e136645f |
disc: emit Pass 1 summary INFO log at sweep exit
Wires the existing PassSummary infrastructure (in read_error.rs as
of
|
||
|
matthew
|
b4229a02ec | v0.18.10: bump version | ||
|
matthew
|
231b9d2cb1 |
disc: structured timing + transition diagnostics for read errors
Adds the observability we need to debug wedge incidents from logs
alone — without needing to enable verbose TRACE-level SCSI tracing.
Goal stated by user: "when error occurs we can debug and code
correctly."
Pre-fix the WARN log on each read error showed only sense codes
and consecutive_failures. Missing: timing context (was the failed
read fast or slow?), gap to previous events (cumulative vs.
immediate failure?), and family transitions (did the drive just
flip into wedge mode, or has it been there?).
New fields on ReadCtx (no caller signature change):
last_success_at: Option<Instant>
last_error_at: Option<Instant>
last_error_family: Option<SenseFamily>
total_errors: u64
total_reads_ok: u64
zones_entered: u64
jumps_taken: u64
in_damage_zone: bool
New SenseFamily enum (NotReady / Medium / Hardware / IllegalRequest
/ Other) with is_wedge_family predicate.
handle_read_error WARN log now carries:
consecutive_failures
consecutive_outer_failures
ms_since_last_error NEW gap between this and previous error
ms_since_last_success NEW gap to last good read
total_errors NEW aggregate this pass
total_reads_ok NEW
wedge_count
sense_family NEW typed category, easier to filter
sense_key / asc / ascq (existing)
NEW WARN log "wedge_transition" fires once when the sense family
changes from non-wedge to wedge (Medium to Hardware/IllegalRequest).
That's the moment the drive's firmware flipped into fast-fail
mode. Single timestamped event in the log so post-mortems can
pinpoint the transition without scanning thousands of TRACE lines.
Worked example: if the next wedge incident shows
read_error ms_since_last_success=18234 ms_since_last_error=null
read_error ms_since_last_success=28000 ms_since_last_error=10000
read_error ms_since_last_success=43000 ms_since_last_error=68
(drive returned <100ms = wedge symptom)
wedge_transition errors_in_zone=5 ms_since_last_success=43000
we can immediately tell cumulative damage, 5 errors over 43 s,
drive went into fast-fail mode at the 5th. If instead we see
read_error ms_since_last_success=200 ms_since_last_error=null sense_family=Hardware
wedge_transition errors_in_zone=1
the wedge was triggered by ONE read at a physically-bricked LBA
(immediate fast-fail, no warm-up).
These two patterns demand different tuning responses (longer
pause vs. larger initial jump), and now we can distinguish them
from a single WARN log line each instead of needing TRACE
verbose for the whole rip.
Plus jumps_taken / zones_entered counters that feed an end-of-pass
INFO summary (PassSummary). Caller invokes pass_summary at sweep
end and logs structured stats: "Pass 1 saw N errors / M ok reads
/ K zones / J jumps". Single-line post-mortem for any rip.
No caller signature change (timing is internal to the handler;
end-of-pass summary is a new method callers opt into). Precommit
green; 433+ tests pass. Staged for the 0.18.10 release once we
have user-validation data on 0.18.9's avoidance tuning.
|
||
|
matthew
|
ff4000f7f7 | v0.18.9: bump version | ||
|
matthew
|
445a15fa25 |
disc: wedge AVOIDANCE on Pass 1 — inter-error pause + larger jumps
Complements the wedge-skip backstop ( |
||
|
matthew
|
5436a9341c |
labels: apply_labels integration tests + class_reader robustness fuzz tests
Closes the final two audit items from this session. labels::apply_labels: factored out of apply() so the matching logic is unit-testable without needing a SectorReader / UdfFs. 11 new tests in apply_tests cover: - codec_hint + variant flow through to AudioStream.label - purpose set on audio with no label English text - name fallback only when purpose=Normal (CLI owns purpose i18n) - subtitle SDH qualifier set; forced flag flipped on Forced - per-type 1-based indexing (audio #2 maps to 2nd audio stream, not 2nd stream overall) - labels for nonexistent streams are no-ops - empty labels list leaves streams untouched - fill_defaults generates audio + video labels; preserves existing class_reader: robustness smoke tests. ClassFile::parse must NEVER panic on adversarial input — only return Err. 9 new tests: - empty input - short magic (0..4 bytes) - wrong magic - truncated after magic - bad CP tag - truncated UTF-8 in CP - 200 random byte buffers (deterministic xorshift) - 100 magic + random tail (most adversarial — magic check passes, everything else garbage) - instructions iter on random code (200 buffers) - instruction_size on every opcode 0..255 with varied tail buffers - modified_utf8 on random byte buffers (500) The xorshift PRNG keeps the tests deterministic (no rand dep) and reproducible — failures will be the same buffer every time. This is the lightweight alternative to a cargo-fuzz setup; if/when we adopt cargo-fuzz, these tests stay as regression cases. All 451 tests passing on cargo +1.86 fmt + clippy + test. |
||
|
matthew
|
d7f186283e |
disc: Pass 1 wedge-skip instead of abort-on-first-wedge
Pre-fix: when the drive returned HARDWARE_ERROR or ILLEGAL_REQUEST during Pass 1 sweep, libfreemkv immediately returned ReadAction:: AbortPass. Autorip surfaced this as a fatal error and stopped the rip at whatever progress percentage Pass 1 had reached — typically 40-50%. On a disc with one physical-damage cluster, the user would see Pass 1 die at ~48% with the cryptic message 'E6000: <lba> 0x02/0x04/0x3e' and have no rip output to work with. Root cause analysis: BU40N firmware transitions into a fast-fail state when it hits cumulative read failures in a small LBA range — returns HARDWARE_ERROR for every subsequent read near that LBA, even sectors that aren't physically damaged. Per CLAUDE.md 'Bad-sector handling' rule #2, 'Recovery requires eject+reload OR significant cool-down.' Aborting on first wedge throws away the rest of the disc; the right response is to SKIP the wedged region (mark as NonTrimmed for Pass N), pause for drive cooldown, and continue. Fix: in handle_read_error, the HARDWARE_ERROR / ILLEGAL_REQUEST arm now branches on bisect_on_marginal: Pass 1 (bisect_on_marginal=false): JumpAhead with WEDGE_JUMP_SECTORS (1 GB at 2048 bytes/sector) and WEDGE_PAUSE_SECS (30 s cooldown). Tracks wedge_count in ReadCtx; resets on any successful read. Truly aborts only after WEDGE_ABORT_THRESHOLD (16) consecutive wedges with no good read in between — generous enough to clear most physical-damage clusters, bounded enough to not loop forever on a permanently bricked drive. Pass N (bisect_on_marginal=true): unchanged AbortPass. Pass N's job is single-sector recovery; if the drive won't talk near a specific LBA, skipping doesn't help. Pass N exits and lets the outer layer decide retry/eject/surface. 5 unit tests cover the new policy: pass_1_hardware_error_jumps_ahead_not_aborts — JumpAhead emitted with correct sectors+pause, wedge_count incremented. pass_1_hardware_error_aborts_after_threshold — AbortPass kicks in on the WEDGE_ABORT_THRESHOLD-th consecutive wedge. pass_1_good_read_resets_wedge_count — on_success clears wedge_count; subsequent wedge gets fresh skip budget. pass_n_hardware_error_still_aborts — Pass N's AbortPass behavior intact. pass_1_illegal_request_also_routes_to_wedge_skip — both wedge sense families get the skip treatment. Impact: on the Dune Pt 2 disc that consistently wedged at 48% (physical damage at LBA ~19.9M), Pass 1 will now jump ahead 1 GB on the wedge, give the drive 30 s cooldown, and continue scanning the rest of the disc. The damaged region becomes Pass N's job to revisit. Worst case if the drive stays wedged: 16 GB of NonTrimmed disc area before honest AbortPass. Precommit (cargo +1.86 fmt + clippy + test) green; 430 passing. |
||
|
matthew
|
5df1dd77fb |
labels/xml: shared tolerant XML helper, paramount + criterion onto it
Replaces two near-duplicate hand-rolled XML scrapers in paramount.rs
and criterion.rs with a single labels::xml module that's robust to:
- Case-insensitive tag / attribute names ('<Playlist>' matches the
same as '<playlist>'; 'Name=...' matches 'name=...').
- XML namespace prefixes (matches '<ns:tag>' for tag='tag').
- Arbitrary whitespace inside open tags and around '=' separators
('<tag name = "X">' works).
- Both quote styles for attribute values (" and ').
- Self-closing tag forms ('<tag/>' and '<tag />').
- '>' chars inside quoted attribute values (no premature end-of-tag).
Three functions:
xml::attr(element, name) -> Option<String>
Extract attribute value from an open-tag fragment.
xml::text(xml, tag) -> Option<String>
Trimmed text content of first <tag>...</tag>.
xml::find_element(xml, tag, from) -> Option<(start, end)>
Locate next <tag>...</tag> for iteration; handles self-closing.
22 unit tests cover the robustness properties: case-insensitivity,
namespace stripping, whitespace tolerance, quote styles, self-close
forms, no-substring-false-positive (looking for 'lang' must NOT
match 'lang_id' or 'language'), '>' inside quoted attrs, iteration
across repeated elements.
paramount.rs: drops local extract_attr; find_feature_playlist now
walks xml::find_element('playlist', ...) so it works regardless of
case and self-closing style. Pre-refactor: required exactly
'<playlist ' (single space, exact case) and '/>' for self-close.
criterion.rs: drops local extract_tag; parse_stream_infos and
parse_playback_config iterate via xml::find_element. Same case-
sensitivity + namespace gains. The 'COMMENTARY' / 'SDH' / 'DS'
content-value match is now case-insensitive too (previously a disc
authored with 'commentary' would have been miscategorized as Normal).
Pre-refactor known failure modes (none observed yet, but trivial
to trip on a future disc): vendor switches whitespace around '=',
uses single quotes, capitalizes a tag, prefixes a namespace. All
now handled.
Out of scope by design: XML entity decoding (&, <), CDATA
sections, comments, processing instructions. None observed in BD-J
authored label data. If a future disc trips them, the entity
decoder is a localized addition.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
matthew
|
3671ad2755 |
labels/deluxe: full Phase B/C/D buildout — codec walk, binding decode
Completes the Deluxe parser pipeline. Phase A (master enums) was
already shipping; this commit lands Phases B/C/D so the parser now
emits per-stream StreamLabel records on Deluxe-authored discs.
Phase B (decode_codec_enum): walks the codec enum's subclass
references (one .class per codec ordinal) and extracts the codec
name string from each subclass's constant pool. Heuristic: pick the
first Utf8 entry that's uppercase + underscored + >=4 chars, or one
of the known codec roots (ATMOS/DOLBY/DTS/TRUEHD/MLP/AC3/EAC3/PCM)
when no underscored candidate is found. CodecTable maps ordinal ->
codec string; empty string for ordinals where extraction failed
(logged via tracing, not fatal).
Phase C (find_binding_class): identifies the class that builds the
per-stream label table by counting getstatic operations targeting
any of the master enum classes from Phase A. Class with the highest
count >= 4 wins. Threshold is empirical (real binding classes have
50+ matches; floor of 4 admits small discs while rejecting incidental
single-reference classes).
Phase D (decode_binding + BindingDecoder): symbolic stack machine
that walks the binding class's <clinit> bytecode. Handles:
- constant pushes: iconst_<n>/bipush/sipush/ldc(Integer)
- new <X>: pushes uninit-object marker
- dup: stack copy
- getstatic <Y.Z>: pushes EnumRef when Y is in MasterEnumTable,
else Unknown
- invokespecial X.<init>(...)V: pops args per descriptor; when the
receiver is NewObj(X), emits a Construction { binding_type: X,
args: [...] }
- invokevirtual/invokestatic/invokeinterface: pop args per
descriptor, push return placeholder unless void
- pop/pop2/aastore/putstatic/putfield: standard stack effects
- branches/returns: clear stack (conservative resync — binding
<clinit> is straight-line in practice)
parse_method_arg_count: JVMS field-descriptor parser, handles
primitives, references (L...;), arrays ([...).
interpret_streams: converts Constructions to StreamLabels using
the master enum table + CodecTable. Each construction with a
Language ref becomes a stream. Audio when codec_hint resolves via
binding_type substring match against CodecTable; subtitle otherwise.
Purpose ordinal -> LabelPurpose via the verified Deluxe Purpose enum
order (Normal/Commentary/PiP/Trivia/Descriptive/Score/NoForced/
NoForcedDescriptive). Stream index = sequential per type. Language
goes through vocab::lang for ISO code + variant.
deluxe::parse now returns Some(ParseResult::medium(labels)) when
all four phases produce labels. Medium confidence — the bytecode
mechanism is rigorously tested but the signal-to-StreamLabel
mapping (which arg is which, audio vs subtitle classification) is
heuristic until corpus binding-class bytecode confirms the exact
pattern.
Test coverage: 13 new unit tests in deluxe.rs
parse_method_arg_count: 3 tests (basic types, references, malformed)
BindingDecoder: 4 tests (simple construction, with int pushes,
skips unmatched invokespecial, resolves master-enum ordinal)
interpret_streams: 4 tests (subtitle on no codec, audio on codec
match, purpose routing, skips no-language)
MasterEnumTable: 3 tests (resolve, value, class_name_set)
extract_codec_name: 1 test (uppercase+underscore matching)
class_reader.rs gained a #[cfg(test)] ConstantPool::from_entries
test-only constructor so Phase D tests can build synthetic CP
fixtures without writing raw .class bytes.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
matthew
|
5253e4e5b7 |
labels: per-parser confidence + highest-confidence-wins registry
Replaces 'first-match-wins by array order' with 'highest-confidence-
wins, array order tiebreaker'. Removes the arbitrariness when more
than one parser can claim a disc (e.g. one with both
bluray_project.bin and playlists.xml).
New types in labels::mod:
pub enum Confidence { Medium, High }
pub struct ParseResult { labels: Vec<StreamLabel>, confidence }
ParseResult::high(labels) / ::medium(labels) constructors
Parser signature change: every parse() now returns
Option<ParseResult> instead of Option<Vec<StreamLabel>>. Updated all
six parsers in lockstep:
paramount: High (fully structured XML)
criterion: High (fully structured XML)
pixelogic: High by default, Medium when an unknown token component
is encountered (the skip-unknown path now propagates the
coverage gap to the caller instead of silently degrading)
ctrm: High (structured key-value)
dbp: High (anchor scan with vocab routing)
deluxe: still returns None pending Phase D — signature aligned
Registry behavior:
extract() iterates all detect-positive parsers, picks highest
Confidence with non-empty labels. Equal confidence falls to array
order (deterministic). Same selection logic in analyze().
LabelAnalysis grew a confidence: Option<Confidence> field so the
diagnostic surface (freemkv-tools labels-analyze) exposes which
confidence tier the selected parser claimed. labels-analyze JSON
and labels-corpus-check structural diff both gained the field.
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
matthew
|
7c3cd29bd4 |
labels: fresh-eyes audit — capture variant, dedupe detect, lock registry
Three targeted fixes from a second-pass audit of the labels module. 1. vocab::lang now returns Option<LangInfo> with both code AND a human-readable variant string. Pre-fix: 'Brazilian Portuguese 5.1' became language=por, variant='', dropping the dialect info the disc had explicitly authored. Post-fix: language=por, variant='Brazilian' — matches the convention pixelogic / ctrm / criterion already use for their region variants. dbp now populates StreamLabel::variant from this. Compound table grew a 3-tuple (needle, code, variant); bare matches still return variant=''. 2. dbp and deluxe had duplicated detect() boilerplate (any top-level .jar in /BDMV/JAR/). Both now call jar::has_any_top_level_jar. The trait-level detect contract — see super::PARSERS — can't peek inside a jar without a SectorReader, so loose-detect-plus-real- check-in-parse is the unavoidable pattern for jar-content parsers. Consolidating in jar.rs at least makes the duplication visible. 3. mod.rs comment about parser ordering said 'dbp last'; deluxe is actually now last. Updated to explain the dbp-before-deluxe order is by cost (cp-iteration cheaper than bytecode walking when Phase D lands). Plus a registry-level lock test in mod.rs::registry_tests — asserts the PARSERS array order is exactly [paramount, criterion, pixelogic, ctrm, dbp, deluxe]. This was previously implicit; if someone reorders the array (which changes which parser wins on overlapping signals), unit tests would have stayed green. Now they fail with an explanatory message about why the order matters. Audit findings deferred to follow-ups (each its own commit + design discussion): - Stronger detect contract — current loose-detect-real-check pattern is forced by SectorReader-not-in-detect-signature; could be fixed by changing the trait to take an Option<&mut dyn SectorReader> or similar. - Per-parser confidence scoring — registry currently first-match-wins. A high-confidence parser ought to beat a low-confidence one regardless of array order. - class_reader fuzzing — handles malformed input via Result but no adversarial corpus yet. Precommit (cargo +1.86 fmt + clippy + test) green. |
||
|
matthew
|
5c3169b4e5 |
labels: refactor pixelogic + ctrm onto shared platform + hardening
Closes the platform unification: every label parser now routes purpose/qualifier/codec classification through one source of truth (vocab.rs) instead of N hand-rolls, and every binary-blob byte scanner goes through one helper (text::extract_ascii_strings). pixelogic.rs: - Drop local extract_strings (~20 lines) — use text::extract_ascii_strings. - HARDENING: replace with skip-unknown-component + trace log. Pre-refactor behavior: any single uncatalogued token part (e.g. a future codec ID, new framework variant) silently dropped the entire stream record. New behavior: skip just the unknown part, surface what we know about the stream. - 8 new unit tests cover basic audio/subtitle paths, commentary, descriptive, region variant, the new skip-unknown-component regression, and the non-audio/non-subtitle early-out. ctrm.rs: - Replace with vocab::purpose(&name). Now word-boundary matched — 'Commenter Pro Track' no longer false-matches Commentary. - Replace with vocab::qualifier(&name). Same word-boundary tightening, plus picks up Forced and DescriptiveService for free. - Preserved structural commentary signal via as a fallback when name is silent (e.g. 'audio_commentary_1.name=Track 2'). - 6 new unit tests including the 'Commenter' false-positive regression and the SDH-only-on-subtitles boundary. text.rs: - Drop module-level dead_code allow now that pixelogic uses extract_ascii_strings. Net: all 5 framework parsers now on the unified platform. Future work (deluxe Phase D, paramount/criterion XML hardening) builds on the same scaffolding. Precommit green. |
||
|
matthew
|
1d46ad02b6 |
labels: add deluxe parser (Phase A — master enum identification)
Closes the 'detected but no parser' gap on Deluxe-authored BD-J discs
(com/bydeluxe/ package signature; ~20% of UHD discs in our corpus per
the 2026-05-10 11-disc capture session).
What ships in this commit:
- detect() — registers the parser in the chain (loose pre-check at the
/BDMV/JAR/ level; real signal in parse via has_path_prefix on
'com/bydeluxe/').
- Phase A master enum identification — walks every .class's <clinit>
ldc sequence and matches against framework-stable fingerprints for:
Language (70 ldcs starting English/French/Spanish/Dutch)
Purpose (8 ldcs starting Normal/Commentary/PiP/Trivia)
VideoFormat (7 ldcs starting HD/HDR10 Plus/HD Dolby)
Region (22 ldcs starting USA_D1/LIC1/LIC2/LIC3 — Disney-only)
Studio (6 ldcs starting Disney/Marvel/Pixar — Disney-only)
All identifications verified out-of-tree on corpus disc-01 (Disney,
The Amateur) and disc-09 (Warner, Dune Part 1) via the standalone
POC.
- Phase B structural skeleton (find_codec_enum) — identifies the
codec enum class by structural signature (>=20 'new' ops, 0 ldcs),
returns the ordered subclass list. Codec string extraction from
subclasses is dead-coded pending the follow-up commit.
What does NOT ship yet:
- Phase D (per-stream binding-class decoder). parse() returns None
intentionally — the master enums alone don't yield StreamLabels
without the streamTable.put(...) bytecode walker. analyze() will
show 'deluxe' in parsers_detected with the enum identification in
tracing logs, so the analyzer reports honestly: 'detected, can't
emit labels yet' rather than silent failure.
Why ship A without D: A is proven on real corpus discs; D's design
needs ground-truth binding bytecode from at least 2 corpus discs
side-by-side to verify the stack-machine pattern. The Phase A
infrastructure (master enum identification + ordinal->name table)
is what D will consume — landing it now unblocks D's design without
holding back the parser registration.
5 unit tests cover the fingerprint matcher logic + a roster lock that
forces explicit consideration when adding/removing fingerprints.
Precommit green.
|
||
|
matthew
|
307bee11e4 |
labels: shared platform (vocab/text/jar) + dbp refactor
Establishes the shared infrastructure layer for label parsers so that
Java-touching parsers (dbp, deluxe) don't reimplement jar walking and
all parsers route language/purpose/qualifier classification through
one source of truth instead of N hand-rolls.
New modules:
vocab.rs expanded from 27 -> ~370 lines
+ lang(text) -> Option<&'static str> (English/multi-word
-> ISO 639-2; ~45
languages, compound
phrases like
'Brazilian Portuguese'
and 'Castilian Spanish')
+ purpose(text) -> LabelPurpose (Commentary,
Descriptive, Score,
Ime; word-boundary
matched)
+ qualifier(text) -> LabelQualifier (SDH, Forced,
DescriptiveService)
+ has_word internal primitive — enforces word-boundary
matching so 'Commenter' no longer matches 'commentary' and
'engineering' no longer matches 'english'. Existing parsers
used .contains() and got lucky on the corpus; vocab now
guarantees the boundary in one place. 20+ unit tests.
text.rs NEW (~85 lines)
+ extract_ascii_strings(data, min_len) — promoted from two
near-duplicate copies (pixelogic min=4, dbp min=5);
threshold passed in. 7 unit tests including
trailing-without-terminator + high-bit-byte handling.
jar.rs NEW (~120 lines)
+ for_each_jar(reader, udf, fn) — walk every top-level
.jar under /BDMV/JAR/,
yield to callback.
+ has_path_prefix(archive, prefix) — cheap 'is this MY
framework's jar?' check
via central-dir filenames.
+ for_each_class(archive, fn) — parse every .class entry
through class_reader,
yield (name, &ClassFile).
+ try_each_class(archive, fn) — same with early-return on
first Some(R) match.
Refactored:
dbp.rs v2 on the new platform:
- dropped extract_printable raw byte scan
- dropped its own English -> ISO 639-2 map
- dropped its own parse_attributes hand-roll
+ iterates CpInfo::Utf8 via class_reader (structurally clean,
no false-positive risk from method bytecode bytes)
+ routes language/purpose/qualifier through vocab
All 7 prior dbp tests still pass; +2 new ones cover
vocab routing.
dead-code allows on text.rs (extract_ascii_strings) and jar.rs
(try_each_class) come off when pixelogic and deluxe land — they're
staged for next steps.
Precommit green (cargo +1.86 fmt + clippy + test).
|
||
|
matthew
|
dab9b9c9db |
labels: add class_reader, hand-rolled JVM .class file parser
Foundation for label parsers that need structured access to .class
files inside /BDMV/JAR/<x>.jar. Replaces noak (~3KLOC dep) with a
~1000-line std-only reader.
Public API:
- ClassFile::parse(&[u8]) -> Result<ClassFile>
- ConstantPool::{get, utf8, class_name, string, integer, member_ref, iter}
- Member::code(&pool) -> Option<CodeAttribute>
- CodeAttribute::instructions() -> Instructions iterator
- Instruction::{name, operand_u8, operand_u16, cp_index}
- Opcode constants (LDC, AASTORE, NEW, GETSTATIC, INVOKESPECIAL, ...)
Spec coverage:
- Constant pool: all 17 tag types incl. Long/Double 2-slot quirk
- Modified UTF-8 incl. 0xC0 0x80 -> U+0000 special case
- Bytecode iteration with full opcode size table
- Variable-length tableswitch / lookupswitch / wide
12 unit tests cover the opcode table edge cases (padded switch tables,
wide-iinc 6-byte form), modified-UTF-8 decoder, and iterator
stop-on-truncated behaviour.
Module is currently #![allow(dead_code)] — the public API is staged
for labels::deluxe (Phases A-E bytecode walker) and a labels::dbp
refactor onto the constant-pool iterator. Tests exercise the API
in isolation. The allow comes off as those callers land.
Also fixes two pre-existing clippy lints that 1.86's stricter checks
flagged after I touched the labels module:
- src/mux/disc.rs: while-let-loop in test fixture
- tests/pass_n_size_aware_skip.rs: type_complexity in helper signature
Precommit (cargo +1.86 fmt + clippy + test) green.
|
||
|
matthew
|
bf1a67706b | v0.18.8: bump version (Pass-1 fast-skip never reached autorip 0.18.7 — Cargo.lock mis-resolved) | ||
|
matthew
|
2495acc677 |
labels: add dbp parser (Magnolia Pictures BD-J framework)
5th BD-J authoring framework recognized. Discriminator: any top-
level .jar in /BDMV/JAR/ that contains com/dbp/ package paths.
Identified during the 2026-05-10 corpus session via string-mining
disc-07's BD-J jar — perm files reference bd-live.magpictures.com
(Magnolia / Magnet Releasing).
Stream labels live as plain ASCII strings inside compiled .class
files in the form
LTextField,Audio1,English Dolby Atmos,Fontstrip_Composite,...
HTextField,Subtitle1,English SDH,Fontstrip_Composite,...
ATextField,Subtitle0,None,Fontstrip_Composite,...
— a quirk of the menu-rendering layer encoding TextField positions
and content as constant strings the Java compiler retained in the
class string pool. The leading single-letter prefix is string-pool
ordering noise; parser anchors on `TextField,`. Subtitle0 is the
disable-subtitles button and is skipped.
The parser:
- reads each top-level .jar via udf.read_file
- opens it with the existing zip dependency
- confirms com/dbp/ presence in the central directory
- walks .class entries, extracts printable strings, matches the
`TextField,(Audio|Subtitle)<N>,<label>,...` pattern
- maps human-readable language names ("English", "Castilian
Spanish", "Brazilian Portuguese", "Canadian French", ...) to
ISO 639-2 codes via a parser-local table (per the rules-of-
engagement memo, each parser knows its own format)
- preserves the full disc-authored label string in `name` so
consumers display it raw without the lib guessing further
structure
- detects SDH / Forced qualifiers and Commentary / Descriptive
purposes from substring matches; everything else falls through
to fill_defaults using BD-spec MPLS data
Verified live on the corpus: disc-07 (Civil War UHD) now matches
parser=dbp with 3 audio + 2 subtitle labels, exactly the count
visible in the disc's authored TextField definitions and what BD
spec MPLS reports.
Limitation: dbp's detect() returns true for ANY top-level .jar in
/BDMV/JAR/ (every BD-J disc has one), since the discriminator
trait function takes only `&UdfFs` and can't read jar contents.
parse() does the real com/dbp/ check — a non-dbp disc gets
parsed-as-dbp, archive_has_dbp returns false, parse() returns
None, and we fall through. Diagnostic noise: parsers_detected
includes "dbp" on non-dbp BD-J discs. Real fix is refactoring the
DetectFn signature to take a SectorReader; deferred.
7 unit tests cover the TextField extraction, language detection
(simple + compound: "Brazilian Portuguese", "Castilian Spanish",
"Canadian French", "Latin American Spanish", "Australian English"
plus the disc-corpus typo "Austrailian English"), SDH/Forced/RNIB
qualifier detection, and Commentary/Descriptive purpose detection.
Don't-guess discipline preserved: unknown languages return ""
(consumer falls back to MPLS spec data via fill_defaults).
|
||
|
matthew
|
f755ca9ea4 |
v0.18.7: Pass 1 fast-skip, defer recovery to Pass N
Pass 1 sweep was grinding through damage zones because the marginal-
media handler returned `Bisect` for every failed 32-sector batch —
forcing 32 single-sector reads per bad block at ~5s each on a real
BU40N-vs-Dune-Pt-2 trace. AND the JumpAhead trigger required a 16-
block damage window to fill before firing, so entry into a
contiguous damage zone took ~40 minutes of grinding before the
first jump fired. Architecturally wrong: Pass 1's job is "fast and
accurate, get the most data in the shortest time." Bisection +
recovery is Pass N's purpose-built role.
ReadCtx now carries two new fields:
- `consecutive_outer_failures: u64` — outer-batch failures since
last outer success. Bisect inner failures don't count.
- `bisect_on_marginal: bool` — whether to return Bisect on a
marginal-media batch failure.
- `fast_jump_threshold: u64` — outer-failures count that triggers
JumpAhead before the damage window has filled.
`for_sweep` (Pass 1) sets `bisect_on_marginal=false`,
`fast_jump_threshold=4`, and zeroes the post-failure pause. Failed
batches become SkipBlock → whole block NonTrimmed → advance, no
sleep. After 4 consecutive outer failures: JumpAhead with the
existing escalating multiplier.
`for_patch` (Pass N) sets `bisect_on_marginal=true`,
`fast_jump_threshold=u64::MAX`, keeps the original cooldown pauses.
Pass N's whole reason to exist is to grind on bad ranges with
proper recovery semantics — single-sector reads, 60s recovery
timeout, retry budget, escalating skip — and that's unchanged.
`on_success` resets `consecutive_outer_failures` only when not
bisecting, so a good single-sector read inside Pass N's bisect
doesn't pretend we've escaped the damaged batch.
Tests:
- `pass_n_marginal_with_batch_gt_1_bisects` — Pass N still bisects.
- `pass_1_marginal_skips_instead_of_bisecting` — Pass 1 doesn't.
- `pass_1_jumps_after_4_consecutive_outer_failures` — fast-entry.
- `pass_n_does_not_fast_jump` — fast-entry is Pass-1-only.
- `outer_success_resets_consecutive_outer_failures` — counter reset.
- `bisect_inner_success_does_not_reset_outer_counter` — semantics.
- `pass_1_does_not_pause_on_skip` — explicit zero-pause contract.
- `long_failure_streak_extends_pause_on_pass_n` — Pass N still
extends pauses on long failure streaks (renamed from the old
sweep-based test).
Integration test `test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed`
updated: it used to assert Pass 1 recovers all sectors via bisect
(bytes_good=total). New contract: Pass 1 marks NonTrimmed; Pass N
recovers. Test now asserts Pass-1-only outcome (bytes_pending=total,
complete=false) consistent with the redesign.
Real-world impact on the user's BU40N + Dune Pt 2 trace from this
session: a damage zone that was on track to take ~40 minutes of
Pass-1 grinding will now jump in ~20 seconds. Pass N still has the
full 7-pass recovery budget to revisit those NonTrimmed ranges.
|
||
|
matthew
|
2984959fe3 |
labels: distinguish "parser detected" from "parser succeeded"
Adds `parsers_detected: Vec<&'static str>` to `LabelAnalysis`. Records
every parser whose discriminator matched, regardless of whether its
parse() then returned Some/None.
Why: when `parser=None` we currently can't tell apart:
(a) no parser recognized this disc — missing parser, candidate for a
new module
(b) a parser recognized it but parse() returned None / empty —
capture truncated, or genuine empty authoring data, or a parser
bug
Surfaced concretely on the 11-disc capture session 2026-05-10:
disc-04 had `bluray_project.bin` in jar_inventory (pixelogic detect()
returned true) but parse() returned None because file content was
past the 1 GB capture window. Old API: parser=None — looked like
"missing parser." New API: detected=[pixelogic], parser=None — clearly
"capture problem, not a parser gap."
The tracing log line on the no-parser-emitted-labels path now
distinguishes the two cases too.
No behavior change to production rip path. extract() is unchanged;
only the diagnostic analyze() returns the richer result.
|
||
|
matthew
|
993d4bf7e1 | v0.18.6: bump version (unified release with bdemu/freemkv/autorip) | ||
|
matthew
|
e61caff4de |
labels: expose analyze() for corpus regression tooling
Promotes `mod labels` to `pub mod labels` and adds `analyze()` plus `LabelAnalysis` (both `#[doc(hidden)]`) so an out-of-tree diagnostic binary (freemkv-tools labels-analyze) can introspect which BD-J parser matched a given disc, what JAR files the discriminators saw, and what labels came out — without going through the production `apply()` path that mutates DiscTitles. Also adds `tracing::info!(parser = name, "label parser matched")` / "no label parser matched" inside `extract()`. Dev-only signal: users get the same seamless behavior; developers can finally tell whether a disc hit a real parser or fell through to the codec-name fallback in fill_defaults(). The new `jar_inventory()` helper deduplicates and sorts filenames under any `/BDMV/JAR/<x>/` subdirectory — same plumbing the existing `jar_file_exists()` discriminators use, just enumerated rather than predicate-tested. Used by `analyze()` to surface unrecognized parser-source files when no parser matches, which is the input to "do we need a new parser?" triage. No behavior change to the production label path. `apply()` and `extract()` remain functionally identical; the new public surface exists alongside. |
||
|
matthew
|
afa04aa6cd |
ci: drop --locked from libfreemkv workflows
libfreemkv is a library — Cargo.lock is gitignored (standard for libs). --locked refuses to create a lockfile on a fresh runner, so it always fails CI. --locked stays in the binary crates (freemkv, autorip, bdemu) which DO track Cargo.lock and benefit from the dependency-race hard-fail behaviour. |
||
|
matthew
|
c6bcce0d67 |
fmt: rustfmt-mandated reflow of canonical_title_order tests
The 0.18.4 commit landed with rustfmt diffs in the new canonical_order tests because my local validation script piped 'cargo fmt --check' to 'tail -1', masking the diff output and reporting green when fmt was actually unhappy. CI's lint job caught it immediately. No code change — pure formatting. |
||
|
matthew
|
455b359e9c | v0.18.4: cargo --locked everywhere — hard-fail dependency races | ||
|
matthew
|
5190c2b063 | v0.18.3: canonical_title_order — main feature first on branching UHDs | ||
|
matthew
|
518c5293c8 |
Disc title order: main feature first on branching UHDs
Disc::titles previously sorted purely by duration_secs descending,
which puts a play-all virtual playlist at index 0 on UHDs that ship
one. Such playlists reference the same source clips multiple times
for seamless alternate-angle / alternate-ending playback and report
inflated duration AND inflated size_bytes that exceeds the disc's
physical capacity.
Concrete observed case (The Amateur 2025 4K UHD, 58.5 GB BD-100):
Title 1 — 00020.mpls — 4h13m — 92.4 GB — 253 clips ← impossible
Title 2 — 00800.mpls — 2h02m — 57.2 GB — 1 clip ← the movie
92.4 GB > 58.5 GB capacity is proof of clip double-counting. With
the duration-only sort, freemkv -t 1 / disc.titles.first() / autorip's
main-feature picker all selected the 4-hour composite instead of the
2-hour movie.
New canonical_title_order:
1. Real titles (size_bytes <= capacity_bytes) before virtual
composites — capacity gate is hard physical truth.
2. Among real titles, fewer clips first (1-clip wins as the
canonical main feature; multi-clip is either chapter-stitched
or composite).
3. Tiebreak on longer duration first.
Behaviour:
- Non-branching discs: unchanged. The longest 1-clip title is
already the movie.
- Branching UHDs: virtual composite drops to the back, the real
movie surfaces at index 0.
Comparator exposed as Disc::canonical_title_order for downstream
consumers that need the same logic on custom title sets.
Three regression tests (disc::tests::canonical_order_*):
- pushes_oversize_play_all_behind_real_main (The Amateur)
- preserves_natural_ranking_on_normal_disc
- fewer_clips_wins_tiebreak
|