The pipeline's adaptive autotuner grows chunk_bytes only when the p95
WAIT_AFTER latency exceeds 200 ms. On NFS, sync_file_range(WAIT_AFTER)
translates to an NFS COMMIT RPC whose ack lands within ~10 ms — so
the autotuner never triggered and the pipeline stayed at the original
32 MiB initial value forever.
That capped sustained mux throughput by paying NFS COMMIT-RPC overhead
roughly once per second of writes. Bidirectional mountstats on rip1
2026-05-15: write side 29 MB/s (RTT 30 ms but exec_time 257 ms — 220 ms
queue/serial waiting) while concurrent dd on the same disk shows
~91 MB/s write + ~65 MB/s read available.
128 MiB initial drops COMMIT cadence 4x while keeping the bounded-
cache invariant intact (worst-case dirty pages ~2 x chunk = 256 MiB,
well under vm.dirty_ratio = 6.6 GB on the 32 GB rig). The adaptive
autotuner can still grow further (up to 256 MiB) or shrink if
WAIT_AFTER ever measures sub-20 ms p95 on faster media.
The WritebackPipeline's WAIT_AFTER + posix_fadvise(DONTNEED) dance
keeps dirty pages bounded at ~2 × chunk_bytes by waiting for each
chunk's writeback to commit before issuing the DONTNEED hint to drop
it from cache. The original 0.18-era design unconditionally skipped
this on NFS on the premise that "NFS clients have their own buffering
and commit semantics that handle dirty-page bounds without us forcing
the issue."
Empirically wrong. On unraid-1 NFS the kernel client buffers dirty
pages up to vm.dirty_ratio (default 20% of RAM = ~6.6 GB on the rip1
host) before the kernel forces writeback and throttles app writes.
Result on 0.21.11 mux measured 2026-05-15: mux throughput cycled
between ~45 MB/s (cache absorbing) and ~7 MB/s (cache draining under
throttle) on a ~100 s period — exactly the burst-flush pathology this
pipeline was built to fix, but disabled on the medium it actually
runs on. /proc/meminfo Dirty: column climbed lockstep with mux
write rate during the slow half of every cycle, confirming the cause.
The original safety concern — `sync_file_range(WAIT_AFTER)` hanging
indefinitely on a wedged NFS server — is already handled by
`wait_after_with_timeout`'s `bounded_syscall` wrapper (30 s deadline).
If a real WAIT_AFTER call exceeds the deadline the pipeline flips to
the `degraded` state and skips WAIT_AFTER + DONTNEED for the rest of
its life — same effect as the old NFS branch, but only triggered when
something is genuinely broken rather than as a blanket exception.
This change is medium-agnostic: every medium goes through the same
path now, every medium gets the same safety net, and the
ADAPTIVE_WINDOW chunk-size autotuner (lines 226-255 — measures p95 of
WAIT_AFTER and resizes between 4 MiB and 256 MiB) finally activates
on NFS where previously it was dead code. Slow medium auto-grows
chunks to amortise per-chunk overhead; fast medium auto-shrinks to
keep cache pressure tight; nothing in the code special-cases the
filesystem type.
`is_nfs` is still detected (for logging + observability) but no
longer keys `skip_wait`. Module doc + startup log line updated to
match.
Pre-coalescing the Phase-2.5 writer thread issued one `file.write_all`
syscall per `Cmd::Write` dequeued. The mux side calls
`WritebackFile::write_all(buf)` per PES frame, typically 30-200 KB.
On NFS that translates to one RPC per syscall, capping per-thread
throughput at `(wsize / rtt) × inflight` — well below what the same
disk delivers under a 1 MiB `dd oflag=direct` workload (empirical
2026-05-15: dd 71 MB/s vs mux ~25 MB/s sustained, with instantaneous
samples bursting 7→108 MB/s as the kernel page cache filled and
drained on its own cadence).
Coalesce instead: dequeue drains consecutive `Cmd::Write` items off
the ring up to a 1 MiB byte budget, returns them as
`DequeuedWork::Writes(Vec<Vec<u8>>)`, and the run loop concatenates
into one contiguous buffer and issues a single `file.write_all`.
Non-write commands (Seek, Flush, SyncAll, Finish) break the run and
are returned one at a time as `DequeuedWork::Other`, preserving their
ordering relative to the writes.
Single-buffer fast path avoids the concat allocation when only one
write is in the queue at dequeue time. A single oversize write (e.g.
the rare matroska cluster larger than 1 MiB) is admitted alone so it
still makes progress — the kernel splits internally.
This is generic across mediums: bigger app writes are at-least-as-
good on local SSD, HDD, or NFS. On fast storage the ring rarely fills
so coalescing is mostly a no-op; on slow storage with significant
per-RPC overhead it materially improves throughput.
Tests in `tests/` (write_then_drop_persists_bytes, sync_all_drains,
seek_then_patch_roundtrip, flush_is_observed_in_order) still pass —
ordering and durability semantics are unchanged.
The depth was conservative because pre-0.21.8 a full sync_file_range
stall on NFS could traverse this channel and pin the producer. With
0.21.8's restored Phase 2.5 writer thread + 128 MiB byte-bounded ring
inside WritebackFile, every blocking syscall happens downstream of
this channel — never on it. The original "smaller buffer reduces
backpressure risk" rationale no longer applies.
Empirical (2026-05-15 Civil War UHD remux on 0.21.8): 30.9 MB/s
sustained but instantaneous samples spanning 9-53 MB/s, stdev 9.3.
Distribution clusters 52% of samples in 25-35 MB/s but has a long
9-15 MB/s tail. The tail corresponds to brief stalls in the matroska
builder when the sink momentarily lags — exactly the case a deeper
inter-thread channel covers. 32 frames at PES-frame sizes is still
well under a megabyte of additional memory, so the cost is zero.
Reverts 523af46. That revert was made on the premise that Phase 2.5
caused a ~60% mux throughput regression on NFS bidirectional workloads.
The premise was wrong: at the time of measurement the producer was
capped at ~8 MB/s by a 50 ms thread::sleep poll in
Pipeline::send_with_halt (fixed in v0.21.7's io/pipeline change), so
the comparison was measuring the polling cap on both sides.
With the polling cap removed, direct passthrough exposes the kernel's
default dirty-page writeback pathology on NFS: writes accumulate, the
kernel periodically bursts a flush, app writes block for the burst.
Observed empirically on Civil War UHD remux 2026-05-14: 5-45 MB/s
spiking around a ~21 MB/s sustained mean, dominated by burst-flush
back-pressure cycles.
Phase 2.5 decouples the mux thread from the file syscall:
* mux writes complete instantly into a 128 MiB byte-bounded SPSC ring,
* a dedicated writer thread executes the real File writes, seeks,
and sync_file_range calls; can sit in a kernel burst without
blocking the mux pipeline,
* backpressure via Condvar notify/wait, no polling primitive,
* the ActiveClusterBuffer fast-path preserves the original MKV
cluster-backpatch optimisation so in-window seeks don't drain
the current writeback chunk.
Halt-safety is preserved: every blocking writeback syscall on the
writer thread still routes through bounded_syscall with a 60 s
deadline. A wedged NFS server cannot trap the writer indefinitely;
the muxer keeps queueing into the ring; the kernel page cache and
the ring together absorb the stall.
The halt-aware send loop polled try_send on a 50 ms sleep slice when
the channel was full. That capped producer throughput at 1 / 50 ms =
20 frames/sec ≈ 1 MB/s at typical PES frame sizes — way below NFS,
let alone local SSD/NVMe. Multi-day diagnostic 2026-05-13/14 surfaced
it as the root cause of the 18 → 3 MB/s mux throughput regression on
0.21.x.
Replaced std::sync::mpsc::sync_channel with crossbeam_channel::bounded
and switched send_with_halt to send_timeout. Producer now BLOCKS on
consumer drain (kernel-wakeup) instead of polling — the timeout slice
(250 ms) only fires when the producer is waiting for stop signal
observation, never on the happy path.
Result: no throughput cap from this primitive at any medium speed.
Mux is bounded by actual storage / network bandwidth, not by our
channel implementation.
Same fix needs to land in autorip's mux.rs producer-side
sync_channel (separate sibling commit).
Documented in (internal)/memory/
feedback_send_with_halt_poll_throttle.md.
Empirical: isolated NFS read 70 MB/s + write 93 MB/s on the rip1 setup
right now, but mux throughput pinned at 2.7 MB/s on 0.21.5. NOT
environmental — code regression.
Root cause: Phase 1 silently dropped the read-side
posix_fadvise(POSIX_FADV_DONTNEED) eviction that the pre-Phase-1 (0.20.7)
hot path had. Without it, an 85 GB streaming ISO read pins the entire
file in the kernel page cache, starving concurrent MKV writeback. 0.21.2
then also dropped the POSIX_FADV_SEQUENTIAL hint on the same theory,
compounding the regression.
Restored both, per-OS split:
- linux: posix_fadvise(SEQUENTIAL) at open + posix_fadvise(DONTNEED)
on consumed 32 MiB windows
- macos: F_RDADVISE hint at open (kept); drop_window no-op (macOS unified
buffer cache less prone to the pin pathology)
- windows / other: both no-op stubs
Target mux speed restored to 20+ MB/s (per concurrent-NFS math:
70/2 read × 0.73 MKV/ISO ratio ≈ 25 MB/s achievable).
The Phase 2.5 writer-thread + bounded ring + ActiveClusterBuffer
architecture introduced a ~60% mux throughput regression on NFS
bidirectional workloads (18 MB/s -> 7-8 MB/s in 0.21.x). Other
candidates (FileSectorSource readahead buffer, per-OS sink split)
ruled out empirically across 0.21.1-0.21.4.
Reverts WritebackFile's hot path to direct File passthrough matching
the 0.20.7 baseline:
* Write/write_all/flush call straight through to self.file.
* Seek calls through to self.file and notifies the pipeline.
* sync_all runs pipeline.finalize() then per-OS durable_sync.
Kept (intentional): the per-OS file split established in earlier work
(writeback_file/{linux,macos,windows,other}.rs). create_with_size_hint
still dispatches to platform::preallocate and sync_all still routes
through platform::durable_sync (bounded_syscall 60s deadline on
Linux/macOS), so halt-safety on a wedged NFS fsync is preserved.
Removed:
* Cmd / RingState / Shared / WriterState / writer_thread_main
* ActiveClusterBuffer (active-cluster window for in-window seeks)
* push_command / dequeue / publish_error / mark_writer_gone
* RING_CAPACITY_BYTES, ACTIVE_CLUSTER_WINDOW_BYTES,
MAX_WRITE_CHUNK_BYTES, WRITER_THREAD_NAME constants
* writer JoinHandle field + Drop join logic
* Tests that asserted internal writer-thread state
(backpressure_blocks_when_ring_full, in/out_of_window_seek_*,
active_cluster_buffer_*, writer_thread_panic_surfaces_on_drop)
Kept tests (correctness-of-output only):
* write_then_drop_persists_bytes
* sync_all_drains_and_flushes (renamed from sync_all_blocks_until_ring_drains)
* seek_then_patch_roundtrip (replaces in_window_seek_then_patch_roundtrip)
* flush_is_observed_in_order
mod.rs: 1026 -> 303 lines. linux/macos/windows/other.rs untouched.
The three tests (multi_sector_read_spanning_buffer_boundary,
backward_seek_rebuffers, partial_buffer_at_eof) were asserting
internal buf_start_lba / buf_len_sectors state. With 0.21.3's
read-path bypass, those fields are no longer mutated. The
byte-level contract assertions (read returns correct bytes for
every scenario the tests cover) remain intact.
The 32 MiB readahead window (0.21.0–0.21.1) regressed mux throughput
on NFS bidirectional workloads vs the pre-Phase-1 0.20.7 baseline
(18 -> 7-8 MB/s). The 0.21.2 4 MiB shrink made it worse (5-6 MB/s).
Both signs point at the application-level buffer itself, not the size.
This commit bypasses the buffer entirely on the read path — every
read_sectors call seeks and pread()s direct to the file. That matches
0.20.7's hot path. Kernel readahead handles the policy; on NFS that
interleaves naturally with concurrent writes on the same TCP
connection.
Buffer state fields and refill/buffer_covers are kept so the
structure is preserved for a future per-source-type policy (e.g. a
local-disk source where batched reads ARE beneficial), and so the
existing tests still exercise that machinery.
Empirical regression observed during 0.21.1 mux test on rip1/unraid-1:
historical 0.20.7 baseline averaged ~18 MB/s mux throughput; 0.21.1
dropped to ~7-8 MB/s flat. Same NFS source + destination, same disc.
Suspect: 32 MiB FileSectorSource readahead + posix_fadvise(SEQUENTIAL)
together saturate the TCP connection on read bursts, starving the
writer thread's concurrent NFS writes (mux reads the source ISO and
writes the MKV over the same connection).
- READAHEAD_BUF_BYTES: 32 MiB -> 4 MiB. Matches NFS rsize=1 MiB * 4
round-trips per refill, interleaves cleanly with writes.
- linux/hint_sequential: now no-op. Kernel's default ~128 KiB
readahead is what we want on NFS-backed ISOs (the dominant case).
Per-OS file stays so we can re-enable a hint cleanly later if a
different path benefits.
CI's lint workflow runs clippy on linux target where:
- platform/fs_type/linux.rs and io/writeback/linux.rs: the i64 cast
on buf.f_type / NFS_SUPER_MAGIC is unnecessary on glibc x86_64 (both
already i64) but required on musl (c_ulong); silence the lint via
inline allow with explanatory comment.
- mux/m2ts_mux/packet.rs: doc comment continuation across lines was
parsed as an unindented list item. Reworded to a single flowing
sentence.
- m2ts_mux: replace u64::is_multiple_of (stable in 1.87+) with %.
CI on the pinned 1.86 toolchain rejected the unstable feature use.
- mux/{fmp4,hevc,m2ts_mux}: rustfmt drift cleanup (test-code wrapping).
SocketSink + UdpSocketSink (`src/io/sink/socket.rs`) — sequential-only
TCP/UDP write destinations. SocketSink wraps BufWriter<TcpStream> with
1 MiB capacity, tunes SO_SNDBUF on construction, calls shutdown(Write)
on finish(). UdpSocketSink emits one datagram per write — caller
packetizes. Both impl Write+Send and thus satisfy SequentialSink via
the Phase 2 blanket; neither impls Seek, so RandomAccessSink is
correctly inaccessible (compile error to mux MKV onto a socket).
New sequential container muxers in src/mux/:
- hevc/ — raw HEVC Annex B elementary stream. Length-prefixed NALU
→ 00 00 00 01 NALU. hvcC parsing emits VPS/SPS/PPS once at stream
head. Fully ships.
- m2ts_mux/ — standard MPEG-TS (188-byte packets). Single program,
HEVC video on PID 0x100, optional AC3/TrueHD audio on PID 0x101.
PAT+PMT re-emitted every 250 packets; PCR stamped on video every
40 packets. Hand-rolled, no new deps. Distinct from the existing
BD-TS (192-byte) `mux::m2ts::M2tsStream` — that path stays as-is.
- fmp4/ — fragmented MP4. STUB: ftyp + minimal moov skeleton with
one HEVC video trak + mvex/trex. Media fragments (moof+mdat) are
TODO for v0.22.0 — write_video accumulates frames into a pending
buffer that finish() clears. Init segment is well-formed enough
that init_segment_starts_with_ftyp_then_moov asserts the box
chain.
17 new unit tests added (socket round-trip, HEVC Annex B conversion,
M2TS packet alignment + PAT/PMT cadence + per-PID CC, fMP4 box chain).
All 514 lib tests + 17 new = pass on Rust 1.86 (fmt + clippy + test
via (internal)/scripts/precommit.sh libfreemkv).
No new dependencies. No version bump. Don't-touch list clean.
WritebackFile now offloads all File I/O to a dedicated writer thread.
Muxer's Write/Seek/sync_all push into a bounded byte ring; the writer
drains the ring and runs the syscalls.
- Commands: Write(Vec<u8>), Seek(SeekFrom), SyncAll(oneshot),
Finish(oneshot). One ring carries all four; ordering preserved.
- ActiveClusterBuffer mirrors the last ACTIVE_CLUSTER_WINDOW_BYTES
written. Seek-back within window → in-memory patch + re-emit
(no forced drain). Seek-back outside window → real drain+seek.
Wins for MKV: cluster size patches almost always land inside the
active window; only the end-of-mux Cues + segment header backpatch
fall outside.
- Ring capacity RING_CAPACITY_BYTES = 128 MiB. Backpressure on full
blocks the muxer (correct semantics for archival workflows).
- All syscalls in writer thread wrapped in bounded_syscall(60s) so
a wedged NFS doesn't pin the thread forever.
- Drop drains via Finish + joins the writer thread; sync_all blocks
until ring drained AND underlying fsync completes.
Introduces the SequentialSink / RandomAccessSink trait pair under
io::sink and an open_for_mkv dispatch helper that picks WritebackFile
on Linux+NFS and LocalFileSink everywhere else. LocalFileSink wraps
BufWriter<File> with a 4 MiB buffer and exposes a per-OS preallocate
path (fallocate on Linux, F_PREALLOCATE on macOS, no-op fallback).
Adds platform::fs_type::detect with a per-OS split (statfs on Linux /
macOS, UNC heuristic on Windows, Unknown elsewhere) so construction-
site dispatch has a single primitive to call.
Blanket impls cover the common shapes: any Write+Send is a
SequentialSink, and any SequentialSink+Seek is a RandomAccessSink.
WritebackFile satisfies the random-access trait via the blanket impl
without needing an explicit per-type impl. No callers wired yet — the
mux::resolve construction sites stay on WritebackFile pending Phase 3.
Tests: 5 new sink/preallocate tests + 3 fs_type tests (1 ignored,
needs a real NFS mount). cargo +1.86 fmt + clippy + tests all green.
Three changes targeting 0.20.9's "muxer never read-stalls on NFS read
latency" invariant:
A. FileSectorSource gets a 32 MiB internal read-ahead buffer
(READAHEAD_BUF_BYTES). Splits out from src/sector/file.rs into
src/io/file_sector_source/ with per-OS open hints (Linux
posix_fadvise(SEQUENTIAL), macOS fcntl(F_RDADVISE) with 64 MiB
cap, Windows TODO stub, BSD/illumos no-op). Backward seeks
rebuffer; partial reads at EOF return only the bytes that exist;
oversize-request bypass for count > BUF_SECTORS.
B. WritebackFile inline #[cfg(target_os = "linux")] blocks split
into per-OS files under src/io/writeback_file/. Linux unchanged
(fallocate KEEP_SIZE, fsync via bounded_syscall). macOS gets a
real F_PREALLOCATE + F_FULLFSYNC impl (was a "skipped (non-linux)"
debug log before). Windows is a stub (FlushFileBuffers via
std sync_all; TODO for SetFileValidData). BSDs/illumos fall back
to std sync_all.
C. New byte_channel module — byte-bounded producer/consumer wrapping
std sync_channel with Mutex/Condvar byte accounting. Sender blocks
when used_bytes + item.byte_size() > capacity. HasByteSize impl
for PesFrame. Default cap BYTE_CHANNEL_DEFAULT_CAPACITY = 64 MiB,
sized to absorb worst-case NFS read p99 (~2 s × UHD peak compressed
~15 MB/s). The mux call site lives in autorip (out of scope here);
this lands the primitive in libfreemkv for autorip to adopt.
Test counts: byte_channel +6, file_sector_source +5, sector::file
round-trip suite (3) preserved. passn_handler_ab.rs A/B fixture
(8 profiles) still green.
precommit.sh libfreemkv: fmt + clippy + test all green on Rust 1.86.
No version bump; no Cargo.lock changes; no forbidden-file edits
(disc/patch.rs, disc/read_error.rs, io/pipeline.rs,
tests/passn_handler_ab.rs).
The 0.20.7 work lives downstream in autorip — process-level safety net
(hard watchdog 5-min mux escalation + std::process::exit, restart-loop
counter + auto-quarantine after 3 attempts, partial-state preservation,
failure_reason surfaced in /api/state).
Bumping libfreemkv to 0.20.7 keeps the 4 crates on the unified release
train per project docs.
Generalizes 0.20.5's hand-written wait_after_with_timeout into a
reusable primitive. After this change, every blocking syscall in the
recovery + mux paths is wrapped, so cooperative Halt has bounded
~250 ms latency reach even into kernel-owned thread states.
New module src/io/bounded.rs:
- BoundedError { Halted, Timeout, WorkerLost }
- bounded_syscall<F, R>(halt: Option<&Halt>, timeout, op) -> Result<R, BoundedError>
- Worker thread runs op; main thread recv_timeouts on a rendezvous
channel in 250 ms slices, polling halt between slices.
- Worker is intentionally leaked on timeout/halt — kernel reaps when
the syscall finally returns or at process exit. Calling thread is
NEVER trapped inside a kernel call.
- 6 unit tests cover the happy path + each error variant.
Refactored callsites:
- src/io/writeback/linux.rs::wait_after_with_timeout now delegates
to bounded_syscall. ~30 LOC of duplicated channel/thread plumbing
deleted. Same semantics, cleaner.
- src/io/writeback_file.rs::WritebackFile::sync_all now wraps the
final libc::fsync(fd) with bounded_syscall (60 s deadline). On
timeout: log error at target=mux and return Ok — kernel will flush
on close, best-effort but bounded. Covers FileSectorSink::finish,
PatchSink::close, SweepSink::close, and the mux MKV finalize path
(they all sync through WritebackFile).
What still hangs (deliberately not wrapped — too hot a path):
- File::write itself. Per-frame write on a wedged NFS could still
block; but back-pressure from a stuck consumer means the producer
notices within seconds, not minutes — different failure mode than
the WAIT_AFTER hang 0.20.5/0.20.6 fix.
Targets the recurring mux hang on NFS dest where the consumer thread
sits indefinitely inside libc::sync_file_range(SYNC_FILE_RANGE_WAIT_AFTER)
because the NFS server never returns a commit ack. The whole rip
wedges; halt is cooperative and can't reach inside a kernel syscall.
A. NFS detection at WritebackPipeline construction (fstatfs f_type ==
NFS_SUPER_MAGIC 0x6969). When NFS:
- Skip SYNC_FILE_RANGE_WAIT_AFTER entirely.
- Skip posix_fadvise(DONTNEED) — NFS client handles its own buffering.
- Still issue async SYNC_FILE_RANGE_WRITE (harmless hint).
Cannot hang on a syscall not made. fstatfs failure fails open (assume
local). Logged at info on construction so operators see which strategy
is active. The whole hang vector is removed for NFS deployments.
B. Hard timeout on WAIT_AFTER for non-NFS (defense in depth, since
even a degraded local disk could in principle hang the syscall).
Each WAIT_AFTER runs on a worker thread; main thread waits on a
sync_channel rendezvous with 30s deadline. On timeout: log error,
set per-pipeline 'degraded' Arc<AtomicBool>, downgrade to NFS-style
skip for the rest of the pipeline's life. Worker thread leaks
intentionally — it'll unwind when the syscall eventually returns or
the process exits. Converts indefinite freeze into 'log loud +
downgrade + keep ripping'.
C. Diagnostic logging for the 73%-of-this-movie reproduction:
- WritebackFile::seek logs every non-trivial seek (from, to, signed
delta) at target=mux so we can see if MkvMuxer seeks back before
a stall.
- WritebackPipeline::finalize logs the chunk being finalised before
any WAIT_AFTER call, so a hung chunk is identifiable by offset.
No new dependencies. macOS / Windows noop stubs unchanged. Net
+198 LOC libfreemkv (mostly writeback/linux.rs).
Four targeted changes to maximize mux throughput regardless of storage
backend (local SSD, local HDD, NFS, network share) and surface enough
log data to diagnose 'mux slow' reports without a re-rip:
1. POSIX_FADV_SEQUENTIAL on FileSectorSource::open (Linux only).
Widens the kernel readahead window for sequential ISO reads. One
syscall at open, free on every storage type.
2. POSIX_FADV_DONTNEED on the ISO read side after every 32 MiB chunk.
Mirrors the writeback DONTNEED that already runs on the write
side. Keeps the read-side page cache bounded during multi-GB ISO
reads — eliminates the OOM-pressure / eviction-storm risk on
long mux runs. Linux only; per-drop trace at target="mux".
3. WritebackFile::create_with_size_hint(path, size_bytes) calls
fallocate(FALLOC_FL_KEEP_SIZE) on Linux to pre-reserve extents
for the output. Reported file size stays 0 (writes grow it
naturally) but the on-disk extent allocation is contiguous —
reduces extent fragmentation for big sequential muxes. Wired
into mkv:// and m2ts:// output paths via DiscTitle::size_bytes.
No-op on macOS/Windows; old create() kept with #[allow(dead_code)]
for callers without a size hint.
4. Adaptive WRITEBACK_CHUNK_BYTES in the Linux writeback pipeline.
Tracks sync_file_range(WAIT_AFTER) elapsed_ms in a rolling
16-sample window. p95 > 200 ms → double chunk size (cap 256 MiB).
p95 < 20 ms → halve (floor 4 MiB). One algorithm, both
fast-storage (small chunks, responsive) and slow-storage (big
chunks, fewer commit round-trips) optimized. Per-chunk trace +
per-32-chunk debug snapshot + info-on-resize so an operator can
see where the autoscaler settled.
All four are universal — no storage-type detection, no env vars to
flip, no per-deploy tuning required. Total +201/-6 across four files.
WO-5 (partial): the patch backtrack inner loop ('while bt_pos <
backtrack_end' in disc/patch.rs) issues per-sector reads to fill the
gap created by a damage-window skip. A long backtrack span can run
minutes; without an inline halt poll, the outer halt only takes
effect when control returns to the per-range loop. Adds a halt poll
at the top of each iteration so cancellation propagates inside the
backtrack span.
Per-sector read failures inside the backtrack already drop through
to the main fail path; this only changes the cancellation latency
between an /api/stop call and the producer actually unwinding. Drops
worst-case unwind from 'whole backtrack span × per-sector recovery
timeout' (minutes) to 'one in-flight SCSI command' (seconds).
The broader Arc<AtomicBool> → Halt migration on CopyOptions /
SweepOptions / PatchOptions / Drive::halt and Pipeline::send halt-
awareness is deferred — separate cycle, larger API impact.
WO-3c: Delete dead non-NOT_READY retry block (~100 LOC). The block
declared retry_count = 0 inside the per-iteration Err arm, so the
'MAX_NON_NOT_READY_RETRIES=3' budget actually fired exactly once
(1s pause + 1 retry) before falling through to NonTrimmed. The
'exponential backoff: 2s, 4s, 8s' comment was wrong by construction.
Cross-pass NonTrimmed retry (each pass gives the same sectors another
shot) already covers the recovery case, and gives the drive minutes
between attempts instead of 1-8 seconds — empirically what stochastic
recovery on the BU40N actually needs.
WO-4 (targeted slice): Add wedge-family cooldown on HARDWARE_ERROR /
ILLEGAL_REQUEST senses. These are what the BU40N's firmware fast-fail
state returns; every subsequent read in that state comes back in
<100ms. Pre-fix patch hammered the drive: mark NonTrimmed, sleep 1s,
advance, hit next wedge, mark, sleep 1s — exactly the rapid-retry
cadence the firmware is sensitive to. Now a wedge-family sense triggers
WEDGE_FAMILY_COOLDOWN_SECS=30 cooldown (matches read_error.rs's
ZONE_ENTRY_COOLDOWN_SECS), and WEDGE_ABORT_THRESHOLD=16 consecutive
wedges aborts the pass for autorip eject+reload. Any non-wedge read
clears the counter.
Also drops the duplicate NonTrimmed dispatch (Mapfile::record is
idempotent so it wasn't a correctness bug, but it doubled per-failure
consumer work).
WO-2 (delete SectorReader trait):
- The 0.18 trait split into SectorSource (read-only) and SectorSink
(write-only) is final; the legacy SectorReader alias was a bridge.
- Renames every internal &mut dyn SectorReader (~25 sites) to
&mut dyn SectorSource. The trait method capacity() becomes
capacity_sectors() with a default of 0 (preserves SectorReader's
default-0 behavior).
- Deletes the SectorReader trait, its blanket-to-Source bridge, and
the FileSectorReader type alias. Adds explicit forwarding impls
for Box<dyn SectorSource> and &mut dyn SectorSource so generic
decorators like DecryptingSectorSource<S: SectorSource> compose.
WO-3a (extract Disc::patch):
- Moves Disc::patch (1230 lines) and bytes_bad_in_title from
disc/mod.rs into disc/patch.rs as a split inherent impl. Zero
behavior change — pure mechanical relocation. disc/mod.rs drops
from 3,945 to 2,714 LOC.
WO-6 (partial):
- Deletes src/labels/png_filenames.rs — was a 72-LOC stub with
detect() returning false, never wired into the PARSERS registry.
project docs doc drift fixes (audited 2026-05-13):
- JUMP_BASE_SECTORS: 256→1024 (64 MB base for UHD, not 8 MB)
- PASSN_DAMAGE_THRESHOLD_PCT: 12→6
- PASSN_SKIP_SECTORS_BASE: 64→32
- MAX_RANGE_SECS=180: replaced by proportional range_sectors × 25,
capped at RANGE_BUDGET_CAP_SECS=1800.
The 0.18 trait split into FrameSource (read-only) and FrameSink
(write-only) was an over-engineered API. Consumers don't think
"frame source backed by MKV" — they think "open MKV for reading".
The split paid a real API-complexity cost (two trait names, two
re-exports, dual impls per bidirectional type, deprecation bridge)
for one marginal property: compile-time direction-safety at the
trait-object boundary. The runtime error path on a wrong-direction
call (StreamReadOnly / StreamWriteOnly) is unambiguous and rare in
practice.
Deletions:
- pes::Stream is no longer #[deprecated]
- pes::FrameSource trait + its blanket-from-Stream bridge
- pes::FrameSink trait + the trampoline impls on every concrete type
- The compile-time-direction-safety test scaffolding
- Crate-root FrameSource / FrameSink re-exports
Additions:
- Stream is now Send-bounded (Stream: Send supertrait). Every
concrete impl was already Send-compliant — Box<dyn Read + Send>
and Box<dyn Write + Send> were already in place on the trait
objects MkvStream / M2tsStream / etc hold internally. Promoting
Send into the trait makes Box<dyn Stream> Send too, which lets
autorip drop its SendStream unsafe newtype.
The public API is now: one Stream trait, one concrete type per
format, two constructors (open/create or input/output). Bidirectional
types route through internal Mode { Read | Write } discriminants.
Net: -347 lines libfreemkv, -38 lines autorip, -5 lines freemkv.
v0.19.0 was tagged with a search-and-replace gone wrong:
rust-version, serde, and zip all had their version strings
replaced with "0.19.0". Edition 2024 rejected rust-version
0.19.0 (< 1.85), failing every CI build. No artifacts shipped
to crates.io.
Repair:
- rust-version: 0.19.0 → 1.86 (CI pin)
- serde: 0.19.0 → 1
- zip: 0.19.0 → 2
Also drops an unused start_lba binding in mux/disc.rs that
clippy 1.86 catches.
scan_with() collapsed every failure path from resolve_encryption() into
None via .ok(), so callers couldn't tell the difference between "no
KEYDB found", "KEYDB failed to parse", "disc hash not in KEYDB and
fallback derivation failed", "AACS files unreadable on disc", and a
handshake that rejected every host cert. autorip's UI was stuck
printing "no decryption keys found (check KEYDB)" for all of them,
which is a particularly bad message when the user has actually loaded
a KEYDB and the real failure is something else.
Changes:
- New pub field Disc.aacs_error: Option<Error>. Populated by scan_with
whenever encrypted && aacs.is_none(). Sentinel KeydbLoad path
"<no keydb in search paths>" distinguishes the no-keydb case from
a real load failure without adding a new Error variant (which would
be a breaking change for downstream exhaustive matches).
- tracing::warn in scan_with at scan_aacs_resolve_failed and
scan_aacs_no_keydb, with error_code and keydb path for grepping.
- tracing in do_handshake: keydb load failure, host-cert exhaustion
(with cert count and last error code), VID read failure post-auth,
and a debug-level success log. Lets us see whether handshake even
got off the ground for a given disc.
Test fixtures updated to set aacs_error: None.
The 'All probes failed — possible wedge condition' log fired during patch
probing whenever 10+ consecutive failures hit AND a probe sweep at the
local zone returned 0 successes. This was distinct from the read_error.rs
'wedge_transition' log that fires when the SCSI sense family ACTUALLY
flips into Hardware/IllegalRequest fast-fail mode.
Two logs both saying 'wedge' caused operator confusion during the
2026-05-11 Dune Pt 2 wedge investigation — was the drive wedged, or was
it just a zone of fully-bad sectors? They mean different things.
Relabel to 'patch_zone_fully_bad' with explicit pointer to read_error.rs
for the canonical wedge detection. Same triggering condition; just clearer
wording in the log stream.
Disc-04 (Top Gun: Maverick) re-test 2026-05-11 surfaced a real-world
bdmt_eng.xml where <di:description> contained no prose, only nested
<di:thumbnail href="…"/> elements. The previous parser surfaced
the raw XML fragment as the description string ("<di:thumbnail
href=\"tgm_meta_sm.jpg\" />\\r\\n <di:thumbnail
href=\"tgm_meta_lg.jpg\" />"). Worse than no description.
Fix: filter description candidates that begin with `<` after
trimming. Real prose never starts with an angle bracket; XML-only
content always does. Net: title extraction unaffected (it uses its
own element-priority path); description field drops when it would
otherwise carry XML noise.
Two new bdmt tests, 12 of 12 passing.
Three layered sources of stream labels now, in precedence order:
1. **Framework parser** (paramount/criterion/pixelogic/ctrm/dbp/deluxe)
— editorial labels with purpose/qualifier ("English Atmos",
"Director's Commentary", "English SDH"). High or Medium confidence.
2. **MPLS gap-fill** (`fill_gaps_from_mpls`) — every stream the
playlist references gets at least a basic lang+codec label, even
when the framework parser missed it.
3. **CLPI orphan append** (`append_clpi_orphans`) — streams in
/BDMV/CLIPINF/*.clpi ProgramInfo that NO MPLS playlist references.
Empirical (2026-05-11): ~5% of streams across the 11-disc corpus,
most dramatic on disc-02 (HDMV-only) at 40% CLPI-only.
Orphan numbering: each appended orphan gets
`stream_number = max(existing per type) + N` so playlist-reachable
streams keep their original positions and orphans sort cleanly at
the tail.
Orphan dedup: (stream_type, language, codec_hint) tuple — fuzzier
than PID matching (PIDs aren't carried on StreamLabel) but it's the
only signal available downstream of the gap-fill. False positives
(genuine orphan that happens to share lang+codec with an existing
entry) silently drop, which is the conservative failure mode — the
user-facing display would just see a confusing duplicate otherwise.
`mpls_universal::language_display_name` and `::codec_name` promoted
from private fn to pub(crate) so this module can build orphan labels
with consistent naming.
Tests: 2 new in gap_fill_tests — synthetic-input verification of the
dedup tuple logic and the stream_number assignment. 6/6 tests in the
gap-fill module now passing.
Rewrites the Pass 1 wedge handling from "slow skip after the drive
has already wedged" to "prevent the wedge transition in the first
place." Driven by 2026-05-11 empirical data: the BU40N transitioned
into IllegalRequest fast-fail mode at exactly 7 medium errors in
6.5 seconds (~1 read/sec retry cadence). Once there, only physical
eject + reload clears it — 30s pauses + 1 GB jumps do not.
The fix is the user's mental model from that session:
"We can detect bad reads, failed reads, and asking to read again
fast after causes a wedge. We need to prevent the wedge in the
first place."
Two changes to the centralized error handler:
1. **`for_sweep().fast_jump_threshold = 1`** (was 4). Pass 1 now
JumpAheads on the FIRST outer-batch failure, not the 4th. The
drive never gets back-to-back retries at the same LBA in Pass 1
— every error → jump 64 MB forward + long cooldown. Pass N keeps
`fast_jump_threshold = u64::MAX` because retries on already-known-
bad LBAs are its whole job.
2. **`ZONE_ENTRY_COOLDOWN_SECS = 30`**. The FIRST error after a
clean run (when `consecutive_outer_failures == 1` and we're not
bisecting) uses this long pause instead of the standard 5 s
FAIL_PAUSE_SECS. Gives the BU40N's firmware / bridge internal
retry counters 30 s of breathing room before the next read,
preventing the "7 errors in 6.5 s" cascade. Subsequent errors
in the same zone use the standard 5 s pause (we've already
jumped past the initial damage; further errors mean we landed
in another bad cluster).
Pass N exempt from the zone-entry cooldown — `bisect_on_marginal=
true` skips the long-pause arm. Pass N's per-sector retries on
known-bad LBAs would multiply uselessly with 30 s/error.
Test updates: 4 tests' expected behavior changed under the new
policy. Renamed `pass_1_marginal_skips_instead_of_bisecting` →
`pass_1_marginal_jumps_immediately_not_bisecting`. Renamed
`pass_1_jumps_after_4_consecutive_outer_failures` →
`pass_1_jumps_immediately_on_first_outer_failure`. Updated
`both_passes_pause_on_failed_read_for_wedge_avoidance` (now
`pass_1_zone_entry_uses_long_cooldown` + `pass_n_pauses_uniformly_on_failed_read`).
Cost analysis:
- Clean disc (no errors): unchanged. 0% overhead.
- Lightly damaged (1-2 zones): +30 s per zone = ~1 min total. Fine.
- Heavily damaged (10+ zones): +5+ min total. The trade for never
wedging the drive and getting a usable Pass N afterwards.
Expected behavior on the next damaged-disc rip:
- Pass 1 hits damage at LBA X → jumps 64 MB forward immediately,
pauses 30 s
- Drive's firmware never accumulates the retry pressure that triggers
IllegalRequest fast-fail
- bytes_maybe accumulates faster (we skip more), but Pass N picks up
the slack with proper per-sector recovery — and Pass N can actually
RUN because the drive isn't wedged
Two layered changes, in service of the empirical question "is CLPI
truly redundant with MPLS for label data?":
1. **clpi.rs ProgramInfo parser**. The existing CLPI parser only
walked the EP map (for sector-range lookups). Added a parser for
the ProgramInfo section's per-stream stream_coding_info table:
pid, coding_type, audio_format/rate, video_format/rate, ISO 639-2
language. Spec layout per libbluray clpi_parse.c. Best-effort —
malformed program_info leaves `streams: vec![]`, EP map keeps
working. `ClipInfo` gains a `streams: Vec<ClpiStream>` field.
2. **labels/clpi_audit.rs**. Diagnostic that walks both
`/BDMV/CLIPINF/*.clpi` (via the new program_info parser) and
`/BDMV/PLAYLIST/*.mpls`, builds a (PID → fields) merged view, and
classifies each row:
- `Match`: both sources agree (same coding_type + language)
- `ClpiOnly`: PID in CLPI but no MPLS playlist references it
(orphan stream on disc — reachable via low-level access, not via menu)
- `MplsOnly`: PID in MPLS but no CLPI lists it (would indicate a
parser bug; verified empirically that this NEVER happens)
- `Divergent`: same PID, different coding_type or language between
sources (playlist re-tagged or attribute encoding mismatch)
Surfaced via `labels-analyze` as `clpi_vs_mpls_audit: {matches,
clpi_only, mpls_only, divergent, total_pids}`. Doesn't affect the
label output — pure diagnostic.
Empirical findings on the 11-disc corpus (excl. disc-04 truncated):
- 226 matches / 0 mpls_only / 8 clpi_only / 5 divergent across 239 PIDs
- 6 of 10 non-truncated discs have CLPI-only streams (orphans)
- disc-02 (HDMV-only) is the most dramatic: 40% of its 5 streams are
CLPI-only — MPLS sees 3, CLPI sees 5
- Conclusion: CLPI is NOT truly redundant. ~5% of streams disc-wide
are CLPI-exclusive. Future work: layer CLPI as a tertiary source
below MPLS in the labels pipeline (orphan streams marked with even
lower confidence than MPLS).
LabelAnalysis gains `chapter_summary: Vec<ChapterSummary>` — one row
per .mpls file in /BDMV/PLAYLIST/, with chapter count (PlaylistMark
entries with mark_type ≤ 1) and approximate playlist duration in
seconds. Sorted by playlist filename.
Sourced from the existing crate::mpls parser (no new format work).
Useful for identifying the main feature playlist at a glance — it's
the one with the longest duration. Verified on disc-11 (Dune Pt 2):
00800.mpls correctly identified as 2h 45m 49s with 18 chapters
amid 30+ shorter playlists.
Doesn't touch the per-title `disc::DiscTitle::chapters` field which
disc::bluray.rs already populates from the same marks during disc
init — this is purely the diagnostic surface for labels-analyze.
When a framework parser (paramount, criterion, pixelogic, ctrm, dbp,
deluxe) is chosen but its label list covers only a subset of the
stream slots MPLS knows about, merge MPLS-derived entries for the
uncovered (stream_type, stream_number) slots. Framework labels keep
their richer fields (purpose=Commentary, codec_hint with "Atmos",
qualifier=Sdh); MPLS only fills slots the framework left unnamed.
Implementation:
- `fn fill_gaps_from_mpls` walks the MPLS label list, pushing any
entry whose (type, number) tuple isn't already in the framework
output. Stable sort by (type, number) groups audios before
subtitles in the merged result.
- Called from both `extract()` and `analyze()`. Skipped when the
chosen parser is itself `mpls_universal` (no gaps possible).
- `LabelAnalysis::gap_fill_added` field reports how many slots got
filled — useful diagnostic from `labels-analyze`.
- `StreamLabelType` gains `Eq + Hash` so the dedup HashSet works.
Tested via 4 new unit tests (155 of 155 labels tests passing, was
151). End-to-end on partial-yield corpus discs:
- disc-05 (Oppenheimer): pixelogic 4/5 already covered, gap_fill_added=0
- disc-11 (Dune Pt 2): pixelogic 8/11 already covered, gap_fill_added=0
(Real-world gap-fill activations are rare in the current corpus because
pixelogic already incorporates MPLS-equivalent data when matching;
the merge is defensive for less-thorough frameworks.)
Per-playlist stream_number counters were resetting between MPLS
files, so a disc with 2 MPLS files each listing the same 8 audio
streams produced labels with stream_number 1..8 then 1..8 again
(dedup kept whichever PID was different, leaving the numbering
visibly broken — multiple "audio1: eng" rows).
Move the counters outside the per-file loop and increment only
when an entry survives dedup. Surviving entries now get dense
1..N numbering across the whole disc per stream_type.
Verified on corpus disc-02 (HDMV-only): was `audio1, audio1, audio1`
for the 3 distinct audio codecs (TrueHD/AC-3/DTS-HD MA), now
`audio1, audio2, audio3`. Same fix applies to disc-01 (12 audio
streams across multiple MPLS) and disc-09 (14 audio streams).
Three new modules in the labels platform, all layered so framework-
specific parsers (paramount, criterion, pixelogic, ctrm, dbp, deluxe)
always take precedence over the universal layer.
**mpls_universal.rs** (~600 LOC, 9 tests): consumes the already-parsed
`crate::mpls::Playlist::streams` and emits StreamLabel entries with
language + codec_hint per stream. Returns `Confidence::Low` (new
variant) so framework parsers' Medium/High always win the registry's
max-by-confidence tiebreaker; MPLS only gets picked when no framework
matched. Closes the "no BD-J disc" case (HDMV-only navigation) that
previously produced zero labels — language and base codec are
spec-mandated in MPLS STN tables on every Blu-ray ever made.
**bdmt.rs** (~350 LOC, 10 tests): reads `/BDMV/META/DL/bdmt_<lang>.xml`
files into a new `DiscMetadata` struct (localized title names per
ISO 639-2 code, descriptions, optional box-set position). Runs
independently of the parser registry — disc-level metadata, not
per-stream, so the registry's confidence selection doesn't apply.
Surfaced on a new `LabelAnalysis::disc_metadata` field.
**png_filenames.rs** (noop stub): pattern documentation + dead-code
detect/parse for future reactivation. Deferred because MPLS already
delivers per-stream lang/codec/type on every disc; PNG filename
language tokens only add studio variant disambiguation (FRC vs FRP,
LAS vs CSP) — niche enough to not justify the implementation cost
right now.
Wiring changes in `mod.rs`:
- New `Confidence::Low` variant (PartialOrd places it below Medium/High)
- New `ParseResult::low()` constructor
- `mpls_universal` appended last to `PARSERS` registry
- `LabelAnalysis::disc_metadata: Option<DiscMetadata>` field
- `analyze()` runs `bdmt::parse` independently and surfaces result
- `pub use bdmt::DiscMetadata` re-export so the labels-analyze tool
in freemkv-tools can construct the JSON payload
Total: 151 of 151 labels tests passing (was 132 — added 19 new).
Pass N now reads at 32 sectors per attempt and drops to 1 only on
batch-read failure to probe each sector individually. After 16
consecutive clean single-sector reads it climbs back to 32. Net
effect: NonTrimmed regions walk ~32x faster in clean stretches
without sacrificing per-sector recovery quality — the drop-to-1
retry from the same cursor position guarantees every sector in a
failed batch is individually attempted.
Design contract:
- A batch-read failure (count > 1) is NOT a recorded failure: no
NonTrimmed mark, no consecutive_failures bump, no damage_window
push, cursor stays put. We just drop current_batch to 1 and the
loop re-attempts the same position at single-sector granularity.
- A single-sector failure (count == 1) follows the existing path:
NonTrimmed mark, consecutive_failures++, damage_window.push(false),
post-failure pause, wedge probes.
- Backtrack always at count=1: this path fills a gap that the main
loop's damage-window skip jumped over. Using batched reads there
would lump good sectors into NonTrimmed marks when the gap
contains even one bad sector.
State machine adds:
- `initial_batch` (from opts.block_sectors, default 32 in patch_internal)
- `current_batch` (mutable, starts at initial_batch, drops to 1 on
batch failure)
- `consecutive_singles_ok` (counter, resets on upscale + failure)
- `ADAPTIVE_UPSCALE_THRESHOLD = 16` (matches sweep's pattern for
"16 consecutive good = back to fast mode")
Tests:
- pass_n_size_aware_skip.rs PatternedSectorReader now fills each
sector with its OWN LBA byte (not the starting LBA's byte). This
matches real drive behavior — the pre-0.18.13 fixture's
"fill whole batch with one byte" was a shortcut that only worked
when patch read 1 sector at a time. Existing recovery-quality
assertions all still pass under adaptive batching.
User spec: "try 32, pass, great, fail -> do 1 sector"
User design call after watching Pass 2 mark ~20 KB as "Cosmetic"
(permanently Unreadable) after just 10 retries within a single pass:
"i think it's good or maybe until all passes are done. then it's
gone."
That contradicts what the multi-pass design promises a user. The
project goal in project docs is "recover 100% of readable data from any
optical disc, automatically." Marking sectors Unreadable after a
SINGLE pass's per-range retry budget gives up on sectors that
subsequent passes might recover — drive reads are stochastic, the
sector that fails 10 times in Pass 2 may succeed on attempt 1 in
Pass 3 after temperature / bus state / prior-read patterns shift.
The patch.rs doc comment already noted ~36% of patch-marked
Unreadable sectors turned out to be readable in re-rip experiments.
Three sites in `Disc::patch` were emitting `PatchItem::Unreadable`
mid-pass:
- backtrack hit damage (line ~2659)
- all-retries-exhausted on a single LBA (line ~2846)
- redundant second mark after the wedge-suspicion log (line ~2970)
All three now emit `PatchItem::NonTrimmed` instead. Failed bytes
stay "maybe" (NonTrimmed) so the next pass gets another shot. The
per-range skip-limit (10) and per-pass wedge-threshold (50) still
bound time-per-pass; they just no longer turn the bytes terminal.
The `PatchItem::Unreadable` variant stays in the enum (with
#[allow(dead_code)]) because the orchestrator-side end-of-recovery
promotion will use it: autorip, after the final retry pass
completes, scans the mapfile and promotes still-NonTrimmed →
Unreadable. That promotion lands in a follow-up commit on the
autorip side — separable from this libfreemkv change.
Loss accounting unchanged: `bytes_pending + bytes_unreadable` is
the "lost or pending" total that `abort_on_lost_secs` consults
(disc/mod.rs:1327). Moving bytes from one bucket to the other
mid-pass doesn't affect whether the rip would abort; it only
affects display (UI shows "Maybe" vs "Cosmetic") and whether
subsequent passes retry the bytes (the actual fix).
Test update: `test_pass_progress_separates_unreadable_from_pending`
was renamed to `test_pass2_leaves_failed_reads_as_pending_not_unreadable`
and rewritten to assert the new invariant — Pass 2 leaves all
failed bytes as bytes_pending (no mid-pass Unreadable promotion).
Original assertions were checking the pre-design-call behavior.
Precommit (cargo +1.86 fmt + clippy + test) green.
User's design call after watching the avoidance work prevent a wedge
on the live rip (no wedge events across 6 read errors): "Pass N
and 1 should both be very very similar in recovery. almost identical
just smaller sectors imo in pass n. pause times the same imo as a
failed read is a failed read."
The error-handling code path was already centralized (one
handle_read_error fn, called by both Disc::sweep and Disc::patch).
The TUNING was split — Pass 1 used 5 s inter-error pauses + a
wedge-skip-and-continue policy; Pass N used 1 s pauses + immediate
AbortPass on HARDWARE_ERROR / ILLEGAL_REQUEST. That asymmetry made
Pass N vulnerable to the same wedge that Pass 1's avoidance fixed.
Changes:
1. FAIL_PAUSE_SECS = 5 — single constant, applied uniformly to both
passes. Dropped PASS_1_FAIL_PAUSE_SECS and POST_FAILURE_PAUSE_SECS
in favor of one value. CONSECUTIVE_FAIL_LONG_PAUSE_SECS kept as a
distinct (but currently equal) value for future tuning escalation.
2. HARDWARE_ERROR / ILLEGAL_REQUEST path is now symmetric:
- Pass 1: JumpAhead WEDGE_JUMP_SECTORS (1 GB) + WEDGE_PAUSE_SECS
cooldown, mark skipped region NonTrimmed.
- Pass N: JumpAhead WEDGE_PASS_N_SKIP_SECTORS (64 sectors / 128 KB)
+ WEDGE_PAUSE_SECS cooldown. Pass N's batch=1 means a 1 GB skip
would abandon the entire current NonTrimmed range; small skip
moves past the bricked LBA + buffer, outer patch loop picks up
the next sector.
- Both share WEDGE_ABORT_THRESHOLD — same 16-skip budget before
real AbortPass on a permanently stuck drive.
3. wedge_skip / wedge_abort tracing logs now include `pass=1|N`
so post-mortems can see which pass hit the wedge condition.
Cost analysis:
Pre-reframe worry was "5 s × 5500 NonTrimmed sectors per Pass N
pass × 7 passes = 53 hours." Reality: most NonTrimmed sectors
recover on first or second retry, so most reads are successful and
pay 0 pause. The few that DON'T recover hit the 10-skip budget and
get marked Unreadable — bounded at 10 × 5 s = 50 s per truly-bad
sector. Worst-case Pass N pause overhead on a typical damaged disc
is single-digit minutes, not hours. And it's strictly cheaper than
the alternative (wedge kills the entire multi-pass recovery).
Tests:
- `both_passes_pause_on_failed_read_for_wedge_avoidance` — locks the
unified pause-tuning policy (was pass_1_pauses_briefly).
- `pass_n_hardware_error_also_skips_not_aborts` — was
`pass_n_hardware_error_still_aborts`. New behavior verified:
JumpAhead with WEDGE_PASS_N_SKIP_SECTORS + WEDGE_PAUSE_SECS.
- `pass_n_hardware_error_aborts_after_threshold` — new. Confirms
Pass N respects the same WEDGE_ABORT_THRESHOLD as Pass 1.
- pass_1_does_not_pause_on_skip is gone (it was the old "Pass 1
pause=0" assertion, irrelevant after the avoidance work).
Empirical validation: avoidance was already proven on a live rip
tonight — 6 read errors on a damaged disc, sense_family=Medium
throughout, wedge_count=0, Pass 1 continued cleanly past 40%
where it previously died at 48%. This commit extends the same
discipline to Pass N's recovery loop.
Precommit (cargo +1.86 fmt + clippy + test) green.
Replaces the speculative arg-position heuristic with type-presence
detection driven by real disc bytecode. Ground truth captured in
(internal)/research/deluxe-poc/data/ via POC v0.3 binding-
bytecode dumps against disc-01 (Disney) and disc-09 (Warner).
What changed:
1. StackVal::CodingType(String) — new variant. getstatic against
org/bluray/ti/CodingType (the BD-J spec codec enum) now pushes
this, carrying the field name (e.g. DOLBY_LOSSLESS_AUDIO). The
pre-fix code was treating codecs as a Deluxe-internal enum
subclass walk (Phase B), which is the wrong model — codecs are
standard BD-J API references.
2. coding_type_to_codec_hint(field) — new function. Maps
org.bluray.ti.CodingType field names to human-readable codec
strings (DOLBY_LOSSLESS_AUDIO -> "Dolby TrueHD", DOLBY_AC3_AUDIO
-> "Dolby Digital", etc.). Unknown field names pass through
verbatim so future codec values still surface something.
3. find_binding_classes — multi-class variant. Some Deluxe discs
split per-stream tables across two binding classes (audio +
subtitle). Returns top-K candidates by getstatic count, filtered
to >=40% of the top count and capped at 4. Replaces the old
single-class find_binding_class (which was unused after this
change).
4. interpret_streams — rewritten. Args identified by TYPE not
position:
- First EnumRef{kind:"Language"} -> language
- First EnumRef{kind:"Purpose"} -> purpose
- First CodingType(name) -> codec_hint
- First Int(n) -> stream index hint (traced
only; per-type sequential stream_number still wins because BD
spec stream-numbering is anchored on MPLS)
- Construction has CodingType -> Audio stream; otherwise Subtitle
- No Language -> skip (not a stream construction)
This handles BOTH the Disney 5-arg pattern (I, Lbe, Llp, I,
LCodingType) and the Warner 4-arg pattern (I, Law, Lgp,
LCodingType) automatically — same code path because args are
identified by type rather than constructor-signature shape.
5. parse() now walks all binding-class candidates and unions
their constructions before calling interpret_streams. Logs each
candidate at INFO with getstatic_count for diagnosis.
Tests:
- 2 new tests verify the CodingType -> codec_hint mapping for
known + unknown field names.
- Existing interpret_streams tests updated to use the new
signature (dropped CodecTable arg).
- Audio-emission test rewritten to use CodingType arg instead of
the old binding_type substring-match approach.
Confidence is still Medium for now (single-corpus verification);
ready to promote to High once tested against a third Deluxe disc.
Precommit (cargo +1.86 fmt + clippy + test) green.
Wires the existing PassSummary infrastructure (in read_error.rs as
of a832bad) into the sweep loop's exit path. One INFO log line per
Pass 1 completion gives operators an at-a-glance damage profile
without grepping per-error WARN lines:
INFO pass1_summary total_reads_ok=384521 total_errors=5
zones_entered=1 jumps_taken=2
bytes_good=38_725_644_288 bytes_pending=46_GB
copy_elapsed_ms=1751650
Particularly useful for post-mortem analysis when combined with
the per-error structured WARN logs (ms_since_last_error /
ms_since_last_success / sense_family / wedge_transition) shipped
in 0.18.10. Single line tells you the pass shape; preceding WARN
lines tell you the per-error detail.
Pass N (Disc::patch) intentionally NOT covered in this commit —
Pass N has its own retry-budget summary semantics that warrant a
separate design pass. Pass 1 sweep is where wedge incidents
originate, so it gets the diagnostic surface first.
Staged for 0.18.11. 0.18.10 already shipped the per-error WARN
layer; this is the finishing companion log.
Adds the observability we need to debug wedge incidents from logs
alone — without needing to enable verbose TRACE-level SCSI tracing.
Goal stated by user: "when error occurs we can debug and code
correctly."
Pre-fix the WARN log on each read error showed only sense codes
and consecutive_failures. Missing: timing context (was the failed
read fast or slow?), gap to previous events (cumulative vs.
immediate failure?), and family transitions (did the drive just
flip into wedge mode, or has it been there?).
New fields on ReadCtx (no caller signature change):
last_success_at: Option<Instant>
last_error_at: Option<Instant>
last_error_family: Option<SenseFamily>
total_errors: u64
total_reads_ok: u64
zones_entered: u64
jumps_taken: u64
in_damage_zone: bool
New SenseFamily enum (NotReady / Medium / Hardware / IllegalRequest
/ Other) with is_wedge_family predicate.
handle_read_error WARN log now carries:
consecutive_failures
consecutive_outer_failures
ms_since_last_error NEW gap between this and previous error
ms_since_last_success NEW gap to last good read
total_errors NEW aggregate this pass
total_reads_ok NEW
wedge_count
sense_family NEW typed category, easier to filter
sense_key / asc / ascq (existing)
NEW WARN log "wedge_transition" fires once when the sense family
changes from non-wedge to wedge (Medium to Hardware/IllegalRequest).
That's the moment the drive's firmware flipped into fast-fail
mode. Single timestamped event in the log so post-mortems can
pinpoint the transition without scanning thousands of TRACE lines.
Worked example: if the next wedge incident shows
read_error ms_since_last_success=18234 ms_since_last_error=null
read_error ms_since_last_success=28000 ms_since_last_error=10000
read_error ms_since_last_success=43000 ms_since_last_error=68
(drive returned <100ms = wedge symptom)
wedge_transition errors_in_zone=5 ms_since_last_success=43000
we can immediately tell cumulative damage, 5 errors over 43 s,
drive went into fast-fail mode at the 5th. If instead we see
read_error ms_since_last_success=200 ms_since_last_error=null sense_family=Hardware
wedge_transition errors_in_zone=1
the wedge was triggered by ONE read at a physically-bricked LBA
(immediate fast-fail, no warm-up).
These two patterns demand different tuning responses (longer
pause vs. larger initial jump), and now we can distinguish them
from a single WARN log line each instead of needing TRACE
verbose for the whole rip.
Plus jumps_taken / zones_entered counters that feed an end-of-pass
INFO summary (PassSummary). Caller invokes pass_summary at sweep
end and logs structured stats: "Pass 1 saw N errors / M ok reads
/ K zones / J jumps". Single-line post-mortem for any rip.
No caller signature change (timing is internal to the handler;
end-of-pass summary is a new method callers opt into). Precommit
green; 433+ tests pass. Staged for the 0.18.10 release once we
have user-validation data on 0.18.9's avoidance tuning.
Complements the wedge-skip backstop (fbdb50c) with proactive
avoidance so we don't HIT the wedge in the first place. User's
take after seeing the Dune Pt 2 rip wedge at 48%: 'we shouldn't be
wedging.'
Empirical observations from the 23:09:12-23:09:55 wedge timeline:
5 read errors over 43 s, ~8 s apart (drive's own ECC recovery
takes 5-10 s per failure). Not 'hammering' in any usual sense,
but cumulative firmware-state buildup over 5 in-cluster errors
was enough to tip the BU40N into wedge mode at the 5th error.
Damage cluster spanned ~140 MB (LBAs 19.898M-19.965M). Current
damage-jump base of 256 sectors × batch=32 = 16 MB first jump,
doubling to 32 MB, 64 MB... Each jump landed BACK INSIDE the
140 MB cluster, exposing the drive to MORE in-cluster errors.
Two avoidance levers:
1. Inter-error pause on Pass 1 (PASS_1_FAIL_PAUSE_SECS = 5 s):
pre-fix Pass 1 ran pause_secs=0 on all errors to 'zoom past'
damage zones. Successful reads still zoom at zero pause — the
pause applies only to FAILED reads, giving the drive's firmware
cool-down between cluster exposures. Cost: ~5 s per scattered
failure (~30-60 s total on a damage cluster); trivial vs.
crashing the rip at 48%.
2. Larger damage-jump base (JUMP_BASE_SECTORS = 1024, up from
256): first jump at batch=32 now covers 64 MB instead of 16 MB,
second jump 128 MB instead of 32 MB. Two jumps clear 192 MB —
well past most single-cluster damage patterns. Smaller jumps
were landing inside the cluster and adding to the wedge counter.
Plus a halt-aware sleep helper (sleep_secs_or_halt) so the new
inter-error pause doesn't degrade halt response time. Halt poll
granularity 100 ms — halt fires within ~100 ms regardless of
remaining pause time. Updated three sleep call sites in disc/mod.rs
(SkipBlock pause, JumpAhead post-pause, Retry pause).
The wedge-SKIP backstop (fbdb50c) stays — combined with this
avoidance work, the flow becomes:
damage cluster encountered →
pause 5 s, mark NonTrimmed →
second failure →
pause 5 s, mark NonTrimmed →
...
threshold hit →
damage-jump 64 MB (clears 95% of clusters) →
if jump lands in another cluster: 128 MB next jump →
only if drive STILL wedges after all this:
wedge-skip kicks in (1 GB jump + 30 s cooldown × 16 budget)
Tests:
pass_1_pauses_briefly_on_skip_for_wedge_avoidance — locks the
new 5 s pause behavior in place (replaces the old pause=0 test).
integration test threshold bumped from 5 s to 60 s with comment
explaining the new bound is 'not infinite' rather than
'milliseconds-fast'.
All 433+ tests green on cargo +1.86 fmt + clippy + test.
Precommit green.
Closes the final two audit items from this session.
labels::apply_labels: factored out of apply() so the matching logic
is unit-testable without needing a SectorReader / UdfFs. 11 new
tests in apply_tests cover:
- codec_hint + variant flow through to AudioStream.label
- purpose set on audio with no label English text
- name fallback only when purpose=Normal (CLI owns purpose i18n)
- subtitle SDH qualifier set; forced flag flipped on Forced
- per-type 1-based indexing (audio #2 maps to 2nd audio stream,
not 2nd stream overall)
- labels for nonexistent streams are no-ops
- empty labels list leaves streams untouched
- fill_defaults generates audio + video labels; preserves existing
class_reader: robustness smoke tests. ClassFile::parse must NEVER
panic on adversarial input — only return Err. 9 new tests:
- empty input
- short magic (0..4 bytes)
- wrong magic
- truncated after magic
- bad CP tag
- truncated UTF-8 in CP
- 200 random byte buffers (deterministic xorshift)
- 100 magic + random tail (most adversarial — magic check passes,
everything else garbage)
- instructions iter on random code (200 buffers)
- instruction_size on every opcode 0..255 with varied tail buffers
- modified_utf8 on random byte buffers (500)
The xorshift PRNG keeps the tests deterministic (no rand dep) and
reproducible — failures will be the same buffer every time. This is
the lightweight alternative to a cargo-fuzz setup; if/when we adopt
cargo-fuzz, these tests stay as regression cases.
All 451 tests passing on cargo +1.86 fmt + clippy + test.
Pre-fix: when the drive returned HARDWARE_ERROR or ILLEGAL_REQUEST
during Pass 1 sweep, libfreemkv immediately returned ReadAction::
AbortPass. Autorip surfaced this as a fatal error and stopped the
rip at whatever progress percentage Pass 1 had reached — typically
40-50%. On a disc with one physical-damage cluster, the user would
see Pass 1 die at ~48% with the cryptic message 'E6000: <lba>
0x02/0x04/0x3e' and have no rip output to work with.
Root cause analysis: BU40N firmware transitions into a fast-fail
state when it hits cumulative read failures in a small LBA range —
returns HARDWARE_ERROR for every subsequent read near that LBA, even
sectors that aren't physically damaged. Per project docs 'Bad-sector
handling' rule #2, 'Recovery requires eject+reload OR significant
cool-down.' Aborting on first wedge throws away the rest of the
disc; the right response is to SKIP the wedged region (mark as
NonTrimmed for Pass N), pause for drive cooldown, and continue.
Fix: in handle_read_error, the HARDWARE_ERROR / ILLEGAL_REQUEST arm
now branches on bisect_on_marginal:
Pass 1 (bisect_on_marginal=false): JumpAhead with WEDGE_JUMP_SECTORS
(1 GB at 2048 bytes/sector) and WEDGE_PAUSE_SECS (30 s cooldown).
Tracks wedge_count in ReadCtx; resets on any successful read.
Truly aborts only after WEDGE_ABORT_THRESHOLD (16) consecutive
wedges with no good read in between — generous enough to clear
most physical-damage clusters, bounded enough to not loop forever
on a permanently bricked drive.
Pass N (bisect_on_marginal=true): unchanged AbortPass. Pass N's
job is single-sector recovery; if the drive won't talk near a
specific LBA, skipping doesn't help. Pass N exits and lets the
outer layer decide retry/eject/surface.
5 unit tests cover the new policy:
pass_1_hardware_error_jumps_ahead_not_aborts — JumpAhead emitted
with correct sectors+pause, wedge_count incremented.
pass_1_hardware_error_aborts_after_threshold — AbortPass kicks in
on the WEDGE_ABORT_THRESHOLD-th consecutive wedge.
pass_1_good_read_resets_wedge_count — on_success clears
wedge_count; subsequent wedge gets fresh skip budget.
pass_n_hardware_error_still_aborts — Pass N's AbortPass behavior
intact.
pass_1_illegal_request_also_routes_to_wedge_skip — both wedge
sense families get the skip treatment.
Impact: on the Dune Pt 2 disc that consistently wedged at 48%
(physical damage at LBA ~19.9M), Pass 1 will now jump ahead 1 GB
on the wedge, give the drive 30 s cooldown, and continue scanning
the rest of the disc. The damaged region becomes Pass N's job to
revisit. Worst case if the drive stays wedged: 16 GB of NonTrimmed
disc area before honest AbortPass.
Precommit (cargo +1.86 fmt + clippy + test) green; 430 passing.
Replaces two near-duplicate hand-rolled XML scrapers in paramount.rs
and criterion.rs with a single labels::xml module that's robust to:
- Case-insensitive tag / attribute names ('<Playlist>' matches the
same as '<playlist>'; 'Name=...' matches 'name=...').
- XML namespace prefixes (matches '<ns:tag>' for tag='tag').
- Arbitrary whitespace inside open tags and around '=' separators
('<tag name = "X">' works).
- Both quote styles for attribute values (" and ').
- Self-closing tag forms ('<tag/>' and '<tag />').
- '>' chars inside quoted attribute values (no premature end-of-tag).
Three functions:
xml::attr(element, name) -> Option<String>
Extract attribute value from an open-tag fragment.
xml::text(xml, tag) -> Option<String>
Trimmed text content of first <tag>...</tag>.
xml::find_element(xml, tag, from) -> Option<(start, end)>
Locate next <tag>...</tag> for iteration; handles self-closing.
22 unit tests cover the robustness properties: case-insensitivity,
namespace stripping, whitespace tolerance, quote styles, self-close
forms, no-substring-false-positive (looking for 'lang' must NOT
match 'lang_id' or 'language'), '>' inside quoted attrs, iteration
across repeated elements.
paramount.rs: drops local extract_attr; find_feature_playlist now
walks xml::find_element('playlist', ...) so it works regardless of
case and self-closing style. Pre-refactor: required exactly
'<playlist ' (single space, exact case) and '/>' for self-close.
criterion.rs: drops local extract_tag; parse_stream_infos and
parse_playback_config iterate via xml::find_element. Same case-
sensitivity + namespace gains. The 'COMMENTARY' / 'SDH' / 'DS'
content-value match is now case-insensitive too (previously a disc
authored with 'commentary' would have been miscategorized as Normal).
Pre-refactor known failure modes (none observed yet, but trivial
to trip on a future disc): vendor switches whitespace around '=',
uses single quotes, capitalizes a tag, prefixes a namespace. All
now handled.
Out of scope by design: XML entity decoding (&, <), CDATA
sections, comments, processing instructions. None observed in BD-J
authored label data. If a future disc trips them, the entity
decoder is a localized addition.
Precommit (cargo +1.86 fmt + clippy + test) green.
Completes the Deluxe parser pipeline. Phase A (master enums) was
already shipping; this commit lands Phases B/C/D so the parser now
emits per-stream StreamLabel records on Deluxe-authored discs.
Phase B (decode_codec_enum): walks the codec enum's subclass
references (one .class per codec ordinal) and extracts the codec
name string from each subclass's constant pool. Heuristic: pick the
first Utf8 entry that's uppercase + underscored + >=4 chars, or one
of the known codec roots (ATMOS/DOLBY/DTS/TRUEHD/MLP/AC3/EAC3/PCM)
when no underscored candidate is found. CodecTable maps ordinal ->
codec string; empty string for ordinals where extraction failed
(logged via tracing, not fatal).
Phase C (find_binding_class): identifies the class that builds the
per-stream label table by counting getstatic operations targeting
any of the master enum classes from Phase A. Class with the highest
count >= 4 wins. Threshold is empirical (real binding classes have
50+ matches; floor of 4 admits small discs while rejecting incidental
single-reference classes).
Phase D (decode_binding + BindingDecoder): symbolic stack machine
that walks the binding class's <clinit> bytecode. Handles:
- constant pushes: iconst_<n>/bipush/sipush/ldc(Integer)
- new <X>: pushes uninit-object marker
- dup: stack copy
- getstatic <Y.Z>: pushes EnumRef when Y is in MasterEnumTable,
else Unknown
- invokespecial X.<init>(...)V: pops args per descriptor; when the
receiver is NewObj(X), emits a Construction { binding_type: X,
args: [...] }
- invokevirtual/invokestatic/invokeinterface: pop args per
descriptor, push return placeholder unless void
- pop/pop2/aastore/putstatic/putfield: standard stack effects
- branches/returns: clear stack (conservative resync — binding
<clinit> is straight-line in practice)
parse_method_arg_count: JVMS field-descriptor parser, handles
primitives, references (L...;), arrays ([...).
interpret_streams: converts Constructions to StreamLabels using
the master enum table + CodecTable. Each construction with a
Language ref becomes a stream. Audio when codec_hint resolves via
binding_type substring match against CodecTable; subtitle otherwise.
Purpose ordinal -> LabelPurpose via the verified Deluxe Purpose enum
order (Normal/Commentary/PiP/Trivia/Descriptive/Score/NoForced/
NoForcedDescriptive). Stream index = sequential per type. Language
goes through vocab::lang for ISO code + variant.
deluxe::parse now returns Some(ParseResult::medium(labels)) when
all four phases produce labels. Medium confidence — the bytecode
mechanism is rigorously tested but the signal-to-StreamLabel
mapping (which arg is which, audio vs subtitle classification) is
heuristic until corpus binding-class bytecode confirms the exact
pattern.
Test coverage: 13 new unit tests in deluxe.rs
parse_method_arg_count: 3 tests (basic types, references, malformed)
BindingDecoder: 4 tests (simple construction, with int pushes,
skips unmatched invokespecial, resolves master-enum ordinal)
interpret_streams: 4 tests (subtitle on no codec, audio on codec
match, purpose routing, skips no-language)
MasterEnumTable: 3 tests (resolve, value, class_name_set)
extract_codec_name: 1 test (uppercase+underscore matching)
class_reader.rs gained a #[cfg(test)] ConstantPool::from_entries
test-only constructor so Phase D tests can build synthetic CP
fixtures without writing raw .class bytes.
Precommit (cargo +1.86 fmt + clippy + test) green.
Replaces 'first-match-wins by array order' with 'highest-confidence-
wins, array order tiebreaker'. Removes the arbitrariness when more
than one parser can claim a disc (e.g. one with both
bluray_project.bin and playlists.xml).
New types in labels::mod:
pub enum Confidence { Medium, High }
pub struct ParseResult { labels: Vec<StreamLabel>, confidence }
ParseResult::high(labels) / ::medium(labels) constructors
Parser signature change: every parse() now returns
Option<ParseResult> instead of Option<Vec<StreamLabel>>. Updated all
six parsers in lockstep:
paramount: High (fully structured XML)
criterion: High (fully structured XML)
pixelogic: High by default, Medium when an unknown token component
is encountered (the skip-unknown path now propagates the
coverage gap to the caller instead of silently degrading)
ctrm: High (structured key-value)
dbp: High (anchor scan with vocab routing)
deluxe: still returns None pending Phase D — signature aligned
Registry behavior:
extract() iterates all detect-positive parsers, picks highest
Confidence with non-empty labels. Equal confidence falls to array
order (deterministic). Same selection logic in analyze().
LabelAnalysis grew a confidence: Option<Confidence> field so the
diagnostic surface (freemkv-tools labels-analyze) exposes which
confidence tier the selected parser claimed. labels-analyze JSON
and labels-corpus-check structural diff both gained the field.
Precommit (cargo +1.86 fmt + clippy + test) green.
Three targeted fixes from a second-pass audit of the labels module.
1. vocab::lang now returns Option<LangInfo> with both code AND a
human-readable variant string. Pre-fix: 'Brazilian Portuguese 5.1'
became language=por, variant='', dropping the dialect info the
disc had explicitly authored. Post-fix: language=por,
variant='Brazilian' — matches the convention pixelogic / ctrm /
criterion already use for their region variants. dbp now
populates StreamLabel::variant from this. Compound table grew a
3-tuple (needle, code, variant); bare matches still return
variant=''.
2. dbp and deluxe had duplicated detect() boilerplate (any top-level
.jar in /BDMV/JAR/). Both now call jar::has_any_top_level_jar.
The trait-level detect contract — see super::PARSERS — can't peek
inside a jar without a SectorReader, so loose-detect-plus-real-
check-in-parse is the unavoidable pattern for jar-content parsers.
Consolidating in jar.rs at least makes the duplication visible.
3. mod.rs comment about parser ordering said 'dbp last'; deluxe is
actually now last. Updated to explain the dbp-before-deluxe order
is by cost (cp-iteration cheaper than bytecode walking when Phase
D lands).
Plus a registry-level lock test in mod.rs::registry_tests — asserts
the PARSERS array order is exactly [paramount, criterion, pixelogic,
ctrm, dbp, deluxe]. This was previously implicit; if someone reorders
the array (which changes which parser wins on overlapping signals),
unit tests would have stayed green. Now they fail with an explanatory
message about why the order matters.
Audit findings deferred to follow-ups (each its own commit + design
discussion):
- Stronger detect contract — current loose-detect-real-check pattern
is forced by SectorReader-not-in-detect-signature; could be fixed
by changing the trait to take an Option<&mut dyn SectorReader> or
similar.
- Per-parser confidence scoring — registry currently first-match-wins.
A high-confidence parser ought to beat a low-confidence one
regardless of array order.
- class_reader fuzzing — handles malformed input via Result but no
adversarial corpus yet.
Precommit (cargo +1.86 fmt + clippy + test) green.
MattJackson