When finish_with_halt's grace period expires it detaches from the
consumer thread and returns an error to the caller, but the leaked
consumer kept running to completion: once its wedged write syscall
returned it would fall through to sink.close(). For the mux writer
close() finalises the MKV (Cues block + segment-header patch), so a
leaked consumer could finalise — and keep writing to — an output file
the caller had already reported as failed, racing a fresh rip for the
same device over the same path.
Add a shared abandonment flag the consumer polls in its drain loop and
again before close(). finish_with_grace sets it before dropping the
JoinHandle, so the moment the wedged syscall returns the consumer skips
any further apply and skips close() entirely, then exits. This does not
interrupt the in-flight syscall (only its return or process exit can),
but it bounds the damage to the write already in flight instead of a
full finalise of an abandoned file.
Regression tests cover both sides: a consumer leaked past the grace
period skips close(), while one that finishes inside the grace window
still calls close() and finalises normally.
Test-hardening release, no runtime changes. Adds spec-grounded unit tests
across the silent-corruption surfaces — UDF/MPLS/CLPI/IFO parsing, BD/DVD
title + extent assembly, AACS/CSS key handling, TS/PS demux + codec parsers,
MKV/EBML container output, the mux pipeline, sector prefetch + decrypt
decorator, drive/SCSI sense decoding, label extraction, and core I/O. Each
test is grounded in the format spec or real on-disc behavior and verified to
fail under a targeted source mutation. No behavior changed.
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
Introduces the freemkv mux throughput highway: a three-stage thread
pipeline that replaces the inline single-thread read path for any
file-backed source (ISO and m2ts file URLs both route through it).
Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher)
Thread B: M2TS demux (DemuxThread)
Thread C: codec parse (PipelinedPesStream, on caller thread)
Each handoff uses a bounded crossbeam channel with a recycled buffer
pool — no allocations or memcpys in the steady-state hot loop.
Component map:
* io/byte_prefetcher.rs (new) — std::io::Read producer thread with
recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector
side) so demux_thread::spawn_zero_copy can wire either upstream.
* sector/prefetched.rs — recycled buffer pool added; into_channels()
peels off the rx/recycle_tx/shell triple for zero-copy demux.
* mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs
feed() on its thread, ships Vec<PesPacket> batches.
* mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls
packets from the demux thread and runs codec parse on the caller.
* mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline
(private) assemble the three stages; iso:// and m2ts:// both
return PipelinedPesStream.
* mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted;
the read direction lives on the highway now).
* mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find.
* mux/codec/hevc.rs — tightened frame_data initial capacity.
* mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB
remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page
first-touch fault tax that the previous 256 KiB pre-alloc paid on
every PES boundary.
* mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as
the legacy autorip ingress (drive + multipass paths still need
on_event / skip_errors before they migrate to the highway).
* io/file_sector_source/* — per-OS prefetch() syscall hook
(Linux readahead, macOS F_RDADVISE, Windows/other no-op).
* decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS;
pool sized to all cores by default.
Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://):
60 → 322 MB/s warm cache (old new_pipeline path)
60 → 660 MB/s warm cache (highway path, this commit)
60 → 126 MB/s sustained disk-bound
The IsoSectorReader baseline reader was deleted in favour of
FileSectorSource so the freemkv CLI and autorip exercise the same
read path.
iter13 strace finding: producer thread (mux ISO reader) spends 80%
of wall-clock in state D (NFS RPC wait), doing 207 preads/sec at
103 us each. Pipe is 78 MB/s isolated read; we use ~25 MB/s.
FileSectorSource was using direct per-sector pread (0.21.3 bypass).
That bypass was justified under Phase 2.5 + 0.21.7 producer-poll cap
("32 MiB refill bursts the TCP connection enough to starve the
writer thread"). Both of those conditions are gone now (iter8
baseline: no Phase 2.5, no producer poll cap).
The 4 MiB buffer infrastructure was preserved with #[allow(dead_code)]
in case re-engagement was ever wanted. iter14 just routes the hot
path through buffer_covers + refill + memcpy. Net diff: ~10 LOC of
business logic, 5 unit tests already cover the contract.
Expected: producer's effective read rate jumps from 25 → 60+ MB/s
(matches dd ceiling for 1+ MiB block reads). If consumer side keeps
up, mean mux climbs from 28.7 → 35-50 MB/s. If consumer is now the
cap, we see a clear ceiling around 30-35 and we know where to look
next.
Critically: this change ONLY affects mux-from-ISO. Disc→ISO sweep,
Pass N bad-sector retry, AACS, drive ops, mapfile, recovery — all
untouched (they use DriveSectorSource which is a separate impl).
iter12 revert restored chunk to 128 (iter11 value) by way of the
revert chain. Setting it back explicitly to 32 for iter13. We're
back to iter8 baseline config (no Phase 2.5, 32 MiB chunks,
DONTNEED, no app-level FileSectorSource buffer, channel depth 32).
iter13 is instrumentation: strace the mux thread during a known-dip
to see where time is actually going.
iter11 (128 MiB) crashed to 16.6 mean — chunk-size sweet spot is
firmly 32 MiB. Locking in.
iter12 hypothesis: each NFS write past the file's EOF triggers a
server-side SETATTR (metadata commit) to update the file length.
With ~62 GiB of MKV output and writes at chunk boundaries, that's
thousands of SETATTRs per rip. By using plain fallocate (mode 0)
the file is pre-extended to size_bytes up front; subsequent writes
overwrite the pre-extended region with no metadata ops.
Adds high_water tracking + truncate_at_sync so we ftruncate down
to actual content size if the size hint was an overestimate.
iter9 (64 MiB on top of no-P2.5) regressed -1.2 from iter8. 32 MiB
remains best on the no-P2.5 path. Net iter10 = iter8 minus DONTNEED
(via the prior commit re-applying f4a881b).
iter8 (Phase 2.5 disabled, 32 MiB chunks) hit 28.7 MB/s mean
— 1.3 MB/s below R2 floor. Without Phase 2.5, WAIT_AFTER blocks the
mux thread directly once per chunk. Halving the WAIT_AFTER frequency
(32 → 64 MiB chunks) should raise the mean without changing the
underlying architecture.
iter7's 64 MiB chunks didn't help (23.1 vs iter4's 24.0). Reverting
to 32 MiB so iter8 differs from iter4 by exactly one variable:
Phase 2.5 disabled / direct passthrough mux thread.
Test question: is Phase 2.5 helping at all? If iter8 mean > iter4,
Phase 2.5 has been a net negative on this rig the whole time. If
iter8 mean < iter4, Phase 2.5 is doing what it claimed.
iter6 (8 MiB chunks) regressed mean -8.2 MB/s vs iter4 (32 MiB).
Trend: bigger = better in this workload, against the page-age
theory. Try 64 MiB to see if fewer/larger syncs raise mean further.
0.21.14 went to 128 MiB and was reverted; 64 is between.
iter5 (256-frame channel) regressed -1.8 MB/s vs iter4. Reverting to
32 frames.
iter4 sample pattern shows clear ~30 s oscillation (peak 45 MB/s →
dip 3 MB/s → recovery). Matches Linux vm.dirty_expire_centisecs
default (30 s). 32 MiB chunks at 25 MB/s issue WAIT_AFTER every
~1.3 s, which can't outrun the kernel's own page-age limit, so
pages buildup then flush in bursts. Smaller 8 MiB chunks
(WAIT_AFTER every ~0.33 s) should keep the dirty-page set young
and eliminate the periodic flush-burst dip.
iter4 measurement (Phase 2.5 + DONTNEED, no FileSectorSource buffer)
showed mean 24.0 MB/s but with dips to 2.8 MB/s. Channel at 32 frames
(~1.6 MiB at 50 KB/frame avg) drains in <100 ms on any producer pause,
starving the consumer.
256 frames = ~12 MiB ≈ 3-4 sec of consumer drain at the rolling-avg
floor we want to hit. Should coast through producer micro-stalls
without idle gaps.
iter1 baseline (Phase 2.5 + DONTNEED restored) measured at 18.4 MB/s
mean on Civil War remux — well below the rig's 37 MB/s concurrent-r+w
ceiling. Per-sector pread is the producer-side bottleneck: ~50us per
pread on NFS = ~19k preads/sec = effective ceiling near what we see.
The 0.21.3 bypass commit cited an A/B test showing the buffer hurt
throughput on NFS bidirectional workloads. That test was taken under
the 0.21.7 producer polling cap; once the cap is gone, the cap was
the bottleneck, not the buffer. Same invalidation pattern as the
0.21.5 Phase 2.5 revert.
Restored the buffered path. DONTNEED page-cache eviction from 0.21.6
stays intact.
Three coupled changes that together meet the "stable max throughput
on any medium" bar:
1. Remove `is_nfs` from `skip_wait`. WAIT_AFTER runs on every medium
now. The `bounded_syscall` 30s safety net (already in place) covers
the wedged-FS case generically — no need to predict NFS-hangs at
compile time. Re-applies the 0.21.12 fix that landed the flat
25 MB/s on NFS in the first place.
2. Drop the `posix_fadvise(DONTNEED)` calls after WAIT_AFTER. Bounded
*dirty* pages (the actual invariant) does not require evicting
*clean* pages. DONTNEED was forcing read-modify-write on any
in-window seek-then-write — exactly the pattern matroska
cluster-size backpatches produce. Empirical 2026-05-15: write side
of mux fed NFS at 43 MB/s while the file grew at 25 MB/s, an 18
MB/s overhead almost certainly composed of those RMW cycles. The
kernel reclaims clean pages under LRU when memory is actually
needed — we don't have to ask.
3. Replace `detect_nfs(fd) -> bool` with `detect_storage_class(fd)
-> (StorageClass, chunk_bytes_seed)`. Medium detection happens
*once* at construction and produces *one* output: the initial
`chunk_bytes` seed for the autotuner. NFS gets 64 MiB (commit ack
~10-30 ms; needs bigger chunks to amortize); other media use the
caller's hint. No hot-path branches on medium. The autotuner drives
all subsequent decisions from measured WAIT_AFTER p95 latency,
identically on every (OS, FS) combination.
Module-level doc rewritten to match: no more "NFS escape hatch", no
"is_nfs" framing. The whole writeback module is now medium-agnostic
except for one labelled detection point.
The pipeline's adaptive autotuner grows chunk_bytes only when the p95
WAIT_AFTER latency exceeds 200 ms. On NFS, sync_file_range(WAIT_AFTER)
translates to an NFS COMMIT RPC whose ack lands within ~10 ms — so
the autotuner never triggered and the pipeline stayed at the original
32 MiB initial value forever.
That capped sustained mux throughput by paying NFS COMMIT-RPC overhead
roughly once per second of writes. Bidirectional mountstats on rip1
2026-05-15: write side 29 MB/s (RTT 30 ms but exec_time 257 ms — 220 ms
queue/serial waiting) while concurrent dd on the same disk shows
~91 MB/s write + ~65 MB/s read available.
128 MiB initial drops COMMIT cadence 4x while keeping the bounded-
cache invariant intact (worst-case dirty pages ~2 x chunk = 256 MiB,
well under vm.dirty_ratio = 6.6 GB on the 32 GB rig). The adaptive
autotuner can still grow further (up to 256 MiB) or shrink if
WAIT_AFTER ever measures sub-20 ms p95 on faster media.
The WritebackPipeline's WAIT_AFTER + posix_fadvise(DONTNEED) dance
keeps dirty pages bounded at ~2 × chunk_bytes by waiting for each
chunk's writeback to commit before issuing the DONTNEED hint to drop
it from cache. The original 0.18-era design unconditionally skipped
this on NFS on the premise that "NFS clients have their own buffering
and commit semantics that handle dirty-page bounds without us forcing
the issue."
Empirically wrong. On unraid-1 NFS the kernel client buffers dirty
pages up to vm.dirty_ratio (default 20% of RAM = ~6.6 GB on the rip1
host) before the kernel forces writeback and throttles app writes.
Result on 0.21.11 mux measured 2026-05-15: mux throughput cycled
between ~45 MB/s (cache absorbing) and ~7 MB/s (cache draining under
throttle) on a ~100 s period — exactly the burst-flush pathology this
pipeline was built to fix, but disabled on the medium it actually
runs on. /proc/meminfo Dirty: column climbed lockstep with mux
write rate during the slow half of every cycle, confirming the cause.
The original safety concern — `sync_file_range(WAIT_AFTER)` hanging
indefinitely on a wedged NFS server — is already handled by
`wait_after_with_timeout`'s `bounded_syscall` wrapper (30 s deadline).
If a real WAIT_AFTER call exceeds the deadline the pipeline flips to
the `degraded` state and skips WAIT_AFTER + DONTNEED for the rest of
its life — same effect as the old NFS branch, but only triggered when
something is genuinely broken rather than as a blanket exception.
This change is medium-agnostic: every medium goes through the same
path now, every medium gets the same safety net, and the
ADAPTIVE_WINDOW chunk-size autotuner (lines 226-255 — measures p95 of
WAIT_AFTER and resizes between 4 MiB and 256 MiB) finally activates
on NFS where previously it was dead code. Slow medium auto-grows
chunks to amortise per-chunk overhead; fast medium auto-shrinks to
keep cache pressure tight; nothing in the code special-cases the
filesystem type.
`is_nfs` is still detected (for logging + observability) but no
longer keys `skip_wait`. Module doc + startup log line updated to
match.
Pre-coalescing the Phase-2.5 writer thread issued one `file.write_all`
syscall per `Cmd::Write` dequeued. The mux side calls
`WritebackFile::write_all(buf)` per PES frame, typically 30-200 KB.
On NFS that translates to one RPC per syscall, capping per-thread
throughput at `(wsize / rtt) × inflight` — well below what the same
disk delivers under a 1 MiB `dd oflag=direct` workload (empirical
2026-05-15: dd 71 MB/s vs mux ~25 MB/s sustained, with instantaneous
samples bursting 7→108 MB/s as the kernel page cache filled and
drained on its own cadence).
Coalesce instead: dequeue drains consecutive `Cmd::Write` items off
the ring up to a 1 MiB byte budget, returns them as
`DequeuedWork::Writes(Vec<Vec<u8>>)`, and the run loop concatenates
into one contiguous buffer and issues a single `file.write_all`.
Non-write commands (Seek, Flush, SyncAll, Finish) break the run and
are returned one at a time as `DequeuedWork::Other`, preserving their
ordering relative to the writes.
Single-buffer fast path avoids the concat allocation when only one
write is in the queue at dequeue time. A single oversize write (e.g.
the rare matroska cluster larger than 1 MiB) is admitted alone so it
still makes progress — the kernel splits internally.
This is generic across mediums: bigger app writes are at-least-as-
good on local SSD, HDD, or NFS. On fast storage the ring rarely fills
so coalescing is mostly a no-op; on slow storage with significant
per-RPC overhead it materially improves throughput.
Tests in `tests/` (write_then_drop_persists_bytes, sync_all_drains,
seek_then_patch_roundtrip, flush_is_observed_in_order) still pass —
ordering and durability semantics are unchanged.
The depth was conservative because pre-0.21.8 a full sync_file_range
stall on NFS could traverse this channel and pin the producer. With
0.21.8's restored Phase 2.5 writer thread + 128 MiB byte-bounded ring
inside WritebackFile, every blocking syscall happens downstream of
this channel — never on it. The original "smaller buffer reduces
backpressure risk" rationale no longer applies.
Empirical (2026-05-15 Civil War UHD remux on 0.21.8): 30.9 MB/s
sustained but instantaneous samples spanning 9-53 MB/s, stdev 9.3.
Distribution clusters 52% of samples in 25-35 MB/s but has a long
9-15 MB/s tail. The tail corresponds to brief stalls in the matroska
builder when the sink momentarily lags — exactly the case a deeper
inter-thread channel covers. 32 frames at PES-frame sizes is still
well under a megabyte of additional memory, so the cost is zero.
Reverts 523af46. That revert was made on the premise that Phase 2.5
caused a ~60% mux throughput regression on NFS bidirectional workloads.
The premise was wrong: at the time of measurement the producer was
capped at ~8 MB/s by a 50 ms thread::sleep poll in
Pipeline::send_with_halt (fixed in v0.21.7's io/pipeline change), so
the comparison was measuring the polling cap on both sides.
With the polling cap removed, direct passthrough exposes the kernel's
default dirty-page writeback pathology on NFS: writes accumulate, the
kernel periodically bursts a flush, app writes block for the burst.
Observed empirically on Civil War UHD remux 2026-05-14: 5-45 MB/s
spiking around a ~21 MB/s sustained mean, dominated by burst-flush
back-pressure cycles.
Phase 2.5 decouples the mux thread from the file syscall:
* mux writes complete instantly into a 128 MiB byte-bounded SPSC ring,
* a dedicated writer thread executes the real File writes, seeks,
and sync_file_range calls; can sit in a kernel burst without
blocking the mux pipeline,
* backpressure via Condvar notify/wait, no polling primitive,
* the ActiveClusterBuffer fast-path preserves the original MKV
cluster-backpatch optimisation so in-window seeks don't drain
the current writeback chunk.
Halt-safety is preserved: every blocking writeback syscall on the
writer thread still routes through bounded_syscall with a 60 s
deadline. A wedged NFS server cannot trap the writer indefinitely;
the muxer keeps queueing into the ring; the kernel page cache and
the ring together absorb the stall.
The halt-aware send loop polled try_send on a 50 ms sleep slice when
the channel was full. That capped producer throughput at 1 / 50 ms =
20 frames/sec ≈ 1 MB/s at typical PES frame sizes — way below NFS,
let alone local SSD/NVMe. Multi-day diagnostic 2026-05-13/14 surfaced
it as the root cause of the 18 → 3 MB/s mux throughput regression on
0.21.x.
Replaced std::sync::mpsc::sync_channel with crossbeam_channel::bounded
and switched send_with_halt to send_timeout. Producer now BLOCKS on
consumer drain (kernel-wakeup) instead of polling — the timeout slice
(250 ms) only fires when the producer is waiting for stop signal
observation, never on the happy path.
Result: no throughput cap from this primitive at any medium speed.
Mux is bounded by actual storage / network bandwidth, not by our
channel implementation.
Same fix needs to land in autorip's mux.rs producer-side
sync_channel (separate sibling commit).
Documented in (internal)/memory/
feedback_send_with_halt_poll_throttle.md.
Empirical: isolated NFS read 70 MB/s + write 93 MB/s on the rip1 setup
right now, but mux throughput pinned at 2.7 MB/s on 0.21.5. NOT
environmental — code regression.
Root cause: Phase 1 silently dropped the read-side
posix_fadvise(POSIX_FADV_DONTNEED) eviction that the pre-Phase-1 (0.20.7)
hot path had. Without it, an 85 GB streaming ISO read pins the entire
file in the kernel page cache, starving concurrent MKV writeback. 0.21.2
then also dropped the POSIX_FADV_SEQUENTIAL hint on the same theory,
compounding the regression.
Restored both, per-OS split:
- linux: posix_fadvise(SEQUENTIAL) at open + posix_fadvise(DONTNEED)
on consumed 32 MiB windows
- macos: F_RDADVISE hint at open (kept); drop_window no-op (macOS unified
buffer cache less prone to the pin pathology)
- windows / other: both no-op stubs
Target mux speed restored to 20+ MB/s (per concurrent-NFS math:
70/2 read × 0.73 MKV/ISO ratio ≈ 25 MB/s achievable).
The Phase 2.5 writer-thread + bounded ring + ActiveClusterBuffer
architecture introduced a ~60% mux throughput regression on NFS
bidirectional workloads (18 MB/s -> 7-8 MB/s in 0.21.x). Other
candidates (FileSectorSource readahead buffer, per-OS sink split)
ruled out empirically across 0.21.1-0.21.4.
Reverts WritebackFile's hot path to direct File passthrough matching
the 0.20.7 baseline:
* Write/write_all/flush call straight through to self.file.
* Seek calls through to self.file and notifies the pipeline.
* sync_all runs pipeline.finalize() then per-OS durable_sync.
Kept (intentional): the per-OS file split established in earlier work
(writeback_file/{linux,macos,windows,other}.rs). create_with_size_hint
still dispatches to platform::preallocate and sync_all still routes
through platform::durable_sync (bounded_syscall 60s deadline on
Linux/macOS), so halt-safety on a wedged NFS fsync is preserved.
Removed:
* Cmd / RingState / Shared / WriterState / writer_thread_main
* ActiveClusterBuffer (active-cluster window for in-window seeks)
* push_command / dequeue / publish_error / mark_writer_gone
* RING_CAPACITY_BYTES, ACTIVE_CLUSTER_WINDOW_BYTES,
MAX_WRITE_CHUNK_BYTES, WRITER_THREAD_NAME constants
* writer JoinHandle field + Drop join logic
* Tests that asserted internal writer-thread state
(backpressure_blocks_when_ring_full, in/out_of_window_seek_*,
active_cluster_buffer_*, writer_thread_panic_surfaces_on_drop)
Kept tests (correctness-of-output only):
* write_then_drop_persists_bytes
* sync_all_drains_and_flushes (renamed from sync_all_blocks_until_ring_drains)
* seek_then_patch_roundtrip (replaces in_window_seek_then_patch_roundtrip)
* flush_is_observed_in_order
mod.rs: 1026 -> 303 lines. linux/macos/windows/other.rs untouched.
The three tests (multi_sector_read_spanning_buffer_boundary,
backward_seek_rebuffers, partial_buffer_at_eof) were asserting
internal buf_start_lba / buf_len_sectors state. With 0.21.3's
read-path bypass, those fields are no longer mutated. The
byte-level contract assertions (read returns correct bytes for
every scenario the tests cover) remain intact.
The 32 MiB readahead window (0.21.0–0.21.1) regressed mux throughput
on NFS bidirectional workloads vs the pre-Phase-1 0.20.7 baseline
(18 -> 7-8 MB/s). The 0.21.2 4 MiB shrink made it worse (5-6 MB/s).
Both signs point at the application-level buffer itself, not the size.
This commit bypasses the buffer entirely on the read path — every
read_sectors call seeks and pread()s direct to the file. That matches
0.20.7's hot path. Kernel readahead handles the policy; on NFS that
interleaves naturally with concurrent writes on the same TCP
connection.
Buffer state fields and refill/buffer_covers are kept so the
structure is preserved for a future per-source-type policy (e.g. a
local-disk source where batched reads ARE beneficial), and so the
existing tests still exercise that machinery.
Empirical regression observed during 0.21.1 mux test on rip1/unraid-1:
historical 0.20.7 baseline averaged ~18 MB/s mux throughput; 0.21.1
dropped to ~7-8 MB/s flat. Same NFS source + destination, same disc.
Suspect: 32 MiB FileSectorSource readahead + posix_fadvise(SEQUENTIAL)
together saturate the TCP connection on read bursts, starving the
writer thread's concurrent NFS writes (mux reads the source ISO and
writes the MKV over the same connection).
- READAHEAD_BUF_BYTES: 32 MiB -> 4 MiB. Matches NFS rsize=1 MiB * 4
round-trips per refill, interleaves cleanly with writes.
- linux/hint_sequential: now no-op. Kernel's default ~128 KiB
readahead is what we want on NFS-backed ISOs (the dominant case).
Per-OS file stays so we can re-enable a hint cleanly later if a
different path benefits.
CI's lint workflow runs clippy on linux target where:
- platform/fs_type/linux.rs and io/writeback/linux.rs: the i64 cast
on buf.f_type / NFS_SUPER_MAGIC is unnecessary on glibc x86_64 (both
already i64) but required on musl (c_ulong); silence the lint via
inline allow with explanatory comment.
- mux/m2ts_mux/packet.rs: doc comment continuation across lines was
parsed as an unindented list item. Reworded to a single flowing
sentence.
SocketSink + UdpSocketSink (`src/io/sink/socket.rs`) — sequential-only
TCP/UDP write destinations. SocketSink wraps BufWriter<TcpStream> with
1 MiB capacity, tunes SO_SNDBUF on construction, calls shutdown(Write)
on finish(). UdpSocketSink emits one datagram per write — caller
packetizes. Both impl Write+Send and thus satisfy SequentialSink via
the Phase 2 blanket; neither impls Seek, so RandomAccessSink is
correctly inaccessible (compile error to mux MKV onto a socket).
New sequential container muxers in src/mux/:
- hevc/ — raw HEVC Annex B elementary stream. Length-prefixed NALU
→ 00 00 00 01 NALU. hvcC parsing emits VPS/SPS/PPS once at stream
head. Fully ships.
- m2ts_mux/ — standard MPEG-TS (188-byte packets). Single program,
HEVC video on PID 0x100, optional AC3/TrueHD audio on PID 0x101.
PAT+PMT re-emitted every 250 packets; PCR stamped on video every
40 packets. Hand-rolled, no new deps. Distinct from the existing
BD-TS (192-byte) `mux::m2ts::M2tsStream` — that path stays as-is.
- fmp4/ — fragmented MP4. STUB: ftyp + minimal moov skeleton with
one HEVC video trak + mvex/trex. Media fragments (moof+mdat) are
TODO for v0.22.0 — write_video accumulates frames into a pending
buffer that finish() clears. Init segment is well-formed enough
that init_segment_starts_with_ftyp_then_moov asserts the box
chain.
17 new unit tests added (socket round-trip, HEVC Annex B conversion,
M2TS packet alignment + PAT/PMT cadence + per-PID CC, fMP4 box chain).
All 514 lib tests + 17 new = pass on Rust 1.86 (fmt + clippy + test
via (internal)/scripts/precommit.sh libfreemkv).
No new dependencies. No version bump. Don't-touch list clean.
WritebackFile now offloads all File I/O to a dedicated writer thread.
Muxer's Write/Seek/sync_all push into a bounded byte ring; the writer
drains the ring and runs the syscalls.
- Commands: Write(Vec<u8>), Seek(SeekFrom), SyncAll(oneshot),
Finish(oneshot). One ring carries all four; ordering preserved.
- ActiveClusterBuffer mirrors the last ACTIVE_CLUSTER_WINDOW_BYTES
written. Seek-back within window → in-memory patch + re-emit
(no forced drain). Seek-back outside window → real drain+seek.
Wins for MKV: cluster size patches almost always land inside the
active window; only the end-of-mux Cues + segment header backpatch
fall outside.
- Ring capacity RING_CAPACITY_BYTES = 128 MiB. Backpressure on full
blocks the muxer (correct semantics for archival workflows).
- All syscalls in writer thread wrapped in bounded_syscall(60s) so
a wedged NFS doesn't pin the thread forever.
- Drop drains via Finish + joins the writer thread; sync_all blocks
until ring drained AND underlying fsync completes.
Introduces the SequentialSink / RandomAccessSink trait pair under
io::sink and an open_for_mkv dispatch helper that picks WritebackFile
on Linux+NFS and LocalFileSink everywhere else. LocalFileSink wraps
BufWriter<File> with a 4 MiB buffer and exposes a per-OS preallocate
path (fallocate on Linux, F_PREALLOCATE on macOS, no-op fallback).
Adds platform::fs_type::detect with a per-OS split (statfs on Linux /
macOS, UNC heuristic on Windows, Unknown elsewhere) so construction-
site dispatch has a single primitive to call.
Blanket impls cover the common shapes: any Write+Send is a
SequentialSink, and any SequentialSink+Seek is a RandomAccessSink.
WritebackFile satisfies the random-access trait via the blanket impl
without needing an explicit per-type impl. No callers wired yet — the
mux::resolve construction sites stay on WritebackFile pending Phase 3.
Tests: 5 new sink/preallocate tests + 3 fs_type tests (1 ignored,
needs a real NFS mount). cargo +1.86 fmt + clippy + tests all green.
Matthew Jackson