Audit finding: both seam gates in MkvMuxer::finish were dead code under test.
set_clips was never called anywhere in the suite, so continuity.dropped_total()
was always zero and neither branch could be reached.
The ordering of the two zero-frame checks is load-bearing and the comment says
so: a title the seam plan emptied also has a zero frame count, and MkvInvalid
is classified by is_skippable_title_stub as an empty nav/menu stub — so
reporting it that way makes an all-titles rip drop a real feature and finish the
rest at exit 0. Swapping the checks left the entire suite green.
The new test builds a two-clip plan (from_clips needs at least two, strictly
increasing) whose marks exclude every frame, and asserts both that the error is
SinkWroteNothing and that it is NOT classified as skippable. Confirmed to FAIL
with the two checks swapped.
Auditing my own fixes found all three. None were in the original code.
The VTS crack sort was byte-wise case-SENSITIVE while the filters that select
those files (vts_group_of / is_title_vob) are case-insensitive. On a
case-sensitive volume a set holding vts_01_1.vob beside VTS_01_2.VOB sorted
part 2 first, because V (0x56) precedes v (0x76) — reintroducing exactly the
budget-exhaustion the ordering exists to prevent. Now sorted on the same
uppercase normalisation the filters apply.
Refusing a name that round-trips to empty aborted the WHOLE plan. A sidecar
folder named with a single emoji made a backup un-rippable that 1.6.0 handled
fine, and reported it as a collision with a file that does not exist. It is now
skipped with a warning: the entry is unaddressable either way, but one
irrelevant file should not cost the user their rip.
The mtime check now applies only to files whose CONTENT the plan read — the
IFOs, whose bytes 0xC0/0xC4 place every VOB. Everything else is planned from
size alone, which is already checked, so comparing mtime there bought nothing
and risked a real false positive: disc backups commonly live on exFAT/FAT32,
which stores local time, so a long rip spanning a DST transition would see a
whole-hour shift on an untouched multi-gigabyte VOB and abort hours in.
Two dirimage findings, both silent-wrong-output.
The per-directory uniqueness check compared raw host names, but the reader does
not see raw host names: parse_udf_name trims leading and trailing whitespace
and drops any code unit char::from_u32 rejects — which is every half of the
surrogate pairs the encoder emits for non-BMP characters. So " 00000.m2ts" and
"00000.m2ts", or "A<astral>.m2ts" and "A.m2ts", were two entries at plan time
and ONE name at read time. find/read_file take the first match, so a title
resolved to the wrong file extents and muxed the wrong bytes at exit 0 — the
exact shadowing DirNameCollision exists to prevent.
The key is now derived by round-tripping the name through the very encoder and
parser that will be used, so it cannot drift from them. A name that survives
that round trip as empty is refused outright: it would exist in the image and
be addressable by nothing. The new test builds a real folder, calls plan, and
was confirmed to FAIL with the fix reverted.
Separately, the plan-vs-read revalidation compared file LENGTH only, while the
plan depends on CONTENT: a DVD VOB placement comes from bytes 0xC0/0xC4 of its
IFO, and IFOs occupy a whole number of sectors so an in-place rewrite keeps the
length. A re-authoring tool touching the folder mid-rip would pass the size
check while every title extent pointed at stale sectors. mtime is now compared
alongside size, and only when both sides report one, so a filesystem without
timestamps falls back to the old behaviour rather than failing every read.
probe_folder_encryption sampled AACS units starting from the largest extent
anywhere in the disc. AACS units are 3 sectors and a unit boundary is only
guaranteed at the START of a clip, so that anchor is only correct when the
largest extent happens to be a clip first extent.
For any clip over ~2 GiB it is not. The planner caps an allocation descriptor
at MAX_AD_BYTES = 524287 sectors, so every full piece of a split file ties on
sector_count and max_by_key returns the LAST tie — an extent beginning
(k-1)*524287 sectors into the file. 524287 % 3 == 1, so that start misses the
unit boundary for two file sizes in three.
The 6144-byte sample windows then begin mid-source-packet and the byte read as
the CPI flag is content. Both verdicts fail in a costly direction: a decrypted
folder is rejected as encrypted (DirImageEncrypted on something perfectly
rippable — the exact case scan_dir was added to rescue), or real ciphertext
reads as clear and the mux writes it out as video at exit 0. is_unit_aligned
cannot catch it: it measures against the same wrong base.
Now the largest TITLE first extent, which is unit-aligned by construction and
is also the more meaningful sample — the main feature rather than whichever
fragment happened to be biggest.
The existing tests could not reach this: their fixture m2ts is 786,432 bytes,
a single extent, which is always its own first.
Round 1 removed a largest-first sort from resolve_vts_key because
largest-first is the 1.5.1 garbage bug. Auditing that fix showed it was only
half right: `planned` comes from walking the UDF directory, which yields File
Identifier Descriptors in on-disc authoring order with nothing sorting them. So
deleting the sort did not restore playback order, it left the order undefined —
whatever the disc happened to list first.
DVD-Video numbers a title sets VOBs in playback order by spec (VTS_xx_1.VOB ..
VTS_xx_9.VOB, single digit), so ascending filename IS playback order and is
deterministic regardless of how the directory is laid out.
Order decides correctness here: crack_key shares one sector budget across the
whole extent list, a CSS DVDs biggest cell opens with a long clear run, and CSS
recovers the title key from scrambled data itself. Starting in the wrong place
can exhaust the budget without ever meeting scrambled data, whereupon the
caller falls back to the disc-wide key and the whole VTS is descrambled wrongly
— corrupt PES behind an intact header, written out at exit 0.
Four fixes from the first audit round. Every finding was verified against a
pinned tree and read directly before being accepted.
resolve_vts_key sorted a VTS title-VOB extents largest-first. That is the 1.5.1
garbage bug, and it grew back in a new code path: the comment claimed it
"matched the scan heuristic", but that heuristic WAS the bug and had already
been fixed in decrypt_keys_for_title, which documents the rule (PLAYBACK ORDER,
never largest-cell-first) and pins it with a regression test. A CSS DVDs biggest
cell opens with a long clear run and crack_key shares one sector budget across
the extent list, so starting there can exhaust it without ever MEETING
scrambled data — and CSS recovers the key from scrambled data itself. The crack
then returns None, the caller falls back to the disc-wide key, and every VOB in
that VTS is descrambled wrongly: corrupt PES behind an intact header, written
out as a complete extract at exit 0.
parse_pgcit dropped titles silently in THREE places — an unparseable PGC, an
out-of-range PGC index, and a truncated entry table. The finder caught one; the
other two turned up on reading the function. parse_vmg already counts and warns
per skipped title SET for exactly this reason, and this was the last place a
disc could quietly report fewer titles than it has.
write_image called flush() and returned Ok. flush() only pushes bytes into the
page cache and promises nothing about durability, so a 6-90 GB image could be
reported complete while still unwritten — a crash or an unmounted volume then
leaves a truncated file the caller was told was finished. Now into_inner (so a
buffered-write error surfaces instead of being dropped by BufWriter::drop)
followed by sync_all.
timeline used abs() on a saturating_sub result. Every other comparison in that
module is saturating because the timestamps come off a disc and are not
trusted; abs() panics on i64::MIN, which saturating_sub can produce.
The 1.6.1 DVD fix removed the USE of the loose predicate but left the
predicate sitting there with the better name. Anyone asking "is this sector
scrambled?" finds is_scrambled before is_scrambled_pack, and reintroduces the
defect that destroyed 1912 bytes of a real VIDEO_TS.IFO — the sector carrying
TT_SRPT — so the disc enumerated 38 titles and an image decrypted from it
enumerated 10, silently, at exit 0.
Byte 0x14 only means "scrambling control" inside an MPEG-2 pack. In an IFO,
UDF or ISO 9660 sector it is whatever that format stores there. The new name
says what the function actually tests and nothing more, so the honest question
has the obvious name and the dangerous one has to be asked for deliberately.
Its doc comment also claimed a caller, decrypt::decrypt_sectors, that does not
exist — so the name was an invitation and the documentation was an argument
for accepting it. It has no production callers at all; it stays public because
an integration test asserts the flag extraction directly.
The first cut ran release-profile tests on all three platforms everywhere,
which invented coverage no crate had ever claimed. bdemu is Linux-only — its
ci.yml has no macOS or Windows job at all, and its tests call
ExitStatus::signal(), which does not exist on Windows, so they do not compile
there let alone run. The rest follow the policy ci.yml already set: tests
execute on Linux and macOS, Windows compiles them so the cfg(windows) halves
still get codegen before release time.
That policy is now explicit in qa.yml as a windows-build job rather than
implied by which jobs happen to exist.
Recorded, not papered over: running the suites on Windows DOES find real
defects — autorip has five Windows path-handling failures in mover.rs and
freemkv-engine has three tests hardcoding /dev/null. freemkv ships a Windows
GUI, so they matter. They are pre-existing rather than 1.6.1 regressions, and
fixing them is its own piece of work.
Also build the binary before cli-parity.sh. cli-integration.sh builds its own;
cli-parity.sh only checks for one and exits 2 with "build it first", so both
legs died before running a single case.
Three release-profile failures on macOS and Windows, all found by the qa gate
on its first run. Release-profile tests on those platforms had never run
before it existed, so none of these were regressions — they had simply never
been visible.
available_space returned None on every non-unix target. That did not merely
skip a test, it skipped the GATE: a Windows user extracting a disc to a full
volume got a confusing failure part-way through instead of a clear refusal up
front, and Windows is where the GUI ships. GetDiskFreeSpaceExW is declared
directly against kernel32, matching how scsi::windows already reaches Win32
rather than pulling in a binding crate for one call. It asks for
FreeBytesAvailableToCaller, which accounts for per-user quotas — the same
question f_bavail answers on unix.
The other two asserted on wall-clock timing with no margin:
- sleep_until_halted_wakes_mid_sleep bounded the wait at 350 ms and measured
377 ms on a loaded runner. That bound measures the scheduler, not the wake.
What the test is for is distinguishing "woke because the flag flipped" from
"woke because the 10 s timeout expired", and 2 s does that just as well.
- abandon_loses_to_a_close_already_committed released at 600 ms against two
300 ms grace windows plus a 250 ms poll cadence, so the windows could expire
first and the caller abandoned — a race, not a defect. The intervals are
scaled up so jitter is small relative to them; the ordering under test is
unchanged, only the margin.
Every push to qa now tags v<version>-rc<N>, N incrementing, before the gates
run. That answers "which build is on qa, and is it the one I tested?" without
anyone having to remember it.
The tag lands whether the run goes green or red, deliberately. A red candidate
needs a name more than a green one does: "rc3 failed release-tests on windows"
is a sentence you can act on, "qa is red" is not. Red on qa is the gate doing
its job — the branch saying this is not production worth yet.
release.yml now excludes v*-rc*. Its trigger was v*, which matches the
candidate tags, so without this every push to qa would have built and PUBLISHED
a GitHub release — including for the candidates that failed.
dev -> qa -> main, across every public repo.
dev is where work lands and is meant to be pushed to often, so ci.yml stays
the fast answer: fmt, clippy, unit tests, leak-guard. qa is the release
candidate, and qa.yml is the claim that a commit is production worth —
release-profile tests on all three platforms, and the Linux cross-target
clippy that the local precommit gate has always run but CI never did.
Release profile matters as its own gate: overflow checks are off, debug_assert
is compiled out, and inlining changes what the optimiser can prove. A test
that only passes in debug never guarded the binary anyone ships.
qa.yml checks siblings out at qa rather than dev. This is not a monorepo, and
a qa run resolving its dependencies from unreleased dev tips would go green on
a combination that is not the one shipping — the exact mismatch the branch
model exists to prevent. ci.yml now tracks whichever branch triggered it for
the same reason, since it also fires on qa.
The consequence is that repos move to qa together, in dependency order. qa is
backfilled from main so an unchanged crate still presents working code to the
crates built on it.
css::is_scrambled reads bits 4-5 of byte 0x14 and nothing else. That is a
sound test once a caller has committed to a title's VOB data, where every
sector is an MPEG-2 PS pack and byte 0x14 always means what it says.
descramble_region is not such a caller: it is handed arbitrary regions of a
disc, so it also sees IFO, UDF and ISO 9660 sectors — raw structures where
byte 0x14 is whatever that format happens to store there.
Measured on a real disc: the second sector of VIDEO_TS.IFO holds 0x15 at
offset 0x14 while starting 00 26 00 00, which is not a pack. The flag test
read it as scrambled, descrambled it, and destroyed 1912 of its 2048 bytes.
That sector carries TT_SRPT, so the title table went with it — the disc
enumerated 38 titles and an image decrypted from it enumerated 10, silently,
at exit 0.
is_scrambled_pack already existed with the right predicate. Use it here. It
costs nothing: a genuinely scrambled VOB sector always carries the pack start
code, and no IFO sector does.
Verified end to end — the decrypted image's `info` output is now identical to
the source disc's, 38 titles both, differing only in the CSS: Encrypted line.
The fixtures moved with it. Four of them built a sector by setting byte 0x14
alone, which no real scrambled sector looks like; they now build packs.
The diagnostic hashed IFO contents but not where they came from, so a
content difference could not be told apart from a path resolving to a
different place. It now prints file_start_lba and file_extents next to the
hash.
That measurement is what inverted this investigation: both a CSS disc and
its decrypted copy resolve VIDEO_TS.IFO to the same extent, and only one
sector of the fourteen differs — by exactly a descrambled payload with the
scrambling-control bits cleared. So the decrypted copy holds the correct
bytes, and an iso:// scan of a CSS disc is parsing a still-scrambled IFO
sector, because scan_iso opens a plain FileSectorSource.
Two opt-in dumps, driven by FMKV_IMAGE, that narrow where a decrypted DVD
image loses titles:
dump_title_sets_for_an_image reports what survives parse_vmg. On one disc
the CSS image yields 13 title sets and 38 titles; its decrypted copy
yields 8 and 10. Sets 8, 9, 10, 12 and 13 are dropped outright, set 11
parses but returns no titles at all, and set 7 returns 3 of 5.
dump_vts_ifo_reads_for_an_image reads every VTS IFO and hashes the
CONTENT. All thirteen are byte-identical across the two images, so
parse_vts is handed the same bytes and the same TT_SRPT info and still
fails on one of them — the divergence is in what it reads from the READER
afterwards, which is file_start_lba and the PGCIT.
An earlier version of the second dump compared only length and magic and
so wrongly reported the images as identical; it hashes the bytes now.
parse_vmg skipped any title set whose parse failed, with no log and no
counter. A real disc enumerated 38 titles from one image and 10 from
another, and the 28 discarded failures were invisible — the symptom read
as a scan difference rather than as dropped reads, which is most of why
it took so long to localise.
Behaviour is unchanged: a disc may legitimately carry placeholder TT_SRPT
entries, so one failure is still not fatal. It now warns per skip with the
title set and the error, and once at the end with kept-versus-declared.
Also adds an opt-in diagnostic that reads every VTS IFO from an image, to
separate a read failure from a parse failure. It reports all 13 sets
reading identically from both a CSS image and its decrypted copy, which is
what proves the 38-to-10 loss is downstream of these reads.
dvd_placement_invariant_on_a_real_folder checks, per title set, the sum
ifo.rs relies on: file_start_lba(VTS_nn_0.IFO) + vtstt_vobs must land on
VTS_nn_1.VOB. It reports all 13 sets correct on a real DVD folder, which
is what excluded placement as the cause.
dump_titles_for_an_image prints every title a scan produces with the
numbers canonical_title_order sorts on. It is what showed the real shape:
the same disc scans to 38 titles as a CSS image and 10 once decrypted,
with identical capacity and a byte-complete image.
Both are #[ignore]d and read their target from the environment, so they
cost the gate nothing and are there for whoever picks the defect up.
The zero-frame check ran before the seam gate, and its error is
classified as a skippable nav stub — so a title the plan dropped
ENTIRELY was reported as an empty stub and an all-titles rip would omit
a real feature and finish the rest at exit 0. The seam case is decided
first now, with a code that is not skippable.
The demux sink read a frame's track kind out of the FILTERED slot, which
is empty for a class the export drops. On an audio:// or sub:// export
the video track was therefore called non-video and handed the permissive
crossing rule — the same defect round 4 fixed for a Dolby Vision layer,
reintroduced one file over. Video tracks are now recorded before the kind
filter, beside the primary-video reference that exists for this reason.
Its frame counter counted frames PLACED, not written, while its name and
doc claimed otherwise. Renamed and documented for what it is, including
that it cannot see a single lost track among many.
Placement: files in a subdirectory of VIDEO_TS were never given data.
They were declared at full size with no extents, so they appeared in the
tree and read as nothing. The same folder under BDMV was always placed
correctly. And the duplicate title-set guard keyed on the constraint maps,
so an IFO declaring no offsets inserted nothing and a colliding second IFO
went undetected — it keys on the groups seen now.
Display for SeamPlanDroppedMost and ShortImageRead discarded their
payloads, and four new variants were missing from the code-uniqueness
test.
The cap was added without a test. Its companion — a modest oversize is
honoured as a gap — already existed, so this pins the other side: an
offset past the ceiling is refused rather than grown into, which is what
keeps a rewritten IFO from planning a multi-terabyte image.
Round 3 counted frames the clip marks excluded and reported them at
finish. Counting is not bounding: the only other gate was a global
zero-frame check, and its error is additionally classified as a skippable
nav stub, so a title whose marks do not line up with its PES clock could
discard almost all of itself and still exit 0 — a two-hour feature
emitting seconds, which is the defect this change set already shipped
once. Dropping more than was kept is never a real join, so it now fails.
The demux sink had no zero-output guard at all, so a fully-dropped title
finished cleanly: a directory of zero-byte track files beside a populated
chapters document. It now refuses, keyed on frames having been OFFERED —
a chapters-only export, or a track class the title does not carry,
legitimately writes none, and two existing tests correctly said so.
A title set's placement group is the parsed number, so VTS_01_0.IFO and
VTS_1_0.IFO land on one key and the second silently overwrote the first's
constraint, placing a VOB where the IFO the reader uses does not point.
Refused rather than resolved by arrival order.
A Dolby Vision enhancement layer is a second video track: it does not
drive epochs, but it does carry B-frame reorder. The crossing rule was
keyed on driving epochs, so the EL took the branch whose premise is that
the track has no reorder — and its ordinary reorder dip near the end of a
clip, which during an overlap also lands inside the next clip's range,
was read as a join. The EL was then placed on the next clip's offset, out
of step with the base-layer frame it must be co-timed with by the width
of the overlap: the same desync the per-track cursor was added to remove,
reintroduced for one track.
The property the rule actually depends on is whether a backward step can
be reorder, so it is now keyed on that. Both sinks derive it from the
track kind rather than from the epoch driver.
Two findings from the same escalation, both silent-wrong-output.
The plan was built for every multi-clip title. Only a Blu-ray PlayItem's
IN/OUT are positions in the clock the PES PTS runs on. HD-DVD fills the
same fields from the XPL's title-relative times and a DVD's come from
cell tables, so a plan built from them is an identity map with a drop
filter: it suppresses the layer-break rebase inference performs, and
drops whatever falls outside marks the PTS was never measured against. An
earlier reading of this called HD-DVD safe because its marks are
contiguous and every computed offset was zero — true, and irrelevant,
because they were zero in the wrong clock. Gated on the content format,
with a test using an HD-DVD-shaped table that the clock check alone
accepts.
The crossing test was also one-shot. A table whose clips restart their
own bases could miss it, and a missed crossing STRANDS the track: every
later frame falls outside the stranded clip's marks and is dropped for
the rest of the title. Counting drops, which is all the previous round
added, does not bound them. A table that is not one advancing clock is
now refused outright and falls back to inference, which is the documented
safe path for those titles.
Also from the same round: read_sectors added an unchecked lba + i, where
callers deliberately saturate their LBAs — a wrap folds the read back to
a low sector and hands the muxer another file's bytes. classify added 1
to two numbers parsed verbatim out of a filename. read_head used a single
read() where a short read on a network mount silently records no
placement constraint at all. And the page-cache eviction added last round
released only the read that crossed its threshold rather than everything
accumulated, so seven eighths of what was read stayed pinned.
The round-1 tests asserted arithmetic about MAX_CS0_NAME_BYTES and never
called plan(), so both would have passed with the guard deleted — which
is the failure mode this audit exists to catch, committed by the audit's
own fix. They now build a real folder containing a 255-byte name and
require the planner to refuse it, plus a companion proving a name at the
cap is still accepted so the guard is not merely refusing everything.
The subdirectory cap keeps its arithmetic-only test — creating 65,535
directories is not reasonable in a unit test — but now says so instead of
implying coverage it does not have.
The folder encryption probe returned "not encrypted" when it had sampled
nothing at all — a title shorter than one aligned unit skipped the loop
entirely. That verdict CLEARS the structural one an AACS directory
raised, so a genuinely encrypted folder would have been ripped as clear
and written ciphertext as video at exit 0. With no evidence it now keeps
the structural verdict, and its bounds arithmetic no longer trusts
disc-derived values not to wrap.
Reading an IFO header swallowed every I/O error and returned an empty
buffer, which sent each placement offset through unwrap_or(0) and
recorded no constraint at all — a permission error on one file produced a
silently misplaced VOB. The directory walk swallowed the same class while
claiming to skip only vanished files. Both now propagate; only NotFound
is skipped.
Four things the round-1 changes left inconsistent: two new error codes had
no doc comments, were absent from the io::Error mapping, printed no path
in Display, and were missing from the test that proves codes are distinct.
The demux sink dropped frames silently while the MKV muxer reported them.
And set_clips had been inserted INTO write_frame's doc comment, leaving
write_frame undocumented and its paragraphs describing the wrong function.
uid/gid used 0 as "not specified"; UDF's sentinel is 0xFFFFFFFF, and 0 is
root.
A DVD title set records where its VOBS begins as an offset inside its own
IFO, and the planner honours that offset because honouring it is what
makes a real backup readable. Nothing bounded it: a regenerated .BUP or a
hand-assembled folder naming an offset far past the content grew the
image to wherever it pointed — a u32 sector count reaches ~8.8 TB, and
writing that to an iso:// destination fills a disk with zeros before
anything notices. Capped at 128 GiB, which clears BD-100 with room.
Metadata is materialized up front and held for the life of the image, at
a 2 KiB File Entry per node, so the 100,000-entry cap alone permitted
~205 MB of it for content of no size at all — and the mux holds two
images at once while probing. The module claimed a budget of a few MiB;
that budget is now enforced rather than asserted.
Host reads had no page-cache eviction. The ISO source documents what that
costs, measured: an 85 GB read pins the whole file, starves the writer,
and collapses the mux to 2.7 MB/s against 70 MB/s isolated. A folder
source reads host files the same way, so it now uses the same eviction —
the hints move from private-to-that-module to crate-internal rather than
being reimplemented.
The placement code skips that field for the Video Manager, which is
right, but said it was skipping it because the field is the menu cell
address table. It is the title search pointer table, and it is an offset
inside the IFO rather than a pointer to another file — which is the
actual reason it constrains nothing. ifo.rs reads the same offset under
the correct name, so the two would have drifted.
A sparse passive track — a subtitle with no event near a clip's mark —
was held to the dense-video crossing window, so it stayed on the previous
clip's offset until its PTS passed that clip's OUT and every event in
between was mistimed by the overlap. Video keeps the tight window,
because its backward steps are also B-frame reorder; passive tracks have
no reorder, so any backward step into the next clip's range is a join.
Frames the marks exclude were dropped without a trace. Dropping is right
at a join, but this codebase has shipped complete-looking wrong output
before, so the count is kept per track and reported when the mux
finishes, alongside the pre-cluster counter that exists for the same
reason.
A File Identifier Descriptor records its name length in one byte, and the
length was narrowed with a cast: a 255-byte name — POSIX NAME_MAX,
entirely ordinary — encodes to 256 and wrote zero, which would read every
later entry in that directory from the wrong offset. A directory's link
count is 16 bits and was computed as 1 + subdirectory count, which the
global entry cap alone permits overflowing. Both are refused while
planning, where the tree can still be rejected cleanly.
The module and struct docs described inference as the whole algorithm;
they now say which path decides what.
The folder reader produces a wrong title list on a real DVD — the title
table is read at a sector offset relative to VIDEO_TS.IFO's start, and a
synthesized layout does not put that file where the offset resolves the
same way. Output is silently wrong at exit 0, so the feature is not
wired to the CLI in this release. The library module stays, unreachable,
and ships once both disc families are verified against real content.
The first cut placed every track with the cursor the video had moved. At
an overlap join the previous clip's audio is still arriving after video
has crossed, and those tail frames sit inside both clips' mark ranges —
so they took the new clip's offset, jumped forward by the overlap, and
collided with the new clip's own audio, which the muxer's monotonic nudge
then flattened. A remux confirmed it: the timeline length was already
correct and the original symptom was still there, 169 audio packets on
the tick floor.
A track's PTS only runs forward inside a clip, so its own backward step
to the next clip's IN is its crossing. That is per track, so the cursor
is too.
A seamless-branching title's PlayItems do not chain contiguously: one
clip's OUT can sit after the next clip's IN, where the disc stores the
join twice, or before it, where the playlist skips material. The mux
never saw those marks — its own header said so — and inferred seams from
PTS jumps instead.
Inference cannot recover this. A forward jump is ambiguous: it means the
playlist skipped, or it means frames were lost to damaged media, and
compressing the latter would falsify timing on exactly the rips that most
need it faithful. An overlap smaller than the B-frame reorder threshold
is invisible to inference entirely, and its duplicate content then
collided in the muxer, where the monotonic nudge flattened a run of audio
onto the tick floor and put sound ahead of picture for the rest of the
film.
Measured on one 11-PlayItem title: the file declared 7893.385 s, which is
what the playlist says the title is, and carried packets to 8029.298 s.
Both numbers came from the same program on the same disc. Four skips
totalling 135.9 s became dead timeline, and a 1.79 s overlap put audio
1.8 s ahead at the half-hour mark. Five of forty-seven titles were
affected; every single-clip title was exact.
So the marks are read. Each clip contributes exactly out - in, laid end to
end, so the output runs as long as the playlist says and a join never
rewinds. Titles without usable marks — DVD, HD-DVD, file sources — keep
the inference path unchanged, and clips that already chain contiguously
produce a constant offset, which is pinned by a test.
Users keep discs as extracted folders — a DVD VIDEO_TS or a Blu-ray BDMV,
usually a backup that is already decrypted. dir:// could only ever be a
destination, so those folders could be produced and never read back.
Everything above the sector layer wants a UdfFs over a SectorSource, and
every UdfFs read re-reads the ICB off that source at call time, so a
folder has to present itself as sectors. It does: dirimage plans a block
layout over the real files, encodes a UDF 1.02 filesystem for the
metadata, and serves data straight from disk. read_filesystem then parses
it exactly as it parses a disc, so nothing above changes — and the
iso:// arm of input() is now shared rather than duplicated, so dir://
inherits its decrypt gates, title selection and stream pruning.
The encoder is validated by more than its own reader: macOS mounts the
synthesized image and the mounted files compare byte-identical to the
originals. A round-trip through our own parser could not have shown that
— the tag CRC seeds at zero, and a wrong seed would satisfy us and no
real driver.
DVD placement is not free packing: a VTS IFO records where its title
VOBS begins relative to itself, so the VOB has to land exactly there.
Unsatisfiable marks fail loudly rather than misplace the file. 3D folders
are refused for now: the scanner detects SSIF and the planner cannot
alias its extents yet, so accepting them would produce quiet nonsense.
Left for later: metadata capture, HD-DVD, FMTS, encrypted folders.
Writing an iso:// meant a raw sector copy off a drive, so an existing
image could only ever be a source. Decrypting one you already had meant
finding the disc again.
The engine's copy is the recovery path — mapfile, multipass, damage-jump,
auto-resume — and all of it exists because optical media returns read
errors. A file does not, so a non-drive source gets write_image instead:
sectors in, bytes out, once, no recovery machinery. Keeping them apart is
not just tidiness. The mapfile identity check compares AACS unit keys and
the VID, both empty for an already-decrypted source, so identity passes
for any such source — a second run with different input to the same
output path would resume over the previous image and report success.
A short read is an error rather than a zero-fill: padding a truncated
source yields an image that looks complete and is not, which is the worst
outcome for a copy someone means to keep.
`dvd_time_t` stores H:M:S:F as non-drop-frame timecode with a rate flag,
not elapsed wall-clock time. On NTSC the seconds field advances every 30
frames, but a frame lasts 1001/30000 s, so 30 frames occupy 1.001 s of
real time. Reading H:M:S as literal seconds under-reports real time by
exactly 0.1% — 3.6 s per hour, growing with elapsed time, which is what
made chapter marks drift ~4 s by the 67-minute mark.
Convert the whole timecode through an integer frame count and apply the
exact 1001/30000 fraction once, instead of reading H:M:S literally and
dividing only the frame remainder by a decimal 29.97. Chapter marks sum
per-cell frame counts as integers and convert once per chapter, so every
mark lands on an exact frame boundary instead of accumulating f64
rounding across a long title.
PAL is arithmetically unaffected: 25 frames = 1.000 s exactly, so the
old and new paths agree to within one ULP. Sweeping all 900k H:M:S:F
combinations under 10 hours, 12 differ, by at most 8.9e-16 s — the new
path divides one exact integer instead of adding a rounded fraction to a
large one, so where they differ it is the more accurate of the two.
Five existing NTSC fixtures asserted the old literal-seconds values and
were updated to real seconds (each is exactly 1.001x its old figure);
`bcd_secs` now documents that its argument is timecode, not real time.
Fixes#1
Reported-by: AnimeFN <admin@animefn.com>
Entries had grown into investigative narratives — root causes, disc
counts, PIDs, percentages, function names — which is the right level for
a post-mortem and the wrong one for a public changelog. Each entry is now
one to three sentences leading with what a user observes.
Halves the file (1538 -> 735 lines) without dropping a version. API
signature changes stay explicit, since a consumer migrates against those.
A vendor label's `stream_number` is a slot in the one stream table its
config blob describes. The labels merged in from the playlists — to
cover streams the vendor named nothing for — carried a different number
entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's
slot numbering at all. Two coordinate systems, one field name. The
merge matched them by equality and the binder then counted streams
against the result.
Measured over the 44-image corpus: 22 discs merge such labels; of the
566 places one lands on a stream, 443 (78%) are a stream it does not
describe — the label states the PID it read itself from and it is a
different one. 142 of those are stopped by the language check 94377c7
added; 301 are applied. Those labels carry no editorial payload, so the
direct damage is confined to codec text — but the polluted list is also
what the anchor gate reads, and on 11 disc/stream-type pairs it is what
decides the anchor, which is how it reaches the vendor's forced and SDH
flags. 53 anchor facts are harvested off a merged slot. The clip-info
orphans had the same shape, numbered from `max + 1` of a list they share
no coordinate system with.
A label now either NAMES its stream — `StreamId { clip, pid }`, read out
of the very table `disc::bluray` builds the stream from — or it does
not, and only the ones that do not are ever reached by counting:
* `label_at` returns vendor labels only, so the two numberings can no
longer be confused by construction.
* Playlist and clip-info labels bind by id. Exact, no anchor, no
sequence, no ordinal.
* A named stream outranks a guessed one, so an editorial flag reaches
a stream only where the disc's own numbering puts it there.
* The presence of an id is the provenance the anchor gate was missing.
It reads the vendor's slots alone now, so a slot the vendor never
named no longer breaks the sequence — under-yield is the normal
shape of these blobs — and a title with fewer streams than the list
has slots is no longer eligible to hold it. Ranking prefers the
title that positively confirms most of the list.
* An orphan is in no playlist, hence in no title, so it binds to
nothing rather than to whatever counted its way.
The MPLS floor is one label per physical stream keyed by `(clip, PID)`,
not by `(type, language, codec, pid)`: the same PID in two clips is two
streams, and the old key collapsed them. Its `stream_number` is now the
entry's real slot in its own playlist's table.
41 of 44 images are byte-identical; all three that move lose a label
they should not have had, and no feature title changes on any image. A
dozen featurette playlists stop reporting a feature subtitle's SDH
marking on their own unrelated subtitle; eleven menu and bonus titles
stop advertising the feature's object-audio format on plain stereo; and
on a disc whose every title carries a single audio stream — too short a
table to anchor anything — a regional-variant tag asserted on all
seventeen titles is now asserted on none, in exchange for every title
stating the codec it actually carries, which none of them did.
Six tests had been asserting the invented numbering, including one
pinning the disc-global counter as a deliberate property.
Also fixes a defect in the same family that the corpus work surfaced:
`pgs_forced_probe::apply_verdicts` set `forced` but left `qualifier`,
so a demoted track shipped with a metadata sidecar calling it forced
next to a Matroska header saying it is not. Only a forced claim is
cleared; an SDH marking is not the probe's to touch.
One vendor's playlists.xml carries a per-subtitle-slot cell that looks
like a boolean, and it was parsed as one: value 1 meant forced, anything
else meant not forced. Across every image in the corpus that uses this
format the cell takes four values, and 1 is not the forced one.
Decoding three of those discs and counting every PGS display set:
* 1 marks a FULL dialogue track that additionally contains some
forced-narrative signs. All nine cells bearing it on one disc are
full tracks of 949-1411 display sets; all seven on another are full
tracks of 1602-1651. Neither disc has a small track among them.
* 2 and 3 mark a DEDICATED forced-narrative track, in its own trailing
stream slot, duplicating a language that already holds a full track.
The two 2 slots measured are 15 and 10 display sets with every one
flagged forced; the four 3 slots are 7, 14, 23 and 59 against
1216-2655 on the tracks they duplicate.
So the old reading was wrong in both directions — it flagged full
dialogue tracks forced, which is how one language came to present as two
identical full subtitle tracks with one of them marked forced, and it
threw away the cells naming the real forced tracks.
Content could not have corrected this afterwards. Clearing a wrong
forced label needs a disc whose authoring sets forced_on_flag, and on
the measured disc carrying four genuine forced tracks not one display
set anywhere sets it — there, the vendor cell is the only evidence there
is. The classification is now explicit: only a dedicated forced slot
earns the flag, an unrecognised value never does, and the
contains-forced-signs value is dropped rather than weakened into a
forced label, since a wrong forced flag on a full dialogue track is the
user-visible defect while a missing hint costs nothing.
Four of the crate's own tests had been asserting the boolean reading;
their subject was positional alignment, so they keep it and now use a
real forced value. The other two parsers that emit a forced qualifier
from vendor metadata were audited and are structurally immune — in both,
the forced marker names a slot of its own rather than hanging off a full
track's entry, so the failure has no encoding there — and each is now
pinned by a test saying so.
A vendor label list describes ONE playlist's stream table, but
apply_labels re-numbered it from 1 inside every title. Where sibling
playlists cover the identical feature clip and enumerate different
subtitle sets, identical ordinals resolve to different PIDs, and the
same physical stream came out flagged forced in one title and not in
the other. On the title such a disc offers as its rip target that put
`forced` on an 873 MB full-dialogue English subtitle track — the
reported "I see English and English (Forced), they are identical".
The blobs are not wrong; the binding was. The list carries no playlist
id, but it carries a language per slot, and that sequence is a
fingerprint: on the corpus exactly one title's per-type language
sequence reproduces the list position for position, and content
confirms that title's binding is the correct one. So binding is now
two-tier:
* The title whose whole per-type language sequence sits on the list
(>= 2 streams, longest wins) is the ANCHOR — the table the list is
describing. Each of its slots yields a `(clip, PID) -> label` fact,
and a PID is the same elementary stream in every playlist that
plays that clip, so sibling playlists bind through the map. A slot
the anchor never showed us is not bound at all.
* Streams no anchor fact reaches still bind by the STN ordinal, but a
label whose language contradicts the stream it would land on is
dropped. Subtitle labels carry nothing but the qualifier, so an
unverifiable one is all risk and no gain: off the authoritative
path they additionally require both sides to STATE a language and
state the same one. Unlabelled beats mislabelled — the muxer's
demotable() guard can only clear a wrong `forced` on discs whose
authoring uses forced_on_flag, and half the measured discs never
set it.
Measured over 44 disc images, 9 change and every cross-title label
conflict goes away: 171 forced flags that contradicted a sibling
playlist are cleared, 61 correct ones are recovered on playlists that
had been missing them, 5 subtitle qualifiers and 1 audio purpose bound
against a contradicting language are dropped. No title gains a label it
did not have.
Known residual: on one disc the featurette playlists keep two forced
flags (down from nine) where a shifted list happens to coincide on
language. Ruling those out needs the list's provenance — which slots
are the vendor's and which were merged in by the MPLS gap-fill — and a
whole-sequence gate without it costs correct flags on discs whose
vendor slots are interleaved with gap-filled ones.
The same scrub as the previous commit, over the files it did not reach:
the variant-MKB layout notes, the 2.1 segment index observations, the
PPS-revert regressions and the playlist-twin tiebreak.
Measurements keep their numbers — "a v70 `0x2d` body = 46_100*2 + 16"
is the useful part, and the title it came from never was.
A forced verdict is an absence claim, and a sampled run sees a fraction
of a track. Measured on real discs: tracks exist that flag about a
quarter of their display sets and leave the rest unflagged, so a sample
that catches one flagged set and nothing else promotes a full dialogue
track to forced -- which players then force on screen. A sampled run now
needs two display sets; a run that read every extent end to end has no
unread gap and may still promote off one.
Fifteen references across six files named the discs a defect was first
seen on. The parser leak found earlier was not an isolated slip — the
same habit runs through the mux comments, the changelog and the AACS
content verdict, where a title name was standing in for the shape of
the problem.
Every one is replaced with the property that actually mattered: a
multi-clip title, a UHD Dolby Vision profile 7 dual-layer stream, a
disc carrying an authored-bad TS packet. The comments are more useful
for it — the reader needs to recognise the shape on a disc they have,
not the one we happened to have.
`SEG_MainFeature` stays: the parser matches on that literal, so it is
a format token rather than a title.
Matthew Jackson