Compare commits

...
73 Commits
Author SHA1 Message Date
Matthew Jackson 8d17904459 v1.6.2: bump version (freemkv-unlock git-pinned for the tag) 2026-08-08 22:18:28 -07:00
Matthew Jackson 1bdb8add2f Stop trusting a byte offset when the feed is not what was measured
A clip's span is measured over the title's full extents when the disc is
scanned, and a frame is placed by the offset it was read from. Those only
agree while the mux reads every byte the scan counted.

On a disc carrying a forensic segment it does not: the read plan omits the
units belonging to another device group, so fewer bytes are fed than the
spans describe, and the shortfall grows through the title. Every frame past
the first segment then looks earlier than it is — placed in a clip it did
not come from, or dropped at a join for failing marks it was never inside.
The spans still tile one another perfectly, so the check that asks whether
they can be trusted cannot see any of it.

The plan is now compared against the extents it was built from. When they
differ the spans are dropped and placement falls back to timestamps, which
is what that path is for and what the surrounding comment already promised
would happen when an offset stops meaning anything. Ordinary discs are
untouched: with no forensic segment the plan IS the extents.
2026-08-08 19:42:53 -07:00
Matthew Jackson c9562c07a3 Make four guards testable, and one guard see both spellings
The provenance guard knew one way to lose a frame's source offset —
writing it as absent. Omitting the field entirely does the same thing,
because the frame type fills it in by default, and the guard read straight
past that. A parser rewritten into the second spelling would have kept its
green light while its track silently went back to being placed by guesswork.
It now reports which spelling it found and where.

The title-count clamp was asserted over a fixture too small to hold more
titles than the cap allows, so the walk stopped when the buffer ran out and
the clamp was never what bounded it — the assertion held with the clamp
deleted. The fixture now carries more entries than the cap.

The subdirectory limit was checked by restating the constant's own
definition; the guard itself had never run, and deleting it changed
nothing. The limit is lowered under test so a real folder can exceed it,
and the test now walks one and requires the refusal.

One test also carried two unrelated grounding notes while the test they
described had none, so reading the note above a test told you about a
different one.
2026-08-08 19:06:37 -07:00
Matthew Jackson 7f9dcf1870 Keep both halves of a clip file that two play items share
A playlist may point at the same .m2ts from several play items — a
seamless split, a looped segment, multiple angles. The file has one set of
bytes, so it has one span, and the byte a frame was read from therefore
identifies the FILE, not which play item's range it falls in. Placement
took the first of them and judged every frame against its marks, so
everything past that range's end was treated as material the playlist
excludes and dropped: half the clip missing from the rip, with the
timeline still charged for its full duration.

The offset narrows a frame to the run; only its timestamp can finish the
job, and each play item carries its own marks. A frame is now matched
against the marks of the entry it actually falls in, falling back to the
first when it falls in none, which leaves genuinely-excluded material
dropped as before.

The test for this construction asserted only that such a playlist is
trusted, and never placed a frame from the second range — so the loss it
described in prose was invisible to it. It now places one from every range
and requires them all to survive.

The deferred-mux replay of buffered frames wrote them without their source
offset, quietly sending the head of every such title down the timestamp
heuristic the rest of this work exists to retire. It passes the offset
through now, and the frame writer that omits it is compiled out of the
library entirely: no production path can discard provenance any more.

Also: the AC-3 buffer-reuse test could not see the regression it named.
Feeding equal-sized packets, a fresh allocation per call yields the same
capacity as a reused one. It now feeds a large packet then a small one,
where only a reused buffer keeps the larger capacity.
2026-08-08 17:18:35 -07:00
Matthew Jackson 3c5c681399 Audit round: an ADTS frame that cannot hold its own CRC, and three drifts
An ADTS header that declares a CRC follows must be at least nine bytes —
seven of header plus the two the CRC occupies — because the declared frame
length counts them. The structural gate compared against a flat seven and
never read the bit that says whether a CRC is there at all, so a frame
whose own header describes something impossible was accepted and handed to
the muxer as decodable.

The pipeline's spawn doc named Sweep, and the thread Sweep would have
created, as callers to look for. Neither has been in this crate since the
recovery passes moved out. The same paragraph already records fixing this
once, for a different departed caller — it simply drifted again a sentence
later, so it now says to name callers that live here or name none.

Whether a disc is structurally AACS-encrypted was spelled out by hand in
both the fast identify and the full scan. They agreed today; nothing made
them agree tomorrow, and disagreeing would mean the same disc reported
encrypted by one path and clear by the other. There is one definition now,
and the comment that pointed at it by line number points at its name.

The AC-3 parser built a fresh buffer on every packet — of the order of a
hundred thousand times per title — to work around a borrow it cannot
avoid. The copy stays; the allocation does not. The buffer is now lent out
and handed back, and a test pins that, because reverting it would be
invisible in behaviour.

Left alone deliberately: send/send_with_halt and finish/finish_with_halt
look like one action under two names, and are not. After the consumer
fails, one must still accept items and the other must refuse them; that
difference is what stops a producer reading an entire disc for a write
that died on its first frame. Collapsing them was tried here and the
existing test caught it. Both now say so where the choice is made.
2026-08-08 16:55:55 -07:00
Matthew Jackson 43d9fb583a Add an error for an image shorter than its recovery data
A recovery resumed against an image that has been truncated since the
previous pass — a full disk, an interrupted transfer, a remount — cannot
repair it: the pass only revisits the ranges the recovery data calls bad,
so everything past the cut stays a hole while the counts still describe a
whole disc. There was no way to say that. Reusing the recovery-data error
would have been wrong, because the recovery data is intact; the image is
not.

Registered in the uniqueness list, the code table and the range check, so
a future variant cannot silently reuse 6015 or map to the wrong code.
2026-08-08 13:35:44 -07:00
Matthew Jackson 60ed45c047 changelog: 1.6.2 2026-08-08 08:53:13 -07:00
Matthew Jackson 07cc9fa5f9 ci: run the dependents' test suites, not just a type-check
The `consumers` job checked out all five dependents against this commit and
ran `cargo check --all-targets`. That proves they still COMPILE, which
catches a changed signature and nothing else. The failures worth catching
here keep every signature intact and change behaviour: the library still
builds, the dependent still builds, and the dependent's tests are what go
red. Those never ran.

Run their suites instead. Same checkouts, same patched paths — only the
verb changes.
2026-08-08 08:46:17 -07:00
Matthew Jackson 1e6bb8babd Give a frame that outruns its epoch's video a provisional offset
The demuxer interleaves, so at a boundary the streams do not reset on the
same frame: audio for the next segment can reach the muxer before the video
frame that opens the epoch it belongs to. Only the primary video track may
open one, so those frames rode the just-ended offset and landed a whole
segment in the past. Downstream the strictly-monotonic block nudge then
crushed the entire run onto one instant a tick apart, which is audible.

Such a frame is recognised on its OWN raw PTS, which within an epoch only
advances, so a large backward step is unambiguous — a different signal from
the shared frontier, which is what the old false-positive ratchet keyed on.
It then rides a provisional offset private to its track, computed the same
way the video path computes a real one, and drops it the moment the video
retires an epoch, so the run rejoins with no seam.

The offset is deliberately private: it never writes offset_ns, never
advances the frontier and never retires an epoch, so it cannot move the
video timeline. Letting a passive track open a REAL epoch was tried first
and inflated a 476.776 s title to 656.216 s, because every track meets a
boundary at its own pace and the video path rebased again on top.

Measured on a real DVD title with 8 cell boundaries: audio frames stamped
inside one cadence 120 -> 0, outsized gaps 7 -> 0, subtitle 1 -> 0, and the
video span byte-identical at 476.484. On the HD-DVD title the remaining
boundary gap on its first audio track falls from 0.999 s to 0.275 s.
Acceptance: 76 pass, 0 fail, 0 skip.
2026-08-08 00:48:10 -07:00
Matthew Jackson 64f894c9f3 Place a straggler by its own epoch's end, not the current frontier
A non-video frame can reach the muxer after the video that opened the next
epoch has already been processed. It carries an old-epoch PTS, so adding
the new offset flings it forward by a whole clip. The remap that exists to
catch this compared the frame against the CURRENT frontier and only
accepted it within one backstep, which is the wrong yardstick: by the time
a straggler is seen the frontier has moved on into a new epoch, and how far
below it the frame lands says nothing about where it belongs.

Retiring an epoch now records the frontier it closed at alongside its
offset, and a straggler is judged against the end of the epoch its PTS came
from. Keeping the whole history rather than one previous offset also fixes
the case a single `prev_offset_ns` cannot express at all: a frame from two
or more epochs back.

Measured on a real HD-DVD title whose second audio track put its final
packet at 12834.587 s in a 6434.100 s file. That frame sits 23 s below the
current frontier — outside the old window, so it was refused — but 0.15 s
below the end of the clip it actually came from. It now lands at 6417.216,
one 32 ms cadence step after its neighbour, and the track spans 6416.160
with no outsized gaps. The other two streams are unchanged.

A title that never rebases retires no epoch, so the lookup finds nothing
and the mapping is bit-identical to before.
2026-08-08 00:02:57 -07:00
Matthew Jackson 3820348d2f test: cover the 1.6.1 provenance work where it was assumed, not asserted
Four parsers (dvdsub, flac, lpcm, mpegaudio) stamp a source offset on
every frame but had no test that read one back, so a regression to
`source: None` would have been caught only by the brace-balanced audit
in codec/mod.rs — a lint, not a behavioural check. Each now asserts the
emitted frame carries the offset of the packet that supplied its first
byte.

The Blu-ray feed spans had no direct test at all. Add one that walks a
multi-item playlist and requires the spans to tile the feed with no gap
or overlap; it catches a one-sector-per-clip drift, which is exactly the
error class that would misattribute frames near a seam.

`no_provenance_still_places_by_marks` asserted only that placement
returned something, which passes for a frame placed in the wrong clip.
Its probe timestamp lands in an overlap between two clips in the real
mark table, so pinning one clip would assert a coin-flip; instead
require the offset to be one that a clip actually containing that
timestamp would produce.
2026-08-07 22:16:02 -07:00
Matthew Jackson 102ae82933 Test the DVD title table's sector-offset arithmetic
The TT_SRPT pointer at 0xC4 is a SECTOR offset from the start of
VIDEO_TS.IFO. Reading it as a byte offset lands 1/2048th of the way in and
does not error — it silently returns a different title list, which is how
a disc enumerating 38 titles and an image enumerating 10 looked like a
scan difference rather than a failure.

The table already had tests for its entry layout, dedup and the 99-title
clamp. What it did not have was anything pinning the offset unit itself,
or the grouping of titles by title set, or a truncated table keeping the
rows it can read rather than failing a whole disc.
2026-08-07 22:01:37 -07:00
Matthew Jackson 9d05f54ddb restore freemkv-unlock path dep for local dev (post-v1.6.1) 2026-08-07 19:08:46 -07:00
Matthew Jackson 47abd1270b v1.6.1: bump version (freemkv-unlock git-pinned for the tag) 2026-08-07 19:08:42 -07:00
Matthew Jackson 96c23ffb87 changelog: mark 1.6.1 UNRELEASED so release.sh can date it
release.sh dates the heading when it cuts the tag, and refuses a bare
'## [1.6.1]' because it cannot tell an undated heading from one it has
already stamped. Shipping a tag whose public changelog says UNRELEASED is
permanent and cannot be corrected inside that tag, so the check is right
to stop rather than guess.
2026-08-07 17:57:29 -07:00
Matthew Jackson f2b0e00bea Guard the invariant that provenance is universal
Every emitted frame must carry the source byte offset of the packet it
came from. That held only for video for as long as it existed, and
nothing asserted it, so nothing caught it: ten parsers built frames with
`source: None` and a multi-clip title could not place audio or subtitles
by byte at all.

Finding them took a brace-balanced scan of the tree by hand — a regex
cannot do it, because a Frame literal contains nested braces and a
non-greedy match stops at the first `}`, which is how five sites survived
the first pass. This is that scan, as a test.

A second test fails if a codec module is added and not listed, since an
unchecked parser is exactly how the gap persists. Modules that emit no
Frame are named explicitly rather than skipped silently.

The guard tripped on itself twice while being written — first on its own
doc comment, then on its own string literals — so it strips line comments
and excludes its own module. A check that matches prose about the defect
rather than the defect is the same mistake in a different place.
2026-08-07 13:37:27 -07:00
Matthew Jackson bca886a9c3 changelog: tighten 1.6.1, and record two entries it was missing
The entries had grown into narrative paragraphs; each is now the outcome
first and the mechanism in a sentence or two. The website changelog page
mirrors libfreemkv's, so it inherits this directly.

Two things that shipped in this cycle were not in it at all:

dir:// as a source. It was pulled from 1.6.1 on 2026-08-05 because the
folder reader produced a wrong title list on a real DVD. That defect was
the CSS descramble bug below, fixed since — the acceptance suite now
shows a folder reporting the same 38 titles as its ISO, with matching
streams, languages and runtime, and the CLI has been wired for it all
along. It ships, so it is listed.

The CSS title-table corruption itself: a disc enumerating 38 titles
produced a decrypted image enumerating 10, silently, at exit 0. That is
a data-integrity fix a user needs to know about.
2026-08-07 13:15:36 -07:00
Matthew Jackson 7cda914f29 Stamp the last frames that were still leaving without provenance
A real rip caught these. The warning added for a track placed from
timestamps under a seam plan fired once, on one subtitle track of a
23-clip title, eighteen minutes in — a display set that arrives as a lone
non-PCS segment is emitted straight through rather than accumulated, and
that path still built its frame with source: None.

Auditing the rest the same way found four more: flac, lpcm, mpegaudio and
the passthrough parser. All are one-PES-one-frame, so the packet's own
facts are the unit's facts.

Every Frame construction in every codec parser now carries a source,
checked by walking balanced braces rather than by eye — the earlier
count was taken with a regex that cannot see a block containing nested
braces, which is how these survived the first pass.
2026-08-07 10:01:39 -07:00
Matthew Jackson 11538981de Place a non-advancing mark table by provenance
Marks that do not advance across a title are normal, not a defect: each
clip file carries its own STC, so one clip's IN has no ordering
relationship to the previous clip's. Refusing those tables dropped
exactly the branched titles this type exists for onto the inference path
that cannot read them, and that is the overrun.

Lifting it was tried once before every track carried a source offset, and
nine audio and subtitle tracks with nothing to place them by pinned
themselves to clip 0 and dropped most of the title. Every parser stamps
provenance now, so the clip comes from the byte offset and the marks keep
one job: whether a frame lies inside its own clip's [in, out].

Still gated on spans_trusted. Without usable spans there is no offset to
place by and inference is all that is left, so a table inference cannot
read is still refused rather than silently truncating a title.

A frame reaching the timestamp path under a plan now says so once per
track. It is not expected any more, and it is how a mostly-dropped track
reached a user without a single line in the log.
2026-08-07 07:49:53 -07:00
Matthew Jackson 8edb53942c Every parser now carries provenance, on the one shared buffer
dts, ac3 and truehd assembled access units across PES packets with three
private implementations of the same bookkeeping, and none of them carried
the source byte offset. They now hold a PesBuf, so a unit takes the
timestamp AND the source of the packet covering its first byte, from the
same mark, and no codec can answer that question its own way again.

dts is the reference case: its pts_marks already implemented the rule
correctly, and all 61 of its existing tests -- including the PTS
attribution ones -- pass unchanged on the shared type. That is the
evidence the type preserves the behaviour dts had right.

ac3 kept a single carry-over timestamp and one anchor offset, so it could
only attribute the first unit in a call; it now resolves each unit at its
own offset. truehd had no attribution at all beyond a running clock.

New tests cover the case that motivated this: a unit whose first bytes
arrive in one packet and whose remainder arrives in the next keeps the
FIRST packet's offset. At a clip seam those two packets belong to
different clips, and taking the later one places the audio in the wrong
one.

Clippy on the pinned toolchain caught an empty `if` block left where dts
used to clear stale marks -- restored as an explicit clear, with why it
is still needed once drain keeps the covering mark.
2026-08-07 07:45:32 -07:00
Matthew Jackson c718cd9e9e One rule for which packet a unit's facts come from
A parser that assembles an access unit across PES packets has to answer
one question for every unit it emits: which packet carried this unit's
FIRST byte? Its timestamp comes from that packet, and so does the source
byte offset that says which clip of a multi-clip title it belongs to.
The packets that complete the unit carry later values that must not
override it.

That question was being answered three ways. DTS kept a deque of
(offset, pts) markers and took the one covering offset 0. AC-3 kept a
single carry-over timestamp. TrueHD kept its own. PGS and DVD subtitles
each held a pending unit with just a start time. None of them carried
the source offset at all, which is why provenance existed only for video
and why nine audio and subtitle tracks on a branched title had nothing
to place them by.

So it lives in one place now. PesBuf owns the bytes AND the marks;
PesFacts returns a packet's timestamp, source and discontinuity
together, so a parser cannot take one from one packet and another from
the next, because it does not assemble them itself.

The type answers WHICH packet. How a codec reads a timestamp out of that
packet stayed the codec's business at first, and that turned out to be
the same drift one level down: dvdsub read pts alone and returned 0 for
a packet carrying only dts, while everything else took pts.or(dts). Now
there is one derivation. It is not a choice between two fields — for
audio and subtitles there is no reordering, so dts IS the presentation
time and reading it is reading the same value from whichever field the
packet used. Reordering video never calls it; that path reconstructs
display order instead.

Migrated: adts, pgs, dvdsub. dts, ac3 and truehd follow.
2026-08-07 07:19:58 -07:00
Matthew Jackson e32f874387 Restore the refusal: only the video parsers carry provenance
Accepting a table whose marks do not advance was wrong, and a real
22-clip title showed exactly how. The rip failed the drop-volume gate at
84 percent with more frames dropped than kept.

The cause is a fact the code never stated: provenance is VIDEO-ONLY.
Every audio and subtitle parser -- dts, ac3, adts, truehd, pgs -- builds
its frames with `source: None`, while hevc, h264 and vc1 propagate it.
So on that title the video track placed correctly by byte offset and
nine audio and subtitle tracks arrived with nothing to place them by.

The branch added to keep an unprovenanced frame from stranding did the
stranding itself. It held such a frame on its track's current clip,
reasoning that the cursor only ever advances under provenance and so
could not be wrong -- which is true only for a track that eventually
receives some. A track that never receives any stays pinned to clip 0
for the whole title, and every frame past clip 0's OUT mark is dropped.
That is what the disc showed: nine tracks pinned to clip 0, the first
drop nine milliseconds past its OUT.

So the refusal goes back, now with the real reason recorded, and the
test asserts it rather than asserting the behaviour that failed.

Lifting it needs the audio and subtitle parsers to carry provenance
first, stamped from the PES that STARTED each access unit -- the same
rule au_assembly already applies to video. The diagnostics that found
this are kept.
2026-08-07 03:21:21 -07:00
Matthew Jackson b1f9adc82b Say which frame and which marks caused a provenance drop
The heuristic path has always logged its first drop per track with the
frame's timestamp and the clip marks it was judged against. The
provenance path did not log at all, so a title that dropped MOST of its
frames there failed the volume gate without a single line saying which
frame, which clip, or which marks — a full-length rip produced zero drop
events and an error, which is not a diagnosable failure.

Also pins the all-identical-spans hazard as a test. Every PlayItem
referencing one clip file gives every clip the same span, which passes
the tiling check and is therefore "trusted" while carrying no
information at all about which PlayItem a byte belongs to. That is not
what the discs on hand do — theirs have one distinct span per clip — but
the check conflates "the spans tile" with "the spans distinguish", and
only the second justifies placing a frame by its byte offset.
2026-08-07 03:05:32 -07:00
Matthew Jackson 16a894f835 Say how many DISTINCT feed spans a title's clips share
A seamlessly branched title re-references one clip file from several
PlayItems with different mark ranges, and those references share a single
feed span because the bytes are read once. A byte offset alone cannot
then tell them apart.

Whether that is happening on a given title is the fact that decides
whether provenance can identify a clip on its own, and it was not
observable. distinct_spans < clips says it directly.
2026-08-07 02:45:57 -07:00
Matthew Jackson 9bcd752216 Say which placement strategy a title got
Whether a title was placed by the seam plan or fell back to inference is
the single most useful fact about a branched rip, and it was invisible.
The two looked identical in the output, so telling them apart meant
rebuilding and re-ripping — which is exactly what it cost to find that
most branched discs were silently on the inference path.

Log both flags once at plan construction, and log each refusal with the
reason that caused it.
2026-08-07 02:01:47 -07:00
Matthew Jackson c1cd2e4b06 Place a restarting-clock playlist by provenance instead of refusing it
A playlist whose clips each restart their own STC has marks that all
cover the same low values, so they are not points on one title-wide
clock. from_clips refused those tables outright and fell back to PTS-jump
inference.

That refusal predates provenance, and it is only about inference. With
the feed spans tiling the title the clip comes from the frame's byte
offset and the marks are never read across clips: each clip's offset_ns
maps its own private clock onto the output timeline, which is exactly
the right operation for a restarting STC.

Refusing unconditionally therefore turned the seam plan off on most of
the branched discs on hand — the very titles it was written for. They
ran on inference with a timeline minutes past the title's real length:
one 2h29m title came out with its audio and video both spanning 3h15m.

So the refusal now applies only when the spans cannot be trusted, where
it is still load-bearing: without a usable clock or a usable byte offset
there is nothing to place with.

Accepting these tables opens one hole, and it is the same stranding the
refusal guarded against. A frame arriving with no byte offset would fall
through to the mark heuristics, which on this kind of table are
meaningless and can strand a track on a clip it has already left,
dropping the rest of the title. Such a frame now stays on its track's
current clip; the cursor only ever advances under provenance, which is
never wrong about which clip a byte came from.
2026-08-07 01:17:04 -07:00
Matthew Jackson cbafe71769 Resolve a frame clip by binary search, not a linear scan
The provenance lookup runs once per frame per track. A linear scan over the
clip list is fine for the 11-PlayItem fixture and is not fine for the real
hoard: two discs there carry 900 PlayItems in one title, and several carry
250-450, so a scan would be hundreds of comparisons on every one of millions of
frames.

spans_trusted already guarantees the spans tile the feed contiguously and in
order, which makes this a partition point. A repeated clip reuses its first
reference span, so the search walks back to the FIRST entry sharing it — the
bytes are read once, so the material is emitted once, at that entry offset, and
the answer is stable regardless of which duplicate the partition lands on.

Also adds a test at the real 900-clip scale: every clip places, output stays
monotonic across all 900, an arbitrary byte resolves to the right clip out of
ascending order, and a byte past the end belongs to no clip.
2026-08-06 23:22:38 -07:00
Matthew Jackson 3c1375a245 Identify a frame clip by provenance, not by inferring it from timestamps
Four audit rounds each fixed one rule in SeamPlan::place and broke another,
because the question the rules were answering has no answer. Inside a
seamless-branching overlap clip k OUT comes AFTER clip k+1 IN — 57.8s of
overlap on the real fixture table — so a single timestamp is legitimately
inside two clips, and a clip file is not trimmed to its marks, so it also
carries material from before its own IN. No rule over timestamps can say which
clip a frame came from, and each attempt was right for one disc layout and
silently wrong for another: 65s of rewind, 17 minutes stranded, 28 minutes
dropped, 55s refused.

Frames already carry the byte offset they were read from (PesFrame::source,
stamped by the TS demuxer). Clip now carries the byte span its stream occupies
in the title feed, recorded while the extents are gathered. So the clip is a
LOOKUP: the offset falls in exactly one span. There is no decision to get wrong.

Every track of a clip lives in the same stream file and therefore shares one
span, so video, audio and subtitles agree by construction. Divergence between
them — each track guessing separately under its own tolerance — is how audio
and video ended up on different clips and drifted apart in the first place.

spans_trusted gates the whole path: unless the spans tile the feed contiguously
from zero, an offset means nothing and provenance is ignored in favour of the
mark heuristics, which is the 1.6.0 behaviour. A broken map degrades instead of
confidently selecting a wrong clip for every frame. A clip referenced twice
reuses its first span (the bytes are read once) and is still trusted.

Sources that stamp no provenance — a mkv:// remux, the deserialize hop — take
the heuristics, which is what they have always used and where they have always
been right, because they have no overlapping clips to be ambiguous about.

36 timeline tests, six of them new and covering: the overlap case marks cannot
see, all tracks agreeing, out-of-marks material dropped AND counted, a holed
span map, a discontiguous one, a repeated clip, and no provenance at all.
2026-08-06 23:08:13 -07:00
Matthew Jackson 651bfaaa2e Say when an entry is omitted for not being a plain file
A symlink to a directory — a normal way to keep tens of gigabytes of streams
off the system disk — was dropped from the plan silently. `entry.file_type()`
reports the LINK, so such a subtree never enters `dirs`, and `metadata()` then
follows it and reports a directory, so it fails the is_file() test and was
skipped with nothing said.

The result is the worst class this release exists to close: PLAYLIST and
CLIPINF still synthesize, so the folder scans and enumerates titles, every clip
resolves to no extents, the mux takes its clean-EOF path, and a near-empty MKV
is written at exit 0.

It is still skipped rather than followed — following link targets invites
cycles and escapes from the folder — but at the same volume as the
unrepresentable-name skip eight lines above.
2026-08-06 22:48:27 -07:00
Matthew Jackson cce4e5effe Enforce the no-rewind invariant instead of guessing at clip crossings
Round 7 enumerated the state space of SeamPlan::place and found three more
holes in the mark heuristics — the third consecutive round to find a defect
here, each time in the guard the previous round added:

- a clip file opening ON its own IN mark, after a tail frame past the previous
  clip OUT had already advanced the cursor, read as another crossing and jumped
  a clip too far: 28 minutes of clip 6 dropped;
- the last clip was unguarded entirely (`clip + 1 < len` skipped the check), so
  the 9->10 seam rewound 75.6s with dropped_total() at 0 — the exact round-5
  signature, reachable without any corrupt input;
- a glitch mid-clip satisfies "inside the current clip", so the round-6
  stranding survived for the rest of that clip.

Three patches to the same heuristic, three wrong in a new way. So stop patching
it. Everything above the placement is still heuristics over marks, but the
property they exist to serve is now checked DIRECTLY: a track output must not
run backwards. A placement that would rewind by more than
DISCONTINUITY_BACKSTEP_NS is refused; the scan then looks for a LATER clip
containing the frame that does not rewind, which is exactly what a genuine
overlap crossing looks like; and if none exists the frame is dropped and
counted, because emitting it anywhere moves the track backwards.

The tolerance is the backstep constant because B-frame reorder legitimately
emits out of order by a fraction of a second, while a rewind that matters is
orders of magnitude larger.

The cursor moves ONLY when a later clip actually accepts the frame. My first
version of this advanced on a failed search, which strands a track exactly as
before — caught immediately by the round-6 regression test.

All 29 previous timeline tests still pass, including
continuity_preserves_forward_gap. The three round-7 scenarios now have a test
each, with the real 00801.mpls numbers.
2026-08-06 12:31:23 -07:00
Matthew Jackson 8e6f1511ea Round 6: a large backstep only crosses when it would be misplaced
The round-5 fix advanced the clip cursor on ANY backward step over 3s. That
also fires for a corrupt PTS, and for a legitimate STC discontinuity inside one
clip — and nothing moves the cursor back, because a forward step matches
neither past_out nor stepped_back. Every later frame then sits below the new
clip IN and is dropped: on the fixture table that is ~17 minutes of one track
gone, and the only volume gate compares total drops against ALL tracks frames,
so it exits 0.

The branch now also requires the frame to be INSIDE the current clip marks,
which is the only case that would otherwise be silently placed at the old
offset — the rewind. A frame outside them needs no help: the containment check
drops and counts it, the cursor stays put, and the next good frame is placed
normally. Both behaviours have a test, each confirmed to fail without the guard.

Separately, the dir:// PES input path scanned the folder and never applied the
encryption verdict scan_dir exists to produce, so the same folder ripped
through one door and failed through the other asking for a key it does not
need. That logic now lives in one function, session::apply_folder_encryption_verdict,
called by both.
2026-08-06 11:02:48 -07:00
Matthew Jackson d68d73e68d Retry a short read instead of skipping past it
The previous commit bounded the crack scan inspection by the bytes actually
read, which stopped it examining stale buffer bytes from an earlier batch. But
the cursor still advanced by the REQUESTED count, so those sectors were skipped
outright — trading "scans the wrong data" for "silently scans less than it
thinks", on exactly the damaged media where a title key is hardest to find.

The cursor now advances by what was read, so the next iteration resumes where
the read stopped. Floored at one sector so a source returning Ok(0) cannot spin.

Found by reading the fix again rather than by the next audit round.
2026-08-06 10:15:53 -07:00
Matthew Jackson 648bcce5b6 Scan only the sectors the crack actually read
read_sectors returns the number of bytes written and a source may return Ok
with fewer than asked — a recovery read over a damaged region does exactly
that. The crack loop discarded the count and inspected all n sectors of a
buffer that is REUSED across batches, so the tail still held the previous
batch sectors.

Cracking a key from those means cracking from data belonging to a different
extent, possibly a different VTS, while crack_span records the CURRENT one. A
key that opens nothing in this region is then installed, and its wrong
descrambles are only partly caught by the per-sector crib.
2026-08-06 09:31:37 -07:00
Matthew Jackson 45af2152a8 Make the public per-sector descramble refuse a non-pack sector
descramble_region was fixed to require the MPEG-2 pack start code, but
descramble_sector — the PUBLIC per-sector entry point, and the one the
module-level example tells callers to use — still keyed on the byte 0x14 flag
bits alone. The crate own documented guidance therefore led straight back into
the defect this release exists to fix: a VIDEO_TS.IFO sector holding 0x15 at
0x14 while starting 00 26 00 00 loses 1912 of its 2048 bytes, and because that
sector carries TT_SRPT the disc enumerates 38 titles while an image decrypted
from it enumerates 10, at exit 0.

It has no callers inside the crate, which is exactly why it survived three
rounds: nothing exercised it. Putting the guard inside the function rather than
in each caller is what keeps the safe path the easy one.

The integration test that covers it built its sector from the flag byte alone,
which no real scrambled sector looks like, so it stopped representing the path
it names — the same fixture-realism gap already fixed in four other places this
release.
2026-08-06 09:26:15 -07:00
Matthew Jackson abbc725acf A clip join must never rewind the output timeline
A hole in the seam fix itself, found in audit round 5 and confirmed by
measurement against the real 00801.mpls marks already in the test file.

Nothing upstream trims a clip stream to its marks, so a clip file can open with
material from BEFORE its IN. With the cursor on clip 5 (7708.99..7910.79) a
frame at 7845.00 — clip 6 pre-mark lead-in, 8s below clip 6 IN of 7853.00 —
sits further from that mark than the 250ms tolerance, so neither crossing rule
fires. The cursor stays on clip 5, and 7845.00 IS inside clip 5 range, so the
frame is PLACED with clip 5 offset.

Measured: output went from 3714.51s to 3649.51s. Backwards 65 seconds, with
dropped_total() still 0 — no counter, no gate, nothing noticed — while the
whole 57.8s overlap band was emitted a second time over clip 5 written tail.
That is the collide-and-flatten symptom this type exists to remove.

A track own PTS only ever runs forward inside a clip, so a backward step larger
than DISCONTINUITY_BACKSTEP_NS is always a clip change. Such a frame now
advances to the first clip that could contain it — which terminates on its own,
rather than running to the end of the list, which is what the next_in bound
exists to prevent — and a frame still below that clip IN is material the
playlist excludes, so the containment check drops and COUNTS it.

All 27 pre-existing timeline tests still pass, including
continuity_preserves_forward_gap, which is the one that catches an over-eager
heuristic here. The new test asserts the output never moves backwards at a
join and was confirmed to FAIL before the fix.
2026-08-06 09:20:39 -07:00
Matthew Jackson 2e3ba374f3 Test the demux seam gates too
The same gap as the MKV muxer: DemuxSink builds its timeline with
TimelineContinuity::with_clips, but no test ever handed it a title carrying
usable PlayItem marks, so frames_mapped and dropped_total() were always zero
and both gates in finish() were unreachable.

Deleting either left the whole suite green while a demux:// export of a
seamless-branching title wrote a directory of zero-byte track files beside a
populated chapters document and reported success at exit 0.

The test gives the title two clips with marks that exclude every frame, and was
confirmed to FAIL with the gate removed.
2026-08-06 09:05:30 -07:00
Matthew Jackson 84b4a6ddae Test the seam gate that stops a drift-broken title being skipped
Audit finding: both seam gates in MkvMuxer::finish were dead code under test.
set_clips was never called anywhere in the suite, so continuity.dropped_total()
was always zero and neither branch could be reached.

The ordering of the two zero-frame checks is load-bearing and the comment says
so: a title the seam plan emptied also has a zero frame count, and MkvInvalid
is classified by is_skippable_title_stub as an empty nav/menu stub — so
reporting it that way makes an all-titles rip drop a real feature and finish the
rest at exit 0. Swapping the checks left the entire suite green.

The new test builds a two-clip plan (from_clips needs at least two, strictly
increasing) whose marks exclude every frame, and asserts both that the error is
SinkWroteNothing and that it is NOT classified as skippable. Confirmed to FAIL
with the two checks swapped.
2026-08-06 08:45:58 -07:00
Matthew Jackson 3f4bf3f290 Round 3: fix three defects introduced by the round-2 fixes
Auditing my own fixes found all three. None were in the original code.

The VTS crack sort was byte-wise case-SENSITIVE while the filters that select
those files (vts_group_of / is_title_vob) are case-insensitive. On a
case-sensitive volume a set holding vts_01_1.vob beside VTS_01_2.VOB sorted
part 2 first, because V (0x56) precedes v (0x76) — reintroducing exactly the
budget-exhaustion the ordering exists to prevent. Now sorted on the same
uppercase normalisation the filters apply.

Refusing a name that round-trips to empty aborted the WHOLE plan. A sidecar
folder named with a single emoji made a backup un-rippable that 1.6.0 handled
fine, and reported it as a collision with a file that does not exist. It is now
skipped with a warning: the entry is unaddressable either way, but one
irrelevant file should not cost the user their rip.

The mtime check now applies only to files whose CONTENT the plan read — the
IFOs, whose bytes 0xC0/0xC4 place every VOB. Everything else is planned from
size alone, which is already checked, so comparing mtime there bought nothing
and risked a real false positive: disc backups commonly live on exFAT/FAT32,
which stores local time, so a long rip spanning a DST transition would see a
whole-hour shift on an untouched multi-gigabyte VOB and abort hours in.
2026-08-06 08:25:54 -07:00
Matthew Jackson edb6f0035b Catch names the reader cannot tell apart, and in-place content changes
Two dirimage findings, both silent-wrong-output.

The per-directory uniqueness check compared raw host names, but the reader does
not see raw host names: parse_udf_name trims leading and trailing whitespace
and drops any code unit char::from_u32 rejects — which is every half of the
surrogate pairs the encoder emits for non-BMP characters. So " 00000.m2ts" and
"00000.m2ts", or "A<astral>.m2ts" and "A.m2ts", were two entries at plan time
and ONE name at read time. find/read_file take the first match, so a title
resolved to the wrong file extents and muxed the wrong bytes at exit 0 — the
exact shadowing DirNameCollision exists to prevent.

The key is now derived by round-tripping the name through the very encoder and
parser that will be used, so it cannot drift from them. A name that survives
that round trip as empty is refused outright: it would exist in the image and
be addressable by nothing. The new test builds a real folder, calls plan, and
was confirmed to FAIL with the fix reverted.

Separately, the plan-vs-read revalidation compared file LENGTH only, while the
plan depends on CONTENT: a DVD VOB placement comes from bytes 0xC0/0xC4 of its
IFO, and IFOs occupy a whole number of sectors so an in-place rewrite keeps the
length. A re-authoring tool touching the folder mid-rip would pass the size
check while every title extent pointed at stale sectors. mtime is now compared
alongside size, and only when both sides report one, so a filesystem without
timestamps falls back to the old behaviour rather than failing every read.
2026-08-06 08:00:23 -07:00
Matthew Jackson 5fc28056c3 Anchor the folder encryption probe on a unit boundary
probe_folder_encryption sampled AACS units starting from the largest extent
anywhere in the disc. AACS units are 3 sectors and a unit boundary is only
guaranteed at the START of a clip, so that anchor is only correct when the
largest extent happens to be a clip first extent.

For any clip over ~2 GiB it is not. The planner caps an allocation descriptor
at MAX_AD_BYTES = 524287 sectors, so every full piece of a split file ties on
sector_count and max_by_key returns the LAST tie — an extent beginning
(k-1)*524287 sectors into the file. 524287 % 3 == 1, so that start misses the
unit boundary for two file sizes in three.

The 6144-byte sample windows then begin mid-source-packet and the byte read as
the CPI flag is content. Both verdicts fail in a costly direction: a decrypted
folder is rejected as encrypted (DirImageEncrypted on something perfectly
rippable — the exact case scan_dir was added to rescue), or real ciphertext
reads as clear and the mux writes it out as video at exit 0. is_unit_aligned
cannot catch it: it measures against the same wrong base.

Now the largest TITLE first extent, which is unit-aligned by construction and
is also the more meaningful sample — the main feature rather than whichever
fragment happened to be biggest.

The existing tests could not reach this: their fixture m2ts is 786,432 bytes,
a single extent, which is always its own first.
2026-08-06 07:37:59 -07:00
Matthew Jackson 51de027d15 Order a VTS crack by filename, not by directory order
Round 1 removed a largest-first sort from resolve_vts_key because
largest-first is the 1.5.1 garbage bug. Auditing that fix showed it was only
half right: `planned` comes from walking the UDF directory, which yields File
Identifier Descriptors in on-disc authoring order with nothing sorting them. So
deleting the sort did not restore playback order, it left the order undefined —
whatever the disc happened to list first.

DVD-Video numbers a title sets VOBs in playback order by spec (VTS_xx_1.VOB ..
VTS_xx_9.VOB, single digit), so ascending filename IS playback order and is
deterministic regardless of how the directory is laid out.

Order decides correctness here: crack_key shares one sector budget across the
whole extent list, a CSS DVDs biggest cell opens with a long clear run, and CSS
recovers the title key from scrambled data itself. Starting in the wrong place
can exhaust the budget without ever meeting scrambled data, whereupon the
caller falls back to the disc-wide key and the whole VTS is descrambled wrongly
— corrupt PES behind an intact header, written out at exit 0.
2026-08-06 07:31:35 -07:00
Matthew Jackson 96282ecda7 Audit round 1: playback order, silent title drops, image durability
Four fixes from the first audit round. Every finding was verified against a
pinned tree and read directly before being accepted.

resolve_vts_key sorted a VTS title-VOB extents largest-first. That is the 1.5.1
garbage bug, and it grew back in a new code path: the comment claimed it
"matched the scan heuristic", but that heuristic WAS the bug and had already
been fixed in decrypt_keys_for_title, which documents the rule (PLAYBACK ORDER,
never largest-cell-first) and pins it with a regression test. A CSS DVDs biggest
cell opens with a long clear run and crack_key shares one sector budget across
the extent list, so starting there can exhaust it without ever MEETING
scrambled data — and CSS recovers the key from scrambled data itself. The crack
then returns None, the caller falls back to the disc-wide key, and every VOB in
that VTS is descrambled wrongly: corrupt PES behind an intact header, written
out as a complete extract at exit 0.

parse_pgcit dropped titles silently in THREE places — an unparseable PGC, an
out-of-range PGC index, and a truncated entry table. The finder caught one; the
other two turned up on reading the function. parse_vmg already counts and warns
per skipped title SET for exactly this reason, and this was the last place a
disc could quietly report fewer titles than it has.

write_image called flush() and returned Ok. flush() only pushes bytes into the
page cache and promises nothing about durability, so a 6-90 GB image could be
reported complete while still unwritten — a crash or an unmounted volume then
leaves a truncated file the caller was told was finished. Now into_inner (so a
buffered-write error surfaces instead of being dropped by BufWriter::drop)
followed by sync_all.

timeline used abs() on a saturating_sub result. Every other comparison in that
module is saturating because the timestamps come off a disc and are not
trusted; abs() panics on i64::MIN, which saturating_sub can produce.
2026-08-06 07:11:52 -07:00
Matthew Jackson 6e2ee9e002 Rename is_scrambled to has_scramble_flag_bits
The 1.6.1 DVD fix removed the USE of the loose predicate but left the
predicate sitting there with the better name. Anyone asking "is this sector
scrambled?" finds is_scrambled before is_scrambled_pack, and reintroduces the
defect that destroyed 1912 bytes of a real VIDEO_TS.IFO — the sector carrying
TT_SRPT — so the disc enumerated 38 titles and an image decrypted from it
enumerated 10, silently, at exit 0.

Byte 0x14 only means "scrambling control" inside an MPEG-2 pack. In an IFO,
UDF or ISO 9660 sector it is whatever that format stores there. The new name
says what the function actually tests and nothing more, so the honest question
has the obvious name and the dangerous one has to be asked for deliberately.

Its doc comment also claimed a caller, decrypt::decrypt_sectors, that does not
exist — so the name was an invitation and the documentation was an argument
for accepting it. It has no production callers at all; it stays public because
an integration test asserts the flag extraction directly.
2026-08-05 22:27:04 -07:00
Matthew Jackson 0f8035b15a Scope the qa matrix to the platforms each crate actually supports
The first cut ran release-profile tests on all three platforms everywhere,
which invented coverage no crate had ever claimed. bdemu is Linux-only — its
ci.yml has no macOS or Windows job at all, and its tests call
ExitStatus::signal(), which does not exist on Windows, so they do not compile
there let alone run. The rest follow the policy ci.yml already set: tests
execute on Linux and macOS, Windows compiles them so the cfg(windows) halves
still get codegen before release time.

That policy is now explicit in qa.yml as a windows-build job rather than
implied by which jobs happen to exist.

Recorded, not papered over: running the suites on Windows DOES find real
defects — autorip has five Windows path-handling failures in mover.rs and
freemkv-engine has three tests hardcoding /dev/null. freemkv ships a Windows
GUI, so they matter. They are pre-existing rather than 1.6.1 regressions, and
fixing them is its own piece of work.

Also build the binary before cli-parity.sh. cli-integration.sh builds its own;
cli-parity.sh only checks for one and exits 2 with "build it first", so both
legs died before running a single case.
2026-08-05 21:22:30 -07:00
Matthew Jackson f67fd92b60 Give Windows a free-space gate, and stop two tests timing the scheduler
Three release-profile failures on macOS and Windows, all found by the qa gate
on its first run. Release-profile tests on those platforms had never run
before it existed, so none of these were regressions — they had simply never
been visible.

available_space returned None on every non-unix target. That did not merely
skip a test, it skipped the GATE: a Windows user extracting a disc to a full
volume got a confusing failure part-way through instead of a clear refusal up
front, and Windows is where the GUI ships. GetDiskFreeSpaceExW is declared
directly against kernel32, matching how scsi::windows already reaches Win32
rather than pulling in a binding crate for one call. It asks for
FreeBytesAvailableToCaller, which accounts for per-user quotas — the same
question f_bavail answers on unix.

The other two asserted on wall-clock timing with no margin:

- sleep_until_halted_wakes_mid_sleep bounded the wait at 350 ms and measured
  377 ms on a loaded runner. That bound measures the scheduler, not the wake.
  What the test is for is distinguishing "woke because the flag flipped" from
  "woke because the 10 s timeout expired", and 2 s does that just as well.

- abandon_loses_to_a_close_already_committed released at 600 ms against two
  300 ms grace windows plus a 250 ms poll cadence, so the windows could expire
  first and the caller abandoned — a race, not a defect. The intervals are
  scaled up so jitter is small relative to them; the ordering under test is
  unchanged, only the margin.
2026-08-05 21:20:29 -07:00
Matthew Jackson 7ced0d9b61 Stamp every qa push as a release candidate
Every push to qa now tags v<version>-rc<N>, N incrementing, before the gates
run. That answers "which build is on qa, and is it the one I tested?" without
anyone having to remember it.

The tag lands whether the run goes green or red, deliberately. A red candidate
needs a name more than a green one does: "rc3 failed release-tests on windows"
is a sentence you can act on, "qa is red" is not. Red on qa is the gate doing
its job — the branch saying this is not production worth yet.

release.yml now excludes v*-rc*. Its trigger was v*, which matches the
candidate tags, so without this every push to qa would have built and PUBLISHED
a GitHub release — including for the candidates that failed.
2026-08-05 21:03:01 -07:00
Matthew Jackson c2c00a874c Add the qa gate: dev for speed, qa for proof
dev -> qa -> main, across every public repo.

dev is where work lands and is meant to be pushed to often, so ci.yml stays
the fast answer: fmt, clippy, unit tests, leak-guard. qa is the release
candidate, and qa.yml is the claim that a commit is production worth —
release-profile tests on all three platforms, and the Linux cross-target
clippy that the local precommit gate has always run but CI never did.

Release profile matters as its own gate: overflow checks are off, debug_assert
is compiled out, and inlining changes what the optimiser can prove. A test
that only passes in debug never guarded the binary anyone ships.

qa.yml checks siblings out at qa rather than dev. This is not a monorepo, and
a qa run resolving its dependencies from unreleased dev tips would go green on
a combination that is not the one shipping — the exact mismatch the branch
model exists to prevent. ci.yml now tracks whichever branch triggered it for
the same reason, since it also fires on qa.

The consequence is that repos move to qa together, in dependency order. qa is
backfilled from main so an unchanged crate still presents working code to the
crates built on it.
2026-08-05 20:59:34 -07:00
Matthew Jackson 87463ed3ee Require a pack start code before descrambling a sector
css::is_scrambled reads bits 4-5 of byte 0x14 and nothing else. That is a
sound test once a caller has committed to a title's VOB data, where every
sector is an MPEG-2 PS pack and byte 0x14 always means what it says.
descramble_region is not such a caller: it is handed arbitrary regions of a
disc, so it also sees IFO, UDF and ISO 9660 sectors — raw structures where
byte 0x14 is whatever that format happens to store there.

Measured on a real disc: the second sector of VIDEO_TS.IFO holds 0x15 at
offset 0x14 while starting 00 26 00 00, which is not a pack. The flag test
read it as scrambled, descrambled it, and destroyed 1912 of its 2048 bytes.
That sector carries TT_SRPT, so the title table went with it — the disc
enumerated 38 titles and an image decrypted from it enumerated 10, silently,
at exit 0.

is_scrambled_pack already existed with the right predicate. Use it here. It
costs nothing: a genuinely scrambled VOB sector always carries the pack start
code, and no IFO sector does.

Verified end to end — the decrypted image's `info` output is now identical to
the source disc's, 38 titles both, differing only in the CSS: Encrypted line.

The fixtures moved with it. Four of them built a sector by setting byte 0x14
alone, which no real scrambled sector looks like; they now build packs.
2026-08-05 20:59:14 -07:00
Matthew Jackson 0c5e419b07 Report the resolved extent alongside each IFO read
The diagnostic hashed IFO contents but not where they came from, so a
content difference could not be told apart from a path resolving to a
different place. It now prints file_start_lba and file_extents next to the
hash.

That measurement is what inverted this investigation: both a CSS disc and
its decrypted copy resolve VIDEO_TS.IFO to the same extent, and only one
sector of the fourteen differs — by exactly a descrambled payload with the
scrambling-control bits cleared. So the decrypted copy holds the correct
bytes, and an iso:// scan of a CSS disc is parsing a still-scrambled IFO
sector, because scan_iso opens a plain FileSectorSource.
2026-08-05 20:24:20 -07:00
Matthew Jackson 218cd1f6f8 Diagnostics that localise the decrypted-DVD title loss
Two opt-in dumps, driven by FMKV_IMAGE, that narrow where a decrypted DVD
image loses titles:

dump_title_sets_for_an_image reports what survives parse_vmg. On one disc
the CSS image yields 13 title sets and 38 titles; its decrypted copy
yields 8 and 10. Sets 8, 9, 10, 12 and 13 are dropped outright, set 11
parses but returns no titles at all, and set 7 returns 3 of 5.

dump_vts_ifo_reads_for_an_image reads every VTS IFO and hashes the
CONTENT. All thirteen are byte-identical across the two images, so
parse_vts is handed the same bytes and the same TT_SRPT info and still
fails on one of them — the divergence is in what it reads from the READER
afterwards, which is file_start_lba and the PGCIT.

An earlier version of the second dump compared only length and magic and
so wrongly reported the images as identical; it hashes the bytes now.
2026-08-05 20:14:46 -07:00
Matthew Jackson 2c4851e6c1 Say when a title set is dropped from a DVD scan
parse_vmg skipped any title set whose parse failed, with no log and no
counter. A real disc enumerated 38 titles from one image and 10 from
another, and the 28 discarded failures were invisible — the symptom read
as a scan difference rather than as dropped reads, which is most of why
it took so long to localise.

Behaviour is unchanged: a disc may legitimately carry placeholder TT_SRPT
entries, so one failure is still not fatal. It now warns per skip with the
title set and the error, and once at the end with kept-versus-declared.

Also adds an opt-in diagnostic that reads every VTS IFO from an image, to
separate a read failure from a parse failure. It reports all 13 sets
reading identically from both a CSS image and its decrypted copy, which is
what proves the 38-to-10 loss is downstream of these reads.
2026-08-05 19:41:43 -07:00
Matthew Jackson 7511adb64e Two opt-in scan diagnostics for the DVD decrypted-image defect
dvd_placement_invariant_on_a_real_folder checks, per title set, the sum
ifo.rs relies on: file_start_lba(VTS_nn_0.IFO) + vtstt_vobs must land on
VTS_nn_1.VOB. It reports all 13 sets correct on a real DVD folder, which
is what excluded placement as the cause.

dump_titles_for_an_image prints every title a scan produces with the
numbers canonical_title_order sorts on. It is what showed the real shape:
the same disc scans to 38 titles as a CSS image and 10 once decrypted,
with identical capacity and a byte-complete image.

Both are #[ignore]d and read their target from the environment, so they
cost the gate nothing and are there for whoever picks the defect up.
2026-08-05 19:33:58 -07:00
Matthew Jackson 84a6c0d373 Round 5: fix the gates added in round 4, and two placement holes
The zero-frame check ran before the seam gate, and its error is
classified as a skippable nav stub — so a title the plan dropped
ENTIRELY was reported as an empty stub and an all-titles rip would omit
a real feature and finish the rest at exit 0. The seam case is decided
first now, with a code that is not skippable.

The demux sink read a frame's track kind out of the FILTERED slot, which
is empty for a class the export drops. On an audio:// or sub:// export
the video track was therefore called non-video and handed the permissive
crossing rule — the same defect round 4 fixed for a Dolby Vision layer,
reintroduced one file over. Video tracks are now recorded before the kind
filter, beside the primary-video reference that exists for this reason.

Its frame counter counted frames PLACED, not written, while its name and
doc claimed otherwise. Renamed and documented for what it is, including
that it cannot see a single lost track among many.

Placement: files in a subdirectory of VIDEO_TS were never given data.
They were declared at full size with no extents, so they appeared in the
tree and read as nothing. The same folder under BDMV was always placed
correctly. And the duplicate title-set guard keyed on the constraint maps,
so an IFO declaring no offsets inserted nothing and a colliding second IFO
went undetected — it keys on the groups seen now.

Display for SeamPlanDroppedMost and ShortImageRead discarded their
payloads, and four new variants were missing from the code-uniqueness
test.
2026-08-05 18:40:14 -07:00
Matthew Jackson 7cc5b0c93d Test the image-size cap against a real folder
The cap was added without a test. Its companion — a modest oversize is
honoured as a gap — already existed, so this pins the other side: an
offset past the ceiling is refused rather than grown into, which is what
keeps a rewritten IFO from planning a multi-terabyte image.
2026-08-05 18:30:23 -07:00
Matthew Jackson f4fc730d83 Make the drop count gate the verdict, not just the log
Round 3 counted frames the clip marks excluded and reported them at
finish. Counting is not bounding: the only other gate was a global
zero-frame check, and its error is additionally classified as a skippable
nav stub, so a title whose marks do not line up with its PES clock could
discard almost all of itself and still exit 0 — a two-hour feature
emitting seconds, which is the defect this change set already shipped
once. Dropping more than was kept is never a real join, so it now fails.

The demux sink had no zero-output guard at all, so a fully-dropped title
finished cleanly: a directory of zero-byte track files beside a populated
chapters document. It now refuses, keyed on frames having been OFFERED —
a chapters-only export, or a track class the title does not carry,
legitimately writes none, and two existing tests correctly said so.

A title set's placement group is the parsed number, so VTS_01_0.IFO and
VTS_1_0.IFO land on one key and the second silently overwrote the first's
constraint, placing a VOB where the IFO the reader uses does not point.
Refused rather than resolved by arrival order.
2026-08-05 18:25:47 -07:00
Matthew Jackson db0ca3367f Key the seam-crossing rule on reorder, not on driving epochs
A Dolby Vision enhancement layer is a second video track: it does not
drive epochs, but it does carry B-frame reorder. The crossing rule was
keyed on driving epochs, so the EL took the branch whose premise is that
the track has no reorder — and its ordinary reorder dip near the end of a
clip, which during an overlap also lands inside the next clip's range,
was read as a join. The EL was then placed on the next clip's offset, out
of step with the base-layer frame it must be co-timed with by the width
of the overlap: the same desync the per-track cursor was added to remove,
reintroduced for one track.

The property the rule actually depends on is whether a backward step can
be reorder, so it is now keyed on that. Both sinks derive it from the
track kind rather than from the epoch driver.
2026-08-05 17:52:13 -07:00
Matthew Jackson cc870a4614 Keep the seam plan to Blu-ray, and stop a missed crossing truncating a title
Two findings from the same escalation, both silent-wrong-output.

The plan was built for every multi-clip title. Only a Blu-ray PlayItem's
IN/OUT are positions in the clock the PES PTS runs on. HD-DVD fills the
same fields from the XPL's title-relative times and a DVD's come from
cell tables, so a plan built from them is an identity map with a drop
filter: it suppresses the layer-break rebase inference performs, and
drops whatever falls outside marks the PTS was never measured against. An
earlier reading of this called HD-DVD safe because its marks are
contiguous and every computed offset was zero — true, and irrelevant,
because they were zero in the wrong clock. Gated on the content format,
with a test using an HD-DVD-shaped table that the clock check alone
accepts.

The crossing test was also one-shot. A table whose clips restart their
own bases could miss it, and a missed crossing STRANDS the track: every
later frame falls outside the stranded clip's marks and is dropped for
the rest of the title. Counting drops, which is all the previous round
added, does not bound them. A table that is not one advancing clock is
now refused outright and falls back to inference, which is the documented
safe path for those titles.

Also from the same round: read_sectors added an unchecked lba + i, where
callers deliberately saturate their LBAs — a wrap folds the read back to
a low sector and hands the muxer another file's bytes. classify added 1
to two numbers parsed verbatim out of a filename. read_head used a single
read() where a short read on a network mount silently records no
placement constraint at all. And the page-cache eviction added last round
released only the read that crossed its threshold rather than everything
accumulated, so seven eighths of what was read stayed pinned.
2026-08-05 17:31:43 -07:00
Matthew Jackson 211d293aee Make the name-cap tests exercise the planner, not the constants
The round-1 tests asserted arithmetic about MAX_CS0_NAME_BYTES and never
called plan(), so both would have passed with the guard deleted — which
is the failure mode this audit exists to catch, committed by the audit's
own fix. They now build a real folder containing a 255-byte name and
require the planner to refuse it, plus a companion proving a name at the
cap is still accepted so the guard is not merely refusing everything.

The subdirectory cap keeps its arithmetic-only test — creating 65,535
directories is not reasonable in a unit test — but now says so instead of
implying coverage it does not have.
2026-08-05 17:21:43 -07:00
Matthew Jackson abf47137ef Audit round 2 fixes: an unsafe default, four omissions, and two swallowed errors
The folder encryption probe returned "not encrypted" when it had sampled
nothing at all — a title shorter than one aligned unit skipped the loop
entirely. That verdict CLEARS the structural one an AACS directory
raised, so a genuinely encrypted folder would have been ripped as clear
and written ciphertext as video at exit 0. With no evidence it now keeps
the structural verdict, and its bounds arithmetic no longer trusts
disc-derived values not to wrap.

Reading an IFO header swallowed every I/O error and returned an empty
buffer, which sent each placement offset through unwrap_or(0) and
recorded no constraint at all — a permission error on one file produced a
silently misplaced VOB. The directory walk swallowed the same class while
claiming to skip only vanished files. Both now propagate; only NotFound
is skipped.

Four things the round-1 changes left inconsistent: two new error codes had
no doc comments, were absent from the io::Error mapping, printed no path
in Display, and were missing from the test that proves codes are distinct.
The demux sink dropped frames silently while the MKV muxer reported them.
And set_clips had been inserted INTO write_frame's doc comment, leaving
write_frame undocumented and its paragraphs describing the wrong function.

uid/gid used 0 as "not specified"; UDF's sentinel is 0xFFFFFFFF, and 0 is
root.
2026-08-05 17:16:03 -07:00
Matthew Jackson 00332e4dba Bound the synthesized image, and stop it pinning the page cache
A DVD title set records where its VOBS begins as an offset inside its own
IFO, and the planner honours that offset because honouring it is what
makes a real backup readable. Nothing bounded it: a regenerated .BUP or a
hand-assembled folder naming an offset far past the content grew the
image to wherever it pointed — a u32 sector count reaches ~8.8 TB, and
writing that to an iso:// destination fills a disk with zeros before
anything notices. Capped at 128 GiB, which clears BD-100 with room.

Metadata is materialized up front and held for the life of the image, at
a 2 KiB File Entry per node, so the 100,000-entry cap alone permitted
~205 MB of it for content of no size at all — and the mux holds two
images at once while probing. The module claimed a budget of a few MiB;
that budget is now enforced rather than asserted.

Host reads had no page-cache eviction. The ISO source documents what that
costs, measured: an 85 GB read pins the whole file, starves the writer,
and collapses the mux to 2.7 MB/s against 70 MB/s isolated. A folder
source reads host files the same way, so it now uses the same eviction —
the hints move from private-to-that-module to crate-internal rather than
being reimplemented.
2026-08-05 17:10:19 -07:00
Matthew Jackson 6152028271 The VMG's 0xC4 is TT_SRPT, not VMGM_C_ADT
The placement code skips that field for the Video Manager, which is
right, but said it was skipping it because the field is the menu cell
address table. It is the title search pointer table, and it is an offset
inside the IFO rather than a pointer to another file — which is the
actual reason it constrains nothing. ifo.rs reads the same offset under
the correct name, so the two would have drifted.
2026-08-05 17:03:19 -07:00
Matthew Jackson abe464d799 Audit round 1 fixes: sparse-track joins, silent drops, and two encoder wraps
A sparse passive track — a subtitle with no event near a clip's mark —
was held to the dense-video crossing window, so it stayed on the previous
clip's offset until its PTS passed that clip's OUT and every event in
between was mistimed by the overlap. Video keeps the tight window,
because its backward steps are also B-frame reorder; passive tracks have
no reorder, so any backward step into the next clip's range is a join.

Frames the marks exclude were dropped without a trace. Dropping is right
at a join, but this codebase has shipped complete-looking wrong output
before, so the count is kept per track and reported when the mux
finishes, alongside the pre-cluster counter that exists for the same
reason.

A File Identifier Descriptor records its name length in one byte, and the
length was narrowed with a cast: a 255-byte name — POSIX NAME_MAX,
entirely ordinary — encodes to 256 and wrote zero, which would read every
later entry in that directory from the wrong offset. A directory's link
count is 16 bits and was computed as 1 + subdirectory count, which the
global entry cap alone permits overflowing. Both are refused while
planning, where the tree can still be rejected cleanly.

The module and struct docs described inference as the whole algorithm;
they now say which path decides what.
2026-08-05 16:58:49 -07:00
Matthew Jackson 862c4e90ab changelog: dir:// is not in 1.6.1
The folder reader produces a wrong title list on a real DVD — the title
table is read at a sector offset relative to VIDEO_TS.IFO's start, and a
synthesized layout does not put that file where the offset resolves the
same way. Output is silently wrong at exit 0, so the feature is not
wired to the CLI in this release. The library module stays, unreachable,
and ships once both disc families are verified against real content.
2026-08-05 16:44:23 -07:00
Matthew Jackson 7cfeadbcd3 changelog: dir:// as a source, and the multi-clip Blu-ray timing fix 2026-08-05 15:47:47 -07:00
Matthew Jackson 9742fe00fd Cross a clip join per track, not on the video's frame
The first cut placed every track with the cursor the video had moved. At
an overlap join the previous clip's audio is still arriving after video
has crossed, and those tail frames sit inside both clips' mark ranges —
so they took the new clip's offset, jumped forward by the overlap, and
collided with the new clip's own audio, which the muxer's monotonic nudge
then flattened. A remux confirmed it: the timeline length was already
correct and the original symptom was still there, 169 audio packets on
the tick floor.

A track's PTS only runs forward inside a clip, so its own backward step
to the next clip's IN is its crossing. That is per track, so the cursor
is too.
2026-08-05 15:46:40 -07:00
Matthew Jackson e814310a3e rustfmt the seam-plan tests
Line-wrapping only, no behaviour change.
2026-08-05 15:28:40 -07:00
Matthew Jackson 480c873be1 Merge branch 'fix/mpls-seam-timeline' into dev 2026-08-05 15:24:37 -07:00
Matthew Jackson 5043baaebe Place Blu-ray clips by the playlist's marks, not by guessing at PTS jumps
A seamless-branching title's PlayItems do not chain contiguously: one
clip's OUT can sit after the next clip's IN, where the disc stores the
join twice, or before it, where the playlist skips material. The mux
never saw those marks — its own header said so — and inferred seams from
PTS jumps instead.

Inference cannot recover this. A forward jump is ambiguous: it means the
playlist skipped, or it means frames were lost to damaged media, and
compressing the latter would falsify timing on exactly the rips that most
need it faithful. An overlap smaller than the B-frame reorder threshold
is invisible to inference entirely, and its duplicate content then
collided in the muxer, where the monotonic nudge flattened a run of audio
onto the tick floor and put sound ahead of picture for the rest of the
film.

Measured on one 11-PlayItem title: the file declared 7893.385 s, which is
what the playlist says the title is, and carried packets to 8029.298 s.
Both numbers came from the same program on the same disc. Four skips
totalling 135.9 s became dead timeline, and a 1.79 s overlap put audio
1.8 s ahead at the half-hour mark. Five of forty-seven titles were
affected; every single-clip title was exact.

So the marks are read. Each clip contributes exactly out - in, laid end to
end, so the output runs as long as the playlist says and a join never
rewinds. Titles without usable marks — DVD, HD-DVD, file sources — keep
the inference path unchanged, and clips that already chain contiguously
produce a constant offset, which is pinned by a test.
2026-08-05 15:24:31 -07:00
Matthew Jackson 4d63a11d63 Read a disc folder as an input: dir:// becomes a source
Users keep discs as extracted folders — a DVD VIDEO_TS or a Blu-ray BDMV,
usually a backup that is already decrypted. dir:// could only ever be a
destination, so those folders could be produced and never read back.

Everything above the sector layer wants a UdfFs over a SectorSource, and
every UdfFs read re-reads the ICB off that source at call time, so a
folder has to present itself as sectors. It does: dirimage plans a block
layout over the real files, encodes a UDF 1.02 filesystem for the
metadata, and serves data straight from disk. read_filesystem then parses
it exactly as it parses a disc, so nothing above changes — and the
iso:// arm of input() is now shared rather than duplicated, so dir://
inherits its decrypt gates, title selection and stream pruning.

The encoder is validated by more than its own reader: macOS mounts the
synthesized image and the mounted files compare byte-identical to the
originals. A round-trip through our own parser could not have shown that
— the tag CRC seeds at zero, and a wrong seed would satisfy us and no
real driver.

DVD placement is not free packing: a VTS IFO records where its title
VOBS begins relative to itself, so the VOB has to land exactly there.
Unsatisfiable marks fail loudly rather than misplace the file. 3D folders
are refused for now: the scanner detects SSIF and the planner cannot
alias its extents yet, so accepting them would produce quiet nonsense.

Left for later: metadata capture, HD-DVD, FMTS, encrypted folders.
2026-08-05 15:24:18 -07:00
Matthew Jackson 2310e65f44 Make iso:// a destination for any image source, not just a drive
Writing an iso:// meant a raw sector copy off a drive, so an existing
image could only ever be a source. Decrypting one you already had meant
finding the disc again.

The engine's copy is the recovery path — mapfile, multipass, damage-jump,
auto-resume — and all of it exists because optical media returns read
errors. A file does not, so a non-drive source gets write_image instead:
sectors in, bytes out, once, no recovery machinery. Keeping them apart is
not just tidiness. The mapfile identity check compares AACS unit keys and
the VID, both empty for an already-decrypted source, so identity passes
for any such source — a second run with different input to the same
output path would resume over the previous image and report success.

A short read is an error rather than a zero-fill: padding a truncated
source yields an image that looks complete and is not, which is the worst
outcome for a copy someone means to keep.
2026-08-05 14:45:01 -07:00
AnimeFN 4bd1c9210c Fix NTSC chapter/duration drift: dvd_time_t is a timecode, not seconds
`dvd_time_t` stores H:M:S:F as non-drop-frame timecode with a rate flag,
not elapsed wall-clock time. On NTSC the seconds field advances every 30
frames, but a frame lasts 1001/30000 s, so 30 frames occupy 1.001 s of
real time. Reading H:M:S as literal seconds under-reports real time by
exactly 0.1% — 3.6 s per hour, growing with elapsed time, which is what
made chapter marks drift ~4 s by the 67-minute mark.

Convert the whole timecode through an integer frame count and apply the
exact 1001/30000 fraction once, instead of reading H:M:S literally and
dividing only the frame remainder by a decimal 29.97. Chapter marks sum
per-cell frame counts as integers and convert once per chapter, so every
mark lands on an exact frame boundary instead of accumulating f64
rounding across a long title.

PAL is arithmetically unaffected: 25 frames = 1.000 s exactly, so the
old and new paths agree to within one ULP. Sweeping all 900k H:M:S:F
combinations under 10 hours, 12 differ, by at most 8.9e-16 s — the new
path divides one exact integer instead of adding a rounded fraction to a
large one, so where they differ it is the more accurate of the two.

Five existing NTSC fixtures asserted the old literal-seconds values and
were updated to real seconds (each is exactly 1.001x its old figure);
`bcd_secs` now documents that its argument is timecode, not real time.

Fixes #1
Reported-by: AnimeFN <admin@animefn.com>
2026-08-05 13:48:11 -07:00
Matthew Jackson bfce73e73c changelog: write it for users, not for the people who fixed it
Entries had grown into investigative narratives — root causes, disc
counts, PIDs, percentages, function names — which is the right level for
a post-mortem and the wrong one for a public changelog. Each entry is now
one to three sentences leading with what a user observes.

Halves the file (1538 -> 735 lines) without dropping a version. API
signature changes stay explicit, since a consumer migrates against those.
2026-08-03 08:31:04 -07:00
Matthew Jackson ca4e6f5a13 restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-03 05:57:31 -07:00
51 changed files with 9419 additions and 1975 deletions
+26 -20
View File
@@ -2,10 +2,12 @@ name: CI
on:
push:
# `dev` is where work lands and where CI must be green. `main` only ever
# moves at release time, to the tagged commit, so a push to it is the
# release validation run rather than day-to-day feedback.
branches: [main, dev]
# dev -> qa -> main. `dev` is where work lands and is meant to be pushed
# to often: these are the FAST checks, so a mistake surfaces in minutes.
# `qa` is the release candidate — it runs these too, plus the expensive
# suite in qa.yml. `main` only ever moves at release time, to a tagged
# commit that was already green on qa.
branches: [main, dev, qa]
pull_request:
jobs:
@@ -27,7 +29,7 @@ jobs:
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
@@ -65,7 +67,7 @@ jobs:
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
@@ -92,7 +94,7 @@ jobs:
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
@@ -119,7 +121,7 @@ jobs:
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: dev
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
@@ -150,19 +152,19 @@ jobs:
- uses: actions/checkout@v5
with: { path: libfreemkv }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-unlock, ref: dev, path: freemkv-unlock }
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-keysources, ref: dev, path: freemkv-keysources }
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-engine, ref: dev, path: freemkv-engine }
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv-i18n, ref: dev, path: freemkv-i18n }
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v5
with: { repository: freemkv/freemkv, ref: dev, path: freemkv }
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v5
with: { repository: freemkv/autorip, ref: dev, path: autorip }
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v5
with: { repository: freemkv/bdemu, ref: dev, path: bdemu }
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
@@ -185,13 +187,17 @@ jobs:
freemkv
autorip
bdemu
- run: cargo check --all-targets
# `cargo check` alone only proves the dependents still COMPILE against
# this commit. It cannot see a behavioural change — the library keeps its
# signatures and a dependent's tests start failing. That is the shape of
# every defect worth catching here, so run their suites too.
- run: cargo test --tests
working-directory: freemkv-keysources
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: freemkv-engine
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: freemkv
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: autorip
- run: cargo check --all-targets
- run: cargo test --tests
working-directory: bdemu
+133
View File
@@ -0,0 +1,133 @@
name: qa
# ── The qa gate: "is this production worth?" ────────────────────────────────
#
# dev -> qa -> main.
#
# `dev` is for committing often. ci.yml answers "is it green" in minutes with
# fmt, clippy and the unit suite, so a mistake surfaces while it is still cheap
# to fix. `qa` is the release-candidate branch, and THIS workflow is the claim
# that a commit is production worth: everything expensive that can run without
# physical media. `main` only ever receives a qa that went green here.
#
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. Those run on a
# self-hosted runner (see the media job at the end) and are the one leg that
# stays on hardware.
on:
push:
branches: [qa]
workflow_dispatch:
jobs:
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
# v<version>-rc<N>, N incrementing. That is the answer to "which build is on
# qa right now, and is it the one I tested?" — a question that otherwise gets
# answered from memory.
#
# This runs FIRST and does not depend on the gates, deliberately. A red
# candidate needs a name more than a green one does: "rc3 failed
# release-tests on windows" is a sentence you can act on; "qa is red" is not.
# Red on qa is a working gate, not an incident — it is the branch saying this
# is not production worth yet. Fix on dev, get dev green, push qa again.
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Stamp the next rc
shell: bash
run: |
set -euo pipefail
v=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
# overflow checks are off, debug_assert! is compiled out, and inlining
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v5
with:
path: libfreemkv
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --release --tests
working-directory: libfreemkv
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
path: libfreemkv
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
working-directory: libfreemkv
# Windows compiles the tests but does not run them, matching the policy
# ci.yml already set. The value here is codegen: the #[cfg(windows)] halves
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
with:
path: libfreemkv
- uses: actions/checkout@v5
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo build --release --tests
working-directory: libfreemkv
+5
View File
@@ -4,6 +4,11 @@ on:
push:
tags:
- 'v*'
# NOT the release-candidate tags. Every push to `qa` stamps a
# v<version>-rc<N> so a run can be named, and 'v*' matches those too —
# which would have this workflow build and PUBLISH a GitHub release for
# every candidate, including the red ones.
- '!v*-rc*'
permissions:
contents: write
+550 -1296
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "1.6.0"
version = "1.6.2"
edition = "2024"
rust-version = "1.97"
license = "MIT"
@@ -28,7 +28,7 @@ cbc = "0.1"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.0" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.2" }
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
-83
View File
@@ -1,83 +0,0 @@
// Minimal ISO dumper — find exact stall point
use libfreemkv::Drive;
use std::io::{BufWriter, Write};
use std::path::Path;
use std::time::Instant;
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("Usage: iso_dump <device> <output>");
std::process::exit(1);
}
let mut drive = Drive::open(Path::new(&args[1])).unwrap();
drive.wait_ready().unwrap();
let _ = drive.init();
let _ = drive.probe_disc();
// AACS handshake — required to read past the protected area
eprint!("Scanning disc... ");
let _ = libfreemkv::Disc::scan(&mut drive, &libfreemkv::ScanOptions::default());
eprintln!("OK");
let cap = drive.read_capacity().unwrap();
let batch = libfreemkv::disc::detect_max_batch_sectors(drive.device_path());
eprintln!("Device: {} | {} sectors | batch {}", args[1], cap, batch);
let file = std::fs::File::create(&args[2]).unwrap();
let mut w = BufWriter::with_capacity(4 * 1024 * 1024, file);
let mut buf = vec![0u8; batch as usize * 2048];
let mut lba: u32 = 0;
let start = Instant::now();
let mut last = Instant::now();
let mut bytes: u64 = 0;
let mut last_bytes: u64 = 0;
while lba < cap {
let count = ((cap - lba) as u16).min(batch);
let n = count as usize * 2048;
// Tiny yield between reads — test if pacing prevents firmware throttle
std::thread::yield_now();
let t0 = Instant::now();
let ok = drive.read(lba, count, &mut buf[..n], true).is_ok();
let read_ms = t0.elapsed().as_millis();
// Flag slow reads
if read_ms > 2000 {
eprintln!("\n SLOW READ: LBA {} took {}ms (ok={})", lba, read_ms, ok);
}
if !ok {
buf[..n].fill(0);
}
w.write_all(&buf[..n]).unwrap();
lba += count as u32;
bytes += n as u64;
if last.elapsed().as_millis() >= 1000 {
let delta = bytes - last_bytes;
let speed = delta as f64 / last.elapsed().as_secs_f64() / 1_048_576.0;
let avg = bytes as f64 / start.elapsed().as_secs_f64() / 1_048_576.0;
let pct = bytes as f64 / (cap as f64 * 2048.0) * 100.0;
eprint!(
"\r {:.1}% LBA {} | {:.0} MB/s (avg {:.0}) | {:.1} GB ",
pct,
lba,
speed,
avg,
bytes as f64 / 1e9
);
last_bytes = bytes;
last = Instant::now();
}
}
w.flush().unwrap();
eprintln!(
"\nDone: {:.1} GB in {:.0}s",
bytes as f64 / 1e9,
start.elapsed().as_secs_f64()
);
}
+187 -19
View File
@@ -277,11 +277,31 @@ fn crack_key_scan(
}
let n = (ext.sector_count - i).min(batch);
let want = n as usize * 2048;
// How far the cursor advances. Set from the bytes actually READ on
// the Ok path so a short read is RETRIED from where it stopped
// rather than skipped: bounding only the inspection (which is what
// stops stale buffer bytes being scanned) would otherwise leave
// those sectors unexamined, quietly shrinking the crack's coverage
// on exactly the damaged media where a key is hardest to find.
let mut advance = n;
match reader.read_sectors(ext.start_lba + i, n as u16, &mut buf[..want], true) {
Ok(_) => {
Ok(got) => {
// A readable batch: the gate is open — reset the locked run.
consecutive_locked = 0;
for s in 0..n as usize {
// Inspect only what was actually READ. The trait returns the
// byte count and a source may return Ok with fewer bytes
// than asked (a recovery read over a damaged region), while
// `buf` is reused across batches — so the tail still holds
// the PREVIOUS batch's sectors. Scanning those means
// cracking a key from data belonging to a different extent,
// and possibly a different VTS, while `crack_span` records
// the CURRENT one: a key that opens nothing here gets
// installed and its wrong descrambles are only partly caught
// by the per-sector crib.
let usable = (got / 2048).min(n as usize);
// At least one, so a source returning Ok(0) cannot spin here.
advance = (usable as u32).max(1);
for s in 0..usable {
tried += 1;
let sect = &buf[s * 2048..(s + 1) * 2048];
// Use the HARDENED pack-gated check (Fix 3): a clear stub
@@ -319,7 +339,7 @@ fn crack_key_scan(
}
}
}
i += n;
i += advance;
}
}
@@ -339,7 +359,25 @@ fn crack_key_scan(
}
/// Descramble a single CSS-encrypted sector in place.
///
/// A no-op unless the sector is a scrambled MPEG-2 PS PACK. The pack start code
/// is checked, not just the byte 0x14 flag bits, for the same reason
/// `descramble_region` checks it: 0x14 only means "scrambling control" inside a
/// pack, and in an IFO it is whatever that format stores there. A real
/// `VIDEO_TS.IFO` sector holds 0x15 there while starting `00 26 00 00`;
/// descrambling it destroyed 1912 of its 2048 bytes, and because that sector
/// carries TT_SRPT the disc enumerated 38 titles while an image decrypted from
/// it enumerated 10, silently, at exit 0.
///
/// This function has no callers inside the crate, but the module-level example
/// above prescribes it — so the crate's own documented guidance led straight
/// into that defect. Making the guard part of the function, rather than
/// something each caller must remember, is what keeps the safe path the easy
/// one.
pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
if !is_scrambled_pack(sector) {
return;
}
lfsr::descramble_sector(&state.title_key, sector);
}
@@ -379,7 +417,21 @@ pub fn descramble_sector(state: &CssState, sector: &mut [u8]) {
/// than apply a neighbouring CPS unit's key.
pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::error::Result<usize> {
for chunk in buf.chunks_mut(2048) {
if chunk.len() < 2048 || !is_scrambled(chunk) {
// `is_scrambled_pack`, NOT the looser `is_scrambled`. The raw flag test
// is only safe once a caller has committed to a title's VOB data, where
// every sector is a pack and byte 0x14 always means what it says. This
// function is handed arbitrary regions of a disc, so it also sees IFO,
// UDF and ISO 9660 sectors — raw structures where byte 0x14 is whatever
// that format stores there.
//
// Measured: `VIDEO_TS.IFO` on one disc holds 0x15 at offset 0x14 of its
// second sector. The raw test read bits 4-5 as "scrambled", descrambled
// it, and destroyed 1912 of 2048 bytes. That sector carries TT_SRPT, so
// the title table went with it: the disc enumerated 38 titles and an
// image decrypted from it enumerated 10, silently, at exit 0. Requiring
// the pack start code first costs nothing — a genuinely scrambled VOB
// sector always has it, and no IFO sector does.
if chunk.len() < 2048 || !is_scrambled_pack(chunk) {
continue;
}
let crib = stevenson::attack_crib(chunk);
@@ -433,15 +485,25 @@ pub fn descramble_region(buf: &mut [u8], title_key: &mut [u8; 5]) -> crate::erro
Ok(0)
}
/// Check if a sector has the CSS scramble flag set.
/// Whether bits 4-5 of the sub-header byte 0x14 are set. NOTHING MORE.
///
/// This is the RAW flag test — bits 4-5 of the sub-header byte 0x14 — used by
/// the descramble loop (`decrypt::decrypt_sectors`), which has already committed
/// to descrambling a known title's VOB data and only needs to skip the clear
/// NAV packs interleaved in it. For the CRACK SCAN's "did this disc actually
/// contain scrambled content?" decision (which must not false-positive on a
/// clear stub), use [`is_scrambled_pack`] instead.
pub fn is_scrambled(sector: &[u8]) -> bool {
/// This is deliberately NOT called `is_scrambled`. Byte 0x14 only means "scrambling control" inside an MPEG-2 Program
/// Stream pack; in an IFO, UDF or ISO 9660 sector it is whatever that format
/// stores there. Treating the flag alone as proof of scrambling is what
/// destroyed 1912 bytes of a real disc's `VIDEO_TS.IFO` — the sector carrying
/// TT_SRPT — so the disc enumerated 38 titles and an image decrypted from it
/// enumerated 10, silently, at exit 0.
///
/// **Callers want [`is_scrambled_pack`].** It asks the same question and also
/// requires the pack start code, which every genuinely scrambled VOB sector
/// carries and no IFO sector does.
///
/// It stays public only because an integration test asserts the flag
/// extraction directly. It has no production callers, and its previous doc
/// comment claimed one (`decrypt::decrypt_sectors`) that did not exist — so
/// the name was an invitation and the documentation was an argument for
/// accepting it.
pub fn has_scramble_flag_bits(sector: &[u8]) -> bool {
sector.len() >= 2048 && (sector[0x14] >> 4) & 0x03 != 0
}
@@ -510,7 +572,10 @@ mod tests {
for (i, b) in sector.iter_mut().enumerate().skip(0x80) {
*b = ((i * 37 + 11) % 251) as u8;
}
assert!(is_scrambled(&sector), "fixture must be a scrambled sector");
assert!(
has_scramble_flag_bits(&sector),
"fixture must be a scrambled sector"
);
assert!(
stevenson::attack_crib(&sector).is_some(),
"fixture must yield a crib, or the mismatch branch is never entered"
@@ -566,9 +631,9 @@ mod tests {
/// 20-byte slice; this test catches it.
#[test]
fn is_scrambled_short_buffer_is_false_no_panic() {
assert!(!is_scrambled(&[]));
assert!(!is_scrambled(&[0u8; 20])); // shorter than 0x14+1 even
assert!(!is_scrambled(&[0xFFu8; 2047])); // one byte short of a sector
assert!(!has_scramble_flag_bits(&[]));
assert!(!has_scramble_flag_bits(&[0u8; 20])); // shorter than 0x14+1 even
assert!(!has_scramble_flag_bits(&[0xFFu8; 2047])); // one byte short of a sector
}
/// is_scrambled keys on bits 4-5 of byte 0x14 (the CSS scramble field).
@@ -593,7 +658,7 @@ mod tests {
] {
s[0x14] = flag;
assert_eq!(
is_scrambled(&s),
has_scramble_flag_bits(&s),
expected,
"flag byte {flag:#04x} scramble detection"
);
@@ -610,7 +675,10 @@ mod tests {
fn is_scrambled_exact_sector_length_accepted() {
let mut s = vec![0u8; 2048];
s[0x14] = 0x30;
assert!(is_scrambled(&s), "exactly 2048 bytes must be eligible");
assert!(
has_scramble_flag_bits(&s),
"exactly 2048 bytes must be eligible"
);
}
/// Fix 3 hardening: `is_scrambled_pack` (the crack-scan evidence gate)
@@ -633,7 +701,10 @@ mod tests {
"0x14 bits without the MPEG-PS pack-start must NOT count as a scrambled pack"
);
// The looser descramble-gate check still sees the raw flag.
assert!(is_scrambled(&s), "is_scrambled keys on the 0x14 flag alone");
assert!(
has_scramble_flag_bits(&s),
"has_scramble_flag_bits keys on the 0x14 flag alone"
);
// A near-miss pack-start (wrong final byte) is still rejected.
s[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBB]);
assert!(
@@ -825,6 +896,103 @@ mod tests {
assert!(crack_key(&mut MockSource::new(0x30), &extents, 1).is_none());
}
/// `descramble_region` is handed arbitrary disc regions, so it also sees
/// IFO/UDF/ISO-9660 sectors — raw structures where byte 0x14 is whatever
/// that format happens to store there, NOT a scrambling-control field.
///
/// Measured on a real disc: the second sector of `VIDEO_TS.IFO` holds 0x15
/// at offset 0x14 (bits 4-5 set) while starting `00 26 00 00`, which is not
/// a pack. Descrambling it destroyed 1912 of its 2048 bytes, and because
/// that sector carries TT_SRPT the whole title table went with it — the
/// disc enumerated 38 titles, an image decrypted from it enumerated 10, at
/// exit 0 with no diagnostic. Guard on the pack start code, not the flag.
/// The PUBLIC per-sector entry point must refuse a non-pack sector too.
///
/// Audit finding: `descramble_region` was fixed to require the pack start
/// code, but `descramble_sector` — which the module-level example tells
/// callers to use — still keyed on the 0x14 flag alone, so the crate's own
/// documented path reached the same defect.
#[test]
fn descramble_sector_refuses_a_non_pack_sector() {
let mut ifo_like = vec![0u8; 2048];
for (i, b) in ifo_like.iter_mut().enumerate() {
*b = (i as u8).wrapping_mul(37).wrapping_add(11);
}
ifo_like[0x00..0x04].copy_from_slice(&[0x00, 0x26, 0x00, 0x00]); // not a pack
ifo_like[0x14] = 0x15; // flag bits set — the trap
assert!(has_scramble_flag_bits(&ifo_like) && !is_scrambled_pack(&ifo_like));
let pristine = ifo_like.clone();
let state = CssState {
title_key: [0x42, 0x13, 0x37, 0xBE, 0xEF],
crack_span: None,
};
descramble_sector(&state, &mut ifo_like);
assert_eq!(
ifo_like, pristine,
"a non-pack sector must survive byte-identical through the public API"
);
}
#[test]
fn descramble_region_leaves_a_non_pack_sector_alone_even_with_the_flag_set() {
let mut ifo_like = vec![0u8; 2048];
for (i, b) in ifo_like.iter_mut().enumerate() {
*b = (i as u8).wrapping_mul(37).wrapping_add(11);
}
ifo_like[0x00..0x04].copy_from_slice(&[0x00, 0x26, 0x00, 0x00]); // not a pack
ifo_like[0x14] = 0x15; // bits 4-5 set: reads as "scrambled" to the raw test
assert!(
has_scramble_flag_bits(&ifo_like) && !is_scrambled_pack(&ifo_like),
"fixture must be exactly the case the two predicates disagree on"
);
let pristine = ifo_like.clone();
let mut key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
descramble_region(&mut ifo_like, &mut key).expect("region descramble");
assert_eq!(
ifo_like, pristine,
"a non-pack sector must survive byte-identical — descrambling it \
corrupts the very structures that enumerate titles"
);
}
/// The other half of the contract: the guard must not cost coverage of
/// sectors that genuinely ARE scrambled VOB data.
#[test]
fn descramble_region_still_descrambles_a_real_scrambled_pack() {
let mut pack = vec![0u8; 2048];
for (i, b) in pack.iter_mut().enumerate() {
*b = (i as u8).wrapping_mul(29).wrapping_add(3);
}
pack[0x00..0x04].copy_from_slice(&PACK_START);
pack[0x14] = 0x30;
assert!(is_scrambled_pack(&pack));
let scrambled = pack.clone();
let mut key = [0x42, 0x13, 0x37, 0xBE, 0xEF];
descramble_region(&mut pack, &mut key).expect("region descramble");
assert_ne!(
pack[0x80..],
scrambled[0x80..],
"the encrypted region of a real pack must actually be transformed"
);
assert_eq!(
(pack[0x14] >> 4) & 0x03,
0,
"the scrambling-control bits must be cleared once descrambled"
);
assert_eq!(
pack[0x15..0x80],
scrambled[0x15..0x80],
"CSS only scrambles from 0x80 on; the rest of the header, byte 0x14 \
aside, must be untouched"
);
assert_eq!(pack[..0x14], scrambled[..0x14]);
}
/// Even when every read FAILS, a scan that never managed to observe a
/// scrambled sector reports `Unencrypted` (we cannot prove encryption from
/// unreadable data alone — the AACS/keydb paths and the disc-level
+52
View File
@@ -695,6 +695,49 @@ mod tests {
// ── `decrypt_sectors_in_content` (now a legacy alias of `decrypt_sectors`) ──
/// `DecryptKeys::None` is a no-op even with a content map + scrambled bytes.
/// A forensic map's read plan is NOT the extents it was given — and that is
/// the precondition the mux's provenance guard keys on.
///
/// A clip's feed span is measured over the FULL extents at scan time, while
/// the mux reads this reduced plan, so the byte offsets stamped on frames
/// and the offsets recorded in the spans describe different streams. The
/// deficit accumulates, so every frame after the first segment resolves to
/// an earlier clip than it came from. The spans still tile each other, so
/// the tiling check cannot see it; the mux compares the plan against the
/// full extents instead and stops trusting provenance when they differ.
#[test]
fn a_forensic_read_plan_drops_units_the_full_extents_include() {
let full = vec![crate::disc::Extent {
start_lba: 1000,
sector_count: 60,
}];
// No forensic segment: the plan IS the extents, byte for byte, so
// provenance stays trustworthy on an ordinary disc.
let plain = AacsKeyMap::from_ranges_phased(vec![(1000, 1060, 5, Phase::All)]);
assert_eq!(
plain.read_plan(&full, 3),
full,
"a non-forensic map must return the extents unchanged"
);
// With alternate phases, units are omitted — fewer sectors are read
// than the spans describe.
let phased = AacsKeyMap::from_ranges_phased(vec![(1000, 1060, 5, Phase::Even)]);
let plan = phased.read_plan(&full, 3);
let planned: u32 = plan.iter().map(|e| e.sector_count).sum();
let whole: u32 = full.iter().map(|e| e.sector_count).sum();
assert!(
planned < whole,
"a forensic segment must drop units: planned {planned} of {whole}"
);
assert_ne!(
plan, full,
"the plan differs from the extents, which is exactly what the mux \
detects before deciding whether a byte offset means anything"
);
}
#[test]
fn content_gate_none_keys_is_noop() {
let mut keys = DecryptKeys::None;
@@ -987,6 +1030,12 @@ mod tests {
/// scramble flag.
fn make_css_sector(title_key: &[u8; 5], seed: &[u8; 5], body_fill: u8) -> (Vec<u8>, Vec<u8>) {
let mut sector = vec![body_fill; 2048];
// A real scrambled DVD sector is an MPEG-2 PS pack, so it begins with
// the pack start code. The descrambler requires it before trusting
// byte 0x14 — without it this fixture is a sector shape that cannot
// occur on a disc, and the test would pass while the production gate
// rejected every sector like it.
sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sector[0x14] = 0x30; // scramble flag (bits 4-5)
sector[0x54..0x59].copy_from_slice(seed);
let plaintext = sector.clone();
@@ -1064,6 +1113,9 @@ mod tests {
period: usize,
) -> (Vec<u8>, Vec<u8>) {
let mut plaintext = vec![0u8; 2048];
// Real scrambled DVD sectors are MPEG-2 PS packs; the scramble policy
// requires the pack start code as well as the flag bits.
plaintext[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
plaintext[0x14] = 0x10; // scramble flag
// Periodic run from 0x59 (just above the seed) through 0x80 and on into
// the encrypted region; phase anchored to offset 0 so it is continuous
+1
View File
@@ -642,6 +642,7 @@ mod tests {
size_bytes,
clips: (0..n_clips)
.map(|i| Clip {
feed_span: None,
clip_id: format!("{i:05}"),
in_time: 0,
out_time: 0,
+739
View File
@@ -0,0 +1,739 @@
//! ECMA-167 / UDF 1.02 descriptor encoder.
//!
//! Turns a [`Layout`](super::layout::Layout) — a directory tree with every
//! ICB, directory-data and file-data block already assigned — into the set of
//! metadata sectors a real UDF volume would carry. Nothing here touches the
//! filesystem: it is a pure function from layout to sectors, which is what
//! makes it testable against the production parser in `udf.rs`.
//!
//! What is emitted, in volume order:
//!
//! | sector | descriptor |
//! |---|---|
//! | 16, 17, 18 | Volume Recognition Sequence — `BEA01`, `NSR02`, `TEA01` (ECMA-167 2/9.1) |
//! | 32… | Main Volume Descriptor Sequence — PVD, IUVD, PD, LVD, USD, TD |
//! | 48… | Reserve VDS (byte-identical but for the tag locations) |
//! | 64, 65 | Logical Volume Integrity Sequence — LVID, TD |
//! | 256 | Anchor Volume Descriptor Pointer |
//! | `part_start` + 0, +1 | File Set Descriptor, TD |
//! | `part_start` + … | File Entries (ICBs) and directory data (FIDs) |
//! | last sector | Anchor Volume Descriptor Pointer (copy) |
//!
//! UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
//! the DVD-Video profile, it is the shape `read_filesystem` takes when
//! `num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
//! entirely. That also means a synthetic image never exercises the Metadata
//! Partition path in `udf.rs` (`:946-991`) — see the module docs on `dirimage`.
use super::layout::{DirNode, Layout};
use crate::error::{Error, Result};
use std::collections::BTreeMap;
/// Logical block / sector size. Fixed for every optical profile this crate reads.
pub(super) const SECTOR: usize = 2048;
/// Descriptor version recorded in every tag. 2 = ECMA-167 2nd edition, which
/// is what UDF revisions up to and including 2.00 require.
const DESC_VERSION: u16 = 2;
/// UDF revision recorded in the domain EntityID suffix (1.02, BCD-ish u16).
const UDF_REVISION: u16 = 0x0102;
/// A fixed recording timestamp, so an image synthesized from the same folder
/// twice is byte-identical. Real mtimes would make every test golden-file
/// comparison and every `dir:// -> iso://` re-run differ for no benefit.
const FIXED_TIME: Timestamp = Timestamp {
year: 2000,
month: 1,
day: 1,
};
struct Timestamp {
year: i16,
month: u8,
day: u8,
}
/// The synthesized metadata: absolute LBA → sector contents. Data sectors are
/// NOT here; they are served from the backing files.
pub(super) type MetaSectors = BTreeMap<u32, Box<[u8; SECTOR]>>;
/// The descriptor-tag CRC of ECMA-167 7.2.4: polynomial 0x1021, initial value
/// ZERO, no reflection, no final XOR — the variant catalogued as CRC-16/XMODEM
/// (check value 0x31C3), NOT CCITT-FALSE, which seeds at 0xFFFF and would make
/// every descriptor this crate writes fail a conformant driver's validation.
fn crc16(data: &[u8]) -> u16 {
let mut crc: u16 = 0;
for &b in data {
crc ^= (b as u16) << 8;
for _ in 0..8 {
crc = if crc & 0x8000 != 0 {
(crc << 1) ^ 0x1021
} else {
crc << 1
};
}
}
crc
}
/// Write an ECMA-167 3/7.2 descriptor tag over `buf[0..16]`.
///
/// `tag_loc` is the block number of the sector holding the descriptor —
/// ABSOLUTE for the volume-space descriptors (AVDP, VDS, LVID) and
/// PARTITION-RELATIVE for everything inside the partition (FSD, File Entries).
/// Getting that wrong is the classic reason a hand-built volume mounts nowhere:
/// a driver that validates the tag location rejects the descriptor outright.
///
/// `desc_len` is the descriptor's total length including the tag; the CRC
/// covers `buf[16..desc_len]`.
fn finish_tag(buf: &mut [u8], tag_id: u16, tag_loc: u32, desc_len: usize) {
buf[0..2].copy_from_slice(&tag_id.to_le_bytes());
buf[2..4].copy_from_slice(&DESC_VERSION.to_le_bytes());
buf[4] = 0; // checksum, filled below
buf[5] = 0; // reserved
buf[6..8].copy_from_slice(&0u16.to_le_bytes()); // tag serial number
let crc_len = desc_len - 16;
let crc = crc16(&buf[16..desc_len]);
buf[8..10].copy_from_slice(&crc.to_le_bytes());
buf[10..12].copy_from_slice(&(crc_len as u16).to_le_bytes());
buf[12..16].copy_from_slice(&tag_loc.to_le_bytes());
// ECMA-167 3/7.2.3: sum of bytes 0..16 EXCLUDING byte 4, modulo 256.
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
buf[4] = (sum % 256) as u8;
}
/// ECMA-167 1/7.2.1 charspec: type 0 (CS0) + "OSTA Compressed Unicode".
fn put_charspec(buf: &mut [u8]) {
buf[0] = 0;
let id = b"OSTA Compressed Unicode";
buf[1..1 + id.len()].copy_from_slice(id);
}
/// ECMA-167 1/7.4 EntityID: flags byte, 23 identifier bytes, 8 suffix bytes.
fn put_entity_id(buf: &mut [u8], id: &[u8], suffix: &[u8]) {
buf[0] = 0;
let n = id.len().min(23);
buf[1..1 + n].copy_from_slice(&id[..n]);
let m = suffix.len().min(8);
buf[24..24 + m].copy_from_slice(&suffix[..m]);
}
/// The `*OSTA UDF Compliant` domain EntityID suffix: UDF revision, domain
/// flags (0 = neither hard nor soft write-protected), reserved.
fn domain_suffix() -> [u8; 8] {
let mut s = [0u8; 8];
s[0..2].copy_from_slice(&UDF_REVISION.to_le_bytes());
s
}
/// This crate's implementation EntityID suffix: OS class / OS identifier
/// (0 = undefined, deliberately — the image is not OS-specific) + 6 free bytes.
fn impl_suffix() -> [u8; 8] {
[0u8; 8]
}
fn put_impl_id(buf: &mut [u8]) {
put_entity_id(buf, b"*freemkv", &impl_suffix());
}
fn put_domain_id(buf: &mut [u8]) {
put_entity_id(buf, b"*OSTA UDF Compliant", &domain_suffix());
}
/// OSTA CS0 d-string: a compression-ID byte, the characters, then the used
/// length in the FIELD'S LAST byte (ECMA-167 1/7.2.12 + UDF 2.1.3). An
/// all-zero field is the empty string.
fn put_dstring(buf: &mut [u8], s: &str) {
if s.is_empty() {
return;
}
let encoded = encode_cs0(s);
// Leave room for the trailing length byte.
let room = buf.len() - 1;
let n = encoded.len().min(room);
buf[..n].copy_from_slice(&encoded[..n]);
buf[buf.len() - 1] = n as u8;
}
/// OSTA CS0: compression ID 8 (one byte per character) when every character
/// is ASCII, otherwise compression ID 16 (UTF-16BE).
///
/// ASCII rather than Latin-1 for the 8-bit form on purpose: `parse_udf_name`
/// (`udf.rs:1467`) decodes a compression-8 name with `from_utf8_lossy`, so a
/// 0x80-0xFF byte — legal CS0 — would come back as U+FFFD. Every character
/// above 0x7F therefore takes the 16-bit form, which that parser decodes
/// correctly.
pub(super) fn encode_cs0(s: &str) -> Vec<u8> {
if s.is_ascii() {
let mut v = Vec::with_capacity(1 + s.len());
v.push(8u8);
v.extend_from_slice(s.as_bytes());
v
} else {
let mut v = vec![16u8];
for u in s.encode_utf16() {
v.extend_from_slice(&u.to_be_bytes());
}
v
}
}
/// ECMA-167 1/7.3 timestamp, 12 bytes. Type 1 (local time) with a zero
/// offset, i.e. UTC.
fn put_timestamp(buf: &mut [u8]) {
buf[0..2].copy_from_slice(&0x1000u16.to_le_bytes());
buf[2..4].copy_from_slice(&FIXED_TIME.year.to_le_bytes());
buf[4] = FIXED_TIME.month;
buf[5] = FIXED_TIME.day;
}
/// ECMA-167 3/7.1 extent_ad: length in BYTES, then location.
fn put_extent_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
/// ECMA-167 4/14.14.2 long_ad: length+type, then lb_addr (block, partition
/// reference), then 6 implementation-use bytes.
fn put_long_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
buf[8..10].copy_from_slice(&0u16.to_le_bytes()); // partition reference 0
}
/// ECMA-167 4/14.14.1 short_ad. The top two bits of the length word are the
/// extent TYPE (0 = recorded and allocated), which is exactly why `udf.rs`
/// masks with `0x3FFF_FFFF` when it reads one back — the mask is the field
/// boundary, not a truncation bug.
fn put_short_ad(buf: &mut [u8], len_bytes: u32, lba: u32) {
debug_assert!(len_bytes <= 0x3FFF_FFFF, "AD length must fit 30 bits");
buf[0..4].copy_from_slice(&len_bytes.to_le_bytes());
buf[4..8].copy_from_slice(&lba.to_le_bytes());
}
fn blank() -> Box<[u8; SECTOR]> {
Box::new([0u8; SECTOR])
}
// ── Volume-space descriptors ────────────────────────────────────────────────
/// ECMA-167 2/9.1 Volume Structure Descriptor: the three-sector recognition
/// sequence an OS looks for before it will even consider the volume UDF.
fn volume_recognition(id: &[u8; 5]) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[0] = 0; // structure type
s[1..6].copy_from_slice(id);
s[6] = 1; // structure version
s
}
/// ECMA-167 3/10.1 Primary Volume Descriptor.
fn primary_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes()); // PVD number
put_dstring(&mut s[24..56], volume_id);
s[56..58].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[58..60].copy_from_slice(&1u16.to_le_bytes()); // max volume sequence number
s[60..62].copy_from_slice(&2u16.to_le_bytes()); // interchange level
s[62..64].copy_from_slice(&2u16.to_le_bytes()); // max interchange level
s[64..68].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[68..72].copy_from_slice(&1u32.to_le_bytes()); // max character set list
// UDF 2.2.2.5: the first 8 characters of the volume set identifier must be
// unique. A fixed hex prefix plus the volume id is sufficient here — the
// image is single-volume and never joins a real volume set.
put_dstring(&mut s[72..200], &format!("46524D4B{volume_id}"));
put_charspec(&mut s[200..264]); // descriptor character set
put_charspec(&mut s[264..328]); // explanatory character set
put_timestamp(&mut s[376..388]);
put_impl_id(&mut s[388..420]);
finish_tag(&mut s[..], 1, lba, 512);
s
}
/// ECMA-167 3/10.4 + UDF 2.2.7 Implementation Use Volume Descriptor
/// (`*UDF LV Info`). Not read by `udf.rs`, required by the spec.
fn impl_use_volume(volume_id: &str, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_entity_id(&mut s[20..52], b"*UDF LV Info", &domain_suffix());
put_charspec(&mut s[52..116]); // LVI charset
put_dstring(&mut s[116..244], volume_id); // logical volume identifier
put_impl_id(&mut s[352..384]);
finish_tag(&mut s[..], 4, lba, 512);
s
}
/// ECMA-167 3/10.5 Partition Descriptor — the descriptor `read_filesystem`
/// takes `partition_start` from (offset 188).
fn partition(part_start: u32, part_sectors: u32, lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..22].copy_from_slice(&1u16.to_le_bytes()); // partition flags: allocated
s[22..24].copy_from_slice(&0u16.to_le_bytes()); // partition number
put_entity_id(&mut s[24..56], b"+NSR02", &[]);
// s[56..184] partition contents use = Partition Header Descriptor. All
// zero: a read-only partition records no unallocated/freed space tables.
s[184..188].copy_from_slice(&1u32.to_le_bytes()); // access type: read only
s[188..192].copy_from_slice(&part_start.to_le_bytes());
s[192..196].copy_from_slice(&part_sectors.to_le_bytes());
put_impl_id(&mut s[196..228]);
finish_tag(&mut s[..], 5, lba, 512);
s
}
/// ECMA-167 3/10.6 Logical Volume Descriptor. Carries the FSD long_ad and the
/// partition map table; `read_filesystem` reads `num_partition_maps` at 268
/// and takes the single-partition path when it is 1.
fn logical_volume(
volume_id: &str,
fsd_lba: u32,
integrity_lba: u32,
integrity_sectors: u32,
lba: u32,
seq: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
put_charspec(&mut s[20..84]);
put_dstring(&mut s[84..212], volume_id);
s[212..216].copy_from_slice(&(SECTOR as u32).to_le_bytes()); // logical block size
put_domain_id(&mut s[216..248]);
// Logical volume contents use = long_ad of the File Set Descriptor,
// partition-relative. One sector.
put_long_ad(&mut s[248..264], SECTOR as u32, fsd_lba);
s[264..268].copy_from_slice(&6u32.to_le_bytes()); // map table length
s[268..272].copy_from_slice(&1u32.to_le_bytes()); // number of partition maps
put_impl_id(&mut s[272..304]);
put_extent_ad(
&mut s[432..440],
integrity_sectors * SECTOR as u32,
integrity_lba,
);
// ECMA-167 3/10.7.2 Type 1 partition map.
s[440] = 1; // map type
s[441] = 6; // map length
s[442..444].copy_from_slice(&1u16.to_le_bytes()); // volume sequence number
s[444..446].copy_from_slice(&0u16.to_le_bytes()); // partition number
finish_tag(&mut s[..], 6, lba, 446);
s
}
/// ECMA-167 3/10.8 Unallocated Space Descriptor with zero extents — the whole
/// volume is accounted for by the partition.
fn unallocated_space(lba: u32, seq: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
s[16..20].copy_from_slice(&seq.to_le_bytes());
s[20..24].copy_from_slice(&0u32.to_le_bytes());
finish_tag(&mut s[..], 7, lba, 24);
s
}
/// ECMA-167 3/10.9 Terminating Descriptor.
fn terminating(lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
finish_tag(&mut s[..], 8, lba, 512);
s
}
/// ECMA-167 3/10.10 + UDF 2.2.6 Logical Volume Integrity Descriptor, closed.
fn integrity(
part_sectors: u32,
files: u32,
dirs: u32,
next_uid: u64,
lba: u32,
) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..32].copy_from_slice(&1u32.to_le_bytes()); // integrity type: close
// s[32..40] next integrity extent: none.
s[40..48].copy_from_slice(&next_uid.to_le_bytes()); // logical volume contents use: next unique id
s[72..76].copy_from_slice(&1u32.to_le_bytes()); // number of partitions
s[76..80].copy_from_slice(&46u32.to_le_bytes()); // length of implementation use
s[80..84].copy_from_slice(&0u32.to_le_bytes()); // free space: none (read-only)
s[84..88].copy_from_slice(&part_sectors.to_le_bytes()); // size table
put_impl_id(&mut s[88..120]);
s[120..124].copy_from_slice(&files.to_le_bytes());
s[124..128].copy_from_slice(&dirs.to_le_bytes());
s[128..130].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min read revision
s[130..132].copy_from_slice(&UDF_REVISION.to_le_bytes()); // min write revision
s[132..134].copy_from_slice(&UDF_REVISION.to_le_bytes()); // max write revision
finish_tag(&mut s[..], 9, lba, 134);
s
}
/// ECMA-167 3/10.2 Anchor Volume Descriptor Pointer. `read_filesystem` reads
/// the main VDS extent from offsets 16..24 and sweeps it.
fn anchor(main_lba: u32, reserve_lba: u32, vds_sectors: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_extent_ad(&mut s[16..24], vds_sectors * SECTOR as u32, main_lba);
put_extent_ad(&mut s[24..32], vds_sectors * SECTOR as u32, reserve_lba);
finish_tag(&mut s[..], 2, lba, 512);
s
}
/// ECMA-167 4/14.1 File Set Descriptor. `read_filesystem` requires tag 256 at
/// the first block of the (metadata =) partition and reads the root ICB block
/// from offset 404.
fn file_set(volume_id: &str, root_icb: u32, lba: u32) -> Box<[u8; SECTOR]> {
let mut s = blank();
put_timestamp(&mut s[16..28]);
s[28..30].copy_from_slice(&3u16.to_le_bytes()); // interchange level
s[30..32].copy_from_slice(&3u16.to_le_bytes()); // max interchange level
s[32..36].copy_from_slice(&1u32.to_le_bytes()); // character set list
s[36..40].copy_from_slice(&1u32.to_le_bytes()); // max character set list
s[40..44].copy_from_slice(&0u32.to_le_bytes()); // file set number
s[44..48].copy_from_slice(&0u32.to_le_bytes()); // file set descriptor number
put_charspec(&mut s[48..112]);
put_dstring(&mut s[112..240], volume_id);
put_charspec(&mut s[240..304]);
put_dstring(&mut s[304..336], volume_id);
put_long_ad(&mut s[400..416], SECTOR as u32, root_icb);
put_domain_id(&mut s[416..448]);
finish_tag(&mut s[..], 256, lba, 512);
s
}
// ── Partition-space descriptors ─────────────────────────────────────────────
/// UDF permission word: read + execute for owner, group and other. No write
/// bit anywhere — the volume is read-only.
const PERM_R_X: u32 = 0x0000_1000 | 0x0000_0400 | 0x0000_0080 | 0x0000_0020 | 0x4 | 0x1;
/// ECMA-167 4/14.9 File Entry (tag 261).
///
/// Tag 261 rather than the Extended File Entry (266) real BD-ROMs use: an EFE
/// requires UDF 2.00+, and this image declares 1.02. `udf.rs` reads both — the
/// 261 field offsets it uses (l_ea 168, l_ad 172, ADs at 176 + l_ea) are the
/// ones written here.
///
/// `extents` are partition-relative (block, byte-length) pairs, already split
/// so no single one exceeds the 30-bit AD length field.
fn file_entry(
is_dir: bool,
info_len: u64,
extents: &[(u32, u32)],
link_count: u16,
unique_id: u64,
lba: u32,
) -> Result<Box<[u8; SECTOR]>> {
let mut s = blank();
// ICB tag (ECMA-167 4/14.6) at offset 16.
s[16..20].copy_from_slice(&0u32.to_le_bytes()); // prior recorded direct entries
s[20..22].copy_from_slice(&4u16.to_le_bytes()); // strategy type 4
s[24..26].copy_from_slice(&1u16.to_le_bytes()); // max number of entries
s[27] = if is_dir { 4 } else { 5 }; // file type: directory / byte sequence
// s[28..34] parent ICB location: not recorded (permitted).
// s[34..36] ICB flags: 0 => short allocation descriptors. `udf.rs:601`
// reads exactly this word to pick its AD stride.
s[34..36].copy_from_slice(&0u16.to_le_bytes());
// UDF's sentinel for "not specified" is 0xFFFFFFFF, not 0 — 0 is a real
// uid/gid (root). A synthesized image has no meaningful owner, and a driver
// that maps these through would otherwise report every file as root-owned.
s[36..40].copy_from_slice(&u32::MAX.to_le_bytes()); // uid: not specified
s[40..44].copy_from_slice(&u32::MAX.to_le_bytes()); // gid: not specified
s[44..48].copy_from_slice(&PERM_R_X.to_le_bytes());
s[48..50].copy_from_slice(&link_count.to_le_bytes());
s[56..64].copy_from_slice(&info_len.to_le_bytes());
let blocks: u64 = extents
.iter()
.map(|(_, len)| (*len as u64).div_ceil(SECTOR as u64))
.sum();
s[64..72].copy_from_slice(&blocks.to_le_bytes()); // logical blocks recorded
put_timestamp(&mut s[72..84]); // access
put_timestamp(&mut s[84..96]); // modification
put_timestamp(&mut s[96..108]); // attribute
s[108..112].copy_from_slice(&1u32.to_le_bytes()); // checkpoint
put_impl_id(&mut s[128..160]);
s[160..168].copy_from_slice(&unique_id.to_le_bytes());
s[168..172].copy_from_slice(&0u32.to_le_bytes()); // length of EAs
let l_ad = extents.len() * 8;
// A short AD is 8 bytes and the entry has 2048 - 176 = 1872 bytes for
// them, i.e. 234 extents — over 200 GiB at the per-AD ceiling. Beyond
// that an Allocation Extent Descriptor chain would be required; refuse
// rather than write a truncated list.
if 176 + l_ad > SECTOR {
return Err(Error::DirImageTooLarge);
}
s[172..176].copy_from_slice(&(l_ad as u32).to_le_bytes());
for (i, (elba, len)) in extents.iter().enumerate() {
let off = 176 + i * 8;
put_short_ad(&mut s[off..off + 8], *len, *elba);
}
finish_tag(&mut s[..], 261, lba, 176 + l_ad);
Ok(s)
}
/// ECMA-167 4/14.4 File Identifier Descriptor, appended to `buf`.
///
/// FIDs are packed with no inter-descriptor padding beyond the 4-byte
/// alignment the spec mandates, and they are allowed to span logical blocks —
/// which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
/// directory extent as one flat byte run and STOPS at the first non-257 tag,
/// so any block-alignment gap would truncate the directory.
fn push_fid(buf: &mut Vec<u8>, name: &str, icb_lba: u32, is_dir: bool, is_parent: bool) {
let start = buf.len();
let name_field: Vec<u8> = if is_parent {
Vec::new()
} else {
encode_cs0(name)
};
let l_fi = name_field.len();
let mut fid = vec![0u8; 38];
fid[16..18].copy_from_slice(&1u16.to_le_bytes()); // file version number
let mut chars = 0u8;
if is_dir {
chars |= 0x02;
}
if is_parent {
chars |= 0x08;
}
fid[18] = chars;
// The planner refuses any name whose encoding exceeds what this byte can
// hold (`layout::MAX_CS0_NAME_BYTES`), so this cannot wrap in practice. The
// assert states the invariant where it is relied on rather than trusting a
// check three files away; a wrap here would desynchronise the directory.
debug_assert!(
l_fi <= u8::MAX as usize,
"FID name length must fit one byte"
);
fid[19] = l_fi as u8;
put_long_ad(&mut fid[20..36], SECTOR as u32, icb_lba);
fid[36..38].copy_from_slice(&0u16.to_le_bytes()); // length of implementation use
buf.extend_from_slice(&fid);
buf.extend_from_slice(&name_field);
let unpadded = buf.len() - start;
let padded = unpadded.div_ceil(4) * 4;
buf.resize(start + padded, 0);
// The tag is written last: its CRC covers the descriptor body, which the
// padding is not part of (ECMA-167 4/14.4.9 counts padding outside the
// CRC'd length).
let tag_loc_placeholder = 0;
finish_tag(
&mut buf[start..start + unpadded],
257,
tag_loc_placeholder,
unpadded,
);
}
/// Serialize one directory's FID list (parent entry first, then children).
pub(super) fn dir_fids(dir: &DirNode) -> Vec<u8> {
let mut buf = Vec::new();
push_fid(&mut buf, "", dir.parent_icb_lba, true, true);
for sub in &dir.dirs {
push_fid(&mut buf, &sub.name, sub.icb_lba, true, false);
}
for f in &dir.files {
push_fid(&mut buf, &f.name, f.icb_lba, false, false);
}
buf
}
/// Patch every FID's tag location to the block it actually lands in. ECMA-167
/// 3/7.2.2 makes the tag location the block of the descriptor, and a FID that
/// spans two blocks records the block it STARTS in.
fn fix_fid_tag_locations(buf: &mut [u8], first_block: u32) {
let mut pos = 0usize;
while pos + 38 <= buf.len() {
let l_fi = buf[pos + 19] as usize;
let l_iu = u16::from_le_bytes([buf[pos + 36], buf[pos + 37]]) as usize;
let unpadded = 38 + l_iu + l_fi;
if pos + unpadded > buf.len() {
break;
}
let block = first_block + (pos / SECTOR) as u32;
finish_tag(&mut buf[pos..pos + unpadded], 257, block, unpadded);
pos += unpadded.div_ceil(4) * 4;
}
}
// ── Whole-image assembly ────────────────────────────────────────────────────
/// Volume-space block of the Volume Recognition Sequence.
const VRS_START: u32 = 16;
/// Volume-space block of the Main Volume Descriptor Sequence.
pub(super) const MAIN_VDS_START: u32 = 32;
/// Volume-space block of the Reserve Volume Descriptor Sequence.
pub(super) const RESERVE_VDS_START: u32 = 48;
/// Sectors reserved for each VDS. ECMA-167 3/10.2.1 requires an anchor to
/// record at least 16.
pub(super) const VDS_SECTORS: u32 = 16;
/// Volume-space block of the Logical Volume Integrity Sequence.
pub(super) const LVID_START: u32 = 64;
/// Sectors reserved for the integrity sequence (LVID + TD).
pub(super) const LVID_SECTORS: u32 = 2;
/// The mandatory anchor block (ECMA-167 3/10.2).
pub(super) const ANCHOR_LBA: u32 = 256;
/// First block a partition may start at. Everything above is volume space.
pub(super) const MIN_PART_START: u32 = 320;
/// Emit the six-descriptor Volume Descriptor Sequence at `start`.
fn write_vds(out: &mut MetaSectors, layout: &Layout, start: u32) {
let vid = &layout.volume_id;
out.insert(start, primary_volume(vid, start, 1));
out.insert(start + 1, impl_use_volume(vid, start + 1, 2));
out.insert(
start + 2,
partition(layout.part_start, layout.part_sectors, start + 2, 3),
);
out.insert(
start + 3,
logical_volume(vid, 0, LVID_START, LVID_SECTORS, start + 3, 4),
);
out.insert(start + 4, unallocated_space(start + 4, 5));
out.insert(start + 5, terminating(start + 5));
}
/// Recursively emit one directory's File Entry and FID list, then its
/// children's.
fn write_dir(out: &mut MetaSectors, layout: &Layout, dir: &DirNode) -> Result<()> {
let mut fids = dir_fids(dir);
fix_fid_tag_locations(&mut fids, dir.data_lba);
debug_assert_eq!(fids.len(), dir.data_bytes as usize);
// A directory's link count is 1 (its own FID in the parent) plus one for
// each child directory's parent FID pointing back at it.
// The planner caps subdirectory fan-out (`layout::MAX_SUBDIRS`) so this
// cannot overflow; saturating rather than wrapping keeps a future change to
// that cap from silently producing a wrong count.
let link_count = (dir.dirs.len() as u16).saturating_add(1);
let fe = file_entry(
true,
fids.len() as u64,
&[(dir.data_lba, fids.len() as u32)],
link_count,
dir.unique_id,
dir.icb_lba,
)?;
out.insert(layout.part_start + dir.icb_lba, fe);
for (i, chunk) in fids.chunks(SECTOR).enumerate() {
let mut s = blank();
s[..chunk.len()].copy_from_slice(chunk);
out.insert(layout.part_start + dir.data_lba + i as u32, s);
}
for f in &dir.files {
let extents: Vec<(u32, u32)> = f.extents.iter().map(|e| (e.lba, e.bytes)).collect();
let fe = file_entry(false, f.size, &extents, 1, f.unique_id, f.icb_lba)?;
out.insert(layout.part_start + f.icb_lba, fe);
}
for sub in &dir.dirs {
write_dir(out, layout, sub)?;
}
Ok(())
}
/// Build every metadata sector of the synthesized volume.
pub(super) fn encode(layout: &Layout) -> Result<MetaSectors> {
let mut out = MetaSectors::new();
out.insert(VRS_START, volume_recognition(b"BEA01"));
out.insert(VRS_START + 1, volume_recognition(b"NSR02"));
out.insert(VRS_START + 2, volume_recognition(b"TEA01"));
write_vds(&mut out, layout, MAIN_VDS_START);
write_vds(&mut out, layout, RESERVE_VDS_START);
out.insert(
LVID_START,
integrity(
layout.part_sectors,
layout.file_count,
layout.dir_count,
layout.next_unique_id,
LVID_START,
),
);
out.insert(LVID_START + 1, terminating(LVID_START + 1));
let avdp = anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, ANCHOR_LBA);
out.insert(ANCHOR_LBA, avdp);
let last = layout.total_sectors - 1;
out.insert(
last,
anchor(MAIN_VDS_START, RESERVE_VDS_START, VDS_SECTORS, last),
);
// Partition block 0 must hold the File Set Descriptor: `read_filesystem`
// reads exactly `metadata_start` (== partition start on a single-partition
// volume) and rejects the volume outright if the tag there is not 256.
out.insert(
layout.part_start,
file_set(&layout.volume_id, layout.root.icb_lba, 0),
);
out.insert(layout.part_start + 1, terminating(1));
write_dir(&mut out, layout, &layout.root)?;
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
/// The reference check value for CRC-16/XMODEM — poly 0x1021 seeded at 0,
/// which is what ECMA-167 7.2.4 specifies: "123456789" → 0x31C3. Seeding
/// at 0xFFFF instead (CCITT-FALSE) yields 0x29B1, and that mutant is
/// invisible to `udf.rs`, which never verifies a tag CRC — it would only
/// show up as a volume no operating system will mount.
#[test]
fn crc16_matches_the_ecma167_check_value() {
assert_eq!(crc16(b"123456789"), 0x31C3);
assert_ne!(crc16(b"123456789"), 0x29B1, "not the 0xFFFF-seeded variant");
}
/// ECMA-167 3/7.2.3: the checksum is the sum of the tag's first 16 bytes
/// EXCLUDING the checksum byte itself, modulo 256.
#[test]
fn tag_checksum_excludes_its_own_byte() {
let mut buf = [0u8; 512];
buf[16..24].copy_from_slice(&[1, 2, 3, 4, 5, 6, 7, 8]);
finish_tag(&mut buf, 261, 0x1234, 512);
let sum: u32 = buf[0..16]
.iter()
.enumerate()
.filter(|(i, _)| *i != 4)
.map(|(_, b)| *b as u32)
.sum();
assert_eq!(buf[4] as u32, sum % 256);
// And the recorded CRC covers the body, not the tag.
let crc = u16::from_le_bytes([buf[8], buf[9]]);
assert_eq!(crc, crc16(&buf[16..512]));
assert_eq!(u16::from_le_bytes([buf[10], buf[11]]), 496);
assert_eq!(
u32::from_le_bytes([buf[12], buf[13], buf[14], buf[15]]),
0x1234
);
}
/// ASCII takes compression ID 8; anything above takes 16 (UTF-16BE),
/// because `parse_udf_name` decodes compression-8 bytes as UTF-8.
#[test]
fn cs0_picks_the_encoding_the_parser_can_decode() {
assert_eq!(encode_cs0("AB"), vec![8, b'A', b'B']);
let e = encode_cs0("Ä");
assert_eq!(e[0], 16);
assert_eq!(&e[1..], &[0x00, 0xC4]);
assert_eq!(crate::udf::parse_udf_name(&e), "Ä");
}
/// A d-string records its used length in the field's LAST byte, and the
/// production parser must read the same string back.
#[test]
fn dstring_round_trips_through_the_production_parser() {
let mut field = [0u8; 32];
put_dstring(&mut field, "FREEMKV");
assert_eq!(field[31], 8, "compid byte + 7 characters");
assert_eq!(crate::udf::parse_dstring_for_test(&field), "FREEMKV");
}
}
File diff suppressed because it is too large Load Diff
+362
View File
@@ -0,0 +1,362 @@
//! `dir://` as an image-level SOURCE: a synthetic UDF volume over a folder.
//!
//! A user's extracted disc — a DVD `VIDEO_TS/` or a Blu-ray `BDMV/`, typically
//! a MakeMKV-style backup — has files but no sectors, and everything above the
//! sector layer in this crate wants sectors: `Disc::scan_image`, `UdfFs`,
//! `ifo.rs`, `mpls.rs`, `clpi.rs` and the mux all read through a
//! [`SectorSource`]. [`DirImage`] supplies one.
//!
//! The trick is that nothing is emulated. A real, minimal, valid UDF 1.02
//! volume is synthesized over the folder:
//!
//! * **Metadata sectors** (anchors, the volume descriptor sequences, the File
//! Set Descriptor, every File Entry, every directory's FID list) are encoded
//! into RAM by [`encode`] — a few MiB even for a large Blu-ray.
//! * **Data sectors** are not materialized at all. Each one maps to a byte
//! range of a real file, read on demand.
//!
//! So `udf::read_filesystem` parses this image by exactly the same code path it
//! parses a real disc with, and every consumer above it is unchanged. The cost
//! is that a single-partition synthetic volume never exercises the UDF 2.50
//! Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
//! this module's tests do not cover that block and must not be read as if they
//! did.
//!
//! What this module deliberately does NOT do:
//!
//! * **3D / SSIF** — rejected up front ([`Error::DirImageSsifUnsupported`]).
//! An SSIF aliases the same sectors as its base and dependent `.m2ts`; the
//! planner allocates disjoint extents, so a 3D folder would produce silently
//! wrong output.
//! * **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
//! * **Encrypted folders** — a folder whose content is still AACS-scrambled is
//! rejected by the caller-side probe, not decrypted here.
mod encode;
mod layout;
use crate::error::{Error, Result};
#[cfg(target_os = "linux")]
use crate::io::file_sector_source::linux::drop_window;
#[cfg(target_os = "macos")]
use crate::io::file_sector_source::macos::drop_window;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use crate::io::file_sector_source::other::drop_window;
#[cfg(target_os = "windows")]
use crate::io::file_sector_source::windows::drop_window;
use crate::sector::SectorSource;
use encode::{MetaSectors, SECTOR};
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
use std::path::{Path, PathBuf};
/// How many host files may be held open at once.
///
/// A Blu-ray `BDMV/` can exceed a thousand files while macOS `RLIMIT_NOFILE`
/// defaults to 256, so "open every file up front" is not available. Reads are
/// overwhelmingly sequential through one large stream file at a time, so a
/// small LRU keeps the hit rate near 1 while bounding descriptors.
const HANDLE_CACHE: usize = 16;
/// One file's bytes at one place in the image.
#[derive(Debug, Clone)]
struct DataRange {
/// Absolute first block.
start_lba: u32,
/// Blocks covered (the last one may be partially used, and is zero-padded).
sectors: u32,
/// Index into [`DirImage::files`].
file: usize,
/// Byte offset within the file at which this range's bytes begin.
offset: u64,
/// Byte length of the range.
bytes: u64,
}
/// A file the image reads through.
#[derive(Debug)]
struct FileRef {
host: PathBuf,
disc_path: String,
size: u64,
/// Host mtime at plan time — see `layout::FileNode::mtime` for why size
/// alone is not enough.
mtime: Option<std::time::SystemTime>,
}
/// A synthesized UDF disc image over a host directory.
///
/// Owns everything it reads through (`PathBuf`s and its own file handles), so
/// it is `Send + 'static` and can be moved into `build_iso_pipeline`, which
/// hands it to `PrefetchedSectorSource`'s producer thread.
pub struct DirImage {
meta: MetaSectors,
/// Sorted by `start_lba`, non-overlapping.
ranges: Vec<DataRange>,
files: Vec<FileRef>,
open: Vec<(usize, File)>,
total_sectors: u32,
volume_id: String,
data_bytes: u64,
}
impl std::fmt::Debug for DirImage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DirImage")
.field("volume_id", &self.volume_id)
.field("total_sectors", &self.total_sectors)
.field("files", &self.files.len())
.field("meta_sectors", &self.meta.len())
.finish()
}
}
impl DirImage {
/// Plan and encode an image over `root`.
///
/// Every error is decided here, at plan time, where it can name the file
/// responsible — the read path is deliberately left with nothing to decide
/// except "this file changed underneath me".
pub fn open(root: &Path) -> Result<Self> {
let plan = layout::plan(root)?;
let meta = encode::encode(&plan)?;
let mut nodes = Vec::new();
layout::flatten(&plan.root, &mut nodes);
let mut files = Vec::with_capacity(nodes.len());
let mut ranges = Vec::new();
for (idx, node) in nodes.iter().enumerate() {
// Carry the plan-time mtime ONLY for files whose CONTENT the plan
// read — the DVD IFOs, whose bytes 0xC0/0xC4 decide where every VOB
// is placed (`layout::place_video_ts` -> `read_head`).
//
// For every other file the plan depends on the SIZE alone, and size
// is already checked. Comparing mtime on those buys nothing and
// costs real false positives: disc backups commonly live on
// exFAT/FAT32, which stores local time, so a long rip spanning a
// DST transition sees a whole-hour shift on a file nobody touched
// and would abort hours in, blaming a change that did not happen.
// The multi-gigabyte VOBs are exactly the files a long rip re-opens
// after the handle cache evicts them.
let content_sensitive = node
.disc_path
.rsplit('.')
.next()
.is_some_and(|e| e.eq_ignore_ascii_case("IFO"));
files.push(FileRef {
host: node.host.clone(),
disc_path: node.disc_path.clone(),
size: node.size,
mtime: content_sensitive.then_some(node.mtime).flatten(),
});
let mut offset = 0u64;
for e in &node.extents {
ranges.push(DataRange {
start_lba: plan.part_start + e.lba,
sectors: (e.bytes as u64).div_ceil(SECTOR as u64) as u32,
file: idx,
offset,
bytes: e.bytes as u64,
});
offset += e.bytes as u64;
}
}
ranges.sort_by_key(|r| r.start_lba);
debug_assert!(
ranges
.windows(2)
.all(|w| w[0].start_lba + w[0].sectors <= w[1].start_lba),
"planned data ranges must not overlap"
);
let data_bytes = layout::total_data_bytes(&plan.root);
tracing::info!(
target: "freemkv::dirimage",
volume_id = %plan.volume_id,
files = files.len(),
dirs = plan.dir_count,
meta_blocks = layout::metadata_block_count(&plan.root),
total_sectors = plan.total_sectors,
"synthesized UDF image over directory"
);
Ok(Self {
meta,
ranges,
files,
open: Vec::new(),
total_sectors: plan.total_sectors,
volume_id: plan.volume_id,
data_bytes,
})
}
/// UDF volume identifier the image declares (the folder's own name).
pub fn volume_id(&self) -> &str {
&self.volume_id
}
/// Total bytes of real file content the image carries — the folder's size,
/// not the image's (which also counts metadata and inter-file gaps).
pub fn data_bytes(&self) -> u64 {
self.data_bytes
}
/// The range covering `lba`, if any.
fn range_at(&self, lba: u32) -> Option<&DataRange> {
let i = self.ranges.partition_point(|r| r.start_lba <= lba);
let r = self.ranges.get(i.checked_sub(1)?)?;
(lba < r.start_lba + r.sectors).then_some(r)
}
/// Borrow an open handle for `file`, opening it (and evicting the
/// least-recently-used handle) if necessary.
///
/// Opening is also where the plan is revalidated. A folder is not a disc:
/// a file can be shortened or replaced between planning and reading, and
/// zero-filling the difference would turn "the user deleted something"
/// into corrupt output at exit 0. The size is re-checked here, and a
/// truncation that happens while the handle is already open is caught by
/// the short read in [`Self::fill`].
fn handle(&mut self, file: usize) -> Result<&mut File> {
if let Some(pos) = self.open.iter().position(|(i, _)| *i == file) {
// `open` is ordered most-recently-used first.
let entry = self.open.remove(pos);
self.open.insert(0, entry);
return Ok(&mut self.open[0].1);
}
let f = File::open(&self.files[file].host).map_err(Error::from)?;
let md = f.metadata().map_err(Error::from)?;
// Size AND mtime. Size alone is content-blind, and this plan depends on
// content: a DVD's VOB placement comes from bytes 0xC0/0xC4 of its IFO,
// and an IFO rewritten in place keeps its length because IFOs occupy a
// whole number of sectors. The size check would pass while every title
// extent pointed at the wrong sectors — corrupt video behind an intact
// structure, reported complete at exit 0.
//
// Only compared when both sides have a timestamp; a platform or
// filesystem that reports none simply falls back to the size check
// rather than failing every read.
let changed_size = md.len() != self.files[file].size;
let changed_mtime = match (self.files[file].mtime, md.modified().ok()) {
(Some(planned), Some(live)) => planned != live,
_ => false,
};
if changed_size || changed_mtime {
return Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
});
}
if self.open.len() >= HANDLE_CACHE {
self.open.pop();
}
self.open.insert(0, (file, f));
Ok(&mut self.open[0].1)
}
/// Fill `out` (a whole number of sectors) from one data range, starting at
/// `lba`. `out` is already zeroed, so a file's tail sector comes back
/// zero-padded — which is exactly what `file_extents`' `div_ceil(2048)`
/// (`udf.rs:816`) makes every consumer expect.
fn fill(&mut self, r: &DataRange, lba: u32, out: &mut [u8]) -> Result<()> {
let within = (lba - r.start_lba) as u64 * SECTOR as u64;
let want = (r.bytes.saturating_sub(within)).min(out.len() as u64) as usize;
if want == 0 {
return Ok(());
}
let at = r.offset + within;
let file = r.file;
let h = self.handle(file)?;
h.seek(SeekFrom::Start(at)).map_err(Error::from)?;
let res = h.read_exact(&mut out[..want]);
if res.is_ok() {
// Release the window just read, every time.
//
// The ISO source accumulates and drops in chunks because it reads
// one file linearly, so a running start offset always names the
// bytes it has consumed. Reads here jump between files, so there is
// no single cursor to accumulate against — an accumulated byte
// count paired with one read's offset names 1/Nth of what was
// actually consumed and leaves the rest pinned, which is how the
// first version of this got it wrong.
//
// Dropping per read costs one advisory syscall per batch (4-16 MiB),
// which is nothing against the read itself, and it is correct
// regardless of how reads interleave across files.
if let Some((_, fh)) = self.open.iter().find(|(i, _)| *i == file) {
drop_window(fh, at, want as u64);
}
}
match res {
Ok(()) => Ok(()),
// The file shrank while the handle was open. Same verdict as the
// size check in `handle`, reached the other way.
Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => {
Err(Error::DirImageFileChanged {
path: self.files[file].disc_path.clone(),
})
}
Err(e) => Err(Error::from(e)),
}
}
}
impl SectorSource for DirImage {
fn capacity_sectors(&self) -> u32 {
self.total_sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let need = count as usize * SECTOR;
if buf.len() < need {
return Err(Error::UdfBufferTooSmall);
}
buf[..need].fill(0);
// Walk the request in RUNS, not sector by sector. A mux batch is 8192
// sectors and almost always lands entirely inside one stream file's
// extent; per-sector seek+read would issue 8192 syscalls for what is
// one 16 MiB sequential read.
let mut i = 0u32;
while i < count as u32 {
// Checked: callers saturate their LBAs (`disc/dvd.rs` builds a cell
// start as `vob_start_sector.saturating_add(cell.first_sector)`, and
// the prefetcher adds an offset the same way), so a crafted IFO can
// present a request at the very top of the address space. Wrapping
// here would fold `at` back to a LOW sector and hand the muxer a
// different file's bytes with nothing reported.
let Some(at) = lba.checked_add(i) else {
break;
};
let off = i as usize * SECTOR;
if let Some(s) = self.meta.get(&at) {
buf[off..off + SECTOR].copy_from_slice(&s[..]);
i += 1;
continue;
}
// Metadata blocks all sit below the data floor, so a data range is
// never interrupted by one.
match self.range_at(at).cloned() {
Some(r) => {
let run = (r.start_lba + r.sectors - at).min(count as u32 - i);
let end = off + run as usize * SECTOR;
self.fill(&r, at, &mut buf[off..end])?;
i += run;
}
// A gap between planned extents. Reads as zeros, exactly as an
// unrecorded sector of a real image does.
None => i += 1,
}
}
Ok(need)
}
}
#[cfg(test)]
mod tests;
File diff suppressed because it is too large Load Diff
+18
View File
@@ -85,6 +85,16 @@ impl Disc {
// Per-PlayItem Clip entries (differing in/out times) still get
// recorded.
let mut seen_clips: std::collections::HashSet<String> = std::collections::HashSet::new();
// Byte offset of the next extent within the TITLE'S FEED — the
// concatenation of `extents` in the order the mux reads them. Each
// clip's span is recorded so a frame's source offset identifies its
// clip by lookup rather than by guessing from timestamps, which is
// ambiguous inside an overlap. A clip referenced a SECOND time pushes
// no extents (see `first_ref`), so it reuses the span of its first
// reference — the same bytes, read once.
let mut feed_pos: u64 = 0;
let mut spans: std::collections::HashMap<String, (u64, u64)> =
std::collections::HashMap::new();
for play_item in &parsed.play_items {
let clip_dur = play_item.out_time.saturating_sub(play_item.in_time) as f64 / 45000.0;
@@ -133,19 +143,27 @@ impl Disc {
}),
};
if let Some(file_exts) = file_exts {
let span_start = feed_pos;
for (lba, sectors) in file_exts {
if sectors > 0 && lba > 0 {
extents.push(Extent {
start_lba: lba,
sector_count: sectors,
});
feed_pos = feed_pos.saturating_add(
sectors as u64 * crate::consts::SECTOR_BYTES as u64,
);
}
}
if feed_pos > span_start {
spans.insert(play_item.clip_id.clone(), (span_start, feed_pos));
}
}
}
}
clips.push(Clip {
feed_span: spans.get(&play_item.clip_id).copied(),
clip_id: play_item.clip_id.clone(),
in_time: play_item.in_time,
out_time: play_item.out_time,
+103 -11
View File
@@ -300,22 +300,62 @@ impl Disc {
) -> DecryptKeys {
// Gather the title VOB extents for this VTS (VTS_xx_1.VOB .. _9.VOB;
// VTS_xx_0.VOB is the menu and is clear, so excluded from the crack).
// Gather this VTS's title VOBs BY NAME, ascending.
//
// `planned` comes from walking the UDF directory, which yields File
// Identifier Descriptors in on-disc order — authoring order, with no
// guarantee it matches playback order. Relying on it would leave the
// crack starting wherever the disc's directory happened to list first.
//
// DVD-Video numbers a title set's VOBs in playback order by spec
// (VTS_xx_1.VOB .. VTS_xx_9.VOB, single digit), so ascending name IS
// playback order and it is deterministic regardless of directory
// layout. That matters because ORDER decides correctness here: see the
// note below on the shared crack budget.
let mut files: Vec<&PlannedFile> = planned
.iter()
.filter(|pf| {
vts_group_of(&pf.disc_name).as_deref() == Some(vts) && is_title_vob(&pf.disc_name)
})
.collect();
// Case-INSENSITIVE, matching `vts_group_of`/`is_title_vob`, which
// selected these files case-insensitively. A byte-wise sort put
// 'V' (0x56) before 'v' (0x76), so a set holding `vts_01_1.vob`
// alongside `VTS_01_2.VOB` — reachable on a case-sensitive volume —
// started the crack at part 2 and could exhaust the budget in a clear
// run, which is the whole failure this ordering exists to prevent.
files.sort_by_key(|f| f.disc_name.to_ascii_uppercase());
let mut extents: Vec<crate::disc::Extent> = Vec::new();
for pf in planned {
if vts_group_of(&pf.disc_name).as_deref() == Some(vts) && is_title_vob(&pf.disc_name) {
for &(abs_lba, byte_len) in &pf.extents {
extents.push(crate::disc::Extent {
start_lba: abs_lba,
sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
});
}
for pf in files {
for &(abs_lba, byte_len) in &pf.extents {
extents.push(crate::disc::Extent {
start_lba: abs_lba,
sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
});
}
}
if extents.is_empty() {
return base_keys.clone();
}
// Largest extent first (movie body), matching the scan heuristic.
extents.sort_by_key(|e| std::cmp::Reverse(e.sector_count));
// PLAYBACK ORDER — do NOT sort. This is the 1.5.1 garbage bug, and it
// grew back here in a new code path: the comment this replaced said it
// "matched the scan heuristic", but that heuristic WAS the bug and was
// already fixed in `Disc::decrypt_keys_for_title`, which documents the
// rule ("PLAYBACK ORDER, never largest-cell-first") and pins it with
// `decrypt_keys_for_title_scans_playback_order_not_largest_first`.
//
// Why order decides correctness: a CSS DVD's biggest cell opens with a
// long CLEAR run, and `crack_key`'s sector budget is shared across the
// whole extent list. Starting at the largest cell can exhaust the budget
// without ever MEETING a scrambled sector — and CSS recovers the title
// key from scrambled data itself, so the scan has to reach some. The
// crack then returns None, this function falls back to `base_keys`, and
// every VOB in the VTS is descrambled with the wrong key: corrupt PES
// behind an intact header, written out as a complete extract at exit 0.
//
// The files above are ordered by name and each file's own extents stay
// in file order, which together is playback order for a DVD title set.
// Crack against the raw (still-scrambled) inner reader, NOT the
// decrypting view — `crack_key` runs the descrambler itself.
match crate::css::crack_key(dec.inner_mut(), &extents, 64) {
@@ -684,7 +724,53 @@ fn available_space(dir: &Path) -> Option<u64> {
Some(avail)
}
#[cfg(not(unix))]
/// Windows has no `statvfs`. This returned `None` unconditionally, which does
/// not merely skip a test — it skipped the free-space GATE, so a Windows user
/// extracting a disc to a full volume got a confusing failure part-way through
/// instead of a clear refusal up front. libfreemkv ships a Windows GUI, so that
/// is the platform where the friendly error matters most.
///
/// `GetDiskFreeSpaceExW` is declared directly against kernel32, matching how
/// `scsi::windows` already reaches the Win32 API rather than pulling in a
/// binding crate for one call.
#[cfg(windows)]
fn available_space(dir: &Path) -> Option<u64> {
use std::os::windows::ffi::OsStrExt;
unsafe extern "system" {
fn GetDiskFreeSpaceExW(
lpDirectoryName: *const u16,
lpFreeBytesAvailableToCaller: *mut u64,
lpTotalNumberOfBytes: *mut u64,
lpTotalNumberOfFreeBytes: *mut u64,
) -> i32;
}
// Wide, NUL-terminated. An interior NUL cannot reach the API, so reject it
// rather than silently truncating the path and measuring the wrong volume.
let mut wide: Vec<u16> = dir.as_os_str().encode_wide().collect();
if wide.contains(&0) {
return None;
}
wide.push(0);
let mut avail: u64 = 0;
// FreeBytesAvailableToCaller, not TotalNumberOfFreeBytes: it accounts for
// per-user quotas, which is what "can I actually write this much" means and
// what `statvfs`'s `f_bavail` gives on the unix side.
let ok = unsafe {
GetDiskFreeSpaceExW(
wide.as_ptr(),
&mut avail,
std::ptr::null_mut(),
std::ptr::null_mut(),
)
};
if ok == 0 { None } else { Some(avail) }
}
/// Neither unix nor Windows: no way to ask, so the gate is skipped.
#[cfg(not(any(unix, windows)))]
fn available_space(_dir: &Path) -> Option<u64> {
None
}
@@ -1288,6 +1374,9 @@ mod tests {
// mirroring css::mod tests' `crackable_sector`.
let seed = [0x11u8, 0x22, 0x33, 0x44, 0x55];
let mut plain = vec![0u8; 2048];
// Pack start code: a real scrambled sector is an MPEG-2 PS pack, and
// the descrambler requires it before trusting byte 0x14.
plain[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
plain[0x14] = 0x10; // scramble flag
let pat: Vec<u8> = (0..8)
.map(|k| (0xA0u8.wrapping_add(k as u8)) ^ 0x5A)
@@ -1759,6 +1848,9 @@ mod tests {
// VTS groups regardless of which extents were gathered, masking
// this exact regression.
plain[0x00..0x04].copy_from_slice(&crate::css::PACK_START);
// Pack start code: a real scrambled sector is an MPEG-2 PS pack, and
// the descrambler requires it before trusting byte 0x14.
plain[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
plain[0x14] = 0x10; // scramble flag
let pat: Vec<u8> = (0..8)
.map(|k| (0xA0u8.wrapping_add(k as u8) ^ marker) ^ 0x5A)
+3
View File
@@ -497,6 +497,7 @@ fn compose_xpl_titles(
extents.extend_from_slice(exts);
size_bytes += *size;
parts.push(Clip {
feed_span: None,
clip_id: orig
.rsplit_once('.')
.map(|(b, _)| b)
@@ -639,6 +640,7 @@ impl Disc {
extents.extend_from_slice(exts);
size_bytes += *size;
parts.push(Clip {
feed_span: None,
clip_id: orig
.rsplit_once('.')
.map(|(b, _)| b)
@@ -690,6 +692,7 @@ impl Disc {
duration_secs: 0.0,
size_bytes: *size,
clips: vec![Clip {
feed_span: None,
clip_id,
in_time: 0,
out_time: 0,
+49 -4
View File
@@ -173,6 +173,17 @@ pub struct Clip {
pub duration_secs: f64,
/// Source packet count (from CLPI, 0 if unavailable)
pub source_packets: u32,
/// Byte range this clip's stream occupies within the TITLE'S FEED — the
/// concatenation of the title's extents, in the order the mux reads them.
/// `None` when it could not be determined.
///
/// This is PROVENANCE, and it is what makes clip assignment a lookup
/// instead of a guess. During an overlap two clips' mark ranges both
/// contain the same timestamp, so no rule over timestamps alone can say
/// which clip a frame came from — four audit rounds each fixed one such
/// rule and broke another. A frame carries its source byte offset
/// (`PesFrame::source`), and that offset falls in exactly one clip's span.
pub feed_span: Option<(u64, u64)>,
}
/// A stream within a title.
@@ -468,6 +479,25 @@ pub struct Extent {
/// Union a set of extents into sorted, merged, disjoint `(start_lba,
/// sector_count)` ranges — the pure, testable core of
/// [`Disc::encrypted_content_ranges`]. Reuses [`crate::udf::merge_ranges`].
/// Is this disc structurally AACS-encrypted — i.e. does it carry an AACS
/// directory?
///
/// THE ONE definition of that question. `Disc::identify` (the fast path, which
/// only reads the filesystem) and `Disc::scan_with` (the full scan) both need
/// it, and both used to spell the same two `find_dir` calls out by hand. They
/// agreed today; nothing made them agree tomorrow. Adding a third AACS
/// location, or excluding an empty placeholder directory, to one copy and not
/// the other would silently desync the fast identify from the full scan — the
/// same disc reported encrypted by one and clear by the other.
///
/// This is STRUCTURAL, not cryptographic: it says the tree looks like an
/// encrypted disc, not that any sector actually is. A folder copied verbatim
/// from a decrypted disc keeps its `AACS/` and answers true here (see
/// `session::scan_dir`, which corrects for exactly that).
pub(crate) fn aacs_dir_present(udf_fs: &crate::udf::UdfFs) -> bool {
udf_fs.find_dir("/AACS").is_some() || udf_fs.find_dir("/BDMV/AACS").is_some()
}
fn merged_extents<'a>(extents: impl Iterator<Item = &'a Extent>) -> Vec<(u32, u32)> {
let mut ranges: Vec<(u32, u32)> = extents.map(|e| (e.start_lba, e.sector_count)).collect();
ranges.sort_by_key(|r| r.0);
@@ -1617,8 +1647,7 @@ impl Disc {
// (no titles needed: BD/UHD/FMTS come from the MKB generation). It no
// longer defaults to BluRay or defers UHD/FMTS to the full scan.
let format = Self::detect_disc_format(&mut buffered, &udf_fs, &[]);
let encrypted =
udf_fs.find_dir("/AACS").is_some() || udf_fs.find_dir("/BDMV/AACS").is_some();
let encrypted = aacs_dir_present(&udf_fs);
let layers = if capacity > 24_000_000 { 2 } else { 1 };
Ok(DiscId {
@@ -1961,6 +1990,21 @@ impl Disc {
Self::read_aacs_inputs_from_reader(&mut reader, &udf_fs)
}
/// Same as [`Disc::read_aacs_inputs`] but over an extracted disc FOLDER.
///
/// `dir://` is an image-level source: `dirimage` synthesizes a real UDF
/// volume over the folder, so the AACS inputs are read by exactly the same
/// reader path an ISO uses. Without this the online key fetch was silently
/// unavailable for folders — the CLI's fetch helper matched only `Iso` and
/// returned None for a folder, so the same disc that fetched its key fine
/// as an ISO failed as an extracted directory. A sink is a sink: any input
/// has to work with any output, and that includes the key path.
pub fn read_aacs_inputs_from_dir(dir: &std::path::Path) -> Result<(Vec<u8>, Vec<u8>, u8)> {
let mut reader = crate::dirimage::DirImage::open(dir)?;
let udf_fs = udf::read_filesystem(&mut reader)?;
Self::read_aacs_inputs_from_reader(&mut reader, &udf_fs)
}
/// Same as [`Disc::read_aacs_inputs`] but reads from a live drive. The
/// out-of-band Unit Key path fetches the disc's key files from the drive,
/// resolves a key from them however it likes, then applies it via
@@ -1989,8 +2033,7 @@ impl Disc {
let scan_with_t0 = std::time::Instant::now();
tracing::info!(target: "freemkv::scan", phase = "scan_with", "begin");
// 2. Resolve encryption (AACS, CSS, or none)
let encrypted =
udf_fs.find_dir("/AACS").is_some() || udf_fs.find_dir("/BDMV/AACS").is_some();
let encrypted = aacs_dir_present(&udf_fs);
let (aacs, aacs_error) = if !encrypted {
(None, None)
@@ -3994,6 +4037,7 @@ mod tests {
t.size_bytes = size_bytes;
t.clips = (0..n_clips)
.map(|i| Clip {
feed_span: None,
clip_id: format!("{i:05}"),
in_time: 0,
out_time: 1,
@@ -4394,6 +4438,7 @@ mod tests {
size_bytes,
clips: (0..n_clips)
.map(|i| Clip {
feed_span: None,
clip_id: format!("{i:05}"),
in_time: 0,
out_time: 0,
+12 -2
View File
@@ -1406,8 +1406,18 @@ mod halt_tests {
let r = sleep_until_halted(&flag, Duration::from_secs(10));
assert!(matches!(r, Err(Error::Halted)));
let waited = t0.elapsed();
// Flag flipped at ~150 ms; we wake within one 100 ms slice → <300 ms.
assert!(waited < Duration::from_millis(350), "waited {waited:?}");
// What this test is for: the sleep must end because the flag flipped,
// NOT because the 10 s timeout expired. Anything comfortably under
// that proves it, and the lower bound proves it did not return early
// for some other reason.
//
// The upper bound used to be 350 ms — flag at ~150 ms plus one 100 ms
// poll slice, with a little slack. That measures the SCHEDULER, not
// this function: a loaded CI runner took 377 ms and failed, which says
// nothing about whether the wake worked. Bound it well below the
// timeout instead, so the assertion still distinguishes the two
// outcomes it exists to distinguish.
assert!(waited < Duration::from_secs(2), "waited {waited:?}");
assert!(waited >= Duration::from_millis(140), "waited {waited:?}");
}
+191
View File
@@ -62,6 +62,7 @@ pub const E_MAPFILE_INVALID: u16 = 6011;
pub const E_SELECTION_PID_UNKNOWN: u16 = 6014;
pub const E_UDF_BUFFER_TOO_SMALL: u16 = 6012;
pub const E_UDF_NOT_FILESYSTEM: u16 = 6013;
pub const E_IMAGE_TRUNCATED: u16 = 6015;
// AACS (7xxx)
pub const E_AACS_NO_KEYS: u16 = 7000;
@@ -170,6 +171,41 @@ pub const E_DIR_INSUFFICIENT_SPACE: u16 = 9027;
pub const E_DIR_NAME_COLLISION: u16 = 9028;
/// A `dir://` create_dir_all / file write / rename failed.
pub const E_DIR_WRITE_FAILED: u16 = 9029;
/// A `dir://` SOURCE folder carries `BDMV/STREAM/SSIF/` (Blu-ray 3D). The
/// scanner detects SSIF unconditionally and would rip it as 3D, but the
/// synthetic-image planner has no extent-aliasing support (an SSIF interleaves
/// the same sectors as the base/dependent `.m2ts`), so the output would be
/// silently wrong. Rejected up front instead.
pub const E_DIR_IMAGE_SSIF_UNSUPPORTED: u16 = 9061;
/// A `dir://` SOURCE `VIDEO_TS` folder's IFO-declared VOB offsets cannot be
/// satisfied by any placement: the required start sector of a VOB lies BELOW
/// the end of the file that must precede it. Carries the offending file's disc
/// path. A silently misplaced VOB would rip the wrong sectors.
pub const E_DIR_IMAGE_PLACEMENT: u16 = 9062;
/// A `dir://` SOURCE folder still carries live AACS-encrypted content: it has
/// an `AACS/` directory AND the sampled content units are genuinely scrambled.
/// `dir://` sources are decrypted-folder only.
pub const E_DIR_IMAGE_ENCRYPTED: u16 = 9063;
/// A `dir://` SOURCE folder holds no disc structure the image synthesizer
/// understands (no `BDMV/`, no `VIDEO_TS/`).
pub const E_DIR_IMAGE_UNSUPPORTED_TREE: u16 = 9064;
/// A file inside a `dir://` SOURCE folder changed (shrank / was removed)
/// between planning and reading. Zero-filling the gap would produce corrupt
/// output at exit 0, so the read fails instead. Carries the disc path.
pub const E_DIR_IMAGE_FILE_CHANGED: u16 = 9065;
/// A `dir://` SOURCE folder does not fit a 32-bit sector address space
/// (> 2^32 sectors ≈ 8 TiB), or holds more entries than a UDF tree can carry.
pub const E_DIR_IMAGE_TOO_LARGE: u16 = 9066;
/// A name in the folder is too long to record in a UDF directory entry: the
/// File Identifier Descriptor stores the encoded length in one byte.
pub const E_DIR_NAME_TOO_LONG: u16 = 9067;
/// One directory in the folder holds more subdirectories than a UDF link count
/// can express (it is 16 bits, one per child plus one for its own entry).
pub const E_DIR_IMAGE_FANOUT: u16 = 9068;
/// A title's clip marks excluded more frames than they kept.
pub const E_SEAM_PLAN_DROPPED_MOST: u16 = 9069;
/// A sink finished having written no frames at all.
pub const E_SINK_WROTE_NOTHING: u16 = 9070;
pub const E_M2TS_PACKET_MALFORMED: u16 = 9021;
/// A `network://` output target resolved to no address that is safe to
/// connect to (every resolved IP was loopback / private / link-local /
@@ -243,6 +279,8 @@ pub const E_SYNC_WORKER_LOST: u16 = 9057;
/// READ CAPACITY returned a short or overflowing transfer.
pub const E_DISC_CAPACITY_MALFORMED: u16 = 9047;
pub const E_DRIVE_INQUIRY_SHORT: u16 = 9058;
pub const E_SHORT_IMAGE_READ: u16 = 9059;
pub const E_EMPTY_IMAGE: u16 = 9060;
// ── Error enum ──────────────────────────────────────────────────────────────
@@ -371,6 +409,22 @@ pub enum Error {
index: usize,
count: usize,
},
/// An image-level write read fewer bytes than the sector count it asked for.
///
/// Distinct from [`Error::IoError`] on purpose: the read SUCCEEDED and simply
/// returned less than a whole sector run, which for a file-backed source means
/// the source is shorter than its declared capacity. Zero-filling the gap
/// would produce an image that looks complete and is not — the worst outcome
/// for a copy someone intends to keep — so it is an error instead.
ShortImageRead {
lba: u32,
expected: u32,
got: u32,
},
/// An image-level write was asked for zero sectors. A zero-byte image is
/// never the intent, and reporting it here names the problem at the point it
/// is knowable rather than leaving an empty file behind.
EmptyImage,
IfoParse,
/// A title produced NO muxable frames: the mux driver's pump ended without
/// any video track's `codec_private` resolving, or the MKV muxer reached
@@ -408,6 +462,14 @@ pub enum Error {
MapfileInvalid {
kind: &'static str,
},
/// The image a mapfile describes is shorter than the mapfile's own total
/// size, so the two no longer agree about the same disc. Resuming against
/// it would treat the absent tail as already recovered. `have` and `want`
/// are byte lengths; a missing file reports `have == 0`.
ImageTruncated {
have: u64,
want: u64,
},
// AACS (7xxx)
AacsNoKeys,
@@ -715,6 +777,58 @@ pub enum Error {
DirWriteFailed {
errno: Option<i32>,
},
/// A `dir://` SOURCE folder carries `BDMV/STREAM/SSIF/` (Blu-ray 3D),
/// which the synthetic-image planner cannot represent. See
/// [`E_DIR_IMAGE_SSIF_UNSUPPORTED`].
DirImageSsifUnsupported,
/// A `dir://` SOURCE `VIDEO_TS` placement constraint is unsatisfiable.
/// `path` is the disc path of the file that could not be placed.
DirImagePlacement {
path: String,
},
/// A `dir://` SOURCE folder still carries live AACS-encrypted content.
DirImageEncrypted,
/// A `dir://` SOURCE folder holds no recognized disc structure.
DirImageUnsupportedTree,
/// A file in a `dir://` SOURCE folder changed between plan and read.
DirImageFileChanged {
path: String,
},
/// A `dir://` SOURCE folder exceeds the addressable image size.
DirImageTooLarge,
/// A name is too long to record in a UDF directory entry.
///
/// The File Identifier Descriptor stores the encoded name length in ONE
/// byte, so a name whose OSTA CS0 encoding exceeds 254 bytes cannot be
/// described. Truncating the length field instead would desynchronise the
/// whole directory — every later entry in it would be read from the wrong
/// offset — so an over-long name is refused while the tree is still being
/// planned.
DirNameTooLong {
path: String,
},
/// One directory holds more subdirectories than a UDF link count can express.
///
/// A directory's File Entry records its link count in 16 bits, and that
/// count is one per child directory plus one for its own entry in its
/// parent. Beyond that the count silently wraps, so the tree is refused.
DirImageFanout {
path: String,
},
/// A title's PlayItem marks excluded more frames than they kept.
///
/// Placing clips by their marks drops whatever falls outside them, which is
/// correct at a join — a disc stores the join twice. Discarding the
/// majority of a title is not a join; it means the marks do not describe
/// the clock the frames are on. Refused rather than written, because the
/// result otherwise looks like a complete file containing seconds of a
/// feature.
SeamPlanDroppedMost {
dropped: u64,
written: u64,
},
/// A sink finished having written no frames at all.
SinkWroteNothing,
}
impl Error {
@@ -744,6 +858,8 @@ impl Error {
Error::UdfNotFilesystem => E_UDF_NOT_FILESYSTEM,
Error::UdfBufferTooSmall => E_UDF_BUFFER_TOO_SMALL,
Error::DiscTitleRange { .. } => E_DISC_TITLE_RANGE,
Error::ShortImageRead { .. } => E_SHORT_IMAGE_READ,
Error::EmptyImage => E_EMPTY_IMAGE,
Error::IfoParse => E_IFO_PARSE,
Error::MkvInvalid => E_MKV_INVALID,
Error::MkvSourceInvalid => E_MKV_SOURCE_INVALID,
@@ -752,6 +868,7 @@ impl Error {
Error::NoStreams => E_NO_STREAMS,
Error::SelectionPidUnknown { .. } => E_SELECTION_PID_UNKNOWN,
Error::MapfileInvalid { .. } => E_MAPFILE_INVALID,
Error::ImageTruncated { .. } => E_IMAGE_TRUNCATED,
Error::AacsNoKeys => E_AACS_NO_KEYS,
Error::AacsCertShort => E_AACS_CERT_SHORT,
Error::AacsAgidAlloc => E_AACS_AGID_ALLOC,
@@ -830,6 +947,16 @@ impl Error {
Error::DirInsufficientSpace { .. } => E_DIR_INSUFFICIENT_SPACE,
Error::DirNameCollision { .. } => E_DIR_NAME_COLLISION,
Error::DirWriteFailed { .. } => E_DIR_WRITE_FAILED,
Error::DirImageSsifUnsupported => E_DIR_IMAGE_SSIF_UNSUPPORTED,
Error::DirImagePlacement { .. } => E_DIR_IMAGE_PLACEMENT,
Error::DirImageEncrypted => E_DIR_IMAGE_ENCRYPTED,
Error::DirImageUnsupportedTree => E_DIR_IMAGE_UNSUPPORTED_TREE,
Error::DirNameTooLong { .. } => E_DIR_NAME_TOO_LONG,
Error::DirImageFanout { .. } => E_DIR_IMAGE_FANOUT,
Error::SeamPlanDroppedMost { .. } => E_SEAM_PLAN_DROPPED_MOST,
Error::SinkWroteNothing => E_SINK_WROTE_NOTHING,
Error::DirImageFileChanged { .. } => E_DIR_IMAGE_FILE_CHANGED,
Error::DirImageTooLarge => E_DIR_IMAGE_TOO_LARGE,
}
}
}
@@ -936,6 +1063,18 @@ impl std::fmt::Display for Error {
},
Error::Halted => write!(f, "E{}", self.code()),
Error::UdfNotFound { path } => write!(f, "E{}: {}", self.code(), path),
Error::SeamPlanDroppedMost { dropped, written } => {
write!(f, "E{} {dropped}/{written}", self.code())
}
Error::ShortImageRead { lba, expected, got } => {
write!(f, "E{} {lba} {expected} {got}", self.code())
}
Error::DirImagePlacement { path }
| Error::DirImageFileChanged { path }
| Error::DirNameTooLong { path }
| Error::DirImageFanout { path } => {
write!(f, "E{}: {}", self.code(), path)
}
Error::DiscTitleRange { index, count } => {
write!(f, "E{}: {}/{}", self.code(), index, count)
}
@@ -1087,6 +1226,22 @@ impl From<Error> for std::io::Error {
// 9027 insufficient space / 9029 write failed: a filesystem-level
// failure, not bad input.
E_DIR_INSUFFICIENT_SPACE | E_DIR_WRITE_FAILED => std::io::ErrorKind::Other,
// dir:// SOURCE gates (9061–9064, 9066–9068): the folder handed in
// cannot be turned into a disc image — a 3D SSIF tree, an
// unsatisfiable VIDEO_TS placement, still-encrypted content, an
// unrecognized tree, one too large to address, a name too long to
// record, or a directory whose fan-out overflows its link count.
// All are properties of the input, decided before a byte is read.
E_DIR_IMAGE_SSIF_UNSUPPORTED
| E_DIR_IMAGE_PLACEMENT
| E_DIR_IMAGE_ENCRYPTED
| E_DIR_IMAGE_UNSUPPORTED_TREE
| E_DIR_IMAGE_TOO_LARGE
| E_DIR_NAME_TOO_LONG
| E_DIR_IMAGE_FANOUT => std::io::ErrorKind::InvalidInput,
// 9065: the folder changed underneath a running read. Not bad input
// at plan time — a mid-flight mutation of the source.
E_DIR_IMAGE_FILE_CHANGED => std::io::ErrorKind::InvalidData,
_ => std::io::ErrorKind::Other,
};
std::io::Error::new(kind, msg)
@@ -1425,6 +1580,27 @@ mod tests {
.code(),
Error::DirNameCollision { host: "x".into() }.code(),
Error::DirWriteFailed { errno: Some(28) }.code(),
Error::DirImageSsifUnsupported.code(),
Error::DirImagePlacement { path: "x".into() }.code(),
Error::DirImageEncrypted.code(),
Error::DirImageUnsupportedTree.code(),
Error::DirImageFileChanged { path: "x".into() }.code(),
Error::DirImageTooLarge.code(),
Error::DirNameTooLong { path: "x".into() }.code(),
Error::DirImageFanout { path: "x".into() }.code(),
Error::SeamPlanDroppedMost {
dropped: 1,
written: 0,
}
.code(),
Error::SinkWroteNothing.code(),
Error::ShortImageRead {
lba: 0,
expected: 1,
got: 0,
}
.code(),
Error::EmptyImage.code(),
];
let mut sorted = codes.to_vec();
sorted.sort();
@@ -1468,6 +1644,13 @@ mod tests {
E_PLATFORM_NOT_IMPLEMENTED,
),
(Error::MapfileInvalid { kind: "hex" }, E_MAPFILE_INVALID),
(
Error::ImageTruncated {
have: 0,
want: 1024,
},
E_IMAGE_TRUNCATED,
),
(Error::DiscUrlNotDirect, E_DISC_URL_NOT_DIRECT),
(Error::ExtentNotUnitAligned, E_EXTENT_NOT_UNIT_ALIGNED),
// Both CSS no-key verdicts: numeric-only Display, no English.
@@ -1654,6 +1837,7 @@ mod tests {
E_NO_STREAMS,
E_HALTED,
E_MAPFILE_INVALID,
E_IMAGE_TRUNCATED,
E_UDF_BUFFER_TOO_SMALL,
E_UDF_NOT_FILESYSTEM,
E_AACS_NO_KEYS,
@@ -1726,6 +1910,12 @@ mod tests {
E_M2TS_PACKET_MALFORMED,
E_EXTENT_NOT_UNIT_ALIGNED,
E_DISC_CAPACITY_MALFORMED,
E_DIR_IMAGE_SSIF_UNSUPPORTED,
E_DIR_IMAGE_PLACEMENT,
E_DIR_IMAGE_ENCRYPTED,
E_DIR_IMAGE_UNSUPPORTED_TREE,
E_DIR_IMAGE_FILE_CHANGED,
E_DIR_IMAGE_TOO_LARGE,
];
let original_len = codes.len();
codes.sort();
@@ -1761,6 +1951,7 @@ mod tests {
assert!((6000..7000).contains(&E_DISC_READ));
assert!((6000..7000).contains(&E_HALTED));
assert!((6000..7000).contains(&E_MAPFILE_INVALID));
assert!((6000..7000).contains(&E_IMAGE_TRUNCATED));
// AACS (7xxx)
assert!((7000..8000).contains(&E_AACS_NO_KEYS));
assert!((7000..8000).contains(&E_NO_DISC_KEY));
+488 -62
View File
@@ -272,41 +272,102 @@ fn sub_slice(data: &[u8], offset: usize, len: usize) -> Result<&[u8]> {
// ── BCD time parsing ────────────────────────────────────────────────────────
/// Convert DVD BCD playback time (4 bytes) to seconds.
/// The nominal (timecode) frame rate and the exact real-time frame duration
/// implied by a DVD `dvd_time_t` rate flag.
///
/// Format: `[hours_bcd, minutes_bcd, seconds_bcd, frames_and_rate]`
/// This distinction is the whole point of the type: `nominal_fps` is the rate
/// at which the BCD *seconds* field advances, while `frame_num / frame_den` is
/// how long a frame actually lasts. For PAL the two agree (25 frames = 1.000 s).
/// For NTSC they do not: the seconds field advances every 30 frames, but the
/// video runs at 30000/1001 fps, so 30 frames occupy 1001/1000 real seconds.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct DvdRate {
/// Frames per timecode second: 25 (PAL) or 30 (NTSC — *not* 29.97).
pub nominal_fps: u32,
/// Real seconds per frame as an exact rational, `frame_num / frame_den`.
pub frame_num: u32,
/// Denominator of the exact frame duration.
pub frame_den: u32,
}
impl DvdRate {
/// Decode bits 7-6 of `dvd_time_t.frame_u`.
///
/// `0b01` = 25 fps (PAL), `0b11` = 30000/1001 fps (NTSC).
/// `0b00` and `0b10` are unspecified/reserved and yield `None`.
pub fn from_flag(flag: u8) -> Option<DvdRate> {
match flag {
0x01 => Some(DvdRate {
nominal_fps: 25,
frame_num: 1,
frame_den: 25,
}),
0x03 => Some(DvdRate {
nominal_fps: 30,
frame_num: 1001,
frame_den: 30000,
}),
_ => None,
}
}
/// Exact real-time seconds for a count of timecode frames.
pub fn frames_to_secs(&self, frames: u64) -> f64 {
(frames as f64) * (self.frame_num as f64) / (self.frame_den as f64)
}
}
/// Total non-drop-frame timecode frames in a 4-byte DVD BCD time.
///
/// Format: `[hours_bcd, minutes_bcd, seconds_bcd, rate_and_frames]`
/// - Byte 0: hours in BCD (e.g. 0x01 = 1 hour, 0x12 = 12 hours)
/// - Byte 1: minutes in BCD
/// - Byte 2: seconds in BCD
/// - Byte 3: bits 7-6 = frame rate flag (01=25fps, 11=29.97fps),
/// bits 5-0 = frame count in BCD
///
/// Returns 0.0 for invalid BCD digits rather than erroring,
/// since some authoring tools produce malformed time fields.
pub fn bcd_to_secs(bcd: &[u8]) -> f64 {
/// **`dvd_time_t` is a timecode, not elapsed wall-clock time.** The seconds
/// field advances once every [`DvdRate::nominal_fps`] frames. On NTSC discs
/// that is every 30 frames, but 30 frames of 30000/1001 fps video last
/// 1001/1000 s — so reading H:M:S as literal seconds under-reports real time
/// by exactly 0.1% (3.6 s per hour). Callers that need real seconds must go
/// through the frame count, which is what [`bcd_to_secs`] does.
///
/// Returns `None` when the slice is short or the rate flag is unspecified,
/// since without a nominal rate the frame count cannot be interpreted.
pub fn bcd_to_frames(bcd: &[u8]) -> Option<(u64, DvdRate)> {
if bcd.len() < 4 {
return 0.0;
return None;
}
let rate = DvdRate::from_flag((bcd[3] >> 6) & 0x03)?;
let hours = bcd_byte(bcd[0]) as u64;
let minutes = bcd_byte(bcd[1]) as u64;
let seconds = bcd_byte(bcd[2]) as u64;
let frames = bcd_byte(bcd[3] & 0x3F) as u64;
let tc_secs = hours * 3600 + minutes * 60 + seconds;
Some((tc_secs * rate.nominal_fps as u64 + frames, rate))
}
let hours = bcd_byte(bcd[0]);
let minutes = bcd_byte(bcd[1]);
let seconds = bcd_byte(bcd[2]);
let rate_flag = (bcd[3] >> 6) & 0x03;
let frame_count = bcd_byte(bcd[3] & 0x3F);
let fps: f64 = match rate_flag {
0x01 => 25.0,
0x03 => 29.97,
_ => 0.0, // unknown rate — ignore frame contribution
};
let total = (hours as f64) * 3600.0 + (minutes as f64) * 60.0 + (seconds as f64);
if fps > 0.0 {
total + (frame_count as f64) / fps
} else {
total
/// Convert DVD BCD playback time (4 bytes) to real elapsed seconds.
///
/// See [`bcd_to_frames`] for the layout and for why the timecode must be
/// converted through a frame count rather than read as literal seconds.
///
/// Returns 0.0 for invalid BCD digits rather than erroring, since some
/// authoring tools produce malformed time fields. When the rate flag is
/// unspecified the nominal rate is unknowable, so this falls back to reading
/// H:M:S as literal seconds and ignoring the frame field.
pub fn bcd_to_secs(bcd: &[u8]) -> f64 {
match bcd_to_frames(bcd) {
Some((frames, rate)) => rate.frames_to_secs(frames),
None => {
if bcd.len() < 4 {
return 0.0;
}
(bcd_byte(bcd[0]) as f64) * 3600.0
+ (bcd_byte(bcd[1]) as f64) * 60.0
+ (bcd_byte(bcd[2]) as f64)
}
}
}
@@ -360,16 +421,40 @@ pub fn parse_vmg(reader: &mut dyn SectorSource, udf: &UdfFs) -> Result<DvdInfo>
// Parse each VTS IFO
let mut title_sets = Vec::new();
let mut skipped = 0usize;
for (&vts_number, titles_info) in &title_set_map {
match parse_vts(reader, udf, vts_number, titles_info) {
Ok(ts) => title_sets.push(ts),
Err(_) => {
// Skip unreadable title sets — some DVDs have placeholder entries.
Err(e) => {
// Some discs carry placeholder TT_SRPT entries for title sets
// that are not really there, so one failure is not fatal. But
// the failure must not be INVISIBLE: every skipped set is a
// title the user will never see, and swallowing the reason made
// a disc that enumerated 38 titles from one image and 10 from
// another look like a scan difference rather than 28 dropped
// reads.
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
vts = vts_number,
titles = titles_info.len(),
error = %e,
"title set could not be parsed; its titles are omitted"
);
continue;
}
}
}
if skipped > 0 {
tracing::warn!(
target: "freemkv::scan",
skipped,
kept = title_sets.len(),
declared = title_set_map.len(),
"some title sets were omitted from the scan"
);
}
Ok(DvdInfo { title_sets })
}
@@ -740,16 +825,36 @@ fn parse_pgcit(
let entries_start = pgcit_offset + 8;
let mut titles = Vec::new();
// Every `continue` below drops a title the user will never see. None of
// them may be silent: `parse_vmg` already counts and warns per skipped
// title SET, and this function was the remaining place where a disc could
// quietly report fewer titles than it has.
let mut skipped = 0usize;
for &(chapter_count, vts_title_num) in titles_info {
// VTS title numbers are 1-based; map to PGC index (typically 1:1)
let pgc_index = vts_title_num.saturating_sub(1) as usize;
if pgc_index >= num_pgcs as usize {
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
pgc_index,
num_pgcs,
"title points past the end of the PGC table; its title is omitted"
);
continue;
}
let entry_offset = entries_start + pgc_index * 8;
if entry_offset + 8 > data.len() {
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
pgc_index,
entry_offset,
len = data.len(),
"PGC entry table is truncated; this title is omitted"
);
continue;
}
@@ -761,13 +866,38 @@ fn parse_pgcit(
match parse_pgc(data, pgc_abs, chapter_count) {
Ok(title) => titles.push(title),
// By design: a single unparseable PGC (truncated/corrupt entry,
// authoring-tool quirk) must not lose the whole title list.
// Skip it and keep collecting the titles that do parse.
Err(_) => continue,
Err(e) => {
// By design a single unparseable PGC (truncated/corrupt entry,
// authoring-tool quirk) must not lose the whole title list. But
// "not fatal" is not the same as "not worth saying": every PGC
// skipped here is a title the user will never see, and this was
// the only remaining silent one — `parse_vmg` above already
// counts and warns per skipped title SET for exactly this
// reason. A disc quietly reporting fewer titles than it has is
// the failure this release exists to stop.
skipped += 1;
tracing::warn!(
target: "freemkv::scan",
pgc = pgc_index + 1,
of = num_pgcs,
error = %e,
"PGC could not be parsed; its title is omitted"
);
continue;
}
}
}
if skipped > 0 {
tracing::warn!(
target: "freemkv::scan",
skipped,
kept = titles.len(),
declared = titles_info.len(),
"some titles were omitted from this title set"
);
}
Ok(titles)
}
@@ -838,15 +968,33 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
let mut times = Vec::new();
if pgm_map_offset > 0 && nr_of_programs > 0 && cell_playback_offset > 0 {
let pgm_base = pgc_offset + pgm_map_offset;
// Collect cell durations
let mut cell_durations = Vec::with_capacity(num_cells);
// Collect per-cell durations as exact timecode FRAME counts. Summing
// frames (integers) and converting once per chapter keeps every mark
// on an exact frame boundary — summing f64 seconds would let rounding
// drift accumulate across a two-hour title.
//
// A cell whose rate flag is unspecified has no frame count, so it
// contributes plain seconds instead; that is tracked separately and
// added on top. In practice a PGC never mixes rates.
let mut cell_frames: Vec<(u64, f64)> = Vec::with_capacity(num_cells);
let mut rate: Option<DvdRate> = None;
let cell_base = pgc_offset + cell_playback_offset;
for i in 0..num_cells {
let co = cell_base + i * 24;
if co + 8 <= data.len() {
cell_durations.push(bcd_to_secs(&data[co + 4..co + 8]));
} else {
cell_durations.push(0.0);
if co + 8 > data.len() {
cell_frames.push((0, 0.0));
continue;
}
let t = &data[co + 4..co + 8];
match bcd_to_frames(t) {
Some((f, r)) => {
// First rate wins; a mixed-rate PGC is malformed, and
// reinterpreting earlier cells would be worse than
// staying on the rate the title started in.
rate.get_or_insert(r);
cell_frames.push((f, 0.0));
}
None => cell_frames.push((0, bcd_to_secs(t))),
}
}
// Program map: each byte is the first cell number (1-based) for that program
@@ -856,10 +1004,12 @@ fn parse_pgc(data: &[u8], pgc_offset: usize, chapters: u16) -> Result<DvdTitle>
}
let first_cell = data[pgm_base + p] as usize;
// Chapter time = sum of cell durations before this program's first cell.
// Clamp to cell_durations.len(): a crafted/corrupt IFO can set first_cell
// Clamp to cell_frames.len(): a crafted/corrupt IFO can set first_cell
// beyond the actual cell count, which would panic the slice index.
let end = first_cell.saturating_sub(1).min(cell_durations.len());
let time: f64 = cell_durations[..end].iter().sum();
let end = first_cell.saturating_sub(1).min(cell_frames.len());
let frames: u64 = cell_frames[..end].iter().map(|&(f, _)| f).sum();
let extra: f64 = cell_frames[..end].iter().map(|&(_, s)| s).sum();
let time = rate.map_or(0.0, |r| r.frames_to_secs(frames)) + extra;
times.push(time);
}
}
@@ -982,12 +1132,60 @@ mod tests {
#[test]
fn bcd_to_secs_with_frames() {
// 0 hours, 1 minute, 30 seconds, 15 frames at 29.97fps
// 0 hours, 1 minute, 30 seconds, 15 frames of NTSC timecode.
let bcd = [0x00, 0x01, 0x30, 0b11_010101];
let secs = bcd_to_secs(&bcd);
// 0b010101 = 0x15, BCD = 15 frames
let expected = 0.0 + 60.0 + 30.0 + 15.0 / 29.97;
assert!((secs - expected).abs() < 0.01, "got {}", secs);
// 0b010101 = 0x15, BCD = 15 frames.
// Timecode 00:01:30:15 is 90*30 + 15 = 2715 frames, and each frame
// lasts 1001/30000 s, so real time is 2715*1001/30000 = 90.5905 s
// (NOT 90.5 s — NTSC timecode runs 0.1% slow against the clock).
let expected = 2715.0 * 1001.0 / 30000.0;
assert!((secs - expected).abs() < 1e-9, "got {}", secs);
assert!((secs - 90.5905).abs() < 1e-9, "got {}", secs);
}
/// The NTSC 0.1% pull-down must be applied to the whole timecode, not just
/// the frame field. Regression test for issue freemkv#25: a title's chapter marks
/// drifted ~3.6 s per hour because H:M:S was read as literal seconds.
#[test]
fn bcd_ntsc_timecode_is_not_literal_seconds() {
// One hour of NTSC timecode = 3603.6 s of real time.
let one_hour = [0x01, 0x00, 0x00, 0b11_000000];
assert!((bcd_to_secs(&one_hour) - 3603.6).abs() < 1e-6);
// 00:01:02:00 -> 1860 frames -> 62.062 s (the issue freemkv#25 chapter 2 value).
let ch2 = [0x00, 0x01, 0x02, 0b11_000000];
assert!((bcd_to_secs(&ch2) - 62.062).abs() < 1e-9);
// A 20-minute NTSC episode cell is 00:19:58:24 = 35964 frames.
let ep = [0x00, 0x19, 0x58, 0b11_100100];
assert_eq!(bcd_to_frames(&ep).unwrap().0, 35964);
assert!((bcd_to_secs(&ep) - 1199.9988).abs() < 1e-6);
}
/// PAL timecode is exact (25 frames = 1.000 s), so the fix must leave every
/// PAL value bit-identical.
#[test]
fn bcd_pal_unaffected_by_ntsc_fix() {
let pal = [0x01, 0x23, 0x45, 0b01_000000];
assert!((bcd_to_secs(&pal) - (3600.0 + 23.0 * 60.0 + 45.0)).abs() < 1e-9);
let pal_frames = [0x00, 0x00, 0x10, 0b01_010010]; // 10 s + 12 frames
assert!((bcd_to_secs(&pal_frames) - 10.48).abs() < 1e-9);
let r = DvdRate::from_flag(0x01).unwrap();
assert_eq!((r.nominal_fps, r.frame_num, r.frame_den), (25, 1, 25));
}
#[test]
fn dvd_rate_from_flag_rejects_reserved() {
assert!(DvdRate::from_flag(0x00).is_none());
assert!(DvdRate::from_flag(0x02).is_none());
let n = DvdRate::from_flag(0x03).unwrap();
assert_eq!((n.nominal_fps, n.frame_num, n.frame_den), (30, 1001, 30000));
}
#[test]
fn bcd_to_frames_none_on_short_or_unknown_rate() {
assert!(bcd_to_frames(&[0x00, 0x00]).is_none());
assert!(bcd_to_frames(&[0x00, 0x00, 0x00, 0b00_000000]).is_none());
assert!(bcd_to_frames(&[0x00, 0x00, 0x00, 0b10_000000]).is_none());
}
#[test]
@@ -1133,12 +1331,16 @@ mod tests {
pgc[co + 23] = 144; // last sector = 400
let title = parse_pgc(&pgc, 0, 5).unwrap();
let expected = 1.0 * 3600.0 + 59.0 * 60.0 + 30.0;
// 01:59:30:00 is NTSC *timecode*, so 7170 timecode seconds = 215100
// frames, and the real running time is 215100 * 1001/30000 = 7177.17 s.
// Reading the BCD as literal seconds (7170.0) was the issue freemkv#25 bug.
let expected = 7170.0 * 30.0 * 1001.0 / 30000.0;
assert!(
(title.duration_secs - expected).abs() < 0.1,
(title.duration_secs - expected).abs() < 1e-6,
"expected ~{expected}s, got {}s",
title.duration_secs
);
assert!((title.duration_secs - 7177.17).abs() < 1e-6);
assert_eq!(title.chapters, 5);
assert_eq!(title.cells.len(), 2);
assert_eq!(title.cells[0].first_sector, 100);
@@ -1393,7 +1595,7 @@ mod tests {
// 0h 0m 0s, 12 frames at 25fps → 12/25 = 0.48s.
let bcd = [0x00, 0x00, 0x00, 0b01_010010]; // frame BCD 0x12 = 12
let secs = bcd_to_secs(&bcd);
assert!((secs - 12.0 / 25.0).abs() < 0.001, "got {secs}");
assert!((secs - 12.0 / 25.0).abs() < 1e-9, "got {secs}");
}
/// BCD rate_flag 0b00 (and 0b10) → fps 0.0 → frame count ignored
@@ -1415,8 +1617,9 @@ mod tests {
#[test]
fn bcd_frame_count_masks_rate_bits() {
let bcd = [0x00, 0x00, 0x00, 0b11_100101]; // 0x25 BCD = 25 frames
assert_eq!(bcd_to_frames(&bcd).unwrap().0, 25);
let secs = bcd_to_secs(&bcd);
assert!((secs - 25.0 / 29.97).abs() < 0.001, "got {secs}");
assert!((secs - 25.0 * 1001.0 / 30000.0).abs() < 1e-9, "got {secs}");
}
/// BCD hours can exceed 12 (long titles): 0x12 BCD = 12 → but test a
@@ -1688,6 +1891,93 @@ mod tests {
);
}
/// Regression test for issue freemkv#25 (NTSC chapter drift).
///
/// Builds a PGC from the real cell table of the DOLL_MASTERS_7 disc in the
/// bug report and checks the chapter marks against MakeMKV's values. Before
/// the fix, chapter 14 landed 4.019 s early; the drift was proportional to
/// elapsed time (0.1%), so a short synthetic fixture would not have caught it.
#[test]
fn pgc_chapter_times_ntsc_no_pulldown_drift() {
// (minutes, seconds, frames) of NTSC non-drop-frame timecode per cell.
let cells: [(u8, u8, u8); 13] = [
(1, 2, 0),
(4, 44, 12),
(1, 32, 11),
(13, 42, 1),
(0, 59, 28),
(0, 31, 8),
(1, 2, 0),
(19, 58, 24),
(0, 59, 28),
(0, 31, 8),
(1, 2, 0),
(19, 58, 26),
(0, 59, 28),
];
// Program map: 14 chapters, one per cell boundary (1-based first cell).
let pgm: [u8; 14] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14];
fn bcd(v: u8) -> u8 {
((v / 10) << 4) | (v % 10)
}
let mut pgc = vec![0u8; 0xEA];
pgc[0x02] = pgm.len() as u8;
pgc[0x03] = cells.len() as u8;
let pgm_off: u16 = 0xEA;
pgc[0xE6] = (pgm_off >> 8) as u8;
pgc[0xE7] = pgm_off as u8;
let cell_off: u16 = pgm_off + pgm.len() as u16;
pgc[0xE8] = (cell_off >> 8) as u8;
pgc[0xE9] = cell_off as u8;
pgc.resize(cell_off as usize + cells.len() * 24, 0);
pgc[pgm_off as usize..pgm_off as usize + pgm.len()].copy_from_slice(&pgm);
for (i, &(m, s, f)) in cells.iter().enumerate() {
let co = cell_off as usize + i * 24;
pgc[co + 4] = 0x00; // hours
pgc[co + 5] = bcd(m);
pgc[co + 6] = bcd(s);
pgc[co + 7] = 0xC0 | bcd(f); // rate flag 0b11 = NTSC
}
let title = parse_pgc(&pgc, 0, pgm.len() as u16).unwrap();
// MakeMKV / source truth, in seconds.
let expected = [
0.0,
62.062,
346.7464,
439.205_433_333,
1262.0608,
1_322.054_066_666,
1353.352,
1415.414,
2615.4128,
2_675.406_066_666,
2706.704,
2768.766,
3_968.831_533_333,
4028.8248,
];
assert_eq!(title.chapter_times.len(), expected.len());
for (i, (&got, &want)) in title.chapter_times.iter().zip(&expected).enumerate() {
assert!(
(got - want).abs() < 1e-6,
"chapter {} drifted: got {got}, want {want}",
i + 1
);
}
// Every mark must sit on an exact 30000/1001 frame boundary.
for (i, &t) in title.chapter_times.iter().enumerate() {
let frames = t * 30000.0 / 1001.0;
assert!(
(frames - frames.round()).abs() < 1e-6,
"chapter {} not frame-aligned: {t}",
i + 1
);
}
}
/// parse_pgc duration: when the PGC-level BCD time is NON-zero it is
/// used directly and NOT overwritten by cell-sum recomputation
/// (the recompute only fires when duration_secs == 0.0).
@@ -2065,7 +2355,14 @@ mod tests {
d
}
/// BCD playback time of `secs` seconds (< 60) at the 29.97 fps flag.
/// BCD playback time of `secs` TIMECODE seconds (< 60) at the NTSC rate
/// flag.
///
/// The argument is timecode, not real time: every fixture built from this
/// helper carries the 0b11 flag, so `secs` timecode seconds are
/// `secs * 30` frames and last `secs * 1.001` real seconds. Expectations
/// below are written in real seconds and so read 1.001x the argument —
/// `bcd_secs(10)` is 10.01 s, not 10 s.
fn bcd_secs(secs: u8) -> [u8; 4] {
assert!(secs < 60);
[0, 0, ((secs / 10) << 4) | (secs % 10), 0b11_000000]
@@ -2088,12 +2385,13 @@ mod tests {
);
let title = parse_pgc(&pgc, 0, 3).unwrap();
assert_eq!(title.cells.len(), 3);
assert_eq!(title.cells[0].duration_secs, 10.0);
assert_eq!(title.cells[1].duration_secs, 20.0);
assert_eq!(title.cells[2].duration_secs, 31.0);
assert_eq!(
title.duration_secs, 61.0,
"recomputed duration must be the sum of the distinct cell times"
assert_eq!(title.cells[0].duration_secs, 10.01);
assert_eq!(title.cells[1].duration_secs, 20.02);
assert_eq!(title.cells[2].duration_secs, 31.031);
assert!(
(title.duration_secs - 61.061).abs() < 1e-9,
"recomputed duration must be the sum of the distinct cell times, got {}",
title.duration_secs
);
}
@@ -2114,7 +2412,7 @@ mod tests {
None,
);
let title = parse_pgc(&pgc, 0, 3).unwrap();
assert_eq!(title.chapter_times, vec![0.0, 10.0, 30.0]);
assert_eq!(title.chapter_times, vec![0.0, 10.01, 30.03]);
}
/// A program map offset of 0 means there is no program map, so no chapter
@@ -2169,7 +2467,7 @@ mod tests {
let title = parse_pgc(&pgc, 0, 2).unwrap();
assert_eq!(title.cells.len(), 2, "only the readable cells are kept");
// Program 2 starts at cell 2, so its time is cell 0's duration.
assert_eq!(title.chapter_times, vec![0.0, 10.0]);
assert_eq!(title.chapter_times, vec![0.0, 10.01]);
}
/// A program map that runs past the end of the data must stop at the
@@ -2194,7 +2492,7 @@ mod tests {
"the program map walk must stop exactly at the buffer end"
);
assert_eq!(title.chapter_times[0], 0.0);
assert_eq!(title.chapter_times[1], 10.0);
assert_eq!(title.chapter_times[1], 10.01);
}
/// The subtitle palette is 16 entries of 4 bytes at PGC+0xA4, each
@@ -2284,19 +2582,19 @@ mod tests {
// vts_title_num is 1-based: title 2 → PGC index 1.
let titles = parse_pgcit(&data, 0, &[(5, 2)]).unwrap();
assert_eq!(titles.len(), 1);
assert_eq!(titles[0].duration_secs, 22.0);
assert_eq!(titles[0].duration_secs, 22.022);
assert_eq!(titles[0].cells[0].first_sector, 50);
let titles = parse_pgcit(&data, 0, &[(5, 1)]).unwrap();
assert_eq!(titles.len(), 1);
assert_eq!(titles[0].duration_secs, 11.0);
assert_eq!(titles[0].duration_secs, 11.011);
assert_eq!(titles[0].cells[0].first_sector, 0);
// Both titles, in order.
let titles = parse_pgcit(&data, 0, &[(5, 1), (7, 2)]).unwrap();
assert_eq!(titles.len(), 2);
assert_eq!(titles[0].duration_secs, 11.0);
assert_eq!(titles[1].duration_secs, 22.0);
assert_eq!(titles[0].duration_secs, 11.011);
assert_eq!(titles[1].duration_secs, 22.022);
}
/// A VTS_PGCIT whose 8-byte header ends exactly at the end of the data is
@@ -2341,4 +2639,132 @@ mod tests {
assert_eq!(map[&1], vec![(7u16, 1u8), (13, 2)]);
assert_eq!(map[&2], vec![(0x0102u16, 1u8)]);
}
// ── TT_SRPT: the title table (`parse_tt_srpt` / `parse_vmg`) ────────────
//
// This is the table that says which titles a DVD has, and it had no tests
// at all until 1.6.2 — while being at the centre of both DVD stories in
// 1.6.1. The CSS descramble bug destroyed the sector carrying it, so a disc
// enumerating 38 titles produced an image enumerating 10, silently, at exit
// 0. And `dir://` was pulled from the release on 2026-08-05 because a
// folder produced the wrong title list — the same table, reached through a
// synthesized image.
//
// Both were found on real discs. Nothing hermetic covered the table itself.
/// Build a VMG with a TT_SRPT at `tt_srpt_sector` declaring `entries`
/// of `(num_chapters, vts_number, vts_title_num)`.
fn vmg_with_tt_srpt(tt_srpt_sector: u32, entries: &[(u16, u8, u8)]) -> Vec<u8> {
let off = tt_srpt_sector as usize * crate::consts::SECTOR_BYTES;
let mut v = vec![0u8; off + 8 + entries.len() * 12 + 16];
v[0..12].copy_from_slice(b"DVDVIDEO-VMG");
v[0xC4..0xC8].copy_from_slice(&tt_srpt_sector.to_be_bytes());
v[off..off + 2].copy_from_slice(&(entries.len() as u16).to_be_bytes());
for (i, &(chapters, vts, title)) in entries.iter().enumerate() {
let b = off + 8 + i * 12;
v[b + 2..b + 4].copy_from_slice(&chapters.to_be_bytes());
v[b + 6] = vts;
v[b + 7] = title;
}
v
}
#[test]
fn tt_srpt_groups_titles_by_their_title_set() {
// Three titles: two in VTS 1, one in VTS 2 — the ordinary shape.
let vmg = vmg_with_tt_srpt(1, &[(12, 1, 1), (5, 1, 2), (20, 2, 1)]);
let map = parse_tt_srpt(&vmg, crate::consts::SECTOR_BYTES).expect("parse");
assert_eq!(map.len(), 2, "two title sets");
assert_eq!(
map[&1],
vec![(12, 1), (5, 2)],
"VTS 1 keeps both titles in order"
);
assert_eq!(map[&2], vec![(20, 1)], "VTS 2 keeps its one");
}
/// The pointer at 0xC4 is a SECTOR offset from the start of VIDEO_TS.IFO,
/// not a byte offset. Reading it as bytes would find the table at 1/2048th
/// of its real position — which is a zero-filled region on most discs, so
/// it would report a disc with no titles rather than fail loudly.
#[test]
fn tt_srpt_pointer_is_a_sector_offset_not_a_byte_offset() {
for sector in [1u32, 2, 5] {
let vmg = vmg_with_tt_srpt(sector, &[(3, 1, 1)]);
let at_sector = parse_tt_srpt(&vmg, sector as usize * crate::consts::SECTOR_BYTES)
.expect("reading at the sector offset works");
assert_eq!(at_sector[&1], vec![(3, 1)], "sector {sector}");
// Reading the same pointer as a BYTE offset lands 1/2048th of the
// way in — inside the header here — and yields a different table.
// It does not error: it silently reports the wrong titles, which is
// how 28 dropped titles looked like a scan difference rather than a
// failure.
let as_bytes = parse_tt_srpt(&vmg, sector as usize).expect("does not error");
assert_ne!(
as_bytes, at_sector,
"sector {sector}: a byte-offset read must not coincidentally \
agree with the correct one, or this test proves nothing"
);
}
}
/// A VTS number of 0 is not a title set; entries carrying it are padding.
#[test]
fn tt_srpt_skips_entries_with_no_title_set() {
let vmg = vmg_with_tt_srpt(1, &[(9, 0, 1), (4, 3, 1)]);
let map = parse_tt_srpt(&vmg, crate::consts::SECTOR_BYTES).expect("parse");
assert_eq!(map.len(), 1, "the vts=0 entry is dropped");
assert_eq!(map[&3], vec![(4, 1)]);
}
/// The same (title set, title) twice is one title, not two. Real discs
/// carry duplicate TT_SRPT rows.
#[test]
fn tt_srpt_collapses_a_duplicate_title() {
let vmg = vmg_with_tt_srpt(1, &[(7, 2, 1), (7, 2, 1), (8, 2, 2)]);
let map = parse_tt_srpt(&vmg, crate::consts::SECTOR_BYTES).expect("parse");
assert_eq!(
map[&2],
vec![(7, 1), (8, 2)],
"the repeat is dropped, the distinct one kept"
);
}
/// A truncated table parses what is there rather than failing the whole
/// disc — one unreadable row should not cost a user every other title.
#[test]
fn tt_srpt_truncated_keeps_the_entries_it_can_read() {
let mut vmg = vmg_with_tt_srpt(1, &[(1, 1, 1), (2, 2, 1), (3, 3, 1)]);
vmg.truncate(crate::consts::SECTOR_BYTES + 8 + 12 + 6); // 1 whole entry + a fragment
let map = parse_tt_srpt(&vmg, crate::consts::SECTOR_BYTES).expect("parse");
assert_eq!(map.len(), 1, "the complete entry survives");
assert_eq!(map[&1], vec![(1, 1)]);
}
/// The count is an untrusted u16 off the disc. DVD-Video caps a disc at 99
/// titles, and each entry re-parses a PGC into a full DvdTitle — so an
/// unclamped 65535 is a ~540 MB allocation from a ~800 KB crafted file.
#[test]
fn tt_srpt_clamps_an_absurd_declared_title_count() {
// The fixture must actually CONTAIN more entries than the cap, or the
// walk stops when the buffer runs out and the clamp is never what
// bounded it. An earlier version of this test declared u16::MAX over a
// two-kilobyte fixture: the loop broke on `base + 12 > len` after three
// iterations and the assertion held with the clamp deleted entirely.
const PRESENT: usize = MAX_TT_SRPT_TITLES + 40;
let entries: Vec<(u16, u8, u8)> = (0..PRESENT)
.map(|i| (1u16, 1u8, (i % 250 + 1) as u8))
.collect();
let mut vmg = vmg_with_tt_srpt(1, &entries);
let off = crate::consts::SECTOR_BYTES;
vmg[off..off + 2].copy_from_slice(&u16::MAX.to_be_bytes());
let map = parse_tt_srpt(&vmg, off).expect("parse");
let total: usize = map.values().map(|v| v.len()).sum();
assert!(
total <= MAX_TT_SRPT_TITLES,
"a declared count of u16::MAX over {PRESENT} real entries must be \
clamped to {MAX_TT_SRPT_TITLES}, got {total}"
);
}
}
+3 -3
View File
@@ -23,7 +23,7 @@
use std::fs::File;
use std::os::unix::io::AsRawFd;
pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
pub(crate) fn hint_sequential(file: &File, _len_bytes: u64) {
// Best-effort: return value ignored. A fadvise failure has no
// user-observable consequence.
unsafe {
@@ -34,7 +34,7 @@ pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
/// Drop pages in the half-open byte range `[start, start+len)` from
/// the page cache. Called periodically by `read_sectors` to bound the
/// read-side page cache pressure.
pub(super) fn drop_window(file: &File, start: u64, len: u64) {
pub(crate) fn drop_window(file: &File, start: u64, len: u64) {
unsafe {
libc::posix_fadvise(
file.as_raw_fd(),
@@ -58,7 +58,7 @@ pub(super) fn drop_window(file: &File, start: u64, len: u64) {
/// can only pre-stage a tiny slice of the next batch. An explicit
/// `readahead()` of the same size as the current batch tells the
/// kernel to queue the full next-batch read now.
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) {
unsafe {
libc::readahead(file.as_raw_fd(), offset as i64, len as usize);
}
+3 -3
View File
@@ -15,7 +15,7 @@ use std::os::unix::io::AsRawFd;
/// pipeline depth.
const RDADVISE_MAX_BYTES: i64 = 64 * 1024 * 1024;
pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
pub(crate) fn hint_sequential(file: &File, len_bytes: u64) {
let bytes = (len_bytes as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory {
ra_offset: 0,
@@ -33,14 +33,14 @@ pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
/// approximation: no-op. macOS's unified buffer cache is generally
/// less prone to the pin-everything pathology that triggers the
/// regression on Linux NFS clients.
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Async-prefetch the byte range `[offset, offset+len)`. macOS uses
/// the same `fcntl(F_RDADVISE, &radvisory)` primitive as the open-
/// time sequential hint, just targeted at a moving window instead of
/// the whole file. The kernel queues I/O for the requested range and
/// returns immediately.
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
pub(crate) fn prefetch(file: &File, offset: u64, len: u64) {
let bytes = (len as i64).min(RDADVISE_MAX_BYTES);
let mut ra = libc::radvisory {
ra_offset: offset as libc::off_t,
+11 -8
View File
@@ -48,22 +48,25 @@
//! far smaller than our 16 MiB app-level batch.
#[cfg(target_os = "linux")]
mod linux;
pub(crate) mod linux;
#[cfg(target_os = "macos")]
mod macos;
pub(crate) mod macos;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
mod other;
pub(crate) mod other;
#[cfg(target_os = "windows")]
mod windows;
pub(crate) mod windows;
// The page-cache hints are shared with any other file-backed sector source:
// `dirimage` reads host files the same way and needs the same eviction, or a
// large rip pins every byte it has read (see this module's DONTNEED note).
#[cfg(target_os = "linux")]
use linux as platform;
pub(crate) use linux as platform;
#[cfg(target_os = "macos")]
use macos as platform;
pub(crate) use macos as platform;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use other as platform;
pub(crate) use other as platform;
#[cfg(target_os = "windows")]
use windows as platform;
pub(crate) use windows as platform;
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
+3 -3
View File
@@ -4,8 +4,8 @@
use std::fs::File;
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {}
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {}
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
+3 -3
View File
@@ -9,7 +9,7 @@ use std::fs::File;
/// No-op stub. `FILE_FLAG_SEQUENTIAL_SCAN` can only be set at
/// `CreateFile` open time, which the plain `File::open` path does not
/// do, so there is no post-open hint to issue here.
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
pub(crate) fn hint_sequential(_file: &File, _len_bytes: u64) {
tracing::debug!(
target: "mux",
"FileSectorSource hint_sequential: windows no-op stub"
@@ -19,10 +19,10 @@ pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
/// Windows page-cache eviction is not exposed via a posix_fadvise
/// equivalent. The kernel does its own working-set management. No-op
/// for now.
pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
pub(crate) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// Windows async-prefetch hint. With FILE_FLAG_SEQUENTIAL_SCAN at
/// open the kernel already prefetches aggressively, so there's no
/// per-range hint we'd add on top. No-op stub for parity with the
/// posix platforms.
pub(super) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
pub(crate) fn prefetch(_file: &File, _offset: u64, _len: u64) {}
+285
View File
@@ -0,0 +1,285 @@
//! `write_image` — write an image-level source out as a sector image.
//!
//! This is the plain image writer: sectors in from any [`SectorSource`], bytes
//! out to a file, in order, once. It is what an `iso://` DESTINATION means when
//! the source is not a physical drive.
//!
//! # Why this is not `freemkv_engine::copy`
//!
//! The engine's `copy` is the RECOVERY path — mapfile sidecar, `--multipass`
//! sweep/patch, damage-jump, ECC-aware batching, auto-resume. Every one of those
//! exists because an optical drive returns read errors on marginal media. A
//! file-backed or synthesized source has no marginal media: a read either
//! succeeds or the underlying file is broken, and retrying it is pointless.
//!
//! Routing a non-drive source through the recovery path is not merely wasteful,
//! it is wrong. Its mapfile identity check compares AACS unit keys and the VID,
//! both of which are empty for an already-decrypted source, so identity passes
//! for ANY such source: a second run with a different input to the same output
//! path would resume over the previous image and produce wrong content at exit
//! zero. Keeping the two paths separate makes that unrepresentable.
//!
//! So: drive sources get `freemkv_engine::copy`. Everything else gets this.
use crate::error::{Error, Result};
use crate::halt::Halt;
use crate::sector::SectorSource;
use std::fs::File;
use std::io::{BufWriter, Write};
use std::path::Path;
/// Sectors per read/write batch. 4 MiB — large enough that per-call overhead
/// disappears against a file-backed source, small enough that the buffer is not
/// a notable allocation and cancellation stays responsive.
const BATCH_SECTORS: u32 = 2048;
/// Bytes per sector. Fixed for every medium this crate reads.
const SECTOR_BYTES: usize = 2048;
/// Write `total_sectors` sectors from `reader` to `dest`.
///
/// Reads sequentially from LBA 0 and writes in order, so the output is a faithful
/// image of whatever the source presents — decrypted if the caller wrapped the
/// source in a [`DecryptingSectorSource`](crate::sector::decrypting::DecryptingSectorSource),
/// ciphertext if it did not. This function performs no decryption itself and makes
/// no decryption decision; that belongs to the caller, which knows whether the run
/// is `--raw`.
///
/// `on_progress` is called after each batch with the cumulative byte count, for
/// front-end progress reporting. It must not block.
///
/// `halt` is checked once per batch. On cancellation the partial file is left in
/// place — the caller decides whether a partial image is worth keeping, and
/// deleting a multi-gigabyte file the user may want to inspect is not this
/// function's call to make.
///
/// Returns the number of bytes written.
///
/// # Errors
///
/// - [`Error::Halted`] if `halt` was cancelled.
/// - [`Error::IoError`] if the destination cannot be created or written.
/// - Whatever the source's `read_sectors` returns. A short read is an error, not
/// a zero-fill: silently padding a truncated source produces an image that
/// looks complete and is not.
pub fn write_image(
reader: &mut dyn SectorSource,
dest: &Path,
total_sectors: u32,
halt: &Halt,
mut on_progress: impl FnMut(u64),
) -> Result<u64> {
if total_sectors == 0 {
return Err(Error::EmptyImage);
}
let file = File::create(dest).map_err(|source| Error::IoError { source })?;
let mut out = BufWriter::with_capacity(BATCH_SECTORS as usize * SECTOR_BYTES, file);
let mut buf = vec![0u8; BATCH_SECTORS as usize * SECTOR_BYTES];
let mut written: u64 = 0;
let mut lba: u32 = 0;
while lba < total_sectors {
if halt.is_cancelled() {
return Err(Error::Halted);
}
let count = BATCH_SECTORS.min(total_sectors - lba);
let want = count as usize * SECTOR_BYTES;
// `recovery = false`: a file-backed source ignores the flag, and a
// retry loop over a local file would only re-read the same bytes.
let got = reader.read_sectors(lba, count as u16, &mut buf[..want], false)?;
if got != want {
return Err(Error::ShortImageRead {
lba,
expected: want as u32,
got: got as u32,
});
}
out.write_all(&buf[..want])
.map_err(|source| Error::IoError { source })?;
written += want as u64;
lba += count;
on_progress(written);
}
// flush() only pushes the BufWriter's bytes into the kernel via write(2).
// It makes no durability promise at all, so returning Ok here would report
// a finished image while up to several gigabytes of it still sit in the
// page cache. A crash, a power loss, or yanking the removable/network
// volume the image was written to then leaves a truncated or empty file
// that the caller was told was complete.
//
// For a 6-90 GB image that is exactly the failure this crate treats as
// worst: success reported over wrong output. `into_inner` is used rather
// than `flush` so a buffered-write error is surfaced instead of being
// dropped on the floor by BufWriter's Drop.
let file = out.into_inner().map_err(|e| Error::IoError {
source: e.into_error(),
})?;
file.sync_all()
.map_err(|source| Error::IoError { source })?;
Ok(written)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::error::Result as FmResult;
/// A source that yields a deterministic byte per sector, so the written
/// image can be checked positionally rather than just by length.
struct PatternSource {
sectors: u32,
/// Sectors after which `read_sectors` reports a short read.
short_after: Option<u32>,
}
impl SectorSource for PatternSource {
fn capacity_sectors(&self) -> u32 {
self.sectors
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> FmResult<usize> {
let want = count as usize * SECTOR_BYTES;
if self.short_after.is_some_and(|after| lba >= after) {
return Ok(want - 1);
}
for s in 0..count as usize {
let byte = ((lba as usize + s) % 251) as u8;
buf[s * SECTOR_BYTES..(s + 1) * SECTOR_BYTES].fill(byte);
}
Ok(want)
}
}
fn tmp(name: &str) -> std::path::PathBuf {
let mut p = std::env::temp_dir();
p.push(format!("fmkv-image-writer-{name}-{}", std::process::id()));
p
}
/// The written image is byte-for-byte what the source presented, at the
/// right offsets — not merely the right length.
#[test]
fn writes_every_sector_in_order() {
let dest = tmp("order");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 5000 * SECTOR_BYTES as u64);
let data = std::fs::read(&dest).expect("read back");
assert_eq!(data.len(), 5000 * SECTOR_BYTES);
// Spot-check across batch boundaries (BATCH_SECTORS = 2048): the last
// sector of batch 0, the first of batch 1, and the final sector.
for lba in [0usize, 2047, 2048, 4095, 4096, 4999] {
let want = (lba % 251) as u8;
assert_eq!(
data[lba * SECTOR_BYTES],
want,
"sector {lba} head byte wrong — batching lost or duplicated a sector"
);
assert_eq!(
data[(lba + 1) * SECTOR_BYTES - 1],
want,
"sector {lba} tail"
);
}
let _ = std::fs::remove_file(&dest);
}
/// A tail shorter than a full batch must still be written whole — the
/// classic off-by-one when `total_sectors` is not a batch multiple.
#[test]
fn writes_a_partial_final_batch() {
let dest = tmp("tail");
let mut src = PatternSource {
sectors: 2049,
short_after: None,
};
let n = write_image(&mut src, &dest, 2049, &Halt::new(), |_| {}).expect("write");
assert_eq!(n, 2049 * SECTOR_BYTES as u64);
assert_eq!(
std::fs::metadata(&dest).expect("stat").len(),
2049 * SECTOR_BYTES as u64
);
let _ = std::fs::remove_file(&dest);
}
/// A short read is an error. Zero-filling would yield an image that looks
/// complete and is not — the single worst outcome for an archival copy.
#[test]
fn short_read_is_an_error_not_a_zero_fill() {
let dest = tmp("short");
let mut src = PatternSource {
sectors: 4096,
short_after: Some(2048),
};
let err = write_image(&mut src, &dest, 4096, &Halt::new(), |_| {}).expect_err("must fail");
assert!(
matches!(err, Error::ShortImageRead { lba: 2048, .. }),
"got {err:?}"
);
let _ = std::fs::remove_file(&dest);
}
/// Cancellation stops the run and reports it, rather than finishing quietly
/// or reporting success on a partial image.
#[test]
fn cancellation_halts_and_reports() {
let dest = tmp("halt");
let mut src = PatternSource {
sectors: 100_000,
short_after: None,
};
let halt = Halt::new();
halt.cancel();
let err = write_image(&mut src, &dest, 100_000, &halt, |_| {}).expect_err("must halt");
assert!(matches!(err, Error::Halted), "got {err:?}");
let _ = std::fs::remove_file(&dest);
}
/// Progress is cumulative and monotonic, and its final value equals the
/// returned byte count — a front-end that trusts the callback must not end
/// up disagreeing with the return value.
#[test]
fn progress_is_cumulative_and_ends_at_the_total() {
let dest = tmp("progress");
let mut src = PatternSource {
sectors: 5000,
short_after: None,
};
let mut seen: Vec<u64> = Vec::new();
let n = write_image(&mut src, &dest, 5000, &Halt::new(), |b| seen.push(b)).expect("write");
assert!(
seen.windows(2).all(|w| w[1] > w[0]),
"not monotonic: {seen:?}"
);
assert_eq!(*seen.last().expect("at least one callback"), n);
let _ = std::fs::remove_file(&dest);
}
/// A zero-sector source is a caller error, not a zero-byte image: an empty
/// ISO is never what anyone wanted, and failing here names the problem.
#[test]
fn zero_sectors_is_an_error() {
let dest = tmp("empty");
let mut src = PatternSource {
sectors: 0,
short_after: None,
};
let err = write_image(&mut src, &dest, 0, &Halt::new(), |_| {}).expect_err("must fail");
assert!(matches!(err, Error::EmptyImage), "got {err:?}");
// The destination must not have been created — a failed run leaves no
// stub for a later run to mistake for output.
assert!(!dest.exists(), "empty run created a file");
}
}
+1
View File
@@ -31,6 +31,7 @@ pub(crate) mod bounded;
pub mod byte_prefetcher;
pub mod file_sector_source;
pub mod fsync;
pub mod image_writer;
pub mod sink;
mod writeback;
mod writeback_file;
+43 -13
View File
@@ -325,17 +325,21 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
///
/// The thread is named `freemkv-pipeline-consumer` so it shows up
/// distinctly in stack traces and `top -H`. Callers that want a
/// more specific name (e.g. `freemkv-sweep-consumer`) should use
/// [`Pipeline::spawn_named`] instead. Returns an `Error::IoError`
/// if the OS refuses the thread spawn (resource exhaustion);
/// callers already operate in fallible context, so this is
/// propagated rather than panicked.
/// more specific name should use [`Pipeline::spawn_named`] instead.
/// Returns an `Error::IoError` if the OS refuses the thread spawn
/// (resource exhaustion); callers already operate in fallible context, so
/// this is propagated rather than panicked.
///
/// Sweep uses [`Pipeline::spawn_named`] directly so the consumer
/// thread shows up as `freemkv-sweep-consumer`; mux uses
/// `freemkv-mux-consumer`. `Pipeline::spawn` (this function, with
/// the default name) is used by `disc::patch` and by the unit
/// tests in this module.
/// Inside this crate the only [`Pipeline::spawn_named`] caller is the mux
/// driver, which names its thread `freemkv-mux-consumer`. `Pipeline::spawn`
/// (this function, with the default name) is used only by the unit tests in
/// this module.
///
/// This paragraph twice named a caller that had left the crate: first
/// `disc::patch`, then Sweep and its `freemkv-sweep-consumer` thread. Both
/// went to freemkv-engine with the recovery passes in 1.6.0, and each in
/// turn sent readers hunting a component that is not here. Name callers
/// that live in THIS crate, or none.
pub fn spawn<S: Sink<I, Output = R>>(depth: usize, sink: S) -> Result<Self, Error> {
Self::spawn_named("freemkv-pipeline-consumer", depth, sink)
}
@@ -343,7 +347,7 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Like [`Pipeline::spawn`] but lets the caller supply the
/// consumer thread's name. Useful when several pipelines run in
/// the same process and stack traces / `top -H` need to tell them
/// apart (e.g. `freemkv-sweep-consumer`, `freemkv-mux-consumer`).
/// apart (e.g. `freemkv-mux-consumer`).
pub fn spawn_named<S: Sink<I, Output = R>>(
name: &str,
depth: usize,
@@ -628,6 +632,14 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// wedged inside an unkillable syscall, the producer can still
/// observe `/api/stop` and unwind within
/// [`SEND_HALT_CHECK_INTERVAL`].
/// NOT a `foo_with_X` variant of [`Pipeline::send`], despite the name.
/// The two encode OPPOSITE policies on the same event, each with its own
/// test: after the consumer's `apply` has failed, `send` still succeeds
/// (the consumer keeps draining, so the channel accepts the item), while
/// this one hands the item straight back — so a producer does not read an
/// hour of disc for a write that died on the first frame. Collapsing them
/// into one Option-parameterised method deletes one of those behaviours;
/// it was tried and `apply_error_drains_then_propagates` caught it.
pub fn send_with_halt(&self, item: I, halt: &Halt, deadline: Duration) -> Result<(), I> {
use crossbeam_channel::SendTimeoutError;
let end = Instant::now() + deadline;
@@ -739,6 +751,12 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Plain [`Pipeline::finish`] is preserved for callers without a
/// halt-token plumbed through; that path still blocks indefinitely
/// on `join()`, matching pre-0.20.8 behaviour.
/// Also not a `foo_with_X` variant: [`Pipeline::finish`] joins and waits
/// however long the consumer needs, while this one gives up after
/// `JOIN_TIMEOUT_SECS` and reports halted. Which is right depends on
/// whether the caller has a user waiting to cancel — the mux driver does
/// and uses this; the unit tests do not and use the plain join. Merging
/// them means picking one of those policies for both.
pub fn finish_with_halt(self, halt: Option<&Halt>) -> Result<R, Error> {
let Pipeline {
tx,
@@ -1839,11 +1857,23 @@ mod tests {
thread::spawn(move || {
// Past the first grace window (and past the 250 ms poll cadence that
// bounds when the window is actually observed), inside the second.
thread::sleep(Duration::from_millis(600));
//
// These intervals used to be 600 ms against a 300 ms grace, which
// left NO margin: two 300 ms windows end at 600 ms, and the 250 ms
// poll cadence can push the observation later still, so on a loaded
// runner the second window expired first and the caller abandoned —
// failing with Err(Halted) against a race, not a defect.
//
// Scaled up so the jitter is small relative to the intervals: the
// first window ends at ~1.0-1.25 s and the second at ~2.0-2.25 s,
// so releasing at 1.6 s sits well inside the second with roughly
// 350 ms of slack on either side. The ordering under test is
// unchanged; only the margin is.
thread::sleep(Duration::from_millis(1600));
rel.store(true, Ordering::SeqCst);
});
let grace = Duration::from_millis(300);
let grace = Duration::from_secs(1);
let res = finish_with_grace(handle, &state, grace, Error::Halted);
assert!(
matches!(res, Ok(42)),
+1
View File
@@ -1887,6 +1887,7 @@ mod apply_tests {
DiscTitle {
playlist: playlist.into(),
clips: vec![crate::disc::Clip {
feed_span: None,
clip_id: clip_id.into(),
in_time: 0,
out_time: 0,
+9 -1
View File
@@ -106,6 +106,7 @@ pub mod consts;
pub mod css;
pub mod decrypt;
pub mod diag;
pub mod dirimage;
pub mod disc;
pub mod drive;
pub mod dvdnav;
@@ -148,7 +149,8 @@ pub use drive::{Drive, DriveStatus, extract_scsi_context, find_drive};
// Owns the `Drive` by value; forwards consumer-built key material into
// `ScanOptions` (the library derives no certs — see `KeySpec`).
pub use session::{
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_iso,
DeviceTarget, DiscSession, KeySourceFactory, KeySpec, ResolvedKeys, resolve_keys_for, scan_dir,
scan_iso,
};
// ─── Errors ─────────────────────────────────────────────────────────────────
@@ -191,6 +193,11 @@ pub use io::pipeline::{
// continuously instead of bursting. General I/O infra, not recovery policy;
// promoted to `pub` so freemkv-engine's relocated sweep/patch can use it too.
pub use io::WritebackFile;
/// Write an image-level source out as a sector image — what an `iso://`
/// DESTINATION means for any source that is not a physical drive. Drive sources
/// go through `freemkv_engine::copy`, which is the recovery path; see
/// [`io::image_writer`] for why the two are deliberately separate.
pub use io::image_writer::write_image;
// ─── Drive events (low-level callbacks) ─────────────────────────────────────
pub use event::{BatchSizeReason, Event, EventKind};
@@ -223,6 +230,7 @@ pub use decrypt::{AacsKeyMap, DecryptKeys, decrypt_sectors, decrypt_threads, set
// — not the `pes::Stream` trait re-exported below as `PesStream`. Two
// different concepts, the same short name; the trait gets the `Pes`
// prefix at the crate root to keep both addressable.
pub use dirimage::DirImage;
pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
DiscFormat, DiscId, DiscTitle, DriveCredentials, Extent, ExtractOptions, ExtractResult,
+164 -26
View File
@@ -62,8 +62,18 @@ const AC3_SAMPLES_PER_FRAME: u32 = 1536;
const MAX_AC3_BUF: usize = 1024 * 1024;
pub struct Ac3Parser {
/// Leftover bytes from previous PES (incomplete frame at end).
buf: Vec<u8>,
/// Leftover bytes from previous PES (incomplete frame at end), each still
/// attributable to the packet that carried it — so an access unit that
/// began in an earlier packet takes THAT packet's source offset, not the
/// one that happened to complete it.
acc: super::pesbuf::PesBuf,
/// Reused working copy of `acc`, kept across calls so the per-PES scan does
/// not allocate. `parse` runs once per PES packet on an audio track — of
/// the order of 10^5 times per title — and previously built a fresh `Vec`
/// each time. The copy itself is unavoidable without restructuring the
/// borrow relationship (the scanner needs `&mut self.tally` while it reads
/// these bytes); the ALLOCATION is not.
scratch: Vec<u8>,
/// PTS (ns) to stamp on the frame that begins the carry-over `buf` — i.e.
/// the running per-frame PTS at the point the partial tail was retained.
/// Used by `flush()` to time the final buffered frame at EOS.
@@ -92,7 +102,8 @@ impl Default for Ac3Parser {
impl Ac3Parser {
pub fn new() -> Self {
Self {
buf: Vec::with_capacity(4096),
scratch: Vec::new(),
acc: super::pesbuf::PesBuf::with_capacity(4096),
flush_pts_ns: 0,
tally: super::dropgate::DropTally::new("ac3"),
saw_extension: false,
@@ -137,6 +148,7 @@ impl Ac3Parser {
base_pts_ns: i64,
anchor: Option<PtsAnchor>,
at_eos: bool,
marks: &[(usize, super::pesbuf::PesFacts)],
) -> (Vec<Frame>, usize, i64) {
let mut frames = Vec::new();
let mut pos = 0usize;
@@ -219,7 +231,7 @@ impl Ac3Parser {
}
} else {
if let Some(au) = pending.take() {
close_access_unit(&mut self.tally, data, &au, &mut frames);
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
// First access unit that starts in this PES's own bytes: adopt
// this PES's timestamp so a genuine PTS jump is followed instead
@@ -264,7 +276,7 @@ impl Ac3Parser {
frame_pts_ns = au.pts_ns;
hold_from = Some(au.start);
} else {
close_access_unit(&mut self.tally, data, &au, &mut frames);
close_access_unit(&mut self.tally, data, &au, marks, &mut frames);
}
}
@@ -334,6 +346,7 @@ fn close_access_unit(
tally: &mut super::dropgate::DropTally,
data: &[u8],
au: &PendingAu,
marks: &[(usize, super::pesbuf::PesFacts)],
out: &mut Vec<Frame>,
) {
if let Some(reason) = au.drop_reason {
@@ -344,7 +357,9 @@ fn close_access_unit(
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
// The packet covering this unit's FIRST byte — which is the packet its
// PTS came from too, when the unit began in an earlier PES.
source: super::pesbuf::facts_for(marks, au.start).source,
pts_ns: au.pts_ns,
keyframe: true,
data: data[au.start..au.end].to_vec(),
@@ -461,7 +476,7 @@ impl CodecParser for Ac3Parser {
// never be stranded by an empty post-gap PES (the demuxer only emits
// non-empty PES today; this is defensive for any future caller).
if pes.discontinuity {
self.buf.clear();
self.acc.clear();
}
if pes.data.is_empty() {
return Vec::new();
@@ -480,7 +495,7 @@ impl CodecParser for Ac3Parser {
// no anchor the running cadence simply continues. The discontinuity-
// carrying PES is a PUSI with a PTS in practice, so this is
// defense-in-depth.
let carry_len = self.buf.len();
let carry_len = self.acc.len();
let anchor = pes.pts.map(|p| PtsAnchor {
at: carry_len,
pts_ns: pts_to_ns(p),
@@ -489,11 +504,20 @@ impl CodecParser for Ac3Parser {
// Prepend leftover from previous PES, then take the whole buffer into a
// local so the scanner can call `self.tally` (the bytes are no longer
// borrowed from `self`). The unconsumed tail is written back at the end.
self.buf.extend_from_slice(&pes.data);
let buf = std::mem::take(&mut self.buf);
self.acc.push(pes);
// Copy the working bytes out so the scanner can borrow `self.tally`;
// the buffer keeps its marks, so the unconsumed tail stays attributed
// to the packet that carried it.
// Take the scratch OUT of `self` so the scanner can borrow `self.tally`
// while reading it; it is put back at the end of the call, keeping its
// capacity for the next PES. There is no early return after this point.
let mut buf = std::mem::take(&mut self.scratch);
buf.clear();
buf.extend_from_slice(self.acc.as_slice());
let marks = self.acc.marks_snapshot();
let data = &buf;
let (frames, keep_from, frame_pts_ns) =
self.scan_access_units(data, self.flush_pts_ns, anchor, false);
self.scan_access_units(data, self.flush_pts_ns, anchor, false, &marks);
if keep_from < data.len() {
let tail = &data[keep_from..];
@@ -506,7 +530,7 @@ impl CodecParser for Ac3Parser {
"ac3: carry-over buffer exceeded {} bytes without a frame; dropping and resyncing",
MAX_AC3_BUF
);
self.buf.clear();
self.acc.clear();
// Advance the cadence, as both sibling branches below do, so the
// three paths out of this block cannot disagree. Defensive: no
// input reaching this parser was found that both parses frames and
@@ -514,7 +538,7 @@ impl CodecParser for Ac3Parser {
// prevents is not currently reachable and has no regression test.
self.flush_pts_ns = frame_pts_ns;
} else {
self.buf = tail.to_vec();
self.acc.drain(keep_from);
// The carried bytes, when later completed and emitted (next call
// or by flush() at EOS), are timed at the PTS the scanner reached
// here: the PTS of the next access unit in presentation order, or
@@ -523,13 +547,16 @@ impl CodecParser for Ac3Parser {
self.flush_pts_ns = frame_pts_ns;
}
} else {
self.buf.clear();
self.acc.clear();
// Nothing carried, but keep the cadence so a following PES with no
// PTS (no anchor) continues the timeline instead of reusing a stale
// value.
self.flush_pts_ns = frame_pts_ns;
}
// Hand the working buffer back so the next PES reuses its capacity.
// `data` borrowed it; that borrow ends here, at its last use.
self.scratch = buf;
frames
}
@@ -539,9 +566,11 @@ impl CodecParser for Ac3Parser {
// with no following PES to close it, and without this drain the last
// ~32 ms of audio is lost. `at_eos` closes the trailing access unit
// instead of holding it; a partial/garbage tail yields nothing.
let buf = std::mem::take(&mut self.buf);
let buf = self.acc.as_slice().to_vec();
let marks = self.acc.marks_snapshot();
self.acc.clear();
let out = self
.scan_access_units(&buf, self.flush_pts_ns, None, true)
.scan_access_units(&buf, self.flush_pts_ns, None, true, &marks)
.0;
// Aggregate drop report at end-of-stream (warn-level, always visible).
self.tally.log_summary();
@@ -794,6 +823,73 @@ mod tests {
frame[n - 1] = (c & 0xFF) as u8;
}
/// The per-PES working buffer must be REUSED, not reallocated.
///
/// `parse` runs once per PES packet — of the order of 10^5 times on a
/// feature's audio track — and used to build a fresh `Vec` every call. The
/// copy itself cannot go without restructuring the borrow relationship, but
/// the allocation can: the buffer is taken out of `self`, filled, and put
/// back with its capacity intact.
///
/// Asserting on capacity is deliberately white-box, because that is exactly
/// what regresses if someone reverts to `to_vec()` — the behaviour would be
/// identical and no other test would notice.
#[test]
fn the_working_buffer_is_reused_across_packets_not_reallocated() {
let mut parser = Ac3Parser::new();
let frame = make_ac3_frame(0, 0);
parser.parse(&PesPacket {
source: None,
pid: 0x1100,
pts: Some(90_000),
dts: None,
data: frame.clone(),
discontinuity: false,
});
let cap_after_first = parser.scratch.capacity();
assert!(
cap_after_first > 0,
"the buffer must be handed back to the parser, not dropped"
);
// The discriminator: a LARGE packet, then a small one. A reused buffer
// keeps the large capacity; a fresh `Vec` sized to each packet drops
// back to the small one. Feeding equal-sized packets cannot tell those
// apart — the first version of this test did exactly that and a
// to_vec()-per-call implementation passed it.
let big = [frame.clone(), frame.clone(), frame.clone(), frame.clone()].concat();
parser.parse(&PesPacket {
source: None,
pid: 0x1100,
pts: None,
dts: None,
data: big.clone(),
discontinuity: false,
});
let cap_after_big = parser.scratch.capacity();
assert!(
cap_after_big >= big.len(),
"the buffer must have grown to hold the large packet"
);
parser.parse(&PesPacket {
source: None,
pid: 0x1100,
pts: None,
dts: None,
data: frame.clone(),
discontinuity: false,
});
assert_eq!(
parser.scratch.capacity(),
cap_after_big,
"after a small packet the buffer must STILL hold the large \
capacity — a fresh allocation per call would have shrunk to the \
small packet's size"
);
}
#[test]
fn parse_empty_pes() {
let mut parser = Ac3Parser::new();
@@ -1030,15 +1126,15 @@ mod tests {
let frames = parser.parse(&pes);
assert!(frames.is_empty());
assert!(
parser.buf.len() <= MAX_AC3_BUF,
parser.acc.len() <= MAX_AC3_BUF,
"buffer grew to {} (cap {})",
parser.buf.len(),
parser.acc.len(),
MAX_AC3_BUF
);
}
// After all that garbage the retained tail is at most a single partial
// syncword byte — never an accumulation of whole PES packets.
assert!(parser.buf.len() <= 1, "retained {} bytes", parser.buf.len());
assert!(parser.acc.len() <= 1, "retained {} bytes", parser.acc.len());
}
#[test]
@@ -1057,7 +1153,11 @@ mod tests {
discontinuity: false,
};
assert!(parser.parse(&pes).is_empty());
assert_eq!(parser.buf, vec![0x0B], "lone trailing 0x0B retained");
assert_eq!(
parser.acc.as_slice(),
vec![0x0B],
"lone trailing 0x0B retained"
);
}
#[test]
@@ -1067,14 +1167,14 @@ mod tests {
// ac3 inherited the no-op default flush and dropped the last frame.
let mut parser = Ac3Parser::new();
let frame_data = make_ac3_frame(0, 2);
parser.buf = frame_data.clone();
parser.acc.seed(&frame_data.clone());
parser.flush_pts_ns = pts_to_ns(99000);
let f = parser.flush();
assert_eq!(f.len(), 1, "complete buffered frame drained at EOS");
assert_eq!(f[0].data.len(), 160);
assert_eq!(f[0].pts_ns, pts_to_ns(99000), "flush uses carried PTS");
assert!(f[0].duration_ns.is_some(), "flush sets duration");
assert!(parser.buf.is_empty(), "buffer consumed by flush");
assert!(parser.acc.is_empty(), "buffer consumed by flush");
}
#[test]
@@ -1107,7 +1207,7 @@ mod tests {
// emitted truncated.
let mut parser = Ac3Parser::new();
let frame_data = make_ac3_frame(0, 2);
parser.buf = frame_data[..80].to_vec(); // half a frame
parser.acc.seed(&frame_data[..80]); // half a frame
assert!(parser.flush().is_empty(), "partial tail dropped");
}
@@ -1511,7 +1611,7 @@ mod tests {
// 100 bytes.
let mut parser = Ac3Parser::new();
let frame = make_ac3_frame(0, 2); // sizes to 160
parser.buf = frame[..100].to_vec();
parser.acc.seed(&frame[..100]);
assert!(
parser.flush().is_empty(),
"incomplete frame must not be emitted truncated at flush"
@@ -1522,7 +1622,7 @@ mod tests {
fn flush_with_no_sync_is_empty() {
// flush on a buffer with no syncword yields nothing and clears.
let mut parser = Ac3Parser::new();
parser.buf = vec![0xAA, 0xBB, 0xCC];
parser.acc.seed(&[0xAA, 0xBB, 0xCC]);
assert!(parser.flush().is_empty());
}
@@ -1839,7 +1939,7 @@ mod tests {
assert_eq!(fr.pts_ns, pts_to_ns(90000) + i as i64 * 32_000_000);
assert_eq!(fr.duration_ns, Some(32_000_000));
}
assert!(parser.buf.is_empty(), "nothing held back for plain AC-3");
assert!(parser.acc.is_empty(), "nothing held back for plain AC-3");
assert!(parser.flush().is_empty(), "flush has nothing left to drain");
}
@@ -2110,4 +2210,42 @@ mod tests {
discontinuity: false,
}
}
/// An access unit that began in an earlier packet keeps THAT packet's
/// source offset. The packet that completes it is a different clip at a
/// seam, and taking its offset places the audio in the wrong one.
#[test]
fn an_access_unit_carries_the_source_of_the_packet_it_began_in() {
let mut parser = Ac3Parser::new();
let frame = make_ac3_frame(0, 4);
let mut p1 = PesPacket {
pid: 0x1100,
pts: Some(90_000),
dts: None,
data: frame[..frame.len() / 2].to_vec(),
source: Some(crate::pes::SourcePos::at_byte(1_000)),
discontinuity: false,
};
p1.data.truncate(frame.len() / 2);
assert!(parser.parse(&p1).is_empty(), "partial frame held");
let mut rest = frame[frame.len() / 2..].to_vec();
rest.extend_from_slice(&make_ac3_frame(0, 4));
let p2 = PesPacket {
pid: 0x1100,
pts: Some(180_000),
dts: None,
data: rest,
source: Some(crate::pes::SourcePos::at_byte(9_000)),
discontinuity: false,
};
let frames = parser.parse(&p2);
assert!(!frames.is_empty(), "the completed unit is emitted");
assert_eq!(
frames[0].source.map(|s| s.byte),
Some(1_000),
"the unit belongs to the packet its FIRST byte came from"
);
}
}
+71 -3
View File
@@ -50,7 +50,15 @@ fn adts_verdict(data: &[u8]) -> AdtsVerdict {
// aac_frame_length: 13 bits = byte3[1:0] | byte4 | byte5[7:5].
let frame_length =
((u32::from(data[3]) & 0x03) << 11) | (u32::from(data[4]) << 3) | (u32::from(data[5]) >> 5);
if frame_length < 7 {
// The floor is the header the frame SAYS it carries, not a constant.
// protection_absent (byte1 bit0) clear means a 16-bit crc_check follows the
// 7-byte fixed+variable header, so the frame cannot be shorter than 9 —
// aac_frame_length counts the header and the CRC, not just the payload.
// Comparing against a flat 7 let a CRC-present frame declaring 7 or 8
// through as structurally Valid, and the muxer then carried a frame whose
// own header says it is impossible.
let header_bytes = if data[1] & 0x01 == 0 { 9 } else { 7 };
if frame_length < header_bytes {
return AdtsVerdict::Invalid;
}
AdtsVerdict::Valid
@@ -112,10 +120,14 @@ impl CodecParser for AdtsParser {
}
self.tally.record_kept();
// One PES is one unit here, so the unit's first byte is in THIS packet
// and its facts are this packet's -- the same rule the buffering
// parsers apply through `PesBuf::front`, with nothing carried over.
let facts = super::pesbuf::PesFacts::of(pes);
vec![Frame {
discontinuity: pes.discontinuity,
discontinuity: facts.discontinuity,
coding: None,
source: None,
source: facts.source,
pts_ns,
keyframe: true,
data: pes.data.clone(),
@@ -148,6 +160,51 @@ mod tests {
}
}
/// A header that CLAIMS a CRC (protection_absent = 0) but declares a
/// frame length too short to contain one.
///
/// `aac_frame_length` counts the header and the CRC, not just the payload,
/// so with a CRC present the smallest structurally possible frame is 9
/// bytes: the 7-byte fixed+variable header plus the 16-bit crc_check.
/// The gate compared against a flat 7 and never read protection_absent at
/// all, so a frame whose own header says it is impossible was classified
/// Valid and forwarded to the muxer.
#[test]
fn a_crc_present_header_shorter_than_its_own_crc_is_invalid() {
for declared in [7u32, 8] {
let mut f = adts_frame(16);
f[1] = 0xF0; // sync + MPEG-4, protection_absent = 0 => CRC present
f[3] = (f[3] & 0xFC) | ((declared >> 11) & 0x03) as u8;
f[4] = ((declared >> 3) & 0xFF) as u8;
f[5] = (f[5] & 0x1F) | ((declared & 0x07) << 5) as u8;
assert!(
matches!(adts_verdict(&f), AdtsVerdict::Invalid),
"protection_absent=0 declaring {declared} bytes cannot hold its \
own 7-byte header plus a 2-byte CRC"
);
}
// 9 is the smallest length that CAN hold header + CRC, so it must pass
// the structural gate — the floor moved, it did not become stricter
// than the spec.
let mut ok = adts_frame(16);
ok[1] = 0xF0;
let nine = 9u32;
ok[3] = (ok[3] & 0xFC) | ((nine >> 11) & 0x03) as u8;
ok[4] = ((nine >> 3) & 0xFF) as u8;
ok[5] = (ok[5] & 0x1F) | ((nine & 0x07) << 5) as u8;
assert!(matches!(adts_verdict(&ok), AdtsVerdict::Valid));
// And with NO CRC the floor is still 7, unchanged.
let mut no_crc = adts_frame(16);
no_crc[1] = 0xF1; // protection_absent = 1
let seven = 7u32;
no_crc[3] = (no_crc[3] & 0xFC) | ((seven >> 11) & 0x03) as u8;
no_crc[4] = ((seven >> 3) & 0xFF) as u8;
no_crc[5] = (no_crc[5] & 0x1F) | ((seven & 0x07) << 5) as u8;
assert!(matches!(adts_verdict(&no_crc), AdtsVerdict::Valid));
}
/// A valid ADTS header (AAC-LC, 44.1 kHz, stereo) + payload, with
/// aac_frame_length set to the total size.
fn adts_frame(payload: usize) -> Vec<u8> {
@@ -272,4 +329,15 @@ mod tests {
let f = p.parse(&make_pes(vec![0xFF, 0xF1, 0x50], Some(0)));
assert_eq!(f.len(), 1, "too short to validate → kept");
}
/// One PES is one unit here, so the frame carries that packet's offset.
#[test]
fn a_frame_carries_its_packets_source() {
let mut parser = AdtsParser::new();
let mut p = make_pes(adts_frame(64), Some(90_000));
p.source = Some(crate::pes::SourcePos::at_byte(4_242));
let frames = parser.parse(&p);
assert!(!frames.is_empty(), "a valid ADTS frame is emitted");
assert_eq!(frames[0].source.map(|s| s.byte), Some(4_242));
}
}
+157 -115
View File
@@ -6,7 +6,7 @@
//! are emitted complete.
use super::startcode::BitReader;
use super::{CodecParser, Frame, PesPacket, pts_to_ns};
use super::{CodecParser, Frame, PesPacket};
const DTS_CORE_SYNC: [u8; 4] = [0x7F, 0xFE, 0x80, 0x01];
/// DTS-HD extension substream syncword. An access unit is delimited by the next
@@ -21,23 +21,16 @@ const DTS_HD_EXT_SYNC: [u8; 4] = [0x64, 0x58, 0x20, 0x25];
/// This preserves the lossless extension data instead of downgrading to lossy
/// core (the lossy-core downgrade bug).
pub struct DtsParser {
buf: Vec<u8>,
/// Bytes assembled across PES packets, each attributable to the packet
/// that carried it. An emitted unit takes the facts of the packet covering
/// its FIRST byte, so an AU whose core arrived in an earlier PES keeps that
/// core's timestamp and source offset when its extensions arrive later.
acc: super::pesbuf::PesBuf,
/// PTS of the access unit currently being assembled in `buf` (the unit
/// starting at the first buffered core sync). Captured when that core
/// frame's PES first arrived; the trailing extension-substream PES
/// packets carry their own (later) PTS which must NOT override it.
pending_pts: i64,
/// PTS markers attributing buffer regions to their source PES. Each entry
/// is `(buffer_offset, pts_ns)` for the PES whose bytes begin at that
/// offset. When an access unit is emitted from the front of `buf`, its PTS
/// is the marker covering offset 0 — NOT the most recent PES's PTS. This is
/// what fixes multi-AU-per-call PTS attribution: if a PES carries the
/// extension substreams (and possibly the next core) for an AU whose own
/// core arrived in an earlier PES, the emitted AU keeps its own core PES's
/// timestamp instead of the later PES's. Offsets are kept relative to the
/// current `buf` start and rebased whenever bytes are drained from the
/// front.
pts_marks: std::collections::VecDeque<(usize, i64)>,
/// The `front_pts` of the PREVIOUS emitted access unit. When the current
/// AU's `front_pts` differs, it began a new PES → re-base to it. When it is
/// unchanged, this AU shares the previous AU's PES → advance one frame
@@ -66,9 +59,8 @@ impl Default for DtsParser {
impl DtsParser {
pub fn new() -> Self {
Self {
buf: Vec::with_capacity(32768),
acc: super::pesbuf::PesBuf::with_capacity(32768),
pending_pts: 0,
pts_marks: std::collections::VecDeque::new(),
last_front_pts: PTS_UNSET,
next_pts_ns: PTS_UNSET,
tally: super::dropgate::DropTally::new("dts"),
@@ -92,7 +84,14 @@ impl DtsParser {
/// PTS clock (which the caller advances whether or not the AU survives), so
/// a drop leaves the following audio on its true timeline — a gap, not a
/// shift. Every drop is logged (fail-loud, never silent).
fn emit_or_drop(&mut self, au: Vec<u8>, au_pts: i64, dur_ns: i64, out: &mut Vec<Frame>) {
fn emit_or_drop(
&mut self,
au: Vec<u8>,
au_pts: i64,
dur_ns: i64,
src: Option<crate::pes::SourcePos>,
out: &mut Vec<Frame>,
) {
let verdict = if self.tally.is_poisoned() {
Err(DropReason::TrackPoisoned)
} else {
@@ -104,7 +103,9 @@ impl DtsParser {
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
// From the SAME packet as `au_pts` — both are the facts of
// the PES covering this unit's first byte.
source: src,
pts_ns: au_pts,
keyframe: true,
data: au,
@@ -148,41 +149,25 @@ impl DtsParser {
base
}
/// Drop `n` bytes from the front of `buf` and rebase the PTS markers so
/// their offsets stay relative to the new buffer start. A marker that now
/// sits at or before offset 0 is clamped to 0 (it still covers the front).
/// Redundant markers all at offset 0 collapse to the last one.
/// Drop `n` bytes from the front, rebasing attribution onto the new front.
fn drain_front(&mut self, n: usize) {
if n == 0 {
return;
}
self.buf.drain(..n);
for m in &mut self.pts_marks {
m.0 = m.0.saturating_sub(n);
}
// Collapse all leading markers that now sit at offset 0 to the last
// such marker — that is the PES whose data currently begins the buffer.
let last_zero = self
.pts_marks
.iter()
.rposition(|&(off, _)| off == 0)
.filter(|&i| i > 0);
if let Some(i) = last_zero {
self.pts_marks.drain(..i);
}
self.acc.drain(n);
}
/// PTS that should be stamped on an access unit currently at the front of
/// `buf` (offset 0): the most recent marker at offset 0, falling back to
/// `pending_pts`.
/// PTS for the access unit at the front of the buffer: the facts of the
/// packet covering offset 0, falling back to the unit's captured base.
fn front_pts(&self) -> i64 {
self.pts_marks
.iter()
.rev()
.find(|&&(off, _)| off == 0)
.map(|&(_, pts)| pts)
self.acc
.front()
.presentation_ns()
.unwrap_or(self.pending_pts)
}
/// Source offset for that same unit — from the SAME packet as its PTS,
/// which is the property the shared buffer exists to guarantee.
fn front_source(&self) -> Option<crate::pes::SourcePos> {
self.acc.front().source
}
}
/// Hard cap on a buffered access unit (core + all its extension substreams).
@@ -227,8 +212,7 @@ impl CodecParser for DtsParser {
// never be stranded by an empty post-gap PES (defensive; the demuxer only
// emits non-empty PES today).
if pes.discontinuity {
self.buf.clear();
self.pts_marks.clear();
self.acc.clear();
self.pending_pts = PTS_UNSET;
// A concealed gap is a timeline discontinuity: let the post-gap AU
// re-base to its own PES PTS rather than the pre-gap cursor.
@@ -242,17 +226,15 @@ impl CodecParser for DtsParser {
// guard at a post-gap continuation) must NOT reset the timeline to 0;
// continue from the most recent known base. Defense-in-depth: the
// discontinuity-carrying PES is a PUSI with a PTS in practice.
let pts_ns = pes.pts.map(pts_to_ns).unwrap_or_else(|| {
self.pts_marks
.back()
.map(|&(_, p)| p)
.filter(|&p| p >= 0)
.unwrap_or(if self.pending_pts >= 0 {
self.pending_pts
} else {
0
})
});
// A PES with no PTS (rare for audio, but legal) must NOT reset the
// timeline to 0 — continue from the most recent known base.
let pts_ns = super::pesbuf::PesFacts::of(pes)
.presentation_ns()
.unwrap_or(if self.pending_pts >= 0 {
self.pending_pts
} else {
0
});
// On Blu-ray, a DTS-HD MA/HRA access unit is a DTS core frame
// (sync 0x7FFE8001) followed by one or more DTS extension substreams
@@ -272,7 +254,7 @@ impl CodecParser for DtsParser {
// prior forced (safety-valve) flush left it invalidated — in the
// forced case the bytes still in `buf` are not a real core frame, so
// the first PES to arrive after the flush carries the correct base.
if self.buf.is_empty() || self.pending_pts == PTS_UNSET {
if self.acc.is_empty() || self.pending_pts == PTS_UNSET {
self.pending_pts = pts_ns;
}
// Mark where THIS PES's bytes begin in the buffer, with its PTS. The
@@ -280,21 +262,22 @@ impl CodecParser for DtsParser {
// (see `front_pts`), so an AU whose core arrived in an earlier PES keeps
// that core's timestamp even when its extensions / the following core
// arrive (with a later PTS) in this same parse() call.
// (pts_marks is bounded implicitly: an empty PES returns above without
// pushing a mark, and a non-empty run grows `buf`, which is cleared —
// along with pts_marks — once it exceeds MAX_AU_BYTES.)
self.pts_marks.push_back((self.buf.len(), pts_ns));
self.buf.extend_from_slice(&pes.data);
// `pts_ns` is this packet's own timestamp, or the carried-forward base
// when it had none; the source offset is always this packet's.
self.acc.push_with(
&pes.data,
super::pesbuf::PesFacts::of(pes).with_pts_ns(pts_ns),
);
let mut frames = Vec::new();
loop {
// Resync to the first core sync; drop any leading junk.
let Some(start) = find_sync(&self.buf, &DTS_CORE_SYNC) else {
let Some(start) = find_sync(self.acc.as_slice(), &DTS_CORE_SYNC) else {
// No core sync at all yet — keep at most a 3-byte tail so a
// sync split across PES packets can still be found next time.
if self.buf.len() > 3 {
let tail = self.buf.len() - 3;
if self.acc.len() > 3 {
let tail = self.acc.len() - 3;
self.drain_front(tail);
}
break;
@@ -305,17 +288,17 @@ impl CodecParser for DtsParser {
// offset 0 by construction, so a re-scan would be a redundant
// O(buf_len) walk per iteration; assert the invariant instead.
debug_assert_eq!(
find_sync(&self.buf, &DTS_CORE_SYNC),
find_sync(self.acc.as_slice(), &DTS_CORE_SYNC),
Some(0),
"drain_front(start) must leave the core sync at offset 0"
);
}
// Need the core header to size the core frame.
if self.buf.len() < CORE_HEADER_MIN_BYTES {
if self.acc.len() < CORE_HEADER_MIN_BYTES {
break;
}
let core_size = dts_core_frame_size(&self.buf);
let core_size = dts_core_frame_size(self.acc.as_slice());
// `dts_core_frame_size` returns a 14-bit `fsize + 1`, so it is
// always in [1, 16384]; the bare `== 0` / `> MAX_AU_BYTES` checks
// can never fire. A real DTS core frame is at least
@@ -330,7 +313,7 @@ impl CodecParser for DtsParser {
self.drain_front(4);
continue;
}
if self.buf.len() < core_size {
if self.acc.len() < core_size {
break; // core frame not fully buffered yet — wait
}
@@ -349,9 +332,9 @@ impl CodecParser for DtsParser {
// flush is an extension-substream PES, carrying its own later
// timestamp) must NOT become the next unit's PTS base.
let mut forced = false;
let (au_end, ext_clean) = match next_core_boundary(&self.buf, core_size) {
let (au_end, ext_clean) = match next_core_boundary(self.acc.as_slice(), core_size) {
NextCore::Found { end, ext_clean } => (end, ext_clean),
NextCore::NeedMore if self.buf.len() <= MAX_AU_BYTES => break,
NextCore::NeedMore if self.acc.len() <= MAX_AU_BYTES => break,
NextCore::NeedMore => {
// A candidate boundary exists but is not fully buffered. Normally
// we wait for more PES; but once the buffer exceeds the AU cap,
@@ -359,7 +342,7 @@ impl CodecParser for DtsParser {
// stream that keeps a boundary perpetually incomplete can't grow
// `buf` without bound (the `break` above never reaches it).
forced = true;
(self.buf.len(), true)
(self.acc.len(), true)
}
NextCore::None => {
// No next core sync buffered yet. The trailing extension
@@ -367,11 +350,11 @@ impl CodecParser for DtsParser {
// them rather than emit a core-only (lossy) frame — unless
// the buffer has grown unreasonably large, in which case
// emit what we have to guarantee forward progress.
if self.buf.len() <= MAX_AU_BYTES {
if self.acc.len() <= MAX_AU_BYTES {
break;
}
forced = true;
(self.buf.len(), true)
(self.acc.len(), true)
}
};
@@ -384,7 +367,7 @@ impl CodecParser for DtsParser {
// rather than shipping it. A recognized-but-unsizeable extension
// (`ext_clean == true`) is preserved in full (lossless).
let emit_end = if ext_clean { au_end } else { core_size };
let au: Vec<u8> = self.buf[..emit_end].to_vec();
let au: Vec<u8> = self.acc.as_slice()[..emit_end].to_vec();
// The AU's own core PES PTS (the PES covering its first byte, even if
// that PES preceded the one(s) carrying its extensions or the next
// core), stamped monotonically: honored when it advances past the
@@ -396,7 +379,10 @@ impl CodecParser for DtsParser {
// would: the following AU keeps its true PTS and the drop is a gap,
// never a shift. `emit_or_drop` decides whether to actually push it.
let au_pts = self.stamp_pts(self.front_pts(), dur_ns);
self.emit_or_drop(au, au_pts, dur_ns, &mut frames);
// Read BEFORE draining: after the drain the front is the NEXT
// unit's packet, not this one's.
let au_src = self.front_source();
self.emit_or_drop(au, au_pts, dur_ns, au_src, &mut frames);
self.drain_front(au_end);
// After draining, the marker covering the new front (if any) carries
// the next AU's PTS; `pending_pts` is only the fallback when no
@@ -408,15 +394,14 @@ impl CodecParser for DtsParser {
// regardless of buffer state, rather than inheriting this
// (non-core) PES's timestamp.
self.pending_pts = PTS_UNSET;
self.pts_marks.clear();
}
}
// Discard markers that no longer reference live buffer bytes (everything
// past the buffer end can't happen, but collapse duplicates at offset 0
// and drop a stale empty-buffer marker set).
if self.buf.is_empty() {
self.pts_marks.clear();
// An empty buffer holds no bytes for a mark to attribute, so drop the
// marks with them. `drain` deliberately keeps the mark covering the new
// front — correct while bytes remain, stale once none do.
if self.acc.is_empty() {
self.acc.clear();
}
frames
@@ -440,26 +425,28 @@ impl DtsParser {
/// streaming), gated through the decodability check. Require a complete core
/// frame; drop a bare partial sync tail.
fn flush_tail(&mut self) -> Vec<Frame> {
if find_sync(&self.buf, &DTS_CORE_SYNC) != Some(0) || self.buf.len() < CORE_HEADER_MIN_BYTES
if find_sync(self.acc.as_slice(), &DTS_CORE_SYNC) != Some(0)
|| self.acc.len() < CORE_HEADER_MIN_BYTES
{
self.buf.clear();
self.acc.clear();
return Vec::new();
}
let core_size = dts_core_frame_size(&self.buf);
let core_size = dts_core_frame_size(self.acc.as_slice());
// `dts_core_frame_size` returns a 14-bit `fsize + 1` (never 0), so the
// old `== 0` check was dead; reject a sub-minimum core like `parse()`.
if core_size < MIN_CORE_FRAME_BYTES || self.buf.len() < core_size {
self.buf.clear();
if core_size < MIN_CORE_FRAME_BYTES || self.acc.len() < core_size {
self.acc.clear();
return Vec::new();
}
// The final AU's PTS is the PES covering the buffer front (its core's
// PES). Fall back to pending_pts, clamping the sentinel to 0.
let au = std::mem::take(&mut self.buf);
let au = self.acc.as_slice().to_vec();
let dur_ns = dts_core_duration_ns(&au) as i64;
let pts_ns = self.stamp_pts(self.front_pts(), dur_ns);
self.pts_marks.clear();
let src = self.front_source();
self.acc.clear();
let mut out = Vec::new();
self.emit_or_drop(au, pts_ns, dur_ns, &mut out);
self.emit_or_drop(au, pts_ns, dur_ns, src, &mut out);
out
}
}
@@ -824,6 +811,7 @@ fn core_header_drop_reason(au: &[u8]) -> Option<DropReason> {
#[cfg(test)]
mod tests {
use super::*;
use crate::mux::codec::pts_to_ns;
use crate::mux::ts::PesPacket;
fn make_pes(data: Vec<u8>, pts: Option<i64>) -> PesPacket {
@@ -899,15 +887,19 @@ mod tests {
fn drain_front_collapses_offset_zero_markers_instead_of_leaking() {
let mut parser = DtsParser::new();
for i in 0..200i64 {
parser.buf.extend_from_slice(&[0u8; 5]);
parser.pts_marks.push_back((5, i));
parser.pts_marks.push_back((5, i));
parser.acc.append_unattributed(&[0u8; 5]);
parser
.acc
.mark_here(crate::mux::codec::pesbuf::PesFacts::default().with_pts_ns(i));
parser
.acc
.mark_here(crate::mux::codec::pesbuf::PesFacts::default().with_pts_ns(i));
parser.drain_front(5);
}
assert!(
parser.pts_marks.len() <= 2,
parser.acc.mark_count() <= 2,
"pts_marks must stay bounded across repeated drains, got {}",
parser.pts_marks.len()
parser.acc.mark_count()
);
}
@@ -1579,7 +1571,7 @@ mod tests {
"NeedMore past the AU cap must force-emit, not stall and balloon the buffer"
);
assert!(
parser.buf.is_empty(),
parser.acc.is_empty(),
"the forced flush drains the buffer instead of growing it unbounded"
);
}
@@ -1751,7 +1743,7 @@ mod tests {
// The first core's bytes are still buffered awaiting the verdict — not
// dropped, not emitted.
assert!(
parser.buf.len() >= 512,
parser.acc.len() >= 512,
"core1 retained while candidate boundary is undecided"
);
}
@@ -1809,8 +1801,8 @@ mod tests {
let mut parser = DtsParser::new();
let f = parser.parse(&make_pes(vec![0x11, 0x22, 0x33, 0x44], Some(90000)));
assert!(f.is_empty());
assert_eq!(parser.buf.len(), 3, "only a 3-byte resync tail retained");
assert_eq!(parser.buf, vec![0x22, 0x33, 0x44]);
assert_eq!(parser.acc.len(), 3, "only a 3-byte resync tail retained");
assert_eq!(parser.acc.as_slice(), &[0x22, 0x33, 0x44]);
}
#[test]
@@ -1825,7 +1817,7 @@ mod tests {
.parse(&make_pes(core[..3].to_vec(), Some(90000)))
.is_empty()
);
assert_eq!(parser.buf.len(), 3, "3-byte sync prefix retained");
assert_eq!(parser.acc.len(), 3, "3-byte sync prefix retained");
// PES 2: the 4th sync byte + the rest of core1, then a 2nd core to close.
let mut rest = core[3..].to_vec();
rest.extend_from_slice(&make_dts_core(640));
@@ -1847,7 +1839,7 @@ mod tests {
let mut data = DTS_CORE_SYNC.to_vec();
data.extend_from_slice(&[0x00, 0x00, 0x00]); // only 7 bytes total < 10
assert!(parser.parse(&make_pes(data, Some(90000))).is_empty());
assert!(!parser.buf.is_empty(), "partial core header retained");
assert!(!parser.acc.is_empty(), "partial core header retained");
}
#[test]
@@ -1859,7 +1851,7 @@ mod tests {
let mut d = vec![0u8; 17];
d[0..4].copy_from_slice(&DTS_CORE_SYNC);
d[6] = 0x01; // fsize → 16 → size 17
parser.buf = d;
parser.acc.seed(&d);
assert!(parser.flush().is_empty(), "sub-spec core rejected at flush");
}
@@ -1869,7 +1861,7 @@ mod tests {
// declared size must be dropped (never emit fewer bytes than declared).
let mut parser = DtsParser::new();
let core = make_dts_core(512);
parser.buf = core[..300].to_vec(); // header says 512, only 300 present
parser.acc.seed(&core[..300]); // header says 512, only 300 present
assert!(
parser.flush().is_empty(),
"incomplete core not emitted truncated"
@@ -1886,9 +1878,9 @@ mod tests {
fn flush_partial_sync_tail_dropped() {
// A bare partial-sync tail (not at offset 0 / not a full core) is dropped.
let mut parser = DtsParser::new();
parser.buf = vec![0x7F, 0xFE, 0x80]; // 3 of 4 sync bytes
parser.acc.seed(&[0x7F, 0xFE, 0x80]); // 3 of 4 sync bytes
assert!(parser.flush().is_empty());
assert!(parser.buf.is_empty(), "buffer cleared on flush");
assert!(parser.acc.is_empty(), "buffer cleared on flush");
}
#[test]
@@ -2311,13 +2303,13 @@ mod tests {
let out = parser.parse(&make_pes(d, Some(90_000)));
assert!(out.is_empty(), "a false sync emits nothing");
assert_eq!(
parser.buf.len(),
parser.acc.len(),
3,
"the false sync was decoded, drained and resynced past — leaving only \
the 3-byte split-sync carry-over"
);
assert_ne!(
find_sync(&parser.buf, &DTS_CORE_SYNC),
find_sync(parser.acc.as_slice(), &DTS_CORE_SYNC),
Some(0),
"and the bogus sync is no longer at the front of the buffer"
);
@@ -2331,14 +2323,14 @@ mod tests {
fn flush_emits_a_core_that_exactly_fills_the_buffer() {
let mut parser = DtsParser::new();
let core = make_dts_core(512);
parser.buf = core.clone();
parser.acc.seed(&core.clone());
parser.pending_pts = 90_000;
let out = parser.flush();
assert_eq!(out.len(), 1, "the final AU is emitted, not dropped");
assert_eq!(out[0].data, core, "and it is the whole core frame");
// One byte short is still refused — the bound is not simply absent.
let mut parser = DtsParser::new();
parser.buf = core[..511].to_vec();
parser.acc.seed(&core[..511]);
parser.pending_pts = 90_000;
assert!(
parser.flush().is_empty(),
@@ -2368,18 +2360,18 @@ mod tests {
Some(0),
"the fixture really has no core sync at the front"
);
parser.buf = broken;
parser.acc.seed(&broken);
parser.pending_pts = 90_000;
assert!(
parser.flush().is_empty(),
"a buffer whose front is not a core sync is discarded, not size-decoded"
);
assert!(parser.buf.is_empty(), "and the junk is dropped");
assert!(parser.acc.is_empty(), "and the junk is dropped");
// The other half of the disjunction: a buffer too short to size, whose
// front IS a core sync, is discarded too.
let mut parser = DtsParser::new();
parser.buf = DTS_CORE_SYNC.to_vec();
parser.acc.seed(DTS_CORE_SYNC.as_ref());
parser.pending_pts = 90_000;
assert!(parser.flush().is_empty(), "a bare sync tail is not an AU");
}
@@ -2416,4 +2408,54 @@ mod tests {
"the bare sync sizes nothing"
);
}
/// The whole point of the shared buffer: an access unit whose core arrived
/// in an EARLIER packet keeps that packet's source offset, not the offset
/// of whichever packet completed it. At a clip boundary the two belong to
/// different clips, and taking the later one puts the unit in the wrong
/// clip -- which is what left nine audio and subtitle tracks unplaceable.
#[test]
fn an_access_unit_carries_the_source_of_the_packet_its_core_arrived_in() {
let mut parser = DtsParser::new();
let core = make_dts_core(512);
// The core starts here, at byte 1000 of the feed.
let mut p1 = make_pes(core[..256].to_vec(), Some(90_000));
p1.source = Some(crate::pes::SourcePos::at_byte(1_000));
assert!(parser.parse(&p1).is_empty(), "partial core held");
// The rest arrives later, at byte 9_000, together with the next core
// that closes the unit.
let mut rest = core[256..].to_vec();
rest.extend_from_slice(&make_dts_core(512));
let mut p2 = make_pes(rest, Some(180_000));
p2.source = Some(crate::pes::SourcePos::at_byte(9_000));
let frames = parser.parse(&p2);
assert!(!frames.is_empty(), "the completed unit is emitted");
assert_eq!(
frames[0].source.map(|s| s.byte),
Some(1_000),
"the unit belongs to the packet its FIRST byte came from"
);
assert_eq!(
frames[0].pts_ns,
pts_to_ns(90_000),
"and its timestamp comes from that same packet"
);
}
/// A unit that begins and ends in one packet takes that packet's offset —
/// the ordinary case, which must not regress while fixing the spanning one.
#[test]
fn a_self_contained_access_unit_carries_its_own_packets_source() {
let mut parser = DtsParser::new();
let mut data = make_dts_core(512);
data.extend_from_slice(&make_dts_core(512));
let mut p = make_pes(data, Some(90_000));
p.source = Some(crate::pes::SourcePos::at_byte(4_242));
let frames = parser.parse(&p);
assert!(!frames.is_empty());
assert_eq!(frames[0].source.map(|s| s.byte), Some(4_242));
}
}
+39 -7
View File
@@ -25,7 +25,10 @@ pub struct DvdSubParser {
/// Pre-formatted VobSub .idx palette header for codec_private.
codec_data: Option<Vec<u8>>,
/// In-progress SPU reassembly: (head PTS in ns, declared SPU_size, bytes).
pending: Option<(i64, usize, Vec<u8>)>,
/// The SPU being accumulated, with the facts of the PES that STARTED it.
/// An SPU spans PES packets, so its timestamp and source offset are the
/// opening packet's — the same rule every other buffering parser applies.
pending: Option<(super::pesbuf::PesFacts, usize, Vec<u8>)>,
}
impl DvdSubParser {
@@ -41,11 +44,12 @@ impl DvdSubParser {
fn take_if_complete(&mut self, force: bool) -> Option<Frame> {
let (_, size, buf) = self.pending.as_ref()?;
if force || buf.len() >= *size {
let (pts_ns, _, data) = self.pending.take().unwrap();
let (facts, _, data) = self.pending.take().unwrap();
let pts_ns = facts.presentation_ns().unwrap_or(0);
return Some(Frame {
discontinuity: false,
coding: None,
source: None,
source: facts.source,
pts_ns,
keyframe: true,
data,
@@ -106,7 +110,7 @@ impl CodecParser for DvdSubParser {
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
source: super::pesbuf::PesFacts::of(pes).source,
pts_ns,
keyframe: true,
data: pes.data.clone(),
@@ -120,7 +124,7 @@ impl CodecParser for DvdSubParser {
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
source: super::pesbuf::PesFacts::of(pes).source,
pts_ns,
keyframe: true,
data: pes.data.clone(),
@@ -133,7 +137,7 @@ impl CodecParser for DvdSubParser {
if buf.len() > MAX_SPU_BYTES {
buf.truncate(MAX_SPU_BYTES);
}
self.pending = Some((pts_ns, declared, buf));
self.pending = Some((super::pesbuf::PesFacts::of(pes), declared, buf));
if let Some(frame) = self.take_if_complete(false) {
out.push(frame);
}
@@ -671,7 +675,17 @@ mod tests {
let f = parser.parse(&make_pes(vec![0x00, 0x10, 0xAA], None));
assert!(f.is_empty(), "incomplete sized segment held, not emitted");
assert!(parser.pending.is_some(), "started a new pending SPU");
assert_eq!(parser.pending.as_ref().unwrap().0, 0, "pts 0 (no PTS)");
assert_eq!(
parser
.pending
.as_ref()
.unwrap()
.0
.presentation_ns()
.unwrap_or(0),
0,
"pts 0 (no PTS)"
);
}
#[test]
@@ -703,4 +717,22 @@ mod tests {
let result = format_palette(&[[0x00, 16, 128, 128]], 0, 0);
assert_eq!(String::from_utf8(result).unwrap(), "palette: 101010\n");
}
/// The text guard in `codec/mod.rs` scans for a literal `source: None` and
/// cannot see a parser that writes `source: facts.source` where the facts
/// carry no offset. Only a runtime check proves an emitted frame really
/// carries the byte it was read from, and without it a multi-clip title
/// places this track by timestamp inference instead of by byte.
#[test]
fn an_emitted_frame_carries_the_packets_source_offset() {
let mut parser = DvdSubParser::new(None);
let mut pes = make_pes(
vec![0x00, 0x0A, 0x00, 0x08, 0x01, 0xFF, 0x02, 0x03, 0x04, 0x05],
Some(90_000),
);
pes.source = Some(crate::pes::SourcePos::at_byte(7_777));
let frames = parser.parse(&pes);
assert!(!frames.is_empty(), "the segment is emitted");
assert_eq!(frames[0].source.map(|s| s.byte), Some(7_777));
}
}
+16 -1
View File
@@ -118,7 +118,7 @@ impl CodecParser for FlacParser {
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
source: super::pesbuf::PesFacts::of(pes).source,
pts_ns,
keyframe: true,
data: pes.data.clone(),
@@ -271,4 +271,19 @@ mod tests {
);
assert_eq!(emitted.len() + tail.len(), 2);
}
/// The text guard in `codec/mod.rs` scans for a literal `source: None` and
/// cannot see a parser that writes `source: facts.source` where the facts
/// carry no offset. Only a runtime check proves an emitted frame really
/// carries the byte it was read from, and without it a multi-clip title
/// places this track by timestamp inference instead of by byte.
#[test]
fn an_emitted_frame_carries_the_packets_source_offset() {
let mut p = FlacParser::new();
let mut pes = make_pes(make_flac_frame(100), Some(90_000));
pes.source = Some(crate::pes::SourcePos::at_byte(7_777));
let f = p.parse(&pes);
assert!(!f.is_empty(), "the frame is emitted");
assert_eq!(f[0].source.map(|s| s.byte), Some(7_777));
}
}
+18 -1
View File
@@ -74,7 +74,7 @@ impl CodecParser for LpcmParser {
vec![Frame {
discontinuity: false,
coding: None,
source: None,
source: super::pesbuf::PesFacts::of(pes).source,
pts_ns,
keyframe: true,
data: pes.data[offset..].to_vec(),
@@ -279,4 +279,21 @@ mod tests {
let f = parser.parse(&make_pes(vec![0xAA, 0xBB], None));
assert_eq!(f[0].pts_ns, 0);
}
/// The text guard in `codec/mod.rs` scans for a literal `source: None` and
/// cannot see a parser that writes `source: facts.source` where the facts
/// carry no offset. Only a runtime check proves an emitted frame really
/// carries the byte it was read from, and without it a multi-clip title
/// places this track by timestamp inference instead of by byte.
#[test]
fn an_emitted_frame_carries_the_packets_source_offset() {
let mut parser = LpcmParser::new();
let mut data = vec![0x00, 0x01, 0x00, 0b1001_0001];
data.extend_from_slice(&[0xDE, 0xAD, 0xBE, 0xEF, 0xCA, 0xFE]);
let mut pes = make_pes(data, Some(90_000));
pes.source = Some(crate::pes::SourcePos::at_byte(7_777));
let frames = parser.parse(&pes);
assert!(!frames.is_empty(), "the frame is emitted");
assert_eq!(frames[0].source.map(|s| s.byte), Some(7_777));
}
}
+170 -1
View File
@@ -34,6 +34,11 @@ pub mod mpeg2;
pub mod mpegaudio;
/// HDMV PGS (Presentation Graphics Stream) subtitle parser.
pub mod pgs;
/// One accumulation buffer for parsers that assemble access units across PES
/// packets, so a unit's timestamp and its source offset always come from the
/// packet that carried its first byte -- and from the SAME packet.
pub(crate) mod pesbuf;
/// Display-order PTS reconstruction for sparse-PTS program-stream video.
pub(crate) mod reorder;
/// Shared MPEG/Annex-B start-code scanning helpers.
@@ -149,7 +154,7 @@ impl CodecParser for PassthroughParser {
// parsers must instead defer the flag to the next emitted frame.)
vec![Frame {
coding: None,
source: None,
source: pesbuf::PesFacts::of(pes).source,
pts_ns,
keyframe: self.keyframe,
discontinuity: pes.discontinuity,
@@ -354,3 +359,167 @@ mod tests {
}
}
}
#[cfg(test)]
mod provenance_guard {
//! Every emitted frame must carry the source byte offset of the packet it
//! came from. That invariant held only for video for as long as it existed:
//! `dts`, `ac3`, `adts`, `truehd`, `pgs`, `dvdsub`, `flac`, `lpcm`,
//! `mpegaudio` and the passthrough parser all built frames with
//! `source: None`, so a multi-clip title could not place audio or subtitles
//! by byte and fell back to inferring the clip from timestamps — which is
//! what made branched titles run minutes long.
//!
//! Nothing asserted it, so nothing caught it. Finding it took a
//! brace-balanced scan of the tree by hand, which also turned up five sites
//! a regex had missed. This is that scan, as a test.
/// Every parser module's source, checked for a `Frame` built without a
/// source. Paired with `every_codec_module_is_covered` below, which fails
/// if a module is added and not listed here.
const PARSER_SOURCES: &[(&str, &str)] = &[
("mod.rs", include_str!("mod.rs")),
("ac3.rs", include_str!("ac3.rs")),
("adts.rs", include_str!("adts.rs")),
("dts.rs", include_str!("dts.rs")),
("dvdsub.rs", include_str!("dvdsub.rs")),
("flac.rs", include_str!("flac.rs")),
("h264.rs", include_str!("h264.rs")),
("hevc.rs", include_str!("hevc.rs")),
("lpcm.rs", include_str!("lpcm.rs")),
("mpeg2.rs", include_str!("mpeg2.rs")),
("mpegaudio.rs", include_str!("mpegaudio.rs")),
("pgs.rs", include_str!("pgs.rs")),
("truehd.rs", include_str!("truehd.rs")),
("vc1.rs", include_str!("vc1.rs")),
];
/// Modules that declare no `Frame` and so need no entry above.
const NON_PARSER_MODULES: &[&str] = &[
"coding",
"crc",
"dropgate",
"pesbuf",
"reorder",
"startcode",
];
/// Walk `Frame { .. }` literals with balanced braces. A regex cannot do
/// this — `Frame` blocks contain nested braces (`coding: Some(..)`, closures)
/// and a non-greedy match stops at the first `}`, which is how five sites
/// survived the first pass.
/// Strip line comments before scanning. A comment that MENTIONS
/// `source: None` is prose, not a construction — this guard's own doc
/// comment tripped it on the first run, which is the same mistake as
/// grepping a file and matching its commentary.
fn code_only(src: &str) -> String {
// The guard itself constructs no Frame; it only talks about them, in
// prose and in string literals. Scanning its own body matches both.
let src = match src.find("mod provenance_guard") {
Some(i) => &src[..i],
None => src,
};
src.lines()
.map(|l| match l.find("//") {
// Not inside a string literal: no quote before the slashes.
Some(i) if !l[..i].contains('"') => &l[..i],
_ => l,
})
.collect::<Vec<_>>()
.join("\n")
}
fn frame_literals(src: &str) -> Vec<&str> {
let mut out = Vec::new();
let b = src.as_bytes();
let mut i = 0;
while let Some(p) = src[i..].find("Frame {") {
let start = i + p;
let open = start + src[start..].find('{').unwrap();
let (mut depth, mut k) = (0usize, open);
while k < b.len() {
match b[k] {
b'{' => depth += 1,
b'}' => {
depth -= 1;
if depth == 0 {
break;
}
}
_ => {}
}
k += 1;
}
out.push(&src[start..(k + 1).min(src.len())]);
i = k + 1;
}
out
}
#[test]
fn no_parser_emits_a_frame_without_provenance() {
let mut offenders = Vec::new();
for (name, src) in PARSER_SOURCES {
let src = &code_only(src);
for blk in frame_literals(src) {
// `PesFrame` in the tests of other modules is not ours; only
// codec `Frame` literals are scanned, and a test fixture that
// builds a PesPacket with `source: None` is legitimate.
// Two spellings, not one. `source: None` is the obvious way to
// lose provenance; OMITTING the field entirely is the quiet
// one, because `Frame` derives Default, so
// `Frame { pts_ns, .. Default::default() }` compiles and
// yields `source: None` while containing no such text. A guard
// that only knew the first spelling would have watched a
// parser be rewritten into the second and stayed green.
let explicit_none = blk.contains("source: None");
let no_source_field = !blk.contains("source:");
if explicit_none || no_source_field {
let line = src[..src.find(blk).unwrap_or(0)].lines().count() + 1;
let how = if explicit_none {
"source: None"
} else {
"no source field (Default fills in None)"
};
offenders.push(format!("{name}:{line} ({how})"));
}
}
}
assert!(
offenders.is_empty(),
"these parsers build a Frame with no source byte offset, so a \
multi-clip title cannot place their tracks by provenance and \
falls back to inferring the clip from timestamps: {offenders:?}"
);
}
/// A parser added without an entry in `PARSER_SOURCES` would be silently
/// unchecked, which is exactly how this class of gap persists.
#[test]
fn every_codec_module_is_covered() {
let modules: Vec<&str> = include_str!("mod.rs")
.lines()
.filter_map(|l| {
let l = l.trim();
let rest = l
.strip_prefix("pub mod ")
.or_else(|| l.strip_prefix("pub(crate) mod "))?;
rest.strip_suffix(';')
})
.collect();
let listed: Vec<&str> = PARSER_SOURCES
.iter()
.map(|(n, _)| n.trim_end_matches(".rs"))
.collect();
let unchecked: Vec<&&str> = modules
.iter()
.filter(|m| !listed.contains(m) && !NON_PARSER_MODULES.contains(m))
.collect();
assert!(
unchecked.is_empty(),
"codec module(s) not covered by the provenance guard — add them to \
PARSER_SOURCES (or to NON_PARSER_MODULES if they emit no Frame): \
{unchecked:?}"
);
}
}
+16 -1
View File
@@ -116,7 +116,7 @@ impl CodecParser for MpegAudioParser {
vec![Frame {
discontinuity: pes.discontinuity,
coding: None,
source: None,
source: super::pesbuf::PesFacts::of(pes).source,
pts_ns,
keyframe: true,
data: pes.data.clone(),
@@ -319,4 +319,19 @@ mod tests {
// a manufactured tail frame would break this even if it were non-empty.
assert_eq!(emitted.len() + tail.len(), 2);
}
/// The text guard in `codec/mod.rs` scans for a literal `source: None` and
/// cannot see a parser that writes `source: facts.source` where the facts
/// carry no offset. Only a runtime check proves an emitted frame really
/// carries the byte it was read from, and without it a multi-clip title
/// places this track by timestamp inference instead of by byte.
#[test]
fn an_emitted_frame_carries_the_packets_source_offset() {
let mut p = MpegAudioParser::new();
let mut pes = make_pes(mp3_frame(400), Some(90_000));
pes.source = Some(crate::pes::SourcePos::at_byte(7_777));
let f = p.parse(&pes);
assert!(!f.is_empty(), "the frame is emitted");
assert_eq!(f[0].source.map(|s| s.byte), Some(7_777));
}
}
+362
View File
@@ -0,0 +1,362 @@
//! One accumulation buffer for parsers that assemble access units across PES
//! packets.
//!
//! A buffering parser has to answer the same question for every unit it emits:
//! *which PES contributed this unit's FIRST byte?* Its timestamp comes from
//! that PES, and so does the source byte offset that identifies which clip of a
//! multi-clip title the unit belongs to. The trailing PES packets that complete
//! the unit carry their own, later, values which must not override it.
//!
//! That question was answered three different ways. DTS kept a deque of
//! `(offset, pts)` markers and took the one covering offset 0 — correct. AC-3
//! kept a single carry-over timestamp. TrueHD kept its own. None of them
//! carried the source offset at all, so provenance existed only for video, and
//! a title whose clip marks could not be read from timestamps alone had nine
//! audio and subtitle tracks with nothing to place them by.
//!
//! Three spellings of one rule is how they drifted, so this is the one place it
//! lives. The buffer owns the bytes AND the marks, and returns a PES's facts
//! together — a parser cannot take the timestamp from one PES and the source
//! from another, because it does not assemble them itself.
use super::PesPacket;
use super::pts_to_ns;
use crate::pes::SourcePos;
/// What a PES contributes to the bytes it carried.
///
/// Returned as a unit so a caller cannot mix fields from different packets.
/// The timestamps are carried RAW, as the packet had them. This type answers
/// *which packet* a unit's facts come from — the question that was being
/// answered three different ways. How a given codec derives a timestamp from
/// that packet stays the codec's business: DVD subtitles read `pts` only and
/// fall back to 0, most audio takes `pts.or(dts)`. Deriving it here would have
/// changed those semantics silently while fixing provenance.
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub(crate) struct PesFacts {
/// Presentation time in NANOSECONDS, already derived (see `of`). Stored
/// derived rather than raw so there is one derivation and no caller can
/// pick a different one.
pub pts_ns: Option<i64>,
/// Byte offset of this PES's first ES byte within the title's feed — what
/// identifies the clip a frame came from. `None` when the demuxer was fed
/// without a base offset.
pub source: Option<SourcePos>,
/// Packets for this stream were lost before this PES.
pub discontinuity: bool,
}
impl PesFacts {
/// The facts a PES packet carries, read straight off it.
///
/// Every parser reads its frame's timestamp, source and discontinuity from
/// a `PesFacts` — never off a `PesPacket` field directly — so the three can
/// never be taken from different packets. Parsers that assemble units
/// across packets get theirs from [`PesBuf::front`]; this is the same value
/// for a parser whose unit begins in the packet it is handed.
pub(crate) fn of(pes: &PesPacket) -> Self {
Self {
pts_ns: pes.pts.or(pes.dts).map(pts_to_ns),
source: pes.source,
discontinuity: pes.discontinuity,
}
}
/// The same facts with the presentation time replaced by one the parser
/// resolved itself — for a packet that carried no timestamp and whose unit
/// continues a base established earlier. The attribution is unchanged:
/// still this packet's bytes, still its source offset.
pub(crate) fn with_pts_ns(self, pts_ns: i64) -> Self {
Self {
pts_ns: Some(pts_ns),
..self
}
}
/// This unit's presentation time in nanoseconds — the ONE derivation.
///
/// PTS and DTS are not two spellings of one value: PTS is when to display,
/// DTS is when to decode, and for a stream that REORDERS (video carrying
/// B-frames) they differ, so reading DTS as a presentation time would be
/// wrong. Reordering is handled by the video path, which reconstructs
/// display order rather than calling this.
///
/// For everything that reaches here — audio and subtitles — there is no
/// reordering, so DTS *is* the presentation time, and falling back to it is
/// reading the same value from whichever field the packet used. A fallback
/// for a missing field, not a second rule: dvdsub read `pts` alone and
/// returned 0 for a packet that carried only DTS.
pub(crate) fn presentation_ns(&self) -> Option<i64> {
self.pts_ns
}
}
/// The facts of the packet covering `off` within a [`PesBuf::marks_snapshot`].
///
/// The same at-or-before rule as [`PesBuf::facts_at`], for a scanner holding a
/// snapshot rather than the buffer.
pub(crate) fn facts_for(marks: &[(usize, PesFacts)], off: usize) -> PesFacts {
let mut found = PesFacts::default();
for &(at, facts) in marks {
if at > off {
break;
}
found = facts;
}
found
}
/// Bytes accumulated across PES packets, each byte attributable to the packet
/// that carried it.
pub(crate) struct PesBuf {
buf: Vec<u8>,
/// `(offset of this PES's first byte within `buf`, its facts)`, ascending.
/// Offsets are relative to the current front and are rebased on `drain`.
marks: std::collections::VecDeque<(usize, PesFacts)>,
}
impl PesBuf {
pub(crate) fn with_capacity(n: usize) -> Self {
Self {
buf: Vec::with_capacity(n),
marks: std::collections::VecDeque::new(),
}
}
/// Append a PES's payload, recording where its bytes begin.
///
/// A PES carrying no payload records nothing: it contributed no byte, so it
/// can never be the answer to "which packet carried the byte at offset N",
/// and admitting a zero-length mark would let it shadow the packet that
/// actually did.
pub(crate) fn push(&mut self, pes: &PesPacket) {
if pes.data.is_empty() {
return;
}
self.marks.push_back((self.buf.len(), PesFacts::of(pes)));
self.buf.extend_from_slice(&pes.data);
}
/// Append a payload under facts the caller resolved — for a parser that
/// carries a timestamp forward across a packet that omitted one. Same
/// attribution rule; only the timestamp differs from what the packet said.
pub(crate) fn push_with(&mut self, data: &[u8], facts: PesFacts) {
if data.is_empty() {
return;
}
self.marks.push_back((self.buf.len(), facts));
self.buf.extend_from_slice(data);
}
/// The facts of the PES that carried the byte at `off`.
///
/// The last mark at or before `off`: bytes belong to the most recent packet
/// that started at or before them. Defaults when the buffer holds bytes
/// that predate any mark (a parser that seeded it directly).
pub(crate) fn facts_at(&self, off: usize) -> PesFacts {
let mut found = PesFacts::default();
for &(at, facts) in &self.marks {
if at > off {
break;
}
found = facts;
}
found
}
/// The facts of the PES that carried the byte at the FRONT of the buffer —
/// the first byte of the access unit a parser is about to emit, which is
/// the whole point of this type.
pub(crate) fn front(&self) -> PesFacts {
self.facts_at(0)
}
pub(crate) fn as_slice(&self) -> &[u8] {
&self.buf
}
pub(crate) fn len(&self) -> usize {
self.buf.len()
}
pub(crate) fn is_empty(&self) -> bool {
self.buf.is_empty()
}
/// Consume `n` bytes from the front, rebasing the marks onto the new front.
///
/// The mark covering the new front byte is RETAINED at offset 0 even though
/// its packet started earlier: those bytes are still that packet's. Dropping
/// it would attribute the remainder of a straddling unit to whichever packet
/// happened to start next — precisely the misattribution this type exists to
/// prevent, and it would land at a clip boundary, the one place it matters.
pub(crate) fn drain(&mut self, n: usize) {
let n = n.min(self.buf.len());
if n == 0 {
return;
}
self.buf.drain(..n);
let covering = self.facts_at(n);
self.marks.retain(|&(at, _)| at > n);
for m in &mut self.marks {
m.0 -= n;
}
if self.marks.front().map(|&(at, _)| at) != Some(0) {
self.marks.push_front((0, covering));
}
}
/// Seed the buffer directly with bytes carrying no packet attribution —
/// for tests that drive a parser's scanner without a demuxer in front of
/// it. Facts for these bytes default to absent, which is what an
/// unattributed byte honestly is.
#[cfg(test)]
pub(crate) fn seed(&mut self, data: &[u8]) {
self.buf.clear();
self.marks.clear();
self.buf.extend_from_slice(data);
}
/// Append unattributed bytes, keeping existing content and marks.
#[cfg(test)]
pub(crate) fn append_unattributed(&mut self, data: &[u8]) {
self.buf.extend_from_slice(data);
}
/// How many packet marks are held — a test hook for the bound on marks.
#[cfg(test)]
pub(crate) fn mark_count(&self) -> usize {
self.marks.len()
}
/// Record a mark at the current end without appending bytes — a test hook
/// for exercising mark bookkeeping directly.
#[cfg(test)]
pub(crate) fn mark_here(&mut self, facts: PesFacts) {
self.marks.push_back((self.buf.len(), facts));
}
/// The marks, for a scanner that must resolve facts at several offsets
/// while the buffer's bytes are borrowed elsewhere. Use with
/// [`facts_for`], which applies the same at-or-before rule as
/// [`PesBuf::facts_at`].
pub(crate) fn marks_snapshot(&self) -> Vec<(usize, PesFacts)> {
self.marks.iter().copied().collect()
}
pub(crate) fn clear(&mut self) {
self.buf.clear();
self.marks.clear();
}
}
#[cfg(test)]
mod tests {
use super::*;
fn pes(data: &[u8], pts: Option<i64>, byte: Option<u64>) -> PesPacket {
PesPacket {
pid: 0x1100,
pts,
dts: None,
data: data.to_vec(),
source: byte.map(SourcePos::at_byte),
discontinuity: false,
}
}
/// The point of the type: a unit assembled from two packets is attributed
/// to the one that carried its FIRST byte, not the one that completed it.
#[test]
fn a_unit_spanning_two_packets_belongs_to_the_packet_it_started_in() {
let mut b = PesBuf::with_capacity(64);
b.push(&pes(&[1, 2, 3], Some(90_000), Some(1_000)));
b.push(&pes(&[4, 5, 6], Some(180_000), Some(2_000)));
let f = b.front();
assert_eq!(f.presentation_ns(), Some(pts_to_ns(90_000)));
assert_eq!(f.source.unwrap().byte, 1_000, "the packet it STARTED in");
}
/// And the timestamp and the source come from the SAME packet — the
/// property that cannot hold when each is derived separately.
#[test]
fn timestamp_and_source_always_come_from_one_packet() {
let mut b = PesBuf::with_capacity(64);
b.push(&pes(&[1, 2], Some(90_000), Some(10)));
b.push(&pes(&[3, 4], Some(180_000), Some(20)));
b.push(&pes(&[5, 6], Some(270_000), Some(30)));
for off in 0..6usize {
let f = b.facts_at(off);
let want = match off {
0 | 1 => (Some(pts_to_ns(90_000)), 10),
2 | 3 => (Some(pts_to_ns(180_000)), 20),
_ => (Some(pts_to_ns(270_000)), 30),
};
assert_eq!(
(f.presentation_ns(), f.source.unwrap().byte),
want,
"offset {off}"
);
}
}
/// Draining a completed unit must leave the REMAINDER attributed to the
/// packet that carried it, not to whichever packet starts next.
#[test]
fn draining_keeps_the_remainder_attributed_to_its_own_packet() {
let mut b = PesBuf::with_capacity(64);
b.push(&pes(&[1, 2, 3, 4], Some(90_000), Some(1_000)));
b.push(&pes(&[5, 6], Some(180_000), Some(2_000)));
// Emit a 2-byte unit: bytes 3 and 4 are still the FIRST packet's.
b.drain(2);
assert_eq!(b.front().source.unwrap().byte, 1_000);
// Emit those two: now the front is genuinely the second packet's.
b.drain(2);
assert_eq!(b.front().source.unwrap().byte, 2_000);
assert_eq!(b.front().presentation_ns(), Some(pts_to_ns(180_000)));
}
/// A payload-less PES contributed no byte, so it must not shadow the packet
/// that did — otherwise the next unit takes a timestamp from a packet whose
/// bytes are not in it.
#[test]
fn an_empty_packet_does_not_claim_the_next_units_bytes() {
let mut b = PesBuf::with_capacity(64);
b.push(&pes(&[1, 2], Some(90_000), Some(1_000)));
b.push(&pes(&[], Some(180_000), Some(2_000)));
assert_eq!(b.front().source.unwrap().byte, 1_000);
assert_eq!(b.len(), 2, "an empty packet adds no bytes");
}
/// Draining everything and refilling must not resurrect a stale mark.
#[test]
fn a_fully_drained_buffer_takes_its_next_packets_facts() {
let mut b = PesBuf::with_capacity(64);
b.push(&pes(&[1, 2], Some(90_000), Some(1_000)));
b.drain(2);
assert!(b.is_empty());
b.push(&pes(&[9], Some(450_000), Some(9_000)));
assert_eq!(b.front().source.unwrap().byte, 9_000);
assert_eq!(b.front().presentation_ns(), Some(pts_to_ns(450_000)));
}
/// The two ways a parser can obtain facts must agree when the unit begins
/// in the packet just handed over — otherwise "one pattern" is two.
#[test]
fn a_unit_starting_in_this_packet_reads_the_same_either_way() {
let p = pes(&[1, 2, 3], Some(90_000), Some(4_242));
let mut b = PesBuf::with_capacity(16);
b.push(&p);
assert_eq!(b.front(), PesFacts::of(&p));
}
/// Over-draining is clamped rather than panicking: a parser that
/// mis-sizes a unit must not take the process down.
#[test]
fn draining_past_the_end_is_clamped() {
let mut b = PesBuf::with_capacity(16);
b.push(&pes(&[1, 2], Some(90_000), Some(1)));
b.drain(99);
assert!(b.is_empty());
}
}
+56 -17
View File
@@ -214,7 +214,12 @@ impl ForcedTracker {
/// Stateful parser that collapses PGS display/clear PCS pairs into
/// duration-bearing Matroska frames. Implements [`CodecParser`].
pub struct PgsParser {
pending: Option<(i64, Vec<u8>)>,
/// The display set being accumulated, with the facts of the PES that
/// STARTED it. A set spans PES packets — it opens on a display PCS and
/// closes on the next one — so its timestamp and its source offset are the
/// opening packet's, never the closing packet's. Same rule the other
/// buffering parsers get from `PesBuf::front`.
pending: Option<(super::pesbuf::PesFacts, Vec<u8>)>,
}
impl Default for PgsParser {
@@ -234,12 +239,13 @@ impl PgsParser {
/// closes or replaces it), clamped to >= 0. Shared by the clear-PCS and
/// replace-PCS arms so the Frame shape stays in one place.
fn emit_pending(&mut self, end_pts_ns: i64) -> Option<Frame> {
let (start_pts, data) = self.pending.take()?;
let (facts, data) = self.pending.take()?;
let start_pts = facts.presentation_ns().unwrap_or(0);
let duration = end_pts_ns.saturating_sub(start_pts).max(0) as u64;
Some(Frame {
discontinuity: false,
coding: None,
source: None,
source: facts.source,
pts_ns: start_pts,
keyframe: true,
data,
@@ -271,12 +277,12 @@ impl CodecParser for PgsParser {
return self
.pending
.take()
.map(|(start_pts, data)| {
.map(|(facts, data)| {
vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: start_pts,
source: facts.source,
pts_ns: facts.presentation_ns().unwrap_or(0),
keyframe: true,
data,
duration_ns: None,
@@ -300,11 +306,11 @@ impl CodecParser for PgsParser {
Some(0) => {
let frame = match pts {
Some(end) => self.emit_pending(end),
None => self.pending.take().map(|(start_pts, data)| Frame {
None => self.pending.take().map(|(facts, data)| Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: start_pts,
source: facts.source,
pts_ns: facts.presentation_ns().unwrap_or(0),
keyframe: true,
data,
duration_ns: None,
@@ -318,17 +324,24 @@ impl CodecParser for PgsParser {
Some(_) => match pts {
Some(start) => {
out.extend(self.emit_pending(start));
self.pending = Some((start, pes.data.clone()));
// The set's facts are THIS packet's — the one that opened
// it. `start` is that packet's PTS by construction.
self.pending = Some((super::pesbuf::PesFacts::of(pes), pes.data.clone()));
debug_assert_eq!(
super::pesbuf::PesFacts::of(pes).presentation_ns(),
Some(start),
"the opening packet's PTS is the set's start"
);
}
// A display PCS with no PTS has an unknown start time. Don't
// store it with a 0 sentinel (wrong start, absurd duration).
// Flush any prior pending undurated and skip storing this one.
None => {
out.extend(self.pending.take().map(|(start_pts, data)| Frame {
out.extend(self.pending.take().map(|(facts, data)| Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: start_pts,
source: facts.source,
pts_ns: facts.presentation_ns().unwrap_or(0),
keyframe: true,
data,
duration_ns: None,
@@ -353,7 +366,10 @@ impl CodecParser for PgsParser {
out.push(Frame {
discontinuity: false,
coding: None,
source: None,
// Emitted straight from THIS packet, so its facts are
// this packet's -- the same rule as a pending set,
// which takes the facts of the packet that opened it.
source: super::pesbuf::PesFacts::of(pes).source,
pts_ns: pts.unwrap_or(0),
keyframe: true,
data: pes.data.clone(),
@@ -379,11 +395,11 @@ impl CodecParser for PgsParser {
// until end of file, which is exactly the desired behavior for
// the final on-screen subtitle (see the module doc).
match self.pending.take() {
Some((start_pts, data)) => vec![Frame {
Some((facts, data)) => vec![Frame {
discontinuity: false,
coding: None,
source: None,
pts_ns: start_pts,
source: facts.source,
pts_ns: facts.presentation_ns().unwrap_or(0),
keyframe: true,
data,
duration_ns: None,
@@ -901,4 +917,27 @@ mod tests {
assert_eq!(t.facts().displays, u32::MAX);
assert_eq!(t.facts().forced_displays, u32::MAX);
}
/// A lone non-PCS segment with a PTS is emitted straight through rather
/// than accumulated, and it must still carry provenance. This path was
/// missed on the first pass and showed up on a real disc as a subtitle
/// track with no source offset -- the one track out of forty that could
/// not be placed by byte.
#[test]
fn a_lone_segment_emitted_directly_still_carries_provenance() {
let mut parser = PgsParser::new();
// A non-PCS segment (type 0x15 = ODS) with a PTS and no pending set.
let mut p = make_pes(vec![0x15, 0x00, 0x00, 0x00, 0x04, 1, 2, 3, 4], Some(90_000));
p.source = Some(crate::pes::SourcePos::at_byte(7_777));
let frames = parser.parse(&p);
assert!(
!frames.is_empty(),
"a lone segment with a PTS is passed through"
);
assert_eq!(
frames[0].source.map(|s| s.byte),
Some(7_777),
"emitted straight from this packet, so it carries this packet's offset"
);
}
}
+65 -34
View File
@@ -63,7 +63,10 @@ const AU_DURATION_NS_441: i64 = 907_029;
const MAX_TRUEHD_BUF: usize = 256 * 1024;
pub struct TrueHdParser {
buf: Vec<u8>,
/// Bytes assembled across PES packets, each attributable to the packet
/// that carried it, so an access unit takes the timestamp AND the source
/// offset of the packet covering its first byte.
acc: super::pesbuf::PesBuf,
next_pts_ns: i64,
/// Per-AU PTS increment. Defaults to the 48 kHz-family value (833_333) and
/// is refined to the 44.1 kHz-family value once the first major sync reveals
@@ -94,7 +97,7 @@ impl Default for TrueHdParser {
impl TrueHdParser {
pub fn new() -> Self {
Self {
buf: Vec::with_capacity(32768),
acc: super::pesbuf::PesBuf::with_capacity(32768),
next_pts_ns: 0,
au_duration_ns: AU_DURATION_NS,
tally: DropTally::new("truehd"),
@@ -195,15 +198,15 @@ impl TrueHdParser {
/// single source of truth shared with the AC-3 parser; a returned `0` there
/// (reserved fscod or out-of-range frmsizecod) is the unmappable case.
fn ac3_frame_at_head(&self) -> Ac3Size {
if self.buf.len() < 6 {
if self.acc.len() < 6 {
return Ac3Size::NeedMore;
}
let frame_bytes = super::ac3::ac3_frame_size(&self.buf);
let frame_bytes = super::ac3::ac3_frame_size(self.acc.as_slice());
if frame_bytes == 0 {
// Reserved fscod or out-of-range frmsizecod → unmappable header.
return Ac3Size::Unmappable;
}
if self.buf.len() < frame_bytes {
if self.acc.len() < frame_bytes {
return Ac3Size::NeedMore;
}
Ac3Size::Frame(frame_bytes)
@@ -363,7 +366,7 @@ impl CodecParser for TrueHdParser {
// never be stranded by an empty post-gap PES (defensive; the demuxer only
// emits non-empty PES today).
if pes.discontinuity {
self.buf.clear();
self.acc.clear();
}
if pes.data.is_empty() {
return Vec::new();
@@ -376,7 +379,7 @@ impl CodecParser for TrueHdParser {
// mid-AU would snap that AU's PTS backward/forward and break the
// monotonic +AU_DURATION_NS cadence (A/V drift). Once the buffer is empty
// the next PES legitimately begins a new AU and seeds the base.
if self.buf.is_empty()
if self.acc.is_empty()
&& let Some(pts) = pes.pts
{
// Resync to the authoritative PES PTS. TrueHD AUs are a fixed
@@ -421,12 +424,12 @@ impl CodecParser for TrueHdParser {
}
}
self.buf.extend_from_slice(&pes.data);
self.acc.push(pes);
let mut frames = Vec::new();
loop {
if self.buf.len() < 4 {
if self.acc.len() < 4 {
break;
}
@@ -438,19 +441,19 @@ impl CodecParser for TrueHdParser {
// when its computed end is corroborated by what follows: end of
// buffer (frame fills the rest), another AC-3 sync, or a plausible
// TrueHD AU header. If none holds, this is treated as a TrueHD AU.
if self.buf[0] == 0x0B && self.buf[1] == 0x77 {
if self.acc.as_slice()[0] == 0x0B && self.acc.as_slice()[1] == 0x77 {
match self.ac3_frame_at_head() {
Ac3Size::Unmappable => {
// Permanently unmappable header at the head would stall
// the parser forever; resync by dropping 2 bytes so one
// bad frame costs one frame, not the whole buffer.
self.buf.drain(..2);
self.acc.drain(2);
continue;
}
Ac3Size::NeedMore => break, // wait for the rest of the frame
Ac3Size::Frame(skip) => {
if ac3_boundary_corroborated(&self.buf, skip) {
self.buf.drain(..skip);
if ac3_boundary_corroborated(self.acc.as_slice(), skip) {
self.acc.drain(skip);
continue;
}
// Not corroborated — fall through and interpret the
@@ -460,19 +463,21 @@ impl CodecParser for TrueHdParser {
}
// TrueHD access unit: lower 12 bits of first 2 bytes = length in words
let unit_words = (((self.buf[0] as usize) << 8) | self.buf[1] as usize) & 0xFFF;
let unit_words = (((self.acc.as_slice()[0] as usize) << 8)
| self.acc.as_slice()[1] as usize)
& 0xFFF;
if unit_words == 0 {
// A zero-length AU is malformed/padding. The AU header is 4 bytes
// (length + timing); drain the whole header, not just the length
// word, otherwise the timing bytes get misread as the next
// length word and produce a spurious parse on the next iteration.
self.buf.drain(..4);
self.acc.drain(4);
continue;
}
// unit_words is masked to 12 bits, so unit_bytes <= 4095 * 2 = 8190;
// no separate oversize-resync guard is reachable.
let unit_bytes = unit_words * 2;
if self.buf.len() < unit_bytes {
if self.acc.len() < unit_bytes {
break; // incomplete access unit, wait for more data
}
@@ -482,10 +487,10 @@ impl CodecParser for TrueHdParser {
// gated on 0xBA alone in `au_check`.
let is_major_sync = unit_bytes >= 8
&& is_mlp_major_sync(u32::from_be_bytes([
self.buf[4],
self.buf[5],
self.buf[6],
self.buf[7],
self.acc.as_slice()[4],
self.acc.as_slice()[5],
self.acc.as_slice()[6],
self.acc.as_slice()[7],
]));
// Decodability gate. MLP/TrueHD decode state persists across access
@@ -493,8 +498,11 @@ impl CodecParser for TrueHdParser {
// major sync (the clean re-init point) rather than excised in place.
// The PTS clock advances across every dropped AU so a drop is a
// silence gap, never a shift.
let au = self.buf[..unit_bytes].to_vec();
let au = self.acc.as_slice()[..unit_bytes].to_vec();
let pts = self.next_pts_ns;
// Read BEFORE the drain below: this unit's source is the packet
// covering the CURRENT front, not the next unit's.
let au_src = self.acc.front().source;
let mut emit_keyframe: Option<bool> = None; // Some(is_keyframe) => emit
let mut drop_reason: Option<(&'static str, bool)> = None; // (reason, verified)
@@ -560,7 +568,7 @@ impl CodecParser for TrueHdParser {
frames.push(Frame {
discontinuity: false,
coding: None,
source: None,
source: au_src,
pts_ns: pts,
keyframe,
data: au,
@@ -575,14 +583,14 @@ impl CodecParser for TrueHdParser {
.record_collateral_drop(pts, self.au_duration_ns, au.len(), reason);
}
}
self.buf.drain(..unit_bytes);
self.acc.drain(unit_bytes);
self.next_pts_ns += self.au_duration_ns;
}
// Bound memory on malformed input: a stream that never yields a
// complete frame must not grow the buffer without limit.
if self.buf.len() > MAX_TRUEHD_BUF {
self.buf.clear();
if self.acc.len() > MAX_TRUEHD_BUF {
self.acc.clear();
}
frames
@@ -1455,7 +1463,7 @@ mod tests {
"TrueHD AU behind a bad header is recovered"
);
assert_eq!(frames[0].data.len(), 200);
assert!(parser.buf.is_empty(), "buffer fully consumed, no stall");
assert!(parser.acc.is_empty(), "buffer fully consumed, no stall");
}
#[test]
@@ -1707,7 +1715,7 @@ mod tests {
f.is_empty(),
"must not emit fewer bytes than the length field"
);
assert_eq!(parser.buf.len(), 100, "partial AU retained");
assert_eq!(parser.acc.len(), 100, "partial AU retained");
}
/// Largest AU the 12-bit length field can declare: 0xFFF words × 2.
@@ -1720,7 +1728,7 @@ mod tests {
//
// The bound that actually holds is MAX_AU_BYTES, not MAX_TRUEHD_BUF:
// `parse`'s loop only breaks with data retained when
// `self.buf.len() < unit_bytes`, and `unit_bytes` is
// `self.acc.len() < unit_bytes`, and `unit_bytes` is
// `((buf[0] << 8 | buf[1]) & 0xFFF) * 2 <= 8190`. Every other exit
// drains. So the post-loop `buf.len() > MAX_TRUEHD_BUF` cap (256 KiB) is
// an unreachable backstop — an exhaustive sweep of all 65536 two-byte
@@ -1740,17 +1748,17 @@ mod tests {
frag[0] = 0xFF;
frag[1] = 0xFF;
let _ = parser.parse(&make_pes(frag, Some(0)));
worst = worst.max(parser.buf.len());
worst = worst.max(parser.acc.len());
assert!(
parser.buf.len() < MAX_AU_BYTES,
parser.acc.len() < MAX_AU_BYTES,
"reassembly buffer exceeded the AU-length ceiling: {} >= {}",
parser.buf.len(),
parser.acc.len(),
MAX_AU_BYTES
);
assert!(
parser.buf.len() <= MAX_TRUEHD_BUF,
parser.acc.len() <= MAX_TRUEHD_BUF,
"reassembly buffer exceeded cap: {} > {}",
parser.buf.len(),
parser.acc.len(),
MAX_TRUEHD_BUF
);
}
@@ -1813,7 +1821,7 @@ mod tests {
fn ac3_frame_at_head_needs_more_when_buffer_short() {
// < 6 bytes buffered → NeedMore (can't read the AC-3 header).
let mut parser = TrueHdParser::new();
parser.buf = vec![0x0B, 0x77, 0x00];
parser.acc.seed(&[0x0B, 0x77, 0x00]);
// Drive through parse: a short 0x0B77 head must wait, not emit.
let f = parser.parse(&make_pes(vec![0x0B, 0x77, 0x00], Some(0)));
assert!(f.is_empty());
@@ -2014,4 +2022,27 @@ mod tests {
assert_eq!(truehd_sample_rate_hz(info.format_info), Some(96000));
assert_eq!(info.is_atmos, Some(true));
}
/// Same rule for TrueHD: a unit spanning two packets belongs to the one
/// that carried its first byte.
#[test]
fn an_access_unit_carries_the_source_of_the_packet_it_began_in() {
let mut parser = TrueHdParser::new();
let unit = make_truehd_unit(512);
let mut p1 = make_pes(unit[..200].to_vec(), Some(90_000));
p1.source = Some(crate::pes::SourcePos::at_byte(1_000));
let first = parser.parse(&p1);
assert!(first.is_empty(), "partial unit held");
let mut p2 = make_pes(unit[200..].to_vec(), Some(180_000));
p2.source = Some(crate::pes::SourcePos::at_byte(9_000));
let frames = parser.parse(&p2);
assert!(!frames.is_empty(), "the completed unit is emitted");
assert_eq!(
frames[0].source.map(|s| s.byte),
Some(1_000),
"the unit belongs to the packet its FIRST byte came from"
);
}
}
+144 -8
View File
@@ -190,7 +190,13 @@ pub(crate) fn codec_label(codec: Codec) -> &'static str {
/// to `.sup`, VobSub records `.idx` entries and emits the sidecar at finish.
trait EsWriter: Send {
/// Write one frame's payload to `w`. Returns the number of bytes written to
/// the main file (used by the VobSub writer for `.idx` filepos tracking).
/// the main file.
///
/// Nothing in production reads this count: the sole caller discards it with
/// `?`, and `VobSubWriter` tracks `.idx` fileposes from its own `pos` field.
/// This previously claimed the VobSub writer depended on the return value,
/// which would have a maintainer believe a wrong count shows up as broken
/// `.idx` output. It does not.
fn write_frame(&mut self, w: &mut dyn Write, f: &PesFrame, pts_ns: i64) -> io::Result<usize>;
/// Finalize. Default: no-op. The VobSub writer serializes its `.idx` here.
@@ -660,6 +666,12 @@ pub struct DemuxSink {
/// Index = track id; `None` for unselected tracks.
tracks: Vec<Option<TrackOut>>,
ref_video_track: Option<usize>,
/// Every VIDEO track index, recorded before the kind filter.
///
/// The filtered `tracks` slots are `None` for a class this export drops, so
/// they cannot answer "is this video" for a frame that still flows through
/// `write()`.
video_tracks: std::collections::HashSet<usize>,
/// First PTS observed on `ref_video_track`, recorded in `write()` REGARDLESS
/// of whether that track has a `TrackOut`. The DELAY reference cannot live in
/// `TrackOut::first_pts_ns`: `audio://` / `sub://` filter the video track's
@@ -671,6 +683,15 @@ pub struct DemuxSink {
ref_first_pts_ns: Option<i64>,
timeline: TimelineContinuity,
finished: bool,
/// Frames the timeline PLACED (not necessarily persisted).
///
/// A frame for a track this export filters out still flows through
/// `write()` and still counts here, so this is "the timeline placed
/// something" rather than "a file received bytes". That is the right
/// denominator for the drop gate below — it asks whether placement worked
/// at all — but it is deliberately NOT a per-track written count, so a
/// single lost track among many is not visible to it.
frames_mapped: u64,
}
impl DemuxSink {
@@ -679,6 +700,7 @@ impl DemuxSink {
std::fs::create_dir_all(dir)?;
let mut tracks: Vec<Option<TrackOut>> = Vec::with_capacity(title.streams.len());
let mut ref_video_track = None;
let mut video_tracks: std::collections::HashSet<usize> = std::collections::HashSet::new();
for (idx, stream) in title.streams.iter().enumerate() {
let selected = opts
@@ -705,6 +727,15 @@ impl DemuxSink {
if kind == TrackKind::Video && ref_video_track.is_none() {
ref_video_track = Some(idx);
}
// Also BEFORE the kind filter, and for the same reason: the seam
// crossing rule differs for a track that carries B-frame reorder,
// and a video track still flows through write() on an `audio://`
// or `sub://` export even though nothing persists it. Reading the
// kind back out of the filtered slot would call it non-video and
// hand it the permissive rule.
if kind == TrackKind::Video {
video_tracks.insert(idx);
}
// Kind filter: `audio://` / `sub://` keep only their class.
if opts.kind_filter.is_some_and(|k| k != kind) {
tracks.push(None);
@@ -741,9 +772,11 @@ impl DemuxSink {
opts: opts.clone(),
tracks,
ref_video_track,
video_tracks,
ref_first_pts_ns: None,
timeline: TimelineContinuity::new(),
timeline: TimelineContinuity::with_clips(&title.clips, title.content_format),
finished: false,
frames_mapped: 0,
})
}
@@ -866,7 +899,23 @@ impl Stream for DemuxSink {
// an audio ES before the video ES, in which case track 0 is audio and a
// non-video epoch driver would ratchet the frontier on sparse/lagging PTS.
let drives = Some(frame.track) == self.ref_video_track;
let pts = self.timeline.adjust(frame.pts, drives);
// Keyed on the track KIND, not on driving epochs: a second video track
// (a Dolby Vision enhancement layer) does not drive epochs but does
// carry B-frame reorder, and the seam-crossing rule differs on exactly
// that property.
let is_video = self.video_tracks.contains(&frame.track);
// See `MkvMuxer::write_frame`: `None` is material outside the
// playlist's clip marks and is dropped rather than emitted.
let Some(pts) = self.timeline.map(
frame.pts,
drives,
frame.track,
is_video,
frame.source.map(|s| s.byte),
) else {
return Ok(());
};
self.frames_mapped = self.frames_mapped.saturating_add(1);
if drives {
// Delay reference: recorded here, not in the track's `TrackOut`, so
// it survives the `audio://` / `sub://` kind filter dropping the
@@ -885,6 +934,34 @@ impl Stream for DemuxSink {
return Ok(());
}
self.finished = true;
// Same reporting as the MKV muxer's finish. Frames the playlist's clip
// marks exclude are dropped on purpose, but the count must not be
// write-only in one sink and reported in the other — an unexpected
// volume here is how a demux ends up quietly short.
let seam_dropped = self.timeline.dropped_total();
// Frames ARRIVED and every one was dropped. That is a fully-dropped
// title, which this sink used to finish cleanly: a directory of
// zero-byte track files beside a populated chapters document, at exit
// 0. Keyed on frames having been offered, because a sink that is never
// given any — a chapters-only export, or a track class the title does
// not carry — legitimately writes none.
if self.frames_mapped == 0 && seam_dropped > 0 {
return Err(crate::error::Error::SinkWroteNothing.into());
}
if seam_dropped > self.frames_mapped {
return Err(crate::error::Error::SeamPlanDroppedMost {
dropped: seam_dropped,
written: self.frames_mapped,
}
.into());
}
if seam_dropped > 0 {
tracing::info!(
target: "mux",
dropped = seam_dropped,
"frames outside the playlist's clip marks were dropped at clip joins"
);
}
// Flush each track's codec writer, then the buffered file.
for slot in self.tracks.iter_mut() {
if let Some(t) = slot.as_mut() {
@@ -1466,14 +1543,14 @@ mod tests {
fn timeline_track0_drives_epoch_others_ride() {
let mut tl = TimelineContinuity::new();
// Clip 1: video 0..10s (track 0 drives the epoch).
assert_eq!(tl.adjust(0, true), 0);
assert_eq!(tl.adjust(0, false), 0); // audio rides the same offset
assert_eq!(tl.adjust(10_000_000_000, true), 10_000_000_000);
assert_eq!(tl.adjust(0, true, 0), 0);
assert_eq!(tl.adjust(0, false, 1), 0); // audio rides the same offset
assert_eq!(tl.adjust(10_000_000_000, true, 0), 10_000_000_000);
// Clip 2 seam: video PTS jumps back to ~0 (> 3s back) → new epoch.
let out = tl.adjust(0, true);
let out = tl.adjust(0, true, 0);
assert!(out >= 10_000_000_000, "epoch must advance past prev high");
// Audio in clip 2 (non-epoch) gets the SAME offset (A/V sync preserved).
let a = tl.adjust(0, false);
let a = tl.adjust(0, false, 1);
assert_eq!(a, out);
}
@@ -1609,6 +1686,65 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
/// A demux export whose seam plan drops every frame must FAIL, not write a
/// directory of empty track files and report success.
///
/// Audit finding: this sink builds its timeline with
/// `TimelineContinuity::with_clips` but no test ever gave it a title with
/// usable PlayItem marks, so `frames_mapped` and `dropped_total()` were
/// always zero and BOTH gates in `finish` were unreachable. Deleting either
/// left the whole suite green while a `demux://` export of a
/// seamless-branching title wrote zero-byte track files beside a populated
/// chapters document, at exit 0.
#[test]
fn a_demux_export_the_seam_plan_emptied_fails() {
let dir = tempdir();
let mut title = title_with(vec![video_stream(Codec::H264)], vec![None]);
// Two clips (SeamPlan needs >= 2 with strictly increasing IN marks)
// covering 100s..200s and 200s..300s in 45 kHz ticks.
title.clips = vec![
crate::disc::Clip {
feed_span: None,
clip_id: "00000".into(),
in_time: 100 * 45_000,
out_time: 200 * 45_000,
duration_secs: 100.0,
source_packets: 0,
},
crate::disc::Clip {
feed_span: None,
clip_id: "00001".into(),
in_time: 200 * 45_000,
out_time: 300 * 45_000,
duration_secs: 100.0,
source_packets: 0,
},
];
let mut sink = DemuxSink::create(&dir, &title, &DemuxOptions::default()).unwrap();
// Frames far before the first IN mark: the plan places none of them.
for (i, pts) in [0i64, 1_000_000_000].iter().enumerate() {
let f = PesFrame {
coding: None,
source: None,
track: 0,
pts: *pts,
keyframe: true,
data: vec![0x00, 0x00, 0x00, 0x01, 0x09, 0x10, i as u8],
duration_ns: None,
};
let _ = Stream::write(&mut sink, &f);
}
let err = Stream::finish(&mut sink).expect_err("an emptied export must fail");
assert_eq!(
crate::error::error_code(&err),
Some(crate::error::E_SINK_WROTE_NOTHING),
"a fully-dropped demux export must report SinkWroteNothing, not success"
);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn sink_read_returns_write_only() {
let dir = tempdir();
+176 -7
View File
@@ -680,8 +680,9 @@ pub struct MkvMuxer<W: Write + Seek> {
cues: Vec<CuePoint>,
frame_count: u64,
/// Frames handed to `write_frame` that were dropped because no cluster was
/// open yet (a cluster only opens on a track-0 video keyframe). See
/// `write_frame` for the track-0 invariant.
/// open yet (a cluster only opens on a keyframe from the primary video
/// track, whatever index that is — not necessarily track 0). See
/// `write_frame` for the cluster-driver invariant.
///
/// The ALL-dropped case is surfaced as an error by `finish()`, but via
/// `frame_count == 0`, not via this counter. A PARTIAL drop — leading audio /
@@ -1323,6 +1324,25 @@ impl<W: Write + Seek> MkvMuxer<W> {
pub fn set_opening_capture(&mut self, capture: Option<crate::diag::OpeningCapture>) {
self.opening_capture = capture;
}
/// Drive seam correction from the title's PlayItem marks instead of
/// inferring it from PTS jumps.
///
/// A multi-clip Blu-ray playlist joins its clips with overlaps and skips
/// that PTS inspection cannot recover: a forward jump is indistinguishable
/// from frames lost to damaged media, and an overlap smaller than the
/// reorder threshold is invisible. Given the marks, each clip is placed at
/// the sum of the earlier clips' durations, so the output runs exactly as
/// long as the playlist says the title is.
///
/// No-op for a title with fewer than two clips or without usable marks —
/// DVD, HD-DVD and file sources keep the inference path.
pub fn set_clips(
&mut self,
clips: &[crate::disc::Clip],
content_format: crate::disc::ContentFormat,
) {
self.continuity = TimelineContinuity::with_clips(clips, content_format);
}
/// Write a single frame.
///
@@ -1348,6 +1368,14 @@ impl<W: Write + Seek> MkvMuxer<W> {
/// Block and the dependent (right-eye) access unit rides as the
/// BlockAdditional under the track's `mvcC` mapping. Such a frame is always a
/// `BlockGroup` (never a SimpleBlock), with a `ReferenceBlock` when it is not
/// TEST-ONLY. Every production path writes through [`MkvMuxer::write_frame_at`]
/// and passes the byte offset the frame was read from; the deferred-mux
/// replay of buffered frames was the last one that did not, and it now
/// does. Keeping this compiled out of the library means there is no
/// production API through which a frame can reach the file with its
/// provenance discarded — the placement logic stopped relying on
/// timestamps precisely so that could not happen silently.
#[cfg(test)]
/// a keyframe. `None` for every non-3D frame.
pub fn write_frame(
&mut self,
@@ -1357,6 +1385,36 @@ impl<W: Write + Seek> MkvMuxer<W> {
data: &[u8],
duration_ns: Option<u64>,
block_additional: Option<&[u8]>,
) -> io::Result<()> {
self.write_frame_at(
track_idx,
pts_ns,
keyframe,
data,
duration_ns,
block_additional,
None,
)
}
/// [`Self::write_frame`] with the frame's SOURCE BYTE OFFSET.
///
/// Under a seam plan that offset identifies which clip the frame came from
/// by lookup rather than by inferring it from the timestamp — which is
/// ambiguous inside an overlap, where two clips' mark ranges both contain
/// the same instant. Callers that have a `PesFrame` should pass
/// `frame.source.map(|s| s.byte)`; `write_frame` is the same call with no
/// provenance, which falls back to the mark heuristics.
#[allow(clippy::too_many_arguments)]
pub fn write_frame_at(
&mut self,
track_idx: usize,
pts_ns: i64,
keyframe: bool,
data: &[u8],
duration_ns: Option<u64>,
block_additional: Option<&[u8]>,
src_byte: Option<u64>,
) -> io::Result<()> {
// --log-level 3: capture the first ~100 coded frames per track to the
// side file BEFORE any timeline mangling, with the codec parser's own
@@ -1391,7 +1449,19 @@ impl<W: Write + Seek> MkvMuxer<W> {
// band of non-monotonic block timestamps. Only the PRIMARY video track
// drives the boundary decision; every other track (audio, subtitle, DV
// EL) rides the current offset. No-op for single-clip titles.
let pts_ns = self.continuity.adjust(pts_ns, drives_epoch);
// `None` means the playlist does not include this frame — material
// outside every clip's IN/OUT marks, which only a seam-plan-driven
// title can report. Dropping it is the point: emitting it is what put
// duplicate content on the timeline at a join.
let Some(pts_ns) = self.continuity.map(
pts_ns,
drives_epoch,
track_idx,
self.track_is_video.get(track_idx).copied().unwrap_or(false),
src_byte,
) else {
return Ok(());
};
let raw_ticks = pts_ns / TIMESTAMP_SCALE_NS;
// Cluster boundaries normally coincide with a video keyframe so every
@@ -1653,11 +1723,17 @@ impl<W: Write + Seek> MkvMuxer<W> {
/// Finish the MKV file: write Cues element.
///
/// # Track-0 invariant
/// # Cluster-driver invariant
///
/// A cluster only opens on a track-0 video keyframe, so the caller must
/// supply track 0 as the video track and deliver a keyframe on it before
/// (or alongside) other-track data. If no track-0 keyframe ever arrives,
/// A cluster only opens on a keyframe from the PRIMARY VIDEO TRACK — the
/// first track whose type is video, at whatever index it occupies (see
/// `cluster_driver`, which is `primary_video_track.unwrap_or(0)`; index 0
/// is only the fallback for a file with no video track at all). The caller
/// must deliver a keyframe on that track before (or alongside) other-track
/// data. This said "track-0" and required the caller to place video at
/// index 0, which the code has never actually required — so a reader
/// debugging dropped frames would suspect track ordering, which is not it.
/// If no such keyframe ever arrives,
/// every `write_frame` is silently dropped; rather than emit a structurally
/// valid but empty MKV (zero clusters, zero frames), `finish` returns
/// `Error::MkvInvalid` when frames were submitted but none were written.
@@ -1667,6 +1743,15 @@ impl<W: Write + Seek> MkvMuxer<W> {
// would otherwise yield a structurally-empty MKV with no clusters or
// cues. Surface that as an error rather than writing valid-but-empty
// output.
// Order matters. A title the seam plan dropped ENTIRELY also has a zero
// frame count, and `MkvInvalid` is classified by
// `is_skippable_title_stub` as an empty nav/menu stub — so reporting it
// that way would make an all-titles rip drop a real feature and finish
// the rest at exit 0. Decide the seam case FIRST, with a code that is
// not skippable.
if self.frame_count == 0 && self.continuity.dropped_total() > 0 {
return Err(crate::error::Error::SinkWroteNothing.into());
}
if self.frame_count == 0 {
return Err(crate::error::Error::MkvInvalid.into());
}
@@ -1682,6 +1767,36 @@ impl<W: Write + Seek> MkvMuxer<W> {
"frames were discarded before the first cluster opened (no track-0 video keyframe had arrived yet); they are absent from the output"
);
}
// Frames the playlist's clip marks excluded are dropped on purpose — a
// join legitimately discards the material a disc stores twice — but the
// count must not be write-only. Same reasoning as the pre-cluster
// counter above: an unexpected VOLUME here is how a title ends up
// quietly short while the run reports success.
let seam_dropped = self.continuity.dropped_total();
// Counting a drop is not the same as bounding it. A join legitimately
// discards the material a disc stores twice — tens of frames — but if a
// title's marks do not line up with its PES clock the plan can discard
// most of it, and the only other gate is a GLOBAL zero-frame check
// whose error is additionally classified as a skippable stub. Between
// them, a title emitting seconds of a two-hour feature exits 0. That is
// the defect this change set already shipped once.
//
// More dropped than kept is never a real join, so it fails.
if seam_dropped > self.frame_count {
return Err(crate::error::Error::SeamPlanDroppedMost {
dropped: seam_dropped,
written: self.frame_count,
}
.into());
}
if seam_dropped > 0 {
tracing::info!(
target: "mux",
dropped = seam_dropped,
frames_written = self.frame_count,
"frames outside the playlist's clip marks were dropped at clip joins"
);
}
// The source declared no duration up-front (DURATION was reserved as a
// placeholder). Derive the real runtime from the muxed timeline so the
// Segment declares it — and so the BPS tags below can be computed.
@@ -4376,6 +4491,60 @@ mod tests {
assert_eq!(err.kind(), io::ErrorKind::InvalidData);
}
/// A seam-plan clip list whose marks exclude every frame must fail with
/// `SinkWroteNothing`, NOT `MkvInvalid`.
///
/// Audit finding: both gates in `finish()` were dead code under test —
/// `set_clips` was never called anywhere in the suite, so
/// `continuity.dropped_total()` was always 0 and neither branch could be
/// reached. Swapping the two `frame_count == 0` checks left the whole suite
/// green, while a title the seam plan emptied came back as `MkvInvalid` —
/// which `is_skippable_title_stub` classifies as an empty nav/menu stub, so
/// an all-titles rip would silently omit a real feature and exit 0.
#[test]
fn a_title_the_seam_plan_emptied_is_not_reported_as_a_skippable_stub() {
let tracks = [make_video_track()];
// TWO clips (SeamPlan::from_clips requires at least two, with strictly
// increasing IN marks), covering 100s..200s and 200s..300s in 45 kHz
// ticks. Every frame below sits far before the first IN, so the plan
// places none of them and drops them all.
let clips = [
crate::disc::Clip {
feed_span: None,
clip_id: "00000".into(),
in_time: 100 * 45_000,
out_time: 200 * 45_000,
duration_secs: 100.0,
source_packets: 0,
},
crate::disc::Clip {
feed_span: None,
clip_id: "00001".into(),
in_time: 200 * 45_000,
out_time: 300 * 45_000,
duration_secs: 100.0,
source_packets: 0,
},
];
let mut muxer = MkvMuxer::new(Cursor::new(Vec::new()), &tracks, None, 100.0, &[]).unwrap();
muxer.set_clips(&clips, crate::disc::ContentFormat::BdTs);
// 0s and 1s — both before the clip's IN mark.
let _ = muxer.write_frame(0, 0, true, &[0x01; 16], None, None);
let _ = muxer.write_frame(0, 1_000_000_000, true, &[0x02; 16], None, None);
let err = muxer.finish().unwrap_err();
assert_eq!(
crate::error::error_code(&err),
Some(crate::error::E_SINK_WROTE_NOTHING),
"an emptied seam title must report SinkWroteNothing"
);
assert!(
!crate::error::is_skippable_title_stub(&err),
"it must NOT be classified as a skippable stub — that is what makes an \
all-titles rip drop a real feature and still exit 0"
);
}
#[test]
fn backjumped_audio_handled_by_i16_split_no_wrap() {
// A NON-VIDEO (audio) frame whose PTS back-jumps far below the open
+14 -2
View File
@@ -302,13 +302,15 @@ fn emit_to_muxer(
frame: &crate::pes::PesFrame,
additional: Option<&[u8]>,
) -> io::Result<()> {
m.write_frame(
m.write_frame_at(
frame.track,
frame.pts,
frame.keyframe,
&frame.data,
frame.duration_ns,
additional,
// Provenance: which clip this frame came from is a lookup, not a guess.
frame.source.map(|s| s.byte),
)
}
@@ -504,17 +506,27 @@ impl MkvStream {
self.disc_title.duration_secs,
&self.disc_title.chapters,
)?;
// Seam correction from the playlist's marks where the title has them.
muxer.set_clips(&self.disc_title.clips, self.disc_title.content_format);
if let Some(path) = &pending.opening_capture_path {
muxer.set_opening_capture(crate::diag::OpeningCapture::new(path, pending.tracks.len()));
}
for (f, additional) in pending.buffered.drain(..) {
muxer.write_frame(
// Provenance must survive the replay. These frames were buffered
// before the muxer existed, but each still carries the byte offset
// it was read from — dropping it here sent them down the timestamp
// heuristic instead, the one path the rest of this change set
// exists to stop relying on. They sit at the head of the title, so
// clip 0 is usually the right guess and the damage was bounded;
// "usually right" is exactly what provenance replaced.
muxer.write_frame_at(
f.track,
f.pts,
f.keyframe,
&f.data,
f.duration_ns,
additional.as_deref(),
f.source.map(|s| s.byte),
)?;
}
self.mode = Mode::Write(WriteMode::Active(Box::new(muxer)));
+289 -182
View File
@@ -55,9 +55,15 @@ pub enum StreamUrl {
Stdio,
/// ISO disc image file.
Iso { path: PathBuf },
/// Decrypted file-tree output directory (`dir://`). A sink that writes
/// per-file decrypted bytes (not muxed PES frames), so it never flows
/// through `output()`; the CLI routes a `Dir` dest to `Disc::extract_tree`.
/// An extracted disc file tree (`dir://`) — a source AND a sink.
///
/// As a SINK it writes per-file decrypted bytes rather than muxed PES
/// frames, so it never flows through `output()`; the CLI routes a `Dir`
/// dest to `Disc::extract_tree`.
///
/// As a SOURCE (1.6.1) it is an image-level source: `crate::dirimage`
/// synthesizes a real UDF volume over the folder, so it reaches the same
/// scan/mux path `iso://` does and every destination follows.
Dir { path: PathBuf },
/// Null sink (write-only, discards data).
Null,
@@ -142,9 +148,22 @@ impl StreamUrl {
}
}
/// Whether this URL represents a disc source (disc:// or iso://).
/// Whether this URL is an IMAGE-level source — one that carries a UDF
/// filesystem, so it can be scanned into a title list, have `-t`/`-a`/`-s`
/// applied, and feed either a PES sink or an image sink.
///
/// `dir://` joined in 1.6.1, when `crate::dirimage` gave a folder a real
/// synthesized UDF volume.
///
/// NOT the same predicate as the CLI's `engine::is_disc_source`, which
/// means "is a live drive" and drives tray/eject behaviour. That one must
/// never gain `Dir` — a directory routed down the live-drive rip path
/// would open, lock and eject a drive that has nothing to do with it.
pub fn is_disc_source(&self) -> bool {
matches!(self, StreamUrl::Disc { .. } | StreamUrl::Iso { .. })
matches!(
self,
StreamUrl::Disc { .. } | StreamUrl::Iso { .. } | StreamUrl::Dir { .. }
)
}
}
@@ -371,170 +390,32 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
// (so the kernel readahead window widens) and the periodic
// DONTNEED page-cache eviction that bounds memory pressure
// when the mux output is being written to the same disk.
let mut reader = crate::io::file_sector_source::FileSectorSource::open(path)?;
let capacity = reader.capacity_sectors();
let mut disc = crate::disc::Disc::scan_image(
&mut reader,
capacity,
&crate::disc::ScanOptions::default(),
)
.map_err(|e| -> io::Error { e.into() })?;
// Apply the caller-resolved keys (lookup-free); decrypt_keys() then
// yields them for the stream below. Propagate a failed application
// rather than silently muxing an undecryptable stream.
if !opts.unit_keys.is_empty() {
// These UKs were already resolved AND validated by the caller
// (the CLI's keydb loop), so no re-validation sample is needed.
disc.decrypt_with(crate::disc::Key::Unit(opts.unit_keys.clone()), &[])
.map_err(|e| -> io::Error { e.into() })?;
}
// Pre-flight decrypt gate (the single, system-wide verdict — see
// `Disc::ensure_decryptable`). Fails fast BEFORE any mux work when
// decryption is needed and unavailable: a scrambled-but-uncracked
// CSS disc (`css_error` set), or an AACS-encrypted disc with no
// usable key (would mux ~100 MB of garbage — encrypted m2ts → no TS
// syncs → demuxer emits nothing → empty/garbage output at exit 0).
// `--raw` and unencrypted/CSS-keyless-success discs pass. This is the
// disc-wide check; the per-title (multi-VTS CSS) check is below, once
// the chosen title's key is resolved.
disc.ensure_decryptable(opts.raw)
.map_err(|e| -> io::Error { e.into() })?;
if disc.titles.is_empty() {
return Err(crate::error::Error::NoStreams.into());
}
let idx = opts.title_index.unwrap_or(0);
if idx >= disc.titles.len() {
return Err(crate::error::Error::DiscTitleRange {
index: idx,
count: disc.titles.len(),
}
.into());
}
// Prune to the selected audio/subtitle streams now, on the scanned
// (pre-`probe_and_remap`) title, so everything downstream — the
// TrueHD channel-correction probe, the final title clone, and
// `build_iso_pipeline`'s demux/track construction — sees the pruned
// list. Video is always kept; a no-op for the default All/All.
opts.selection
.apply(&mut disc.titles[idx])
.map_err(|e| -> io::Error { e.into() })?;
// Per-title key resolution. DVD CSS is resolved at exactly ONE site —
// `build_iso_pipeline`'s per-title crack (below), which decrypts a
// crackable title, passes a genuinely-clear one through, and
// hard-fails an uncrackable one with CssKeyMissing. So for a DVD we do
// NOT pre-crack here: pass `None` and let the pipeline own it.
// Pre-cracking would re-open the ISO and re-scan every clear title
// (`decrypt_keys_for_title` → None → the pipeline re-cracks anyway).
// AACS / unencrypted resolve from `decrypt_keys()` with NO read; `--raw`
// (any format) is deliberate ciphertext passthrough — also `None`.
let is_dvd = disc.format == crate::disc::DiscFormat::Dvd;
let (keys, title_is_clear) = if opts.raw || is_dvd {
(crate::decrypt::DecryptKeys::None, false)
} else {
(disc.decrypt_keys(), false)
};
// Decrypt gate for the AACS / non-DVD path: a None key means no usable
// disc key, which would mux scrambled ciphertext verbatim — fail loudly
// (NoDiscKey). The DVD path is gated inside `build_iso_pipeline` (its
// CSS hard-fail), and `--raw` passes.
if !is_dvd {
disc.ensure_title_decryptable(opts.raw, &keys, title_is_clear)
.map_err(|e| -> io::Error { e.into() })?;
}
// FMTS (AACS 2.1) forensic segments are sourced + fail-loud-checked
// downstream by `resolve_mux_key_map`/`resolve_fmts_key_map`, which hold
// the key-fetch closure and can actually attempt resolution. (An older
// upfront blanket-reject gate lived here; it predated the resolver and
// rejected every 2.1 disc before a source could be tried.)
// Correct TrueHD channel counts (MPLS understates 7.1/Atmos as 5.1)
// by probing the first DECRYPTED access units of the chosen title.
// A fresh reader avoids disturbing the mux reader below. Skipped in
// --raw mode: the probe would re-open + decrypt for nothing (on an
// AACS disc with no key the correction is a no-op on ciphertext, and
// raw output isn't decoded anyway).
if !opts.raw {
match crate::io::file_sector_source::FileSectorSource::open(path) {
Ok(mut probe) => {
// The probe DECRYPTS the title head, so it needs the SAME
// up-front key map the mux read installs below. An AACS
// `DecryptingSectorSource` with no map fails loud on the
// first unit (`decrypt_sectors_mapped` is the only AACS
// decrypt path) — without resolving one here the correction
// is silently skipped on every AACS disc and 7.1/Atmos stays
// understated as the MPLS-declared 5.1. Resolution failure is
// non-fatal (`.ok()`): leave channels uncorrected, never
// fail the mux.
let mut probe_keys = keys.clone();
let probe_title = disc.titles[idx].clone();
let probe_map = match &probe_keys {
crate::decrypt::DecryptKeys::Aacs { .. } => resolve_mux_key_map(
&mut probe,
&probe_title,
&mut probe_keys,
opts.key_fetch.as_ref(),
disc.content_format,
// File-backed, bounded probe (best-effort `.ok()`);
// no live drive to protect from a stuck stop here.
None,
)
.ok()
.map(std::sync::Arc::new),
_ => None,
};
let mut dec = crate::sector::DecryptingSectorSource::new(probe, probe_keys);
if let Some(map) = probe_map {
dec = dec.with_key_map(map);
}
crate::disc::correct_truehd_channels(&mut dec, &mut disc.titles[idx]);
}
Err(e) => {
// Non-fatal: a failed re-open just leaves MPLS 7.1/Atmos
// channel counts uncorrected (understated as 5.1). Log so
// the uncorrected path is diagnosable rather than silent.
tracing::debug!(
target: "mux",
"TrueHD channel-correction probe re-open failed: {e}"
);
}
}
}
let title = disc.titles[idx].clone();
let format = disc.content_format;
// ISO file: 8192-sector batch (16 MiB at 2048 B/sector) —
// sequential read from fast storage, no bad sectors. Empirically
// optimal; bumping to 16384 sectors (32 MiB) regressed (more cache
// pressure, longer per-batch latency starves the consumer between
// iterations). Physical drives keep smaller batches for adaptive
// error handling.
const ISO_MUX_BATCH_SECTORS: u16 = 8192;
// Pass `DecryptKeys::None` to the decrypt decorator when
// --raw is set — the read stack still flows through the
// same producer+demux+parse pipeline, just without the
// AACS / CSS step. Single highway for all ISO reads.
let effective_keys = if opts.raw {
crate::decrypt::DecryptKeys::None
} else {
keys
};
// Install the shared fetch closure (if the app supplied one) so a
// unit no held key decrypts is re-tried via the app's key source.
// Suppressed in --raw (no decrypt step to recover).
let fetch = if opts.raw {
None
} else {
opts.key_fetch.clone()
};
let stream = build_iso_pipeline(
let reader = crate::io::file_sector_source::FileSectorSource::open(path)?;
let probe_path = path.clone();
let stream = image_input(
reader,
title,
effective_keys,
ISO_MUX_BATCH_SECTORS,
format,
opts.raw,
None,
None,
fetch,
opts,
move || crate::io::file_sector_source::FileSectorSource::open(&probe_path).ok(),
false,
)?;
Ok(Box::new(stream))
}
// `dir://` as a SOURCE: an extracted disc folder, presented as a
// synthetic UDF image (`crate::dirimage`). It reaches EXACTLY the same
// body as `iso://` above — scan, title select, key resolution, mux —
// because by the time `DirImage` exists it is just another
// `SectorSource`. That is the point of synthesizing a real filesystem
// rather than faking a tree: every destination follows for free, with
// no per-scheme mux path to keep in step.
StreamUrl::Dir { ref path } => {
validate_file_path(path, "dir")?;
let reader = crate::dirimage::DirImage::open(path)?;
let probe_path = path.clone();
let stream = image_input(
reader,
opts,
move || crate::dirimage::DirImage::open(&probe_path).ok(),
true,
)?;
Ok(Box::new(stream))
}
@@ -556,9 +437,6 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
Ok(Box::new(NetworkStream::listen(addr)?))
}
StreamUrl::Stdio => Ok(Box::new(StdioStream::input())),
// `dir://` is an output-only sink (decrypted file tree); it is never a
// PES source. Mirror `null://` → write-only.
StreamUrl::Dir { .. } => Err(crate::error::Error::StreamWriteOnly.into()),
StreamUrl::Null => Err(crate::error::Error::StreamWriteOnly.into()),
// `mp4://` as a source: demux a progressive MP4 back into PES frames, so
// `mp4://` flows to every sink (mkv://, audio://, json://, …).
@@ -578,6 +456,202 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result<Box<dyn crate::pes::S
}
}
/// Shared body of every IMAGE-level PES source.
///
/// `iso://` and `dir://` differ only in how the sectors are produced — a file
/// versus a synthesized UDF volume over a folder — and not at all in what is
/// done with them: scan, apply caller-resolved keys, gate on decryptability,
/// select the title, prune streams, correct TrueHD channel counts, mux. One
/// body means a `dir://` source cannot drift away from the `iso://` behaviour
/// that has years of fixes in it.
///
/// `reopen` yields a SECOND, independent reader for the TrueHD channel probe,
/// which must not disturb the mux reader's position. It returns `Option`
/// because a failed re-open is non-fatal: the correction is skipped, not the
/// mux.
fn image_input<S, F>(
mut reader: S,
opts: &InputOptions,
reopen: F,
// A FOLDER's `encrypted` flag comes from tree shape and can be wrong; an
// image's cannot. See `session::apply_folder_encryption_verdict`.
is_folder: bool,
) -> io::Result<PipelinedPesStream>
where
S: SectorSource + Send + 'static,
F: FnOnce() -> Option<S>,
{
let capacity = reader.capacity_sectors();
let mut disc =
crate::disc::Disc::scan_image(&mut reader, capacity, &crate::disc::ScanOptions::default())
.map_err(|e| -> io::Error { e.into() })?;
// Without this a folder reached here with a tree-shape verdict while
// `session::scan_dir` reached the opposite one from its CONTENT, so the
// same folder ripped through one door and failed through the other.
if is_folder {
crate::session::apply_folder_encryption_verdict(&mut reader, &mut disc)
.map_err(|e| -> io::Error { e.into() })?;
}
// Apply the caller-resolved keys (lookup-free); decrypt_keys() then
// yields them for the stream below. Propagate a failed application
// rather than silently muxing an undecryptable stream.
if !opts.unit_keys.is_empty() {
// These UKs were already resolved AND validated by the caller
// (the CLI's keydb loop), so no re-validation sample is needed.
disc.decrypt_with(crate::disc::Key::Unit(opts.unit_keys.clone()), &[])
.map_err(|e| -> io::Error { e.into() })?;
}
// Pre-flight decrypt gate (the single, system-wide verdict — see
// `Disc::ensure_decryptable`). Fails fast BEFORE any mux work when
// decryption is needed and unavailable: a scrambled-but-uncracked
// CSS disc (`css_error` set), or an AACS-encrypted disc with no
// usable key (would mux ~100 MB of garbage — encrypted m2ts → no TS
// syncs → demuxer emits nothing → empty/garbage output at exit 0).
// `--raw` and unencrypted/CSS-keyless-success discs pass. This is the
// disc-wide check; the per-title (multi-VTS CSS) check is below, once
// the chosen title's key is resolved.
disc.ensure_decryptable(opts.raw)
.map_err(|e| -> io::Error { e.into() })?;
if disc.titles.is_empty() {
return Err(crate::error::Error::NoStreams.into());
}
let idx = opts.title_index.unwrap_or(0);
if idx >= disc.titles.len() {
return Err(crate::error::Error::DiscTitleRange {
index: idx,
count: disc.titles.len(),
}
.into());
}
// Prune to the selected audio/subtitle streams now, on the scanned
// (pre-`probe_and_remap`) title, so everything downstream — the
// TrueHD channel-correction probe, the final title clone, and
// `build_iso_pipeline`'s demux/track construction — sees the pruned
// list. Video is always kept; a no-op for the default All/All.
opts.selection
.apply(&mut disc.titles[idx])
.map_err(|e| -> io::Error { e.into() })?;
// Per-title key resolution. DVD CSS is resolved at exactly ONE site —
// `build_iso_pipeline`'s per-title crack (below), which decrypts a
// crackable title, passes a genuinely-clear one through, and
// hard-fails an uncrackable one with CssKeyMissing. So for a DVD we do
// NOT pre-crack here: pass `None` and let the pipeline own it.
// Pre-cracking would re-open the ISO and re-scan every clear title
// (`decrypt_keys_for_title` → None → the pipeline re-cracks anyway).
// AACS / unencrypted resolve from `decrypt_keys()` with NO read; `--raw`
// (any format) is deliberate ciphertext passthrough — also `None`.
let is_dvd = disc.format == crate::disc::DiscFormat::Dvd;
let (keys, title_is_clear) = if opts.raw || is_dvd {
(crate::decrypt::DecryptKeys::None, false)
} else {
(disc.decrypt_keys(), false)
};
// Decrypt gate for the AACS / non-DVD path: a None key means no usable
// disc key, which would mux scrambled ciphertext verbatim — fail loudly
// (NoDiscKey). The DVD path is gated inside `build_iso_pipeline` (its
// CSS hard-fail), and `--raw` passes.
if !is_dvd {
disc.ensure_title_decryptable(opts.raw, &keys, title_is_clear)
.map_err(|e| -> io::Error { e.into() })?;
}
// FMTS (AACS 2.1) forensic segments are sourced + fail-loud-checked
// downstream by `resolve_mux_key_map`/`resolve_fmts_key_map`, which hold
// the key-fetch closure and can actually attempt resolution. (An older
// upfront blanket-reject gate lived here; it predated the resolver and
// rejected every 2.1 disc before a source could be tried.)
// Correct TrueHD channel counts (MPLS understates 7.1/Atmos as 5.1)
// by probing the first DECRYPTED access units of the chosen title.
// A fresh reader avoids disturbing the mux reader below. Skipped in
// --raw mode: the probe would re-open + decrypt for nothing (on an
// AACS disc with no key the correction is a no-op on ciphertext, and
// raw output isn't decoded anyway).
if !opts.raw {
match reopen() {
Some(mut probe) => {
// The probe DECRYPTS the title head, so it needs the SAME
// up-front key map the mux read installs below. An AACS
// `DecryptingSectorSource` with no map fails loud on the
// first unit (`decrypt_sectors_mapped` is the only AACS
// decrypt path) — without resolving one here the correction
// is silently skipped on every AACS disc and 7.1/Atmos stays
// understated as the MPLS-declared 5.1. Resolution failure is
// non-fatal (`.ok()`): leave channels uncorrected, never
// fail the mux.
let mut probe_keys = keys.clone();
let probe_title = disc.titles[idx].clone();
let probe_map = match &probe_keys {
crate::decrypt::DecryptKeys::Aacs { .. } => resolve_mux_key_map(
&mut probe,
&probe_title,
&mut probe_keys,
opts.key_fetch.as_ref(),
disc.content_format,
// File-backed, bounded probe (best-effort `.ok()`);
// no live drive to protect from a stuck stop here.
None,
)
.ok()
.map(std::sync::Arc::new),
_ => None,
};
let mut dec = crate::sector::DecryptingSectorSource::new(probe, probe_keys);
if let Some(map) = probe_map {
dec = dec.with_key_map(map);
}
crate::disc::correct_truehd_channels(&mut dec, &mut disc.titles[idx]);
}
None => {
// Non-fatal: a failed re-open just leaves MPLS 7.1/Atmos
// channel counts uncorrected (understated as 5.1). Log so
// the uncorrected path is diagnosable rather than silent.
tracing::debug!(
target: "mux",
"TrueHD channel-correction probe re-open failed"
);
}
}
}
let title = disc.titles[idx].clone();
let format = disc.content_format;
// ISO file: 8192-sector batch (16 MiB at 2048 B/sector) —
// sequential read from fast storage, no bad sectors. Empirically
// optimal; bumping to 16384 sectors (32 MiB) regressed (more cache
// pressure, longer per-batch latency starves the consumer between
// iterations). Physical drives keep smaller batches for adaptive
// error handling.
const ISO_MUX_BATCH_SECTORS: u16 = 8192;
// Pass `DecryptKeys::None` to the decrypt decorator when
// --raw is set — the read stack still flows through the
// same producer+demux+parse pipeline, just without the
// AACS / CSS step. Single highway for all ISO reads.
let effective_keys = if opts.raw {
crate::decrypt::DecryptKeys::None
} else {
keys
};
// Install the shared fetch closure (if the app supplied one) so a
// unit no held key decrypts is re-tried via the app's key source.
// Suppressed in --raw (no decrypt step to recover).
let fetch = if opts.raw {
None
} else {
opts.key_fetch.clone()
};
let stream = build_iso_pipeline(
reader,
title,
effective_keys,
ISO_MUX_BATCH_SECTORS,
format,
opts.raw,
None,
None,
fetch,
)?;
Ok(stream)
}
/// Open a PES output stream (consumes PES frames).
///
/// `source` is the provenance of the material being written — the INPUT the
@@ -2106,10 +2180,25 @@ pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
// decrypted, or handed to the demux, so the demux sees one gapless our-variant
// stream (no ciphertext to trip a concealed-gap resync). A non-forensic map
// returns the extents unchanged, so the common disc reads exactly as before.
let full_extents = extents.clone();
let extents = match &key_map {
Some(map) => map.read_plan(&extents, unit_align as u32),
None => extents,
};
// The plan and the clips' feed spans must describe the SAME bytes.
//
// A clip's span was measured over the title's full extents at scan time,
// and a frame is placed by the offset it was read from. When a forensic
// segment makes the plan drop alternate-phase units, the mux feeds fewer
// bytes than the spans describe and the two drift apart cumulatively —
// every frame after the first segment looks earlier than it is, and near a
// join it is placed in the wrong clip or dropped. The spans still tile each
// other perfectly, so the trust check cannot see it.
//
// Provenance is only meaningful when the feed matches. When it does not,
// say so and let placement fall back to timestamps, which is what the
// untrusted path exists for.
let feed_matches_spans = extents == full_extents;
let mut decrypting =
crate::sector::DecryptingSectorSource::new(Box::new(reader) as Box<dyn SectorSource>, keys);
if let Some(map) = key_map {
@@ -2128,6 +2217,17 @@ pub fn build_iso_pipeline<S: SectorSource + Send + 'static>(
// correct `0x8x` sub-streams. No-op for non-DVD or an empty probe. Reset the
// unit base afterward so the prefetcher's first batch starts clean.
let mut title = title;
if !feed_matches_spans {
tracing::info!(
target: "freemkv::mux",
planned = extents.len(),
full = full_extents.len(),
"read plan omits units the clip spans include; placing by timestamps"
);
for c in &mut title.clips {
c.feed_span = None;
}
}
crate::disc::dvd_audio_probe::probe_and_remap(&mut decrypting, &mut title);
decrypting.set_unit_base(0);
@@ -2547,9 +2647,12 @@ mod tests {
}
/// `dir://PATH/` parses to `StreamUrl::Dir` with the raw remainder as the
/// path; it is a SINK (not a disc source), so `is_disc_source()` is false.
/// path. Unlike the other directory schemes it IS an image-level source
/// (1.6.1): `crate::dirimage` synthesizes a UDF volume over the folder, so
/// `is_disc_source()` — "has a filesystem to scan" — is true for it and
/// false for the write-only `demux://` / `fvi://` directory sinks.
#[test]
fn parse_dir_url_is_sink_not_disc_source() {
fn parse_dir_url_is_an_image_source_unlike_the_directory_sinks() {
match parse_url("dir://out/movie/") {
StreamUrl::Dir { path } => {
assert_eq!(path, PathBuf::from("out/movie/"));
@@ -2565,8 +2668,8 @@ mod tests {
"demux:// is a sink, never a disc source"
);
assert!(
!parse_url("dir://x").is_disc_source(),
"dir:// is a sink, never a disc source"
parse_url("dir://x").is_disc_source(),
"dir:// carries a filesystem, so selection flags and image sinks apply"
);
// fvi:// parses to Fvi with the raw remainder as the path, and is a
// sink (never a disc source) — parallel to the demux:// coverage above.
@@ -2594,15 +2697,19 @@ mod tests {
);
}
/// `dir://` is output-only: `input()` rejects it (StreamWriteOnly →
/// Unsupported), and `output()` rejects it too (StreamReadOnly →
/// Unsupported) because it is NOT a PES sink — the CLI routes it to
/// `Disc::extract_tree` before the mux path.
/// `dir://` is never a PES SINK — it writes raw decrypted files, not muxed
/// frames, so `output()` still rejects it (StreamReadOnly → Unsupported)
/// and the CLI routes a `dir://` dest to `Disc::extract_tree`.
///
/// As a SOURCE it is no longer rejected out of hand: it is an image source,
/// and a missing folder now fails as a missing folder (NotFound) rather
/// than as "this scheme cannot be read".
#[test]
fn dir_url_is_not_a_pes_stream_either_direction() {
fn dir_url_is_an_input_but_never_a_pes_sink() {
assert_eq!(
input_err_kind("dir://out/"),
std::io::ErrorKind::Unsupported
input_err_kind("dir://definitely/not/here/"),
std::io::ErrorKind::NotFound,
"a dir:// source that does not exist must report a missing path"
);
let t = DiscTitle::empty();
assert_eq!(
+2164 -31
View File
File diff suppressed because it is too large Load Diff
+3
View File
@@ -682,6 +682,9 @@ mod tests {
for (i, b) in template.iter_mut().enumerate() {
*b = (i as u8).wrapping_mul(29).wrapping_add(3);
}
// Real scrambled DVD sectors are MPEG-2 PS packs; the scramble policy
// requires the pack start code as well as the flag bits.
template[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
template[0x14] = 0x30; // scramble bits (4-5) set → flags == 0x03
let pristine = template;
+152
View File
@@ -439,6 +439,158 @@ pub fn scan_iso(path: &Path, opts: ScanOptions) -> Result<(Disc, Box<dyn SectorS
Ok((disc, Box::new(reader)))
}
/// Sampled 6144-byte aligned units when deciding whether a folder that still
/// carries `AACS/` actually holds encrypted content. Enough to survive a clip
/// whose opening units happen to be unflagged (a clear leader), few enough to
/// stay a handful of reads.
const AACS_PROBE_UNITS: usize = 8;
/// [`Disc`] together with a [`SectorSource`] over a synthesized image of an
/// extracted disc FOLDER — the `dir://` counterpart to [`scan_iso`].
///
/// The extra step over `scan_iso` is the encryption verdict.
/// `Disc::scan_with` decides `encrypted` STRUCTURALLY, from the presence of an
/// `/AACS` or `/BDMV/AACS` directory (see `disc::aacs_dir_present`). For the common
/// case — a MakeMKV-style backup, which strips `AACS/` — that already gives the
/// right answer, and `DecryptKeys::None` is a pass-through. But a folder copied
/// verbatim from a decrypted disc keeps `AACS/`, and the tree shape then claims
/// encryption over content that is already in the clear: the rip would fail
/// asking for a key it does not need.
///
/// So for a folder, tree shape is not the evidence — CONTENT is. Several
/// aligned units at the largest title's start are sampled and judged by
/// `aacs_unit_needs_decrypt`, the same authority the mux read path uses:
///
/// * none need decryption → the folder is decrypted; `encrypted` is forced
/// false and the reason is logged.
/// * any unit does → the folder is a raw encrypted copy, which `dir://` does
/// not support; [`Error::DirImageEncrypted`].
///
/// This lives HERE and not in `Disc::scan_image`, which is shared with the ISO
/// and drive paths: an ISO that carries `AACS/` and clear content is a
/// different situation (it may be mid-decrypt, or `--raw` output), and the
/// verdict must not change underneath those callers.
pub fn scan_dir(path: &Path, opts: ScanOptions) -> Result<(Disc, Box<dyn SectorSource>)> {
let mut reader = crate::dirimage::DirImage::open(path)?;
let capacity = reader.capacity_sectors();
let mut disc = Disc::scan_image(&mut reader, capacity, &opts)?;
apply_folder_encryption_verdict(&mut reader, &mut disc)?;
Ok((disc, Box::new(reader)))
}
/// Re-judge a FOLDER's encryption verdict from its CONTENT.
///
/// `Disc::scan_with` decides `encrypted` from tree shape — whether an `AACS/`
/// directory is present. That is right for an image and wrong for a verbatim
/// copy of an already-decrypted disc that kept the directory: the rip would
/// fail asking for a key it does not need.
///
/// Shared by [`scan_dir`] and by the `dir://` PES input path in
/// `mux::resolve`. It lives in one place because the two disagreed: a folder
/// that ripped through `scan_dir` failed through `input()`, which is the exact
/// failure this probe was written to prevent, reachable by the other door.
pub(crate) fn apply_folder_encryption_verdict(
reader: &mut dyn SectorSource,
disc: &mut Disc,
) -> Result<()> {
// `css.is_some()` is the DVD path, and that verdict came from actually
// cracking scrambled sectors — real evidence about content, not tree shape.
// Only the AACS-by-tree-shape verdict is re-judged here.
if disc.encrypted && disc.css.is_none() && disc.css_error.is_none() {
match probe_folder_encryption(reader, disc)? {
true => return Err(Error::DirImageEncrypted),
false => {
tracing::warn!(
target: "freemkv::scan",
phase = "folder_verdict",
"folder carries an AACS directory but its sampled content units \
are already in the clear; treating it as decrypted"
);
disc.encrypted = false;
disc.aacs = None;
disc.aacs_error = None;
}
}
}
Ok(())
}
/// `true` when any sampled content unit still needs decryption.
///
/// Anchored at the largest title's first extent, because AACS unit alignment
/// is measured from the clip FILE's start (`aacs::content::is_unit_aligned`),
/// not from an absolute `lba % 3` — sampling off a boundary would mis-judge a
/// perfectly clear unit.
fn probe_folder_encryption(reader: &mut dyn SectorSource, disc: &Disc) -> Result<bool> {
use crate::aacs::content::{aacs_unit_needs_decrypt, is_unit_aligned};
use crate::consts::SECTOR_BYTES;
const UNIT_SECTORS: u32 = 3;
// Anchor on the largest TITLE's FIRST extent, not on the largest extent
// anywhere.
//
// AACS units are 3 sectors, and a unit boundary is only guaranteed at the
// START of a clip. `max_by_key` over every extent picked a mid-file one for
// any clip big enough to be split: the planner caps an allocation
// descriptor at MAX_AD_BYTES = 524287 sectors, every full piece of a split
// file therefore ties on sector_count, and `max_by_key` returns the LAST
// tie — an extent starting (k-1)*524287 sectors in. 524287 % 3 == 1, so
// that start is off the unit boundary for two file sizes in three.
//
// The sampling then reads 6144-byte windows that begin mid-source-packet,
// so the CPI byte it thinks it is testing is content. Both verdicts are
// wrong in a costly direction: a decrypted folder gets rejected as
// encrypted (DirImageEncrypted on something perfectly rippable), or
// genuine ciphertext reads as clear and the mux writes it out as video at
// exit 0. `is_unit_aligned` cannot catch it, because it measures against
// this same wrong base.
let Some(extent) = disc
.titles
.iter()
.max_by_key(|t| {
t.extents
.iter()
.fold(0u64, |a, e| a.saturating_add(e.sector_count as u64))
})
.and_then(|t| t.extents.first())
else {
// No content to judge. A folder with an AACS directory and no titles
// has nothing to rip either way; leave the structural verdict alone.
return Ok(true);
};
let base = extent.start_lba;
let mut unit = vec![0u8; UNIT_SECTORS as usize * SECTOR_BYTES];
let mut sampled = 0u32;
for i in 0..AACS_PROBE_UNITS as u32 {
// Saturating: `start_lba` and `sector_count` come off the medium, and a
// crafted or corrupt extent must not wrap this bound into a read past
// the end of the content.
let Some(lba) = base.checked_add(i.saturating_mul(UNIT_SECTORS)) else {
break;
};
let end = base.saturating_add(extent.sector_count);
if lba.saturating_add(UNIT_SECTORS) > end {
break;
}
debug_assert!(is_unit_aligned(lba, base));
reader.read_sectors(lba, UNIT_SECTORS as u16, &mut unit, false)?;
sampled += 1;
if aacs_unit_needs_decrypt(&unit, disc.content_format) {
return Ok(true);
}
}
// Nothing was actually sampled — the largest title is shorter than one
// aligned unit, so there is no evidence either way. "Not encrypted" is the
// dangerous default here: it would clear the structural verdict an `AACS`
// directory raised and rip ciphertext as though it were video, at exit 0.
// With no evidence, keep the structural verdict.
if sampled == 0 {
return Ok(true);
}
Ok(false)
}
#[cfg(test)]
mod tests {
use super::*;
+8
View File
@@ -1553,6 +1553,14 @@ fn parse_dstring(data: &[u8]) -> String {
}
}
/// Test-only view of [`parse_dstring`], so the `dirimage` encoder can assert
/// that the d-strings it writes are the ones this parser reads back rather
/// than re-implementing the decode in its own tests.
#[cfg(test)]
pub(crate) fn parse_dstring_for_test(data: &[u8]) -> String {
parse_dstring(data)
}
/// Buffered sector reader — reduces SCSI round-trips by coalescing
/// single-sector reads into `batch`-sized SCSI commands. Per-command
/// latency dominates on USB drives, so serving many adjacent single-sector
+16 -6
View File
@@ -20,6 +20,10 @@ fn css_descramble_sector_roundtrip_via_public_api() {
};
let mut sector = vec![0xAAu8; 2048];
// Real scrambled DVD sectors are MPEG-2 PS packs, and the public API now
// requires the pack start code as well as the flag bits — byte 0x14 means
// something else entirely in an IFO or UDF sector.
sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sector[0x14] = 0x30;
sector[0x54..0x59].copy_from_slice(&[0xDE, 0xAD, 0xBE, 0xEF, 0x42]);
let original = sector.clone();
@@ -37,30 +41,36 @@ fn css_descramble_sector_roundtrip_via_public_api() {
assert_ne!(&sector[128..256], &original[128..256]);
}
/// Test: css_is_scrambled detects scramble flags correctly.
/// Test: has_scramble_flag_bits detects the raw flag bits correctly.
#[test]
fn css_is_scrambled_detection() {
let mut sector = vec![0u8; 2048];
assert!(
!css::is_scrambled(&sector),
!css::has_scramble_flag_bits(&sector),
"empty sector should not be scrambled"
);
sector[0x14] = 0x10; // bit 4 set
assert!(css::is_scrambled(&sector), "bit 4 set should be detected");
assert!(
css::has_scramble_flag_bits(&sector),
"bit 4 set should be detected"
);
sector[0x14] = 0x20; // bit 5 set
assert!(css::is_scrambled(&sector), "bit 5 set should be detected");
assert!(
css::has_scramble_flag_bits(&sector),
"bit 5 set should be detected"
);
sector[0x14] = 0x30; // both bits set
assert!(
css::is_scrambled(&sector),
css::has_scramble_flag_bits(&sector),
"both bits set should be detected"
);
sector[0x14] = 0xCF; // bits 4-5 clear, other bits set
assert!(
!css::is_scrambled(&sector),
!css::has_scramble_flag_bits(&sector),
"bits 4-5 clear should not be scrambled"
);
}
+4
View File
@@ -42,6 +42,10 @@ fn css_decrypt_of_an_uncrackable_sector_still_descrambles() {
// unit key either opens a unit or does not, whereas a CSS title key is
// recovered from data whose recoverability varies sector by sector.
let mut sector = vec![0xFFu8; 2048];
// Scrambled DVD sectors are MPEG-2 PS packs. The descramble policy requires
// the pack start code as well as the flag bits, because byte 0x14 means
// something else entirely in an IFO, UDF or ISO 9660 sector.
sector[0x00..0x04].copy_from_slice(&[0x00, 0x00, 0x01, 0xBA]);
sector[0x14] |= 0x30; // CSS scramble flag, bits 4-5
let title_key: [u8; 5] = [0x42, 0x13, 0x37, 0xBE, 0xEF];