libfreemkv/docs/dts.md

# DTS / DTS-HD elementary stream parser (`src/mux/codec/dts.rs`)

Internal design notes moved out of source comments (see `ci/comment-guard.py`
prose caps). Each section is pointed to by a short `//` comment at the
corresponding site in `dts.rs`.

## `DTS_HD_EXT_SYNC`

An access unit is delimited by the next CORE sync; the parser locates and
exactly sizes each extension substream (via `exss_frame_size`) so a false
core sync inside the EXSS payload can't split the AU and truncate the
lossless extension.

## `emit_or_drop`

Gates an assembled access unit through the decodability check and either
pushes it or drops it. `au_pts`/`dur_ns` are already stamped on the shared
PTS clock (which the caller advances whether or not the AU survives), so a
drop leaves the following audio on its true timeline — a gap, not a shift.
Every drop is logged (fail-loud, never silent).

## `stamp_pts`

`front` is the AU's own core-PES PTS (from `front_pts`); `dur_ns` its
decoded duration.

The model matches the (correct) AC-3 path: **re-base to each PES's own
container timestamp, and advance by one frame duration ONLY within a run of
AUs that share the same PES.** A new PES (its `front` differs from the
previous AU's) trusts its own timestamp — so the emitted timeline tracks the
container and never drifts. Advancing within a PES is what fixes the DVD
case (several DTS core frames packed in one PES, which otherwise all
collided on that single PES timestamp → "non monotonically increasing dts").
A global running clock was WRONG here: once accumulated frame durations
exceeded the PES spacing it never re-based, drifting the track minutes past
the real length over a feature-long title.

## `CORE_HEADER_MIN_BYTES`

Number of leading bytes that must be buffered before the core `fsize` field
(bytes 5-7) can be decoded. This is a HEADER-LAYOUT minimum — "enough bytes
to read the size field" — and is deliberately distinct from
`MIN_CORE_FRAME_BYTES` (the decoded-frame-size validity floor). They must
not be conflated: this one gates buffer reads of the header, the other
rejects implausible decoded sizes.

## `MIN_CORE_FRAME_BYTES`

Minimum plausible decoded DTS core frame size, per ETSI TS 102 114: the
on-wire FSIZE floor is 95, so a real core frame is at least 96 bytes. A
decoded `core_size` below this means we matched a false/corrupt core sync
(a lucky 0x7FFE8001 in extension-substream payload whose 14-bit `fsize`
decoded to a tiny value) rather than a real frame, so we resync instead of
closing an access unit at a junk boundary and dropping the DTS-HD extension
tail.

## `PTS_UNSET`

Sentinel for "no valid PTS base captured yet". Real PTS-in-ns values are
non-negative (derived from the unsigned 90 kHz PES timestamp), so a
negative value can never collide with a genuine timestamp. Used to mark the
PTS base invalid after a forced flush so the next PES sets it regardless of
buffer state.

## `flush_tail`

Emits the final buffered access unit (the last core + its extension
substreams, which had no following core sync to close it during
streaming), gated through the decodability check. Requires a complete core
frame; drops a bare partial sync tail.

## `SYNCWORD_BYTES`

Byte length shared by the DTS core sync and the DTS-HD EXSS sync (both
32-bit words). `next_core_boundary` finds the next *valid* core sync after
the current core frame, to delimit the access unit: extension-substream
payload can contain byte sequences that match the core syncword, so each
candidate is validated by decoding its core size — a match whose decoded
size is implausible (`< MIN_CORE_FRAME_BYTES` or `> MAX_AU_BYTES`) is a
false sync and is skipped, continuing the search.

## EXSS header field bit widths

`EXSS_USER_DEFINED_BITS`, `EXSS_INDEX_BITS`, `EXSS_HEADER_SIZE_TYPE_BITS`,
etc. are the DTS-HD extension-substream (EXSS) header field bit widths
(ETSI TS 102 114, ExtSS header). `bHeaderSizeType` selects the short form
(`nuExtSSHeaderSize` 8 bits, `nuExtSSFsize` 16 bits) or, for larger
substreams, the long form (12 / 20 bits).

## `exss_frame_size`

DTS-HD extension substream (EXSS) total byte size — INCLUDING the
`0x64582025` syncword — read precisely from its header. `buf` must begin
with `DTS_HD_EXT_SYNC`. Returns `None` when the size fields aren't fully
buffered.

`nuExtSSFsize` is the total frame size in bytes minus one. Parsing it lets
the AU framer skip the extension by its exact length instead of scanning
its (arbitrary) payload for a core sync.

## `next_core_boundary`

Offset where the current access unit ends (the start of the next core
frame). The AU is the core frame plus its trailing DTS-HD extension
substreams, which are skipped PRECISELY by their declared size — so a
chance core syncword inside the XLL lossless payload can never be mistaken
for the next AU boundary (the bug that truncated the extension and produced
the "Failed to decode block code(s)" class). Falls back to the heuristic
core-sync scan only when an extension can't be sized (malformed / truncated
input).

## `scan_for_next_core`

Heuristic fallback (the pre-fix behaviour): scan forward for the next core
syncword whose decoded size is plausible. Used only when precise extension
skipping can't proceed; a chance core syncword in extension payload usually
decodes to an implausible size and is skipped.

## `dts_core_frame_size`

`fsize` is the 14-bit field at bits 46-59 of the header (bytes 5-7). On the
wire `fsize` is the frame length minus one, so this returns `fsize + 1`,
i.e. the core frame length in bytes (range 1..=16384). Callers treat the
result as the actual byte length and the MIN..=MAX range checks assume so.

Returns `0` when `data` is shorter than `CORE_HEADER_MIN_BYTES` — every call
site rejects that via the minimum-frame lower bound, so a `0` is never
mistaken for a valid tiny frame.

## `dts_core_samples`

Samples in one DTS core frame: `(NBLKS + 1) * 32`. `NBLKS` (7 bits) is the
core-header PCM-sample-block count (ETSI TS 102 114) that fixes the frame's
decoded sample count. Bit layout after the 32-bit sync: FTYPE(1) SHORT(5)
CPF(1) **NBLKS(7)** FSIZE(14) …, so NBLKS = byte4 bit0 + byte5 bits7-2.

## `DTS_PCMBLOCK_SAMPLES` / `DTS_SUBBAND_SAMPLES` (core-header validity)

DTS core-header validity constants (ETSI TS 102 114). For a NORMAL frame
`deficit_samples` must equal `DTS_PCMBLOCK_SAMPLES` — a termination frame
may carry fewer; `npcmblocks` must be a multiple of `DTS_SUBBAND_SAMPLES`;
`audio_mode` must be below `DTS_AMODE_COUNT`; `lfe_present ==
DTS_LFE_FLAG_INVALID` is rejected.

## `DTS_AMODE_COUNT`

Number of LEGAL `AMODE` (channel-arrangement) codes. The 6-bit AMODE field
(ETSI TS 102 114 §5.3.1) has 16 defined channel arrangements, codes 0-15;
only 16-63 are reserved/user-defined and undecodable. The per-AMODE channel
counts in ETSI TS 102 114 §5.3.1 cover all 16, confirming they are
decodable — codes 10-15 are the 6/7/8-channel layouts. A frame is dropped
only when `audio_mode >= DTS_AMODE_COUNT` (i.e. a truly reserved 16-63
code); dropping a legal 10-15 multichannel core would silence recoverable
audio.

## `DTS_CORE_SR_VALID`

Sample rate (Hz) per core `SFREQ` code (ETSI TS 102 114 Table 6-4); a `0`
entry marks a reserved code that fails header validation as an invalid
sample rate. Valid entries are locked to the spec by
`dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
{0, 4, 5, 9, 10}.

## `core_header_drop_reason`

Decodability gate: the core-frame header validity checks from ETSI TS 102
114. Returns `Some(reason)` when the DTS core-frame header is invalid — in
which case the packet is undecodable ("Invalid data found") and dropping it
loses nothing a decoder could have used. Returns `None` (keep) for a
decodable header OR if the header can't be fully read (never false-drop on
our own buffer underrun; the framer only emits AUs whose core is fully
buffered and ≥ 96 bytes).

The 4-byte core sync is already validated by the framer, so this reads the
header fields that follow it. The 16-bit CPF header CRC is not verified (we
skip past it) and the audio-header/side-info CRCs are likewise not checked,
because decoders treat those bytes as optional/ignored — verifying them
would drop frames that decode fine (false positives).

## Test: `stamp_pts_reuses_front_when_no_running_cursor_yet`

`stamp_pts`'s third arm (`else if front != PTS_UNSET`) is only reached once
the first arm's `front != self.last_front_pts` has already failed (i.e.
`front == self.last_front_pts`) and no within-PES running cursor is
available yet (`next_pts_ns == PTS_UNSET`). In that state the AU must still
be stamped with its own (repeated) front timestamp, not silently collapsed
to 0 — a `!=` -> `==` typo on the guard would only take this arm when
`front` IS the unset sentinel, producing 0 instead here and the sentinel
value itself if front really were unset.

## Test: `drain_front_collapses_offset_zero_markers_instead_of_leaking`

Every drained access unit that shares its PES with the previous one pushes
a new `(0, pts)` marker once rebased; `drain_front` must collapse those down
to the single most-recent one each time, or `pts_marks` grows once per
access unit for the life of the track — an unbounded allocation on a
multi-hour disc. `front_pts()` alone can't observe this: it always finds the
last offset-0 entry regardless of how many duplicates precede it, so the
leak is invisible unless something checks that the collapse actually ran.

## Test: `make_exss`

Builds a real DTS-HD EXSS substream of `total` bytes (short header form),
with an optional false DTS core syncword embedded in its payload (decoding
to a plausible core size) — to prove precise sizing, not a payload scan,
bounds the extension.

## Test: `make_dts_ext`

Builds a minimal DTS-HD extension substream of `size` bytes (just the sync
+ zero-padding). The parser delimits extensions by the next CORE sync, not
by the extension's own size header, so a valid header isn't required — only
that the bytes carry no spurious core sync.

## Test: `bogus_tiny_core_sync`

Builds 4 bytes that look like a DTS core sync but whose `fsize` field
decodes to a tiny `core_size` (< MIN_CORE_FRAME_BYTES). With the dead-code
guards this passed validation and could close an access unit at a junk
boundary; with the fix it must be drained and resynced past.

## Test: `next_core_boundary_exact_syncword_length_is_not_need_more`

Only the 4-byte EXSS syncword is buffered after the core frame — no header
fields at all. The guard `buf.len() < pos + SYNCWORD_BYTES` must be strict
`<`: at exactly `pos + SYNCWORD_BYTES` the sync bytes ARE fully present, so
the function proceeds to identify them as an extension sync (then fails to
size the header and falls back, ending in `NextCore::None` here since
nothing after it looks like a core sync either). A `<=` typo would instead
return `NeedMore` at this exact length without ever inspecting what the 4
buffered bytes are.

## Test: `make_bad_dts_core`

Builds a structurally-framed but UNDECODABLE core: a valid `make_dts_core`
whose LFE flag is set to the reserved value 3 (`DTS_LFE_FLAG_INVALID`). It
still sizes and syncs correctly (so the framer delimits it normally), but
the core-frame header validity check rejects it as an invalid LFE flag
(ETSI TS 102 114 §5.3.1: an LFF value of 3 is invalid). LFE is byte10
bits2-1, and does NOT feed the frame duration (NBLKS + SFREQ only), so a
dropped bad core still carries the same `DTS_CORE_DUR_NS` as its good peers.

## Test: `reparse_real_dts_file`

Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
through `DtsParser` and writes the emitted access units back out, so the
garbage-extension → core-only drop can be validated against an actual
damaged stream (e.g. an extracted DTS-HD MA track) end-to-end with an
external DTS decoder. Env: `DTS_IN` (input), `DTS_OUT` (output).

    cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture

## Test: `find_sync_matches_a_buffer_that_is_exactly_the_syncword`

`find_sync` scans `0..=len-4`, so a buffer that is EXACTLY the syncword
still matches. The existing tests cover 0, 3 and "longer than 4", which
leaves the `len == 4` boundary open — and that is the case a syncword split
across PES packets lands on the moment its last byte arrives.

## Test: `a_core_header_of_exactly_the_minimum_length_is_decoded_not_deferred`

`CORE_HEADER_MIN_BYTES` is "enough bytes to DECODE the size field", so a
buffer holding exactly that many must be decoded, not deferred. The
distinction is visible precisely at the boundary: with 10 bytes of a core
sync whose decoded size is sub-spec, the parser must recognise the false
sync, drain past it and resync down to the 3-byte carry-over tail. Waiting
instead leaves the false sync sitting at the front of the buffer, where it
blocks every later real core behind it.

## Test: `flush_emits_a_core_that_exactly_fills_the_buffer`

The end-of-stream flush must emit a final access unit whose core is exactly
as long as the buffer — the ordinary case, since the last AU is closed by
end-of-stream rather than by a following core sync. Rejecting it at the
boundary silently drops the last frame of every DTS track.

## Test: `flush_discards_a_long_buffer_that_does_not_begin_with_a_core_sync`

The flush guard is a DISJUNCTION: a buffer that does not BEGIN with a core
sync is discarded whatever its length. Requiring both conditions instead
lets a long run of junk through to `dts_core_frame_size`, which happily
decodes a 14-bit size out of arbitrary bytes — and the flush then emits an
"access unit" that is not DTS at all.

The junk here is sized so that mis-decoding it yields a plausible core size
that the buffer fully covers, which is exactly when the wrong answer looks
like a right one.

## Test: `exss_frame_size_needs_the_worst_case_header_and_no_more`

`EXSS_HEADER_MIN_BYTES` is the WORST-CASE header length — the long form's
43 bits after the syncword, rounded up to 6 bytes, plus the 4-byte sync. It
gates whether an extension substream can be sized precisely, and that is
what keeps a chance core syncword inside XLL payload from being mistaken
for the next AU boundary; too large and every extension falls back to the
payload scan, too small and the reader runs off a truncated header.

Pinned at both sides of the boundary rather than by value, so the field
widths it is summed from stay honest.

## Test: `an_access_unit_carries_the_source_of_the_packet_its_core_arrived_in`

The whole point of the shared buffer: an access unit whose core arrived in
an EARLIER packet keeps that packet's source offset, not the offset of
whichever packet completed it. At a clip boundary the two belong to
different clips, and taking the later one puts the unit in the wrong clip —
which is what left nine audio and subtitle tracks unplaceable.
</content>
</invoke>