End a label section where the next one's stream list begins

The pixelogic walk finds the feature playlist's section by name and ends
it at the next `SEG_`/`SF_`/`FPL_` marker. Those markers are section
NAMES, and a project's trailing sections — the per-language notice,
disclaimer and dub-credit cards — carry none. On 8 of the 11
affected-format discs in the corpus the feature playlist is the last
NAMED section in the blob, so the terminator never fires and the walk
consumes the whole tail of the file as more of the feature's stream
list.

The card names are `{lang3}_{card}`, which passes `is_stream_token`, so
each one advances an STN counter, and a card whose name collides with a
catalogued component emits a label outright. Measured on the worst disc:
95 entries past the end of a 9-audio/21-PG list, five phantom audio
labels at STN 10-14 from `*_AC` notice cards (`AC` reads as the AC-3
codec), and 94 uncatalogued-component occurrences — which also took the
parse from High to Medium confidence and fired the vocabulary-gap
warning on four components that are deliberately not catalogued. A
second disc fabricated one subtitle label from a token in a following
playlist section named `FP_SingAlong`, which `FPL_` does not match.

What every section has, named or not, is a stream list that opens with
its video slots. So a `Video Stream N` entry repeating one this section
already listed is the first entry of the NEXT section, and ends this
one. Distinct video entries are kept, since a section may legitimately
list a secondary video stream; the memo of them is bounded at the BD STN
table's ceiling so disc bytes cannot grow it.

Replaying all 11 blobs through `assign_labels` before and after: the two
discs above lose exactly their phantom labels (11→6 and 5→4), the other
nine are byte-identical.

One residue is pinned rather than papered over: a card's name precedes
its own section's video slot, so a forward-only walk can still count the
FIRST card after the last real slot. It sits at the tail of a list
nothing follows in, so it can renumber nothing — at worst it costs a
parse its High confidence.

No other parser in src/labels/ walks a flat entry sequence with a
terminator set; the rest scope each stream to a structural range or read
its number off the entry itself. paramount and criterion gain immunity
pins for the boundary property specifically: a stream list cannot run
into the next element's, and a missing element boundary shortens the
list rather than extending it.
This commit is contained in:
Matthew Jackson
2026-08-02 16:55:08 -07:00
parent d0d8e2c9bf
commit c94e9f4fb7
4 changed files with 272 additions and 2 deletions
+42
View File
@@ -276,6 +276,48 @@ mod tests {
///
/// Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
/// real audio streams renumber to 1 and 2.
/// Immunity pin, section-boundary half. Each stream here is one closed XML
/// element, and every field is read out of `&text[start..end]` — the range
/// `xml::find_element` returned — so one element can never absorb the next
/// one's fields, however the document is malformed around it. Contrast the
/// flat-string walk in pixelogic, where a section whose end marker is
/// missing keeps consuming entries as STN slots.
///
/// The missing-boundary case fails closed. An element with no close tag of
/// its own ends at the NEXT close tag, so it absorbs the element behind it
/// — the list comes back SHORTER. It cannot come back longer: nothing
/// outside a returned range is ever read as a stream, and `find_element`
/// yields `None` rather than a range running to EOF when no close tag
/// exists at all. A malformed document can cost this parser a slot; it can
/// never invent one.
///
/// Mutation: read fields from the document rather than the element's
/// range, or let a close-less element run to EOF → the trailing elements
/// re-enter the list as extra streams.
#[test]
fn an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it() {
let sp = concat!(
"<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID></AudioStreamInfos>",
// No `</AudioStreamInfos>` for this one.
"<AudioStreamInfos><ID>a1</ID><LangInfoID>FRA</LangInfoID>",
"<AudioStreamInfos><ID>a2</ID><LangInfoID>DEU</LangInfoID></AudioStreamInfos>",
);
let infos = parse_stream_infos(sp);
assert_eq!(
infos.iter().map(|i| i.id.as_str()).collect::<Vec<_>>(),
vec!["a0", "a1"],
"the close-less element absorbs the one behind it — two slots, not \
three, and never four"
);
assert_eq!(infos[1].language, "fra", "and keeps its own leading fields");
// With no close tag anywhere behind it, the element is not returned at
// all and the walk ends — the tail of the document never becomes a
// stream list.
let no_close = "<AudioStreamInfos><ID>a0</ID><LangInfoID>ENG</LangInfoID>";
assert!(parse_stream_infos(no_close).is_empty());
}
#[test]
fn unusable_stream_element_still_occupies_its_position() {
let sp = r#"