Group whole E-AC-3 frame sets, and keep short reads unit-aligned
Two defects in fixes landed the same day, both found by round 4 auditing round 3's work rather than trusting it. **The E-AC-3 grouping ignored substreamid, so the timeline still doubled.** Confirmed independently by the correctness and conformance lenses and verified by hand: substreamid appeared only in test helpers, never in the production path. Per ETSI TS 102 366 (A/52) Annex E a frame set is independent substream 0 — mandatory, always first — with its dependents, then the OPTIONAL additional independent substreams 1..7 with theirs, all covering the same time period. Treating an additional independent substream as a new access unit advanced the clock a second time for the same 32 ms, which is exactly the doubling the grouping fix existed to prevent. No fixture caught it because every fixture used substreamid 0. is_dependent_substream becomes substream_role -> Starts | Extends: strmtyp 1 extends; strmtyp 0/2 with substreamid != 0 now extends (this was the bug); strmtyp 0/2 with substreamid == 0, legacy AC-3, and reserved strmtyp 3 start. Reserved 3 starts regardless of its id bits, because its BSI layout is undefined so those bits cannot be trusted — an unknown frame is neither merged into an unrelated programme nor silently discarded. The frame set stays ONE sample rather than being split into a separate track for the associated service, and the reasoning is in the module doc: a substream numbered 1..7 with no substream 0 is not conforming, so extracting one would mean renumbering ids and rebuilding frame sets — a transcode, not a remux. Programme selection is the player's job. A stream joined mid-frame-set (first sync is substreamid 3) is skipped with a debug and resyncs at the next id-0, mirroring the orphan-dependent rule: its mandatory id-0 substream was never seen, so it is neither decodable alone nor timeable. MAX_AC3_BUF 128 KiB -> 1 MiB, because an AU is now a whole frame set: worst case 8 independent x 9 substreams x 8192 B = 576 KiB, which the old cap could have dropped mid-hold. **The forced probe's two round-3 fixes cancelled each other.** CHUNK_SECTORS = 1023 exists (with a const assert) so every read starts on a 3-sector AACS aligned-unit boundary; the short-read fix advanced by actual bytes, making the advance a non-multiple of 3. Every later read was then misaligned, DecryptingSectorSource refused it before reading, the stop became ReadFailed, and no verdict was asserted — so content-based forced detection silently fell back to the vendor label on exactly the encrypted discs the 1023 change was written for. A partially-satisfied read now advances only by whole aligned units and re-reads the <=2 residue sectors from the next boundary, feeding only the aligned prefix so nothing is double-fed and no partial unit reaches the parsers. A read that fully satisfies its request still advances by all of it. When less than one aligned unit comes back the bytes are fed and the same LBA is retried twice before stopping, so a starved source cannot spin — verified by raising the retry limit and watching the test hang. Verified red independently here: reverting substream_role to strmtyp-only fails eac3_additional_independent_substream_stays_in_the_frame_set (6 access units where 3 are correct — the doubling, literally) and the mid-frame-set resync test. Reported, not fixed: dec3_box still hardcodes num_ind_sub - 1 = 0 and num_dep_sub = 0, so it under-declares any stream carrying additional independent or dependent substreams now that frame sets arrive whole. DolbyConfig has no fields for either; a real fix needs the parser to surface observed substream counts. That file is another lens's this round. Unverified: no real multi-programme DD+ stream exists here, so defect 1 rests on synthetic Annex-E fixtures. The retail DD+ check (No Time to Die, all substreamid 0) confirms single-programme discs are unaffected.
This commit is contained in:
+166
-19
@@ -46,6 +46,14 @@ const _: () = assert!(
|
|||||||
"probe chunks must be a whole number of AACS aligned units"
|
"probe chunks must be a whole number of AACS aligned units"
|
||||||
);
|
);
|
||||||
|
|
||||||
|
/// How many times a read that came back with less than one AACS aligned unit — so
|
||||||
|
/// the read position could not advance without leaving the unit grid — is retried
|
||||||
|
/// at the same LBA before the run is declared truncated. A couple of retries covers
|
||||||
|
/// a source whose batching straddles the request (a short call followed by a
|
||||||
|
/// satisfying one); a source that can never yield a whole unit must not spin, so
|
||||||
|
/// the count is small and the stop is `ReadFailed` (inconclusive, not memoised).
|
||||||
|
const STALL_RETRY_LIMIT: u32 = 2;
|
||||||
|
|
||||||
/// Hard ceiling on sectors read per probe call (256 MiB).
|
/// Hard ceiling on sectors read per probe call (256 MiB).
|
||||||
///
|
///
|
||||||
/// The probe's natural exit is "every track has shown a non-forced display set",
|
/// The probe's natural exit is "every track has shown a non-forced display set",
|
||||||
@@ -237,6 +245,9 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
|
|||||||
// `None` = this extent was read to its end, so its evidence is complete
|
// `None` = this extent was read to its end, so its evidence is complete
|
||||||
// and may be memoised. `Some(reason)` = the read stopped early.
|
// and may be memoised. `Some(reason)` = the read stopped early.
|
||||||
let mut cut_short: Option<StopReason> = None;
|
let mut cut_short: Option<StopReason> = None;
|
||||||
|
// Consecutive reads that came back with less than one AACS aligned unit, so
|
||||||
|
// the read position could not move (see the short-read handling below).
|
||||||
|
let mut stalled: u32 = 0;
|
||||||
while remaining > 0 {
|
while remaining > 0 {
|
||||||
// Bounded work and a responsive cancel: without these the probe
|
// Bounded work and a responsive cancel: without these the probe
|
||||||
// reads the entire title whenever a track really is forced.
|
// reads the entire title whenever a track really is forced.
|
||||||
@@ -266,15 +277,49 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
|
|||||||
// used to advance `lba`/`remaining`/`sectors_read` by the full
|
// used to advance `lba`/`remaining`/`sectors_read` by the full
|
||||||
// `count`, silently SKIPPING the unread tail of the chunk while
|
// `count`, silently SKIPPING the unread tail of the chunk while
|
||||||
// `stop` stayed `Exhausted` — so an absence-based forced verdict was
|
// `stop` stayed `Exhausted` — so an absence-based forced verdict was
|
||||||
// asserted (and memoised) over data that was never seen. The partial
|
// asserted (and memoised) over data that was never seen.
|
||||||
// trailing sector, if any, is left for the next read rather than fed
|
let served = (n.min(want) / SECTOR_BYTES) as u32;
|
||||||
// twice.
|
// ...but advancing by the raw sector count breaks the OTHER invariant
|
||||||
let got = (n.min(want) / SECTOR_BYTES) as u32;
|
// this loop rests on: with a decrypting source every read must BEGIN a
|
||||||
|
// whole number of AACS aligned units past `ext.start_lba`, which is why
|
||||||
|
// `CHUNK_SECTORS` is a multiple of `ALIGNED_UNIT_SECTORS` (see the
|
||||||
|
// const-assert above). A short read of, say, 64 sectors is not a whole
|
||||||
|
// number of units, so the next `lba` would be off the unit grid,
|
||||||
|
// `DecryptingSectorSource` would reject it with `DecryptFailed` before
|
||||||
|
// reading, and the run would end `ReadFailed` — inconclusive, nothing
|
||||||
|
// memoised, forced detection silently back to the vendor label on
|
||||||
|
// exactly the encrypted discs the aligned chunk size was chosen for.
|
||||||
|
//
|
||||||
|
// So a read that did NOT satisfy the whole request advances only by
|
||||||
|
// whole aligned units, and the residue sectors are simply RE-READ from
|
||||||
|
// the next unit boundary on the following pass: at most two sectors of
|
||||||
|
// duplicated drive work, no gap, no double-feed of any byte to the
|
||||||
|
// demuxer (only the aligned prefix is fed), and no trailing partial
|
||||||
|
// unit — whose plaintext a unit-anchored decrypt cannot produce anyway
|
||||||
|
// — is ever handed to the parsers.
|
||||||
|
//
|
||||||
|
// A read that satisfied the whole request advances by all of it even
|
||||||
|
// when `count` itself was not unit-aligned: `count` is only ever below
|
||||||
|
// `CHUNK_SECTORS` on the extent's final chunk or at the sector budget,
|
||||||
|
// and both end the loop before another read of this extent.
|
||||||
|
let got = if served >= u32::from(count) {
|
||||||
|
u32::from(count)
|
||||||
|
} else {
|
||||||
|
served - served % crate::aacs::content::ALIGNED_UNIT_SECTORS
|
||||||
|
};
|
||||||
if got == 0 {
|
if got == 0 {
|
||||||
// Less than one whole sector: the bytes are real, so feed them,
|
// Less than one whole aligned unit came back, so the read position
|
||||||
// but the loop cannot advance (re-reading the same partial sector
|
// cannot move: the next aligned boundary IS the one just read. The
|
||||||
// would feed it twice) — so this is the truncated prefix an error
|
// bytes are real, so feed them (never lose an observation), then
|
||||||
// is. A source that claims sectors and yields none is the same case.
|
// RETRY the same aligned lba — a source that short-changed one call
|
||||||
|
// commonly satisfies the next, and only when it repeatedly cannot
|
||||||
|
// yield a whole unit is the run the truncated prefix an error is.
|
||||||
|
// Bounded retries are what keep a source that never yields a unit
|
||||||
|
// (including one that claims sectors and returns none) from
|
||||||
|
// spinning here for ever. A retry that re-serves the same bytes
|
||||||
|
// feeds them twice; the evidence a [`ForcedTracker`] keeps is
|
||||||
|
// monotone (observed / saw-a-non-forced-set), so a repeat cannot
|
||||||
|
// change a verdict.
|
||||||
for pes in demux.feed(&buf[..n.min(want)]) {
|
for pes in demux.feed(&buf[..n.min(want)]) {
|
||||||
if let (Some(parser), Some(tracker)) =
|
if let (Some(parser), Some(tracker)) =
|
||||||
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
|
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
|
||||||
@@ -284,9 +329,21 @@ pub(crate) fn probe_and_set_forced<S: SectorSource + ?Sized>(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
stalled += 1;
|
||||||
|
if stalled > STALL_RETRY_LIMIT {
|
||||||
|
tracing::debug!(
|
||||||
|
target: "freemkv::scan",
|
||||||
|
lba,
|
||||||
|
requested = count,
|
||||||
|
served,
|
||||||
|
"forced-subtitle probe stalled below one aligned unit; stopping"
|
||||||
|
);
|
||||||
cut_short = Some(StopReason::ReadFailed);
|
cut_short = Some(StopReason::ReadFailed);
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
stalled = 0;
|
||||||
for pes in demux.feed(&buf[..got as usize * SECTOR_BYTES]) {
|
for pes in demux.feed(&buf[..got as usize * SECTOR_BYTES]) {
|
||||||
if let (Some(parser), Some(tracker)) =
|
if let (Some(parser), Some(tracker)) =
|
||||||
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
|
(parsers.get_mut(&pes.pid), trackers.get_mut(&pes.pid))
|
||||||
@@ -1089,11 +1146,11 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A source that serves only `frac` of the sectors requested, never erroring —
|
/// A source that never serves more than `batch` sectors per call, never
|
||||||
/// what `PrefetchedSectorSource` does (it returns its producer's batch, not
|
/// erroring — what `PrefetchedSectorSource` does (it returns its producer's
|
||||||
/// `count * 2048`). Records the LBAs it actually served.
|
/// batch, not `count * 2048`). Records what it actually served, per call.
|
||||||
struct ShortReader {
|
struct ShortReader {
|
||||||
frac: u32,
|
batch: u32,
|
||||||
served: Vec<(u32, u32)>,
|
served: Vec<(u32, u32)>,
|
||||||
}
|
}
|
||||||
impl SectorSource for ShortReader {
|
impl SectorSource for ShortReader {
|
||||||
@@ -1104,7 +1161,7 @@ mod tests {
|
|||||||
buf: &mut [u8],
|
buf: &mut [u8],
|
||||||
_recovery: bool,
|
_recovery: bool,
|
||||||
) -> crate::error::Result<usize> {
|
) -> crate::error::Result<usize> {
|
||||||
let give = (count as u32 / self.frac).max(1);
|
let give = u32::from(count).min(self.batch);
|
||||||
self.served.push((lba, give));
|
self.served.push((lba, give));
|
||||||
let n = give as usize * SECTOR_BYTES;
|
let n = give as usize * SECTOR_BYTES;
|
||||||
buf[..n].fill(0);
|
buf[..n].fill(0);
|
||||||
@@ -1124,7 +1181,7 @@ mod tests {
|
|||||||
let pid = 0x1200u16;
|
let pid = 0x1200u16;
|
||||||
let count = CHUNK_SECTORS as u32 * 2;
|
let count = CHUNK_SECTORS as u32 * 2;
|
||||||
let mut reader = ShortReader {
|
let mut reader = ShortReader {
|
||||||
frac: 4,
|
batch: 64,
|
||||||
served: Vec::new(),
|
served: Vec::new(),
|
||||||
};
|
};
|
||||||
let mut title = pgs_title(pid, true);
|
let mut title = pgs_title(pid, true);
|
||||||
@@ -1134,15 +1191,105 @@ mod tests {
|
|||||||
}];
|
}];
|
||||||
probe_and_set_forced(&mut reader, &mut title, &mut ForcedProbeCache::new(), None);
|
probe_and_set_forced(&mut reader, &mut title, &mut ForcedProbeCache::new(), None);
|
||||||
|
|
||||||
// The served ranges must tile the extent exactly: contiguous, no gaps.
|
// The served ranges must cover the extent with NO GAP. A residue re-read
|
||||||
let mut next = 0u32;
|
// (the sectors of a partial aligned unit, read again from the unit
|
||||||
|
// boundary) is allowed — what must never happen is an unread sector.
|
||||||
|
let mut covered = 0u32;
|
||||||
for &(lba, given) in &reader.served {
|
for &(lba, given) in &reader.served {
|
||||||
assert_eq!(lba, next, "gap: sectors {next}..{lba} were never read");
|
assert!(
|
||||||
next += given;
|
lba <= covered,
|
||||||
|
"gap: sectors {covered}..{lba} were never read"
|
||||||
|
);
|
||||||
|
covered = covered.max(lba + given);
|
||||||
}
|
}
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
next, count,
|
covered, count,
|
||||||
"every sector of the extent must be read when the source short-reads"
|
"every sector of the extent must be read when the source short-reads"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A short read must not break the aligned-unit invariant the chunk size
|
||||||
|
/// exists to hold. `CHUNK_SECTORS` is a multiple of `ALIGNED_UNIT_SECTORS` so
|
||||||
|
/// that every read BEGINS on an AACS aligned-unit boundary measured from the
|
||||||
|
/// extent base; a source that serves fewer sectors than requested (a 64-sector
|
||||||
|
/// prefetch batch: `64 % 3 == 1`) used to advance `lba` by that raw count, so
|
||||||
|
/// every subsequent read of the extent was off the unit grid.
|
||||||
|
/// `DecryptingSectorSource` rejects those before reading (`DecryptFailed`) →
|
||||||
|
/// `ReadFailed` → `absence_is_conclusive()` false → no verdict asserted and
|
||||||
|
/// nothing memoised, i.e. content-based forced detection silently degraded to
|
||||||
|
/// the vendor label on precisely the encrypted discs it was fixed for.
|
||||||
|
#[test]
|
||||||
|
fn short_reads_stay_on_aacs_unit_boundaries() {
|
||||||
|
let pid = 0x1200u16;
|
||||||
|
// A base that is NOT itself 3-aligned, so only the base-relative gate is
|
||||||
|
// satisfiable — absolute `lba % 3` would disagree.
|
||||||
|
let base = 4_001u32;
|
||||||
|
let count = CHUNK_SECTORS as u32 * 2;
|
||||||
|
let mut reader = ShortReader {
|
||||||
|
batch: 64,
|
||||||
|
served: Vec::new(),
|
||||||
|
};
|
||||||
|
let mut title = pgs_title(pid, true);
|
||||||
|
title.extents = vec![Extent {
|
||||||
|
start_lba: base,
|
||||||
|
sector_count: count,
|
||||||
|
}];
|
||||||
|
let mut cache = ForcedProbeCache::new();
|
||||||
|
probe_and_set_forced(&mut reader, &mut title, &mut cache, None);
|
||||||
|
|
||||||
|
assert!(reader.served.len() > 2, "several short reads happened");
|
||||||
|
for &(lba, _) in &reader.served {
|
||||||
|
assert!(
|
||||||
|
crate::aacs::content::is_unit_aligned(lba, base),
|
||||||
|
"read at lba {lba} is off the aligned-unit grid from base {base}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
// ...and the extent still gets read to its end, so the run reaches a
|
||||||
|
// designed stop and its (absence-based) evidence is memoisable.
|
||||||
|
let last = reader.served.last().copied().unwrap_or_default();
|
||||||
|
assert_eq!(
|
||||||
|
last.0 + last.1,
|
||||||
|
base + count,
|
||||||
|
"the extent must still be read to its end"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
cache.contains_key(&(base, count, pid)),
|
||||||
|
"a fully-read extent must be memoised, not discarded as inconclusive"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A source that can never yield a whole aligned unit cannot be advanced past
|
||||||
|
/// without leaving the unit grid — so the loop retries a bounded number of
|
||||||
|
/// times and then stops. It must NOT spin: the test simply completing is the
|
||||||
|
/// assertion, plus a bounded read count and an inconclusive (uncached,
|
||||||
|
/// vendor-flag-preserving) outcome.
|
||||||
|
#[test]
|
||||||
|
fn a_source_below_one_aligned_unit_stops_instead_of_spinning() {
|
||||||
|
let pid = 0x1200u16;
|
||||||
|
let mut reader = ShortReader {
|
||||||
|
batch: 1, // one sector: less than an aligned unit, for ever
|
||||||
|
served: Vec::new(),
|
||||||
|
};
|
||||||
|
let mut title = pgs_title(pid, true);
|
||||||
|
title.extents = vec![Extent {
|
||||||
|
start_lba: 0,
|
||||||
|
sector_count: CHUNK_SECTORS as u32,
|
||||||
|
}];
|
||||||
|
let mut cache = ForcedProbeCache::new();
|
||||||
|
probe_and_set_forced(&mut reader, &mut title, &mut cache, None);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
reader.served.len() as u32 <= STALL_RETRY_LIMIT + 1,
|
||||||
|
"a stalled source must be retried a bounded number of times, got {} reads",
|
||||||
|
reader.served.len()
|
||||||
|
);
|
||||||
|
for &(lba, _) in &reader.served {
|
||||||
|
assert_eq!(lba, 0, "a stalled read never advances off the unit grid");
|
||||||
|
}
|
||||||
|
let Stream::Subtitle(s) = &title.streams[0] else {
|
||||||
|
panic!()
|
||||||
|
};
|
||||||
|
assert!(s.forced, "inconclusive run keeps the vendor flag");
|
||||||
|
assert!(cache.is_empty(), "inconclusive run is not memoised");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+215
-56
@@ -2,13 +2,29 @@
|
|||||||
//!
|
//!
|
||||||
//! Every (E-)AC-3 syncframe starts with syncword 0x0B77. A legacy AC-3
|
//! Every (E-)AC-3 syncframe starts with syncword 0x0B77. A legacy AC-3
|
||||||
//! syncframe is a complete access unit on its own, but an E-AC-3 access unit
|
//! syncframe is a complete access unit on its own, but an E-AC-3 access unit
|
||||||
//! (ETSI TS 102 366 / ATSC A/52 Annex E) is an INDEPENDENT substream frame
|
//! (ETSI TS 102 366 / ATSC A/52 Annex E) is a whole FRAME SET: the mandatory
|
||||||
//! plus every DEPENDENT substream frame that follows it — a decoder needs the
|
//! independent substream (`substreamid` 0) plus every DEPENDENT substream frame
|
||||||
//! whole set to reconstruct the programme (a 5.1 independent substream plus a
|
//! that follows it, plus any ADDITIONAL independent substreams (`substreamid`
|
||||||
//! dependent substream carrying the extra channels of a 7.1 programme, and the
|
//! 1..7) and their own dependents. A decoder needs the whole set to reconstruct
|
||||||
//! AC-3-core + E-AC-3-dependent arrangement used for backwards-compatible
|
//! the programme (a 5.1 independent substream plus a dependent substream
|
||||||
//! Dolby Digital Plus). This parser therefore groups syncframes into access
|
//! carrying the extra channels of a 7.1 programme, and the AC-3-core +
|
||||||
//! units by `strmtyp` rather than emitting one frame per syncframe.
|
//! E-AC-3-dependent arrangement used for backwards-compatible Dolby Digital
|
||||||
|
//! Plus), and every substream of the frame set covers the SAME time period.
|
||||||
|
//! This parser therefore groups syncframes into access units at each
|
||||||
|
//! independent substream whose `substreamid` is 0, rather than emitting one
|
||||||
|
//! frame per syncframe or breaking at every independent substream.
|
||||||
|
//!
|
||||||
|
//! WHY the whole frame set is ONE sample, rather than splitting an additional
|
||||||
|
//! independent substream (an associated / commentary service) into a track of its
|
||||||
|
//! own: Annex E orders the substreams of a frame set inside a single elementary
|
||||||
|
//! stream, all covering one time period, and a decoder handed a frame set renders
|
||||||
|
//! the programme it is asked for from it — that is exactly what the container's
|
||||||
|
//! E-AC-3 sample and its `dec3`/`num_ind_sub` description denote. Splitting would
|
||||||
|
//! mean rewriting the bitstream (re-numbering `substreamid` so the extracted
|
||||||
|
//! service becomes substream 0, and rebuilding its frame sets), because a
|
||||||
|
//! substream numbered 1..7 with no substream 0 is not a conforming stream. That
|
||||||
|
//! is a transcode, not a remux; this parser is lossless, so the frame set stays
|
||||||
|
//! whole and player-side programme selection decides what is heard.
|
||||||
//!
|
//!
|
||||||
//! Buffers across PES boundaries so access units that span two PES packets
|
//! Buffers across PES boundaries so access units that span two PES packets
|
||||||
//! are emitted complete, not truncated or split.
|
//! are emitted complete, not truncated or split.
|
||||||
@@ -36,13 +52,14 @@ const MIN_FRAME_BYTES: usize = 6;
|
|||||||
const AC3_SAMPLES_PER_FRAME: u32 = 1536;
|
const AC3_SAMPLES_PER_FRAME: u32 = 1536;
|
||||||
|
|
||||||
/// Hard cap on the carry-over buffer. An AC-3/E-AC-3 syncframe is at most 8192
|
/// Hard cap on the carry-over buffer. An AC-3/E-AC-3 syncframe is at most 8192
|
||||||
/// bytes (the `frame_size > 8192` reject below) and an access unit is at most one
|
/// bytes (the `frame_size > 8192` reject below) and an access unit is a whole
|
||||||
/// independent substream plus 8 dependent substreams (ETSI TS 102 366 Annex E),
|
/// frame set: up to 8 independent substreams, each with up to 8 dependent
|
||||||
/// so a single straddling access unit plus slack never needs more than this. If
|
/// substreams (ETSI TS 102 366 Annex E) — 72 syncframes worst case. A cap of one
|
||||||
/// the buffer grows past the cap without yielding a frame (pathological /
|
/// straddling frame set plus slack therefore has to exceed 72 × 8192 = 576 KiB.
|
||||||
|
/// If the buffer grows past the cap without yielding a frame (pathological /
|
||||||
/// never-syncing input) we drop it and resync rather than accumulate one PES
|
/// never-syncing input) we drop it and resync rather than accumulate one PES
|
||||||
/// worth of data per call for the whole title.
|
/// worth of data per call for the whole title.
|
||||||
const MAX_AC3_BUF: usize = 128 * 1024;
|
const MAX_AC3_BUF: usize = 1024 * 1024;
|
||||||
|
|
||||||
pub struct Ac3Parser {
|
pub struct Ac3Parser {
|
||||||
/// Leftover bytes from previous PES (incomplete frame at end).
|
/// Leftover bytes from previous PES (incomplete frame at end).
|
||||||
@@ -56,13 +73,14 @@ pub struct Ac3Parser {
|
|||||||
/// the running PTS is advanced across it (see the emit loop) so the drop is a
|
/// the running PTS is advanced across it (see the emit loop) so the drop is a
|
||||||
/// silence gap, never a shift of the following audio.
|
/// silence gap, never a shift of the following audio.
|
||||||
tally: super::dropgate::DropTally,
|
tally: super::dropgate::DropTally,
|
||||||
/// Set once a dependent substream (E-AC-3 `strmtyp` == 1) has been seen on
|
/// Set once a syncframe that EXTENDS an access unit (an E-AC-3 dependent
|
||||||
/// this track. Until then a trailing LEGACY AC-3 syncframe is closed and
|
/// substream, or an additional independent substream with `substreamid` != 0)
|
||||||
/// emitted in-call (a plain AC-3 / DVD track has no substreams at all, so
|
/// has been seen on this track. Until then a trailing LEGACY AC-3 syncframe is
|
||||||
/// holding it back would only add latency); afterwards it is held open
|
/// closed and emitted in-call (a plain AC-3 / DVD track has no substreams at
|
||||||
|
/// all, so holding it back would only add latency); afterwards it is held open
|
||||||
/// across the PES boundary because it may be the core of an AC-3-core +
|
/// across the PES boundary because it may be the core of an AC-3-core +
|
||||||
/// E-AC-3-dependent access unit whose dependent half is in the next PES.
|
/// E-AC-3-dependent frame set whose remaining substreams are in the next PES.
|
||||||
saw_dependent: bool,
|
saw_extension: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Default for Ac3Parser {
|
impl Default for Ac3Parser {
|
||||||
@@ -77,7 +95,7 @@ impl Ac3Parser {
|
|||||||
buf: Vec::with_capacity(4096),
|
buf: Vec::with_capacity(4096),
|
||||||
flush_pts_ns: 0,
|
flush_pts_ns: 0,
|
||||||
tally: super::dropgate::DropTally::new("ac3"),
|
tally: super::dropgate::DropTally::new("ac3"),
|
||||||
saw_dependent: false,
|
saw_extension: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -93,13 +111,15 @@ impl Ac3Parser {
|
|||||||
|
|
||||||
/// Scan `data` for (E-)AC-3 syncframes and group them into access units.
|
/// Scan `data` for (E-)AC-3 syncframes and group them into access units.
|
||||||
///
|
///
|
||||||
/// Per ETSI TS 102 366 (ATSC A/52) Annex E an access unit is one INDEPENDENT
|
/// Per ETSI TS 102 366 (ATSC A/52) Annex E an access unit is one FRAME SET:
|
||||||
/// substream frame plus every DEPENDENT substream frame that follows it, up
|
/// the mandatory independent substream (`substreamid` 0) plus every DEPENDENT
|
||||||
/// to the next independent substream. The access unit therefore closes only
|
/// substream that follows it, plus any ADDITIONAL independent substreams
|
||||||
/// when the NEXT independent substream (or, with `at_eos`, the end of the
|
/// (`substreamid` 1..7 — associated/commentary services) with their own
|
||||||
/// stream) is seen, and it carries the INDEPENDENT substream's PTS and
|
/// dependents. The access unit therefore closes only when the next
|
||||||
/// duration: dependent substreams describe the same time period and add no
|
/// `substreamid`-0 independent substream (or, with `at_eos`, the end of the
|
||||||
/// duration of their own.
|
/// stream) is seen, and it carries THAT substream's PTS and duration: every
|
||||||
|
/// other substream of the frame set describes the SAME time period and adds no
|
||||||
|
/// duration of its own.
|
||||||
///
|
///
|
||||||
/// `base_pts_ns` times the access unit that begins at `data[0]` — i.e. the
|
/// `base_pts_ns` times the access unit that begins at `data[0]` — i.e. the
|
||||||
/// running cadence carried over from the previous call. `anchor` re-anchors
|
/// running cadence carried over from the previous call. `anchor` re-anchors
|
||||||
@@ -165,26 +185,35 @@ impl Ac3Parser {
|
|||||||
// vice versa, so the whole access unit is dropped as one silence gap.
|
// vice versa, so the whole access unit is dropped as one silence gap.
|
||||||
let reason = ac3_drop_reason(&self.tally, frame, bsid);
|
let reason = ac3_drop_reason(&self.tally, frame, bsid);
|
||||||
|
|
||||||
if is_dependent_substream(remaining, bsid) {
|
if substream_role(remaining, bsid) == SubstreamRole::Extends {
|
||||||
match pending.as_mut() {
|
match pending.as_mut() {
|
||||||
// A dependent substream extends the access unit it directly
|
// A dependent substream — or an additional independent
|
||||||
// follows. Requiring byte contiguity keeps skipped junk out of
|
// substream (`substreamid` 1..7) — extends the frame set it
|
||||||
// the emitted access unit.
|
// directly follows. Requiring byte contiguity keeps skipped
|
||||||
|
// junk out of the emitted access unit.
|
||||||
Some(au) if au.end == start => {
|
Some(au) if au.end == start => {
|
||||||
au.end = start + frame_size;
|
au.end = start + frame_size;
|
||||||
if au.drop_reason.is_none() {
|
if au.drop_reason.is_none() {
|
||||||
au.drop_reason = reason;
|
au.drop_reason = reason;
|
||||||
}
|
}
|
||||||
self.saw_dependent = true;
|
self.saw_extension = true;
|
||||||
}
|
}
|
||||||
// A dependent substream with no independent parent (a mid-AU
|
// A substream with no open access unit — a mid-frame-set
|
||||||
// resync, or a stream that starts inside an access unit) is not
|
// resync, or a stream whose first syncframe is not
|
||||||
// decodable on its own; skip it rather than ship a frame a
|
// `substreamid` 0 (Annex E orders a frame set independent
|
||||||
// decoder cannot use. No PTS advance: it carries no duration.
|
// substream 0 first, so joining mid-set is exactly the case
|
||||||
|
// here). It belongs to a frame set whose mandatory
|
||||||
|
// `substreamid`-0 substream was never seen, so it is neither
|
||||||
|
// decodable on its own (a dependent substream) nor timeable
|
||||||
|
// (an additional independent substream carries the frame set's
|
||||||
|
// time period, not its own PTS). Skip it rather than ship a
|
||||||
|
// fragment a decoder cannot use and re-time the whole
|
||||||
|
// timeline around; resync at the next `substreamid`-0
|
||||||
|
// substream. No PTS advance: it carries no duration.
|
||||||
_ => {
|
_ => {
|
||||||
tracing::debug!(
|
tracing::debug!(
|
||||||
target: "mux",
|
target: "mux",
|
||||||
"ac3: dependent substream with no independent parent; skipped"
|
"ac3: substream with no open access unit (frame set joined mid-set); skipped"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -210,24 +239,28 @@ impl Ac3Parser {
|
|||||||
drop_reason: reason,
|
drop_reason: reason,
|
||||||
bsid,
|
bsid,
|
||||||
});
|
});
|
||||||
// Only the independent substream advances the timeline.
|
// Only the frame set's `substreamid`-0 independent substream
|
||||||
|
// advances the timeline: every other substream of the set covers
|
||||||
|
// the same time period (Annex E).
|
||||||
frame_pts_ns += duration_ns as i64;
|
frame_pts_ns += duration_ns as i64;
|
||||||
}
|
}
|
||||||
|
|
||||||
pos = start + frame_size;
|
pos = start + frame_size;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Close or HOLD the trailing access unit. Its dependent substreams may
|
// Close or HOLD the trailing access unit. The rest of its frame set — its
|
||||||
|
// dependent substreams and any additional independent substreams — may
|
||||||
// still be in the next PES, so an access unit that can still grow is
|
// still be in the next PES, so an access unit that can still grow is
|
||||||
// held: the carry-over rewinds to its first byte and the whole access
|
// held: the carry-over rewinds to its first byte and the whole access
|
||||||
// unit is re-scanned (and only then counted/emitted) next call. An
|
// unit is re-scanned (and only then counted/emitted) next call. An
|
||||||
// E-AC-3 independent substream can always gain dependents; a legacy
|
// E-AC-3 `substreamid`-0 substream can always gain more of its frame set;
|
||||||
// AC-3 syncframe only in the AC-3-core + E-AC-3-dependent arrangement,
|
// a legacy AC-3 syncframe only in the AC-3-core + E-AC-3-dependent
|
||||||
// so it is held only once this track has actually shown a dependent
|
// arrangement, so it is held only once this track has actually shown a
|
||||||
// substream — a plain AC-3 track keeps emitting every frame in-call.
|
// substream that extends an access unit — a plain AC-3 track keeps
|
||||||
|
// emitting every frame in-call.
|
||||||
let mut hold_from = None;
|
let mut hold_from = None;
|
||||||
if let Some(au) = pending {
|
if let Some(au) = pending {
|
||||||
if !at_eos && (au.bsid >= 11 || self.saw_dependent) {
|
if !at_eos && (au.bsid >= 11 || self.saw_extension) {
|
||||||
frame_pts_ns = au.pts_ns;
|
frame_pts_ns = au.pts_ns;
|
||||||
hold_from = Some(au.start);
|
hold_from = Some(au.start);
|
||||||
} else {
|
} else {
|
||||||
@@ -274,18 +307,22 @@ struct PtsAnchor {
|
|||||||
pts_ns: i64,
|
pts_ns: i64,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// An access unit under construction: `data[start..end]` is an independent
|
/// An access unit (frame set) under construction: `data[start..end]` is the
|
||||||
/// substream frame plus the dependent substream frames appended so far.
|
/// `substreamid`-0 independent substream frame plus every substream appended to it
|
||||||
|
/// so far — its dependents, and any additional independent substreams 1..7 with
|
||||||
|
/// their own dependents.
|
||||||
struct PendingAu {
|
struct PendingAu {
|
||||||
start: usize,
|
start: usize,
|
||||||
end: usize,
|
end: usize,
|
||||||
/// PTS of the independent substream — the PTS the whole access unit carries.
|
/// PTS of the `substreamid`-0 independent substream — the PTS the whole frame
|
||||||
|
/// set carries.
|
||||||
pts_ns: i64,
|
pts_ns: i64,
|
||||||
/// Duration of the independent substream; dependent substreams add none.
|
/// Duration of the `substreamid`-0 independent substream; the frame set's other
|
||||||
|
/// substreams cover the same time period and add none.
|
||||||
duration_ns: u64,
|
duration_ns: u64,
|
||||||
/// First decodability failure among the access unit's substreams, if any.
|
/// First decodability failure among the access unit's substreams, if any.
|
||||||
drop_reason: Option<&'static str>,
|
drop_reason: Option<&'static str>,
|
||||||
/// bsid of the independent substream (< 11 = legacy AC-3 core).
|
/// bsid of the substream that opened the access unit (< 11 = legacy AC-3 core).
|
||||||
bsid: u8,
|
bsid: u8,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -315,20 +352,61 @@ fn close_access_unit(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether a syncframe is an E-AC-3 DEPENDENT substream, i.e. one that must be
|
/// What a syncframe does to the access unit (frame set) being assembled.
|
||||||
/// decoded together with the preceding independent substream.
|
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
|
||||||
|
enum SubstreamRole {
|
||||||
|
/// Begins a new access unit.
|
||||||
|
Starts,
|
||||||
|
/// Belongs to the access unit already open — it covers the same time period
|
||||||
|
/// and must not close it or advance the timeline.
|
||||||
|
Extends,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Classify a syncframe for access-unit assembly.
|
||||||
///
|
///
|
||||||
/// Byte 2 of an E-AC-3 syncframe is `strmtyp(2) | substreamid(3) | frmsiz[10:8]`
|
/// Byte 2 of an E-AC-3 syncframe is `strmtyp(2) | substreamid(3) | frmsiz[10:8]`
|
||||||
/// (ETSI TS 102 366 Annex E BSI). `strmtyp` 0 and 2 are independent substreams
|
/// (ETSI TS 102 366 Annex E BSI). `strmtyp` 0 and 2 are independent substreams
|
||||||
/// (type 2 being an independent substream that is not the first of the bit
|
/// (type 2 being an independent substream that is not the first of the bit
|
||||||
/// stream); `strmtyp` 1 is the dependent substream. `strmtyp` 3 is reserved and
|
/// stream); `strmtyp` 1 is the dependent substream.
|
||||||
/// is treated as independent, so an unknown type starts a fresh access unit
|
///
|
||||||
/// rather than being merged into an unrelated one.
|
/// Annex E defines a FRAME SET as independent substream 0 — mandatory, always
|
||||||
|
/// first — with its dependent substreams, followed by the OPTIONAL additional
|
||||||
|
/// independent substreams 1..7, each with their own dependents; all of them carry
|
||||||
|
/// the same time period. So the access-unit boundary is an independent substream
|
||||||
|
/// with `substreamid` == 0, and NOT merely "an independent substream": an
|
||||||
|
/// additional independent substream (an associated or commentary service) sits
|
||||||
|
/// INSIDE the frame set already open. Keying the boundary on `strmtyp` alone made
|
||||||
|
/// every such substream close the access unit and advance the running PTS a
|
||||||
|
/// second time over the same ~32 ms, doubling the timeline (about a second of A/V
|
||||||
|
/// drift per second of audio).
|
||||||
|
///
|
||||||
|
/// `strmtyp` 3 is reserved: its BSI layout is not defined, so its `substreamid`
|
||||||
|
/// bits cannot be trusted and it is treated as starting a fresh access unit —
|
||||||
|
/// an unknown frame is never merged into an unrelated programme, and never
|
||||||
|
/// discarded as an orphan either.
|
||||||
///
|
///
|
||||||
/// Legacy AC-3 (`bsid < 11`) has no substream structure — byte 2 there is the
|
/// Legacy AC-3 (`bsid < 11`) has no substream structure — byte 2 there is the
|
||||||
/// crc1 field, never `strmtyp` — so it is never dependent.
|
/// crc1 field, never `strmtyp` — so it always starts an access unit.
|
||||||
fn is_dependent_substream(data: &[u8], bsid: u8) -> bool {
|
///
|
||||||
bsid >= 11 && data.len() >= 3 && ((data[2] >> 6) & 0x03) == 1
|
/// A frame set MAY carry a substream as several consecutive syncframes of fewer
|
||||||
|
/// than six blocks each (Annex E allows numblkscod < 3). Those extra syncframes
|
||||||
|
/// are `substreamid` 0 too, so each starts its own access unit here rather than
|
||||||
|
/// merging into one frame set. That is deliberate: each carries its OWN
|
||||||
|
/// numblkscod-derived duration (see `frame_duration_ns`), so the timeline total
|
||||||
|
/// stays exact, and the substreams are still delivered in bitstream order.
|
||||||
|
fn substream_role(data: &[u8], bsid: u8) -> SubstreamRole {
|
||||||
|
if bsid < 11 || data.len() < 3 {
|
||||||
|
return SubstreamRole::Starts;
|
||||||
|
}
|
||||||
|
let strmtyp = (data[2] >> 6) & 0x03;
|
||||||
|
let substreamid = (data[2] >> 3) & 0x07;
|
||||||
|
match strmtyp {
|
||||||
|
// Dependent substream: always part of the open frame set.
|
||||||
|
1 => SubstreamRole::Extends,
|
||||||
|
// Independent substream: only id 0 begins a frame set.
|
||||||
|
0 | 2 if substreamid != 0 => SubstreamRole::Extends,
|
||||||
|
_ => SubstreamRole::Starts,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
use super::crc::crc16_ansi;
|
use super::crc::crc16_ansi;
|
||||||
@@ -1873,6 +1951,87 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn eac3_additional_independent_substream_stays_in_the_frame_set() {
|
||||||
|
// THE FIX: an Annex E frame set is independent substream 0 (mandatory,
|
||||||
|
// first) with its dependents, then the OPTIONAL additional independent
|
||||||
|
// substreams 1..7 with theirs — ALL covering the SAME time period. An
|
||||||
|
// additional independent substream (an associated / commentary service)
|
||||||
|
// therefore belongs to the frame set already open; keying the access-unit
|
||||||
|
// boundary on `strmtyp` alone made it close the AU and advance the running
|
||||||
|
// PTS a SECOND time over the same 32 ms, doubling the timeline (~1 s of
|
||||||
|
// A/V drift per second of audio).
|
||||||
|
let mut parser = Ac3Parser::new();
|
||||||
|
let ind0 = make_eac3_frame(0, 0, 160); // main programme
|
||||||
|
let dep0 = make_eac3_frame(1, 0, 96); // its dependent (7.1 extension)
|
||||||
|
let ind1 = make_eac3_frame(0, 1, 128); // associated service
|
||||||
|
let dep1 = make_eac3_frame(1, 1, 96); // its dependent
|
||||||
|
let mut set = ind0.clone();
|
||||||
|
set.extend_from_slice(&dep0);
|
||||||
|
set.extend_from_slice(&ind1);
|
||||||
|
set.extend_from_slice(&dep1);
|
||||||
|
|
||||||
|
// Three consecutive frame sets: the third closes the second, and the
|
||||||
|
// third itself is held for a possible continuation and drained by flush().
|
||||||
|
let mut data = Vec::new();
|
||||||
|
for _ in 0..3 {
|
||||||
|
data.extend_from_slice(&set);
|
||||||
|
}
|
||||||
|
let mut f = parser.parse(&make_eac3_pes(data));
|
||||||
|
f.extend(parser.flush());
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
f.len(),
|
||||||
|
3,
|
||||||
|
"one access unit per frame set — NOT one per independent substream"
|
||||||
|
);
|
||||||
|
let base = pts_to_ns(90000);
|
||||||
|
for (i, fr) in f.iter().enumerate() {
|
||||||
|
assert_eq!(
|
||||||
|
fr.data, set,
|
||||||
|
"frame set {i} emerges whole, all four substreams in bitstream order"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
fr.pts_ns,
|
||||||
|
base + i as i64 * 32_000_000,
|
||||||
|
"frame set {i} advances by ONE 32 ms period, not two"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
fr.duration_ns,
|
||||||
|
Some(32_000_000),
|
||||||
|
"the additional independent substream adds no duration"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assert_eq!(parser.dropped_frames(), 0, "nothing dropped");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn eac3_stream_joined_mid_frame_set_resyncs_at_substreamid_0() {
|
||||||
|
// A stream whose first syncframe is an ADDITIONAL independent substream
|
||||||
|
// (here substreamid 3) joined a frame set whose mandatory substreamid-0
|
||||||
|
// substream was never seen. Mirroring the orphan-dependent rule, it is
|
||||||
|
// skipped: it carries the frame set's time period rather than its own, so
|
||||||
|
// emitting it as an access unit would invent a period for audio whose
|
||||||
|
// main programme is missing. Grouping resyncs at the next substreamid-0
|
||||||
|
// substream, and the orphan consumes none of the timeline.
|
||||||
|
let mut parser = Ac3Parser::new();
|
||||||
|
let orphan = make_eac3_frame(0, 3, 128);
|
||||||
|
let orphan_dep = make_eac3_frame(1, 3, 96);
|
||||||
|
let ind0 = make_eac3_frame(0, 0, 160);
|
||||||
|
let mut data = orphan;
|
||||||
|
data.extend_from_slice(&orphan_dep);
|
||||||
|
data.extend_from_slice(&ind0);
|
||||||
|
let mut f = parser.parse(&make_eac3_pes(data));
|
||||||
|
f.extend(parser.flush());
|
||||||
|
assert_eq!(f.len(), 1, "only the frame set that has substreamid 0");
|
||||||
|
assert_eq!(f[0].data, ind0, "the orphan substreams are not shipped");
|
||||||
|
assert_eq!(
|
||||||
|
f[0].pts_ns,
|
||||||
|
pts_to_ns(90000),
|
||||||
|
"orphans consumed no time — the resynced audio keeps its true PTS"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn eac3_strmtyp2_starts_a_new_access_unit() {
|
fn eac3_strmtyp2_starts_a_new_access_unit() {
|
||||||
// strmtyp 2 is an INDEPENDENT substream (Annex E), and strmtyp 3 is
|
// strmtyp 2 is an INDEPENDENT substream (Annex E), and strmtyp 3 is
|
||||||
|
|||||||
Reference in New Issue
Block a user