diff --git a/docs/FVI_FORMAT.md b/docs/FVI_FORMAT.md new file mode 100644 index 0000000..f15055e --- /dev/null +++ b/docs/FVI_FORMAT.md @@ -0,0 +1,290 @@ +# FVI — Freemkv Video Index Format + +**Specification version:** 1.0 (DRAFT) +**File extension:** `.fvi` **Media type:** `application/vnd.freemkv.fvi+jsonl` +**Status:** Draft for review. This document is the normative reference for the FVI +format; implementations and downstream tools cite it by section. + +--- + +## 1. Scope and purpose + +FVI is an open, codec-agnostic, byte-exact **index of the coded pictures** in a +video bitstream, together with **provenance** back to the source medium. + +An FVI document answers, for every picture in a stream, three questions: + +1. **Where is it?** — the byte-exact offset of its first byte in the *source* + (the disc/ISO/file), so a reader can extract or seek to any picture without + re-parsing the whole bitstream. +2. **What is it?** — coding type, random-access capability, GOP boundary, and + (where the codec defines them) field/pulldown attributes. +3. **When is it?** — decode and presentation timestamps on a declared timescale. + +FVI is **not** a container, a codec, or a copy of the bitstream. It indexes; it +never stores coded samples. It is the serialized form of an indexer's per-picture +truth — carried from the demuxer, **never reconstructed** (§9). + +### 1.1 Relationship to prior art + +Legacy MPEG-only project-index formats from the AviSynth frameserving ecosystem +solve a narrow version of (1) and (2) for MPEG-1/2 only, in a bespoke, +single-tool text encoding. FVI generalizes that idea: codec-agnostic, JSON-based, +provenance-native, and openly specified so any tool may read or write it. + +## 2. Conformance + +The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, +**SHOULD**, **SHOULD NOT**, **MAY**, and **OPTIONAL** are to be interpreted as +described in BCP 14 (RFC 2119, RFC 8174) when, and only when, they appear in all +capitals. + +A **conformant writer** MUST emit a document that satisfies §4–§10. A +**conformant reader** MUST accept any such document and MUST ignore unknown +object members (§11) so that forward-compatible extensions do not break it. + +## 3. Terminology + +- **Picture** — one coded video frame (or pair of fields coded as a frame). The + unit FVI indexes. +- **Access unit (AU)** — the set of bitstream bytes that decode to exactly one + picture (ISO/IEC 14496-10 §3; ISO/IEC 23008-2 §3). +- **Coded order** — the order pictures appear in the bitstream. FVI records are + emitted in coded order. +- **GOP / coded video sequence** — a self-contained run beginning at a + random-access point. +- **Provenance** — the mapping from an AU back to the exact bytes of the physical + source it was read from (§9). +- **Source position (`src`)** — `{ file, sector, byte }`, the provenance anchor of + an AU. + +## 4. Encoding + +An FVI document is a sequence of **UTF-8** text lines separated by a single LF +(`U+000A`). Each non-empty line is exactly one JSON value (RFC 8259), forming a +**JSON Lines / NDJSON** stream. A writer MUST NOT emit a UTF-8 BOM. A writer MUST +NOT pretty-print: each JSON value occupies exactly one line. + +The first line MUST be the **Header** object (§6). Each subsequent line is one +**Picture record** (§7), in coded order. + +Rationale: line-delimited JSON is streamable (a writer appends as it indexes; a +reader processes without loading the whole file), line-addressable (picture *n* +is near line *n+1*), append-safe, and parseable by every language without a +custom grammar — while remaining a precisely specified format, not an ad-hoc dump. + +A document MAY be concatenated for multiple elementary streams: each stream is its +own header line followed by its records. Readers MUST treat a Header line as the +start of a new stream section. + +## 5. Document structure + +``` +
line 1 (exactly one Header object) + line 2 .. N (one Picture record per picture, coded order) +[
…] (OPTIONAL further stream sections) +``` + +## 6. Header object + +| Member | JSON type | Req | Semantics / reference | +|---|---|---|---| +| `format` | string | MUST | Constant `"freemkv/video-index"`. Signature: a document begins with these bytes. | +| `fvi_version` | integer | MUST | Document format version. This spec defines `1`. | +| `generator` | string | SHOULD | Producing tool + version, e.g. `"freemkv/1.0.0-rc.6"`. | +| `stream` | object | MUST | The indexed elementary stream (§6.1). | +| `source` | object | MUST | Provenance root (§6.2). | +| `timescale` | integer | MUST | Ticks per second for all `pts`/`dts` (§10). E.g. `90000`. | +| `picture_count` | integer | MAY | Total pictures, if known at header time; OMITTED when streaming. | + +### 6.1 `stream` object + +| Member | JSON type | Req | Semantics / reference | +|---|---|---|---| +| `codec` | string | MUST | Registered codec id (Appendix B), e.g. `"mpeg2video"`, `"hevc"`. | +| `width`,`height` | integer | MUST | Coded luma dimensions in pixels. | +| `dar` | `[int,int]` | SHOULD | Display aspect ratio as `[num,den]`. | +| `frame_rate` | `[int,int]` | SHOULD | Nominal rate as exact rational `[num,den]` (e.g. `[24000,1001]`). | +| `scan` | string | MUST | `"progressive"` \| `"interlaced"` \| `"mbaff"`. | +| `colour` | object | SHOULD | CICP per ITU-T H.273: `primaries`,`transfer`,`matrix` (integer CICP codes or registered names), `range` (`"limited"`\|`"full"`). HDR: `mastering_display`, `max_cll`, `max_fall` per ITU-T H.273 / SMPTE ST 2086. | +| `language` | string | MAY | BCP 47 tag, if known. | + +### 6.2 `source` object + +| Member | JSON type | Req | Semantics | +|---|---|---|---| +| `medium` | string | MUST | `"disc"` \| `"iso"` \| `"file"` \| `"stream"`. | +| `path` | string | MAY | Source path/label. | +| `title` | integer | MAY | Title/program number. | +| `playlist` | string | MAY | Playlist/PGC identifier. | +| `volume_id` | string | MAY | Disc volume identifier, if read. | +| `sector_size` | integer | SHOULD | Bytes per `src.sector` unit (e.g. `2048`). Lets readers convert `src` to an absolute byte offset. | + +## 7. Picture record + +One JSON object per coded picture, in coded order. + +| Member | JSON type | Req | Semantics / reference | +|---|---|---|---| +| `n` | integer | MUST | Coded-order index, 0-based, contiguous. | +| `src` | object | MUST | Provenance: `{ "file": int?, "sector": uint, "byte": uint }` — the offset of this AU's **first byte** in the source (§9). MUST be carried from demux, never reconstructed. | +| `type` | string | MUST | Coding type: `"I"` \| `"P"` \| `"B"` (ISO/IEC 13818-2 §6.3.9; H.264/H.265 slice types collapsed to frame type). | +| `key` | boolean | MUST | `true` iff this picture is an intra (I) picture / parser-flagged decode-restart point (IDR / IRAP / I-picture). MPEG-2 open-GOP clean-RAP precision (`closed_gop`) is not currently distinguished — see note below. | +| `gop` | boolean | SHOULD | `true` iff this picture begins a GOP / coded video sequence. Omitted when the implementation does not carry a distinct GOP-boundary signal. | +| `pts` | integer\|null | SHOULD | Presentation timestamp in `timescale` ticks; `null` if unknown. | +| `dts` | integer\|null | MAY | Decode timestamp in `timescale` ticks. | +| `size` | integer | MAY | AU length in bytes; enables byte-range extraction with `src`. | +| `recovered` | boolean | MAY | `true` iff any byte of this AU came from a retried/marginal read (§9.1). Default `false`. | +| codec ext | object | MAY | Codec-specific members under the codec's namespace (§8). | + +The `type` and `key` members are **codec-agnostic** and MUST be populated for +every codec. `type` is the I/P/B coding type the parser decoded (collapsing +H.264/H.265 slice types to a frame type); where no per-picture coding is carried +(audio / synthetic frames), `type` is `"I"` for a key picture else `"P"`. `key` +is the picture's random-access flag as the codec parser sets it (IDR / IRAP / +I-picture). A writer MUST NOT emit a degraded record (`type:"?"` or `src:null`) +merely because a codec lacks per-picture coding info — those fallbacks are +reserved for a field that is genuinely unavailable (e.g. provenance absent on a +synthetic source). + +> **Limitation (honest random-access).** `key` is set from the picture's +> intra / decode-restart flag. The per-picture coding model this index carries +> does **not** distinguish MPEG-2 open-GOP clean random-access points +> (`closed_gop`) from any other I-picture, so `key` is the parser-flagged +> decode-restart point, not a verified clean-RAP claim. A future revision MAY +> tighten `key` for codecs/profiles that carry that signal; readers MUST NOT +> assume present `key` precision beyond "intra / decode-restart point". + +### 7.1 Interlace / pulldown fields + +Codec-agnostic interlace/pulldown attributes, derived through the indexer's +per-picture coding accessors (MPEG-2: ISO/IEC 13818-2 §6.3.10). Emitted as +top-level members of the record, and ONLY when the codec actually measured the +signal — an OPTIONAL member that is omitted (not defaulted) when unknown: + +| Member | JSON type | Req | Semantics | +|---|---|---|---| +| `field_order` | string | MAY | Display field order: `"tff"` (top field first) \| `"bff"` (bottom field first) \| `"progressive"` (no field order applies). Omitted when the codec did not signal it. | +| `progressive` | boolean | MAY | `true` iff the picture is progressive. Omitted when the codec did not signal it. | +| `nb_fields` | integer | MAY | Number of displayed field periods this picture occupies (the soft-telecine / 2:3 pulldown basis): `1` for a single field picture, `2` for a normal frame, `3`/`4`/`6` for `repeat_first_field` pulldown per §6.3.10. | + +Codecs that carry only a coding type (e.g. H.264 / HEVC / VC-1 through this +pipeline) omit `field_order` and `progressive` rather than guessing a default. + +## 8. Codec model and extensibility + +Core record members (§7) are codec-agnostic and present for every codec. +Codec-specific data is either (a) promoted to top-level members for a small, +registered set per codec profile (e.g. MPEG-2 §7.1), or (b) placed under an +`ext` object keyed by codec id for richer/optional data: + +```json +{"n":42,"type":"P","key":false,"src":{"sector":17,"byte":924}, + "ext":{"hevc":{"temporal_id":0,"nal_type":1}}} +``` + +New codecs and members are added through Appendix B (codec registry) without a +breaking version bump, provided readers continue to ignore unknown members (§11). + +## 9. Provenance and recovery semantics + +`src` is **byte-exact** to the source as read. `src.sector` counts in +`source.sector_size`-byte units; `src.byte` is the offset within that sector of +the AU's first byte. For multi-file sources, `src.file` indexes a writer-declared +file list. Provenance MUST be the value observed at demux time; an implementation +MUST NOT recompute `src` by re-parsing — the point of FVI is to *carry* the truth. + +### 9.1 Recovery + +Because FVI is provenance-native, it can record reliability. A record with +`"recovered":true` indicates the AU's source bytes required retry/marginal-read +recovery. This lets downstream tools surface or quarantine pictures whose bytes +are not byte-identical to a clean read — a capability legacy index formats lack. + +## 10. Time model + +All `pts`/`dts` are integers in units of `1/timescale` seconds. `pts` is +presentation (display) time; `dts` is decode time. Records are in **coded** +(decode) order, so `pts` is not necessarily monotonic across records (B-pictures +reorder); `dts` is non-decreasing. Readers needing display order sort by `pts`. + +## 11. Versioning and forward compatibility + +- `fvi_version` is the document version; this spec defines `1`. +- **Additive** changes (new OPTIONAL members, new registered codecs) do NOT bump + `fvi_version`. Readers MUST ignore members they do not recognize. +- A change that alters the meaning of an existing member or makes a new member + REQUIRED bumps `fvi_version`. +- A reader encountering a higher `fvi_version` than it implements SHOULD process + the members it understands and MUST NOT reject the document solely for the + version being higher, unless a member it relies on is absent. + +## 12. Conformance requirements (summary) + +A conformant **writer** MUST: emit a Header first; emit records in coded order +with contiguous `n`; populate `src` from demux; use named/registered codec ids; +encode one JSON value per UTF-8 LF-terminated line. + +A conformant **reader** MUST: accept any §4–§10 document; ignore unknown members; +not assume `picture_count`, `pts`, or `size` are present unless required above. + +--- + +## Appendix A — JSON Schema (informative) + +Header: + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "type": "object", + "required": ["format","fvi_version","stream","source","timescale"], + "properties": { + "format": {"const": "freemkv/video-index"}, + "fvi_version": {"type": "integer", "minimum": 1}, + "timescale": {"type": "integer", "minimum": 1}, + "stream": {"type": "object", "required": ["codec","width","height","scan"]}, + "source": {"type": "object", "required": ["medium"]} + } +} +``` + +Record: + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "type": "object", + "required": ["n","src","type","key"], + "properties": { + "n": {"type": "integer", "minimum": 0}, + "type": {"enum": ["I","P","B"]}, + "key": {"type": "boolean"}, + "src": {"type":"object","required":["sector","byte"], + "properties":{"file":{"type":"integer"}, + "sector":{"type":"integer","minimum":0}, + "byte":{"type":"integer","minimum":0}}} + } +} +``` + +## Appendix B — Registered codec identifiers + +| `codec` | Bitstream | Field profile | +|---|---|---| +| `mpeg2video` | ISO/IEC 13818-2 | §7.1 (field_order/progressive/nb_fields) | +| `mpeg1video` | ISO/IEC 11172-2 | §7.1 | +| `h264` | ISO/IEC 14496-10 | core + `ext.h264` | +| `hevc` | ISO/IEC 23008-2 | core + `ext.hevc` | +| `vc1` | SMPTE 421M | core | + +## Appendix C — Normative references + +- RFC 2119, RFC 8174 — Requirement keywords (BCP 14). +- RFC 8259 — JSON. +- ISO/IEC 13818-2 — MPEG-2 video (picture coding, §6.3.9–6.3.10). +- ISO/IEC 14496-10 — H.264/AVC. ISO/IEC 23008-2 — H.265/HEVC. +- ITU-T H.273 — Coding-independent code points (colour primaries/transfer/matrix). +- SMPTE ST 2086 — Mastering display colour volume (HDR). +- BCP 47 — Language tags. +- RFC 9559 — Matroska (alignment of colour/field-order semantics). diff --git a/src/mux/fvi_sink.rs b/src/mux/fvi_sink.rs new file mode 100644 index 0000000..1c1ed30 --- /dev/null +++ b/src/mux/fvi_sink.rs @@ -0,0 +1,460 @@ +//! `fvi://` sink — freemkv's own native video-index output. +//! +//! This is a write-only [`crate::pes::Stream`] that, instead of muxing frames +//! into a container, emits one machine-readable *video-index* record per coded +//! picture of the title's primary video track. It is a thin consumer of the +//! reusable, pure-data [`VideoMap`](crate::mux::videomap) model +//! ([`MapHeader`]/[`PictureRecord`]): the sink builds the header from the title, +//! then writes one record per video [`PesFrame`] straight to disk — nothing here +//! re-parses the elementary stream, and the whole index is never buffered. +//! +//! The on-disk shape is the freemkv FVI format (normative public spec +//! `docs/FVI_FORMAT.md`): JSON Lines — a header object on line 1, then one +//! record object per picture. Serialization is inlined here in the sink. +//! +//! A different output format would be a DIFFERENT sink reusing the same +//! [`VideoMap`](crate::mux::videomap) model (e.g. a future `fvi2://`), not a +//! pluggable encoder — extensibility is by adding a sink, like every other +//! sink in this crate. +//! +//! The sink is purely additive — it does NOT touch the MKV mux path. + +use crate::disc::{DiscTitle, Stream as DiscStream}; +use crate::mux::videomap::{ + FVI_FORMAT, FVI_GENERATOR, FVI_SECTOR_SIZE, FVI_TIMESCALE, FVI_VERSION, MapHeader, + PictureRecord, SourceInfo, field_order_label, is_random_access, type_label, +}; +use crate::pes::{PesFrame, Stream}; +use std::fs::File; +use std::io::{self, BufWriter, Write}; +use std::path::Path; + +/// Write the FVI header row (JSON Lines, `docs/FVI_FORMAT.md` §6) into `w`. +fn write_fvi_header(w: &mut dyn Write, h: &MapHeader) -> io::Result<()> { + let mut source = serde_json::json!({ + "medium": h.source.medium.as_str(), + "path": h.source.path, + "title": h.source.title, + "sector_size": FVI_SECTOR_SIZE, + }); + // playlist / volume_id are MAY — emit only when known. + if !h.source.playlist.is_empty() { + source["playlist"] = serde_json::Value::String(h.source.playlist.clone()); + } + if !h.source.volume_id.is_empty() { + source["volume_id"] = serde_json::Value::String(h.source.volume_id.clone()); + } + + let mut obj = serde_json::json!({ + "format": FVI_FORMAT, + "fvi_version": FVI_VERSION, + "generator": FVI_GENERATOR, + "stream": { + "codec": h.stream.codec, + "width": h.stream.width, + "height": h.stream.height, + "dar": [h.stream.dar.0, h.stream.dar.1], + "frame_rate": [h.stream.frame_rate.0, h.stream.frame_rate.1], + "scan": h.stream.scan.as_str(), + "colour": { + "primaries": h.stream.colour.primaries, + "transfer": h.stream.colour.transfer, + "matrix": h.stream.colour.matrix, + "range": if h.stream.colour.full_range { "full" } else { "limited" }, + }, + }, + "source": source, + "timescale": FVI_TIMESCALE, + }); + // picture_count is MAY — omitted when streaming (unknown at header time). + if let Some(pc) = h.picture_count { + obj["picture_count"] = serde_json::json!(pc); + } + serde_json::to_writer(&mut *w, &obj)?; + w.write_all(b"\n") +} + +/// Write one FVI per-picture record (JSON Lines, `docs/FVI_FORMAT.md` §7) into +/// `w`. `type`/`key` are codec-agnostic and always emitted; the coding-derived +/// members (`field_order`, `progressive`, `nb_fields`) are emitted ONLY when the +/// codec actually measured them — an honest absence, never a guessed default. +fn write_fvi_record(w: &mut dyn Write, r: &PictureRecord) -> io::Result<()> { + // `src` is REQUIRED by the record schema (Appendix A); when provenance is + // absent the member is still emitted as null — a reader treats null as + // "position unknown". + let src = match r.source { + Some(s) => serde_json::json!({ "sector": s.sector, "byte": s.byte }), + None => serde_json::Value::Null, + }; + + let mut obj = serde_json::json!({ + "n": r.n, + "src": src, + "type": type_label(r.coding, r.keyframe), + "key": is_random_access(r.coding, r.keyframe), + }); + + // pts is SHOULD — emit when present. + if let Some(pts) = r.pts_ns { + obj["pts"] = serde_json::json!(pts); + } + // dts is MAY — the highway carries no DTS on a frame, so it is omitted. + // TODO(provenance→recovery join): a `recovered` MAY member belongs here, + // sourced from the sweep/patch mapfile's bad-range overlap with this AU's + // `src`. Not reachable at the PesFrame today; omitted (spec MAY). + + // Coding-derived members (§7.1), emitted as top-level members for ANY frame + // whose parser decoded the signal — derived through the codec-agnostic + // `PictureInfo` accessors, never the raw bitstream: + // - `field_order` only when measured (TFF/BFF/Progressive); OMITTED on a + // codec-type-only codec (HEVC/H.264/VC-1) — honest absence. + // - `progressive` only when the codec signalled it (Option). + // - `nb_fields` (displayed field periods, soft-telecine basis) when coding + // is present. + if let Some(c) = r.coding { + if let Some(fo) = field_order_label(r.coding) { + obj["field_order"] = serde_json::json!(fo); + } + if let Some(prog) = c.progressive() { + obj["progressive"] = serde_json::json!(prog); + } + obj["nb_fields"] = serde_json::json!(c.nb_fields()); + } + + serde_json::to_writer(&mut *w, &obj)?; + w.write_all(b"\n") +} + +/// `fvi://` sink: streams the title's primary-video per-picture index to a +/// `.fvi` (or `.jsonl` / `.json`) file as JSON Lines. +pub struct FviSink { + title: DiscTitle, + /// Index of the title's primary video track — only frames on this track are + /// indexed; audio / subtitle / secondary-video frames are ignored. + video_track: Option, + /// The sink owns the destination file. + w: BufWriter, + /// The header row, written lazily on the first `write`/`finish` so an + /// empty / audio-only title still emits a valid single-line file. + header: MapHeader, + /// 0-based picture counter (the record `n`), incremented per indexed frame. + next_n: u64, + header_written: bool, + finished: bool, +} + +impl FviSink { + /// Create the sink at `path`, assembling the header from `title`'s primary + /// video stream. + /// + /// `source_path` / `source_title` record where the index was built from + /// (the input URL path + the 0-based title index); they are carried into the + /// header's `source` object. The medium defaults to `file` — callers with a + /// known medium / playlist / volume use [`FviSink::create_with_source`]. + pub fn create( + path: &Path, + title: &DiscTitle, + source_path: String, + source_title: usize, + ) -> io::Result { + Self::create_with_source( + path, + title, + SourceInfo { + path: source_path, + title: source_title, + ..SourceInfo::default() + }, + ) + } + + /// Create the sink with a fully-specified [`SourceInfo`] provenance root. + pub fn create_with_source( + path: &Path, + title: &DiscTitle, + source: SourceInfo, + ) -> io::Result { + let file = File::create(path)?; + + let video_track = title + .streams + .iter() + .position(|s| matches!(s, DiscStream::Video(_))); + let header = MapHeader::from_title(title, source); + + Ok(Self { + title: title.clone(), + video_track, + w: BufWriter::new(file), + header, + next_n: 0, + header_written: false, + finished: false, + }) + } + + /// Write the header row once, lazily. + fn ensure_header(&mut self) -> io::Result<()> { + if self.header_written { + return Ok(()); + } + write_fvi_header(&mut self.w, &self.header)?; + self.header_written = true; + Ok(()) + } +} + +impl Stream for FviSink { + fn read(&mut self) -> io::Result> { + // Write-only sink, per the Stream trait contract. + Err(crate::error::Error::StreamWriteOnly.into()) + } + + fn write(&mut self, frame: &PesFrame) -> io::Result<()> { + // Only index pictures of the primary video track. Audio / subtitle / + // secondary-video frames carry no PictureInfo and are not part of the + // video index. + if Some(frame.track) != self.video_track { + return Ok(()); + } + self.ensure_header()?; + let rec = PictureRecord { + n: self.next_n, + coding: frame.coding, + keyframe: frame.keyframe, + pts_ns: Some(frame.pts), + source: frame.source, + }; + write_fvi_record(&mut self.w, &rec)?; + self.next_n += 1; + Ok(()) + } + + fn finish(&mut self) -> io::Result<()> { + if self.finished { + return Ok(()); + } + self.finished = true; + // Emit the header even for a title that produced no records, so the + // output is always a valid (if record-less) `.fvi` file. JSON Lines has + // no footer. + self.ensure_header()?; + self.w.flush() + } + + fn info(&self) -> &DiscTitle { + &self.title + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::disc::{ + Codec, ColorSpace, ContentFormat, FrameRate, HdrFormat, Resolution, VideoStream, + }; + use crate::mux::codec::PictureInfo; + use crate::mux::codec::coding::{CodingType, Mpeg2Coding}; + use crate::pes::SourcePos; + use std::path::PathBuf; + + /// Tiny unique temp dir helper (avoids a dev-dependency on `tempfile`). + fn tempdir() -> PathBuf { + use std::sync::atomic::{AtomicU64, Ordering}; + static N: AtomicU64 = AtomicU64::new(0); + let n = N.fetch_add(1, Ordering::Relaxed); + let p = std::env::temp_dir().join(format!("fmkv_fvi_test_{}_{}", std::process::id(), n)); + std::fs::create_dir_all(&p).unwrap(); + p + } + + fn mpeg2_title() -> DiscTitle { + let mut t = DiscTitle::empty(); + t.streams = vec![DiscStream::Video(VideoStream { + pid: 0x1011, + codec: Codec::Mpeg2, + resolution: Resolution::R480i, + frame_rate: FrameRate::F29_97, + hdr: HdrFormat::Sdr, + color_space: ColorSpace::Smpte170m, + display_aspect: Some((16, 9)), + secondary: false, + label: String::new(), + measured_cicp: None, + })]; + t.content_format = ContentFormat::MpegPs; + t + } + + fn hevc_title() -> DiscTitle { + let mut t = DiscTitle::empty(); + t.streams = vec![DiscStream::Video(VideoStream { + pid: 0x1011, + codec: Codec::Hevc, + resolution: Resolution::R2160p, + frame_rate: FrameRate::F23_976, + hdr: HdrFormat::Sdr, + color_space: ColorSpace::Bt2020, + display_aspect: None, + secondary: false, + label: String::new(), + measured_cicp: None, + })]; + t.content_format = ContentFormat::BdTs; + t + } + + fn i_pic() -> PictureInfo { + // Interlaced (tff) MPEG-2 I-frame picture. + PictureInfo::mpeg2( + CodingType::I, + Mpeg2Coding { + top_field_first: true, + repeat_first_field: false, + progressive_frame: false, + progressive_sequence: false, + frame_picture: true, + }, + ) + } + + fn vframe(track: usize, coding: Option, source: Option) -> PesFrame { + let keyframe = coding.map(|c| c.keyframe()).unwrap_or(false); + vframe_kf(track, coding, keyframe, source) + } + + fn vframe_kf( + track: usize, + coding: Option, + keyframe: bool, + source: Option, + ) -> PesFrame { + PesFrame { + track, + pts: 0, + keyframe, + data: vec![0u8; 4], + duration_ns: None, + source, + coding, + } + } + + #[test] + fn sink_is_write_only() { + let dir = tempdir(); + let mut sink = + FviSink::create(&dir.join("x.fvi"), &mpeg2_title(), String::new(), 0).unwrap(); + let err = Stream::read(&mut sink).expect_err("read must error"); + assert_eq!(err.kind(), io::ErrorKind::Unsupported); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn sink_writes_header_and_only_video_records() { + let dir = tempdir(); + let path = dir.join("movie.fvi"); + let mut sink = FviSink::create(&path, &mpeg2_title(), "iso://m.iso".into(), 1).unwrap(); + // Video frame on track 0 → indexed. + sink.write(&vframe(0, Some(i_pic()), Some(SourcePos::at_byte(2048)))) + .unwrap(); + // Audio frame on a non-video track → ignored. + sink.write(&vframe(7, None, Some(SourcePos::at_byte(9999)))) + .unwrap(); + sink.finish().unwrap(); + + let text = std::fs::read_to_string(&path).unwrap(); + let lines: Vec<_> = text.lines().collect(); + assert_eq!(lines.len(), 2, "header + one video record only"); + + let header: serde_json::Value = serde_json::from_str(lines[0]).unwrap(); + assert_eq!(header["format"], "freemkv/video-index"); + assert_eq!(header["fvi_version"], 1); + assert_eq!(header["stream"]["dar"], serde_json::json!([16, 9])); // anamorphic + assert_eq!(header["stream"]["scan"], "interlaced"); // 480i + assert_eq!(header["stream"]["codec"], "mpeg2video"); + assert_eq!(header["timescale"], 1_000_000_000u64); + assert_eq!(header["source"]["title"], 1); + assert_eq!(header["source"]["medium"], "file"); // default medium + + let rec: serde_json::Value = serde_json::from_str(lines[1]).unwrap(); + assert_eq!(rec["n"], 0); + assert_eq!(rec["type"], "I"); + assert_eq!(rec["key"], true); // I-picture (frame keyframe) → random-access + // Interlaced tff frame → field_order "tff", progressive false, 2 fields. + assert_eq!(rec["field_order"], "tff"); + assert_eq!(rec["progressive"], false); + assert_eq!(rec["nb_fields"], 2); + assert_eq!(rec["pts"], 0); + assert_eq!(rec["src"]["sector"], 1); + assert!(rec.get("dts").is_none(), "no DTS on a frame → omitted"); + assert!( + rec.get("gop").is_none(), + "no GOP-closure signal → gop omitted" + ); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn empty_title_still_emits_valid_header() { + let dir = tempdir(); + let path = dir.join("empty.fvi"); + let mut sink = FviSink::create(&path, &mpeg2_title(), String::new(), 0).unwrap(); + sink.finish().unwrap(); // no frames + let text = std::fs::read_to_string(&path).unwrap(); + assert_eq!(text.lines().count(), 1, "header only"); + let header: serde_json::Value = serde_json::from_str(text.lines().next().unwrap()).unwrap(); + assert_eq!(header["format"], "freemkv/video-index"); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn extension_jsonl_is_still_json_lines() { + // Output is always JSON Lines regardless of extension (one format today). + let dir = tempdir(); + let path = dir.join("idx.jsonl"); + let mut sink = FviSink::create(&path, &mpeg2_title(), String::new(), 0).unwrap(); + sink.write(&vframe(0, Some(i_pic()), None)).unwrap(); + sink.finish().unwrap(); + let text = std::fs::read_to_string(&path).unwrap(); + // null src on a provenance-absent frame. + let rec: serde_json::Value = serde_json::from_str(text.lines().nth(1).unwrap()).unwrap(); + assert_eq!(rec["src"], serde_json::Value::Null); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn codec_agnostic_non_mpeg2_records() { + // A non-MPEG2 stream (coding None) whose parser sets keyframe must still + // produce USEFUL records: key/type from the frame keyframe flag, src + + // pts populated, and NO mpeg2-only field members. + let dir = tempdir(); + let path = dir.join("uhd.fvi"); + let mut sink = FviSink::create(&path, &hevc_title(), "disc://".into(), 0).unwrap(); + // HEVC IDR (keyframe) with real provenance. + sink.write(&vframe_kf(0, None, true, Some(SourcePos::at_byte(12288)))) + .unwrap(); + // Non-key HEVC picture. + sink.write(&vframe_kf(0, None, false, Some(SourcePos::at_byte(20480)))) + .unwrap(); + sink.finish().unwrap(); + + let text = std::fs::read_to_string(&path).unwrap(); + let recs: Vec = text + .lines() + .skip(1) + .map(|l| serde_json::from_str(l).unwrap()) + .collect(); + assert_eq!(recs[0]["key"], true, "HEVC IDR → key from frame.keyframe"); + assert_eq!(recs[0]["type"], "I"); + assert_eq!(recs[0]["src"]["sector"], 6); // 12288 / 2048 + assert!( + recs[0].get("field_order").is_none() && recs[0].get("nb_fields").is_none(), + "coding-absent frame omits field_order/progressive/nb_fields" + ); + assert_eq!(recs[1]["key"], false); + assert_eq!(recs[1]["type"], "P"); + assert_eq!(recs[1]["src"]["sector"], 10); // 20480 / 2048 + let _ = std::fs::remove_dir_all(&dir); + } +} diff --git a/src/mux/mod.rs b/src/mux/mod.rs index 2e80e9e..59bd885 100644 --- a/src/mux/mod.rs +++ b/src/mux/mod.rs @@ -52,6 +52,10 @@ pub(crate) mod demux_thread; // network/stdio implementations that no external caller had business // reaching for. pub(crate) mod ebml; +/// `fvi://` sink — freemkv's native per-picture video index (see +/// `docs/FVI_FORMAT.md`). A write-only PES sink that emits one JSON-Lines record +/// per coded picture; reuses the pure-data [`videomap`] model. +pub(crate) mod fvi_sink; pub(crate) mod m2ts; /// FMKV metadata header (used by `M2tsStream` / `NetworkStream` / `StdioStream` /// to round-trip codec_privates that don't fit inside the underlying format). @@ -93,9 +97,18 @@ pub(crate) mod stdio; pub(crate) mod timeline; pub(crate) mod ts; pub(crate) mod tsmux; +/// Reusable, pure-data per-picture video index (the FVI logical model) consumed +/// by the [`fvi_sink`]. Serialization-independent. `#[allow(dead_code)]`: the +/// `VideoMap` accumulator is a standalone primitive staged for the side-channel +/// (mux-while-indexing) reuse described in its module doc; the `fvi://` sink +/// today builds `PictureRecord`s directly, so the accumulator is covered only by +/// its own unit tests until that wiring lands. +#[allow(dead_code)] +pub(crate) mod videomap; pub use demux_sink::{ChaptersFmt, DelayMode, DemuxOptions, DemuxSink, Naming}; pub use disc::DiscStream; +pub use fvi_sink::FviSink; pub use m2ts::M2tsStream; pub use mkvstream::MkvStream; pub use network::NetworkStream; diff --git a/src/mux/pipelined_stream.rs b/src/mux/pipelined_stream.rs index 8bf4b7a..c3dc900 100644 --- a/src/mux/pipelined_stream.rs +++ b/src/mux/pipelined_stream.rs @@ -208,7 +208,11 @@ impl PipelinedPesStream { continue; }; let pes = PesPacket { - source: None, + // Carry the PS demuxer's byte-exact source stamp through to the + // codec parser, exactly as the TS path does — provenance must + // survive the PsPacket → PesPacket seam so the frame's `source` + // reaches the mux/index (FVI `src`), never reconstructed. + source: ps.source, pid, pts: ps.pts.map(|p| p as i64), dts: ps.dts.map(|d| d as i64), diff --git a/src/mux/resolve.rs b/src/mux/resolve.rs index 23fb7fb..2b76064 100644 --- a/src/mux/resolve.rs +++ b/src/mux/resolve.rs @@ -12,6 +12,7 @@ //! | network:// | Yes (listen) | Yes (connect) | host:port (required) | //! | stdio:// | Yes (stdin) | Yes (stdout) | empty | //! | null:// | -- | Yes | empty | +//! | fvi:// | -- | Yes | file path (required) — per-picture video index | //! //! Bare paths without a scheme are rejected. //! For disc→ISO (raw sector copy), use `Disc::copy()` instead. @@ -60,6 +61,10 @@ pub enum StreamUrl { /// chapters + delay metadata). Like `dir://` it targets a directory; the /// CLI constructs the `DemuxSink` with full options before the mux loop. Demux { dir: PathBuf }, + /// freemkv native per-picture video index (`fvi://`). A write-only PES sink + /// that emits one JSON-Lines record per coded picture of the title's primary + /// video track to a `.fvi` file (normative spec `docs/FVI_FORMAT.md`). + Fvi { path: PathBuf }, /// Unrecognized URL. Unknown { raw: String }, } @@ -77,6 +82,7 @@ impl StreamUrl { StreamUrl::Dir { .. } => "dir", StreamUrl::Null => "null", StreamUrl::Demux { .. } => "demux", + StreamUrl::Fvi { .. } => "fvi", StreamUrl::Unknown { .. } => "unknown", } } @@ -90,7 +96,8 @@ impl StreamUrl { | StreamUrl::Mkv { path } | StreamUrl::Iso { path } | StreamUrl::Dir { path } - | StreamUrl::Demux { dir: path } => path.to_str().unwrap_or(""), + | StreamUrl::Demux { dir: path } + | StreamUrl::Fvi { path } => path.to_str().unwrap_or(""), StreamUrl::Network { addr } => addr, StreamUrl::Stdio | StreamUrl::Null => "", StreamUrl::Unknown { raw } => raw, @@ -163,6 +170,11 @@ pub fn parse_url(url: &str) -> StreamUrl { dir: PathBuf::from(rest), }; } + if let Some(rest) = url.strip_prefix("fvi://") { + return StreamUrl::Fvi { + path: PathBuf::from(rest), + }; + } StreamUrl::Unknown { raw: url.to_string(), } @@ -403,6 +415,8 @@ pub fn input(url: &str, opts: &InputOptions) -> io::Result Err(crate::error::Error::StreamWriteOnly.into()), // `demux://` is an output-only sink (per-track ES files); never a source. StreamUrl::Demux { .. } => Err(crate::error::Error::StreamWriteOnly.into()), + // `fvi://` is an output-only sink (per-picture video index); never a source. + StreamUrl::Fvi { .. } => Err(crate::error::Error::StreamWriteOnly.into()), StreamUrl::Unknown { ref raw } => { Err(crate::error::Error::StreamUrlInvalid { url: raw.clone() }.into()) } @@ -479,6 +493,20 @@ pub fn output( dir, title, &opts, )?)) } + // `fvi://` writes the per-picture video index (`docs/FVI_FORMAT.md`). + // The bare `output()` arm records the resolver path as the provenance + // `source.path` and defaults the title index to 0 (the resolver carries + // no title-index context — the CLI follow-up passes the real medium / + // title via `FviSink::create_with_source`). + StreamUrl::Fvi { ref path } => { + validate_file_path(path, "fvi")?; + Ok(Box::new(super::fvi_sink::FviSink::create( + path, + title, + path.to_string_lossy().into_owned(), + 0, + )?)) + } StreamUrl::Unknown { ref raw } => { Err(crate::error::Error::StreamUrlInvalid { url: raw.clone() }.into()) } diff --git a/src/mux/videomap.rs b/src/mux/videomap.rs new file mode 100644 index 0000000..146f20b --- /dev/null +++ b/src/mux/videomap.rs @@ -0,0 +1,596 @@ +//! `VideoMap` — freemkv's reusable, pure-data per-picture video index ("the +//! FVI object"). +//! +//! A [`VideoMap`] is a header (per-title video facts + provenance root) plus an +//! ordered list of per-picture records. Each record carries the per-picture +//! coding truth ([`PictureInfo`], off `frame.coding`) and the byte-exact source +//! provenance ([`SourcePos`], off `frame.source`) that the highway already +//! stamps — this module never re-parses the elementary stream. +//! +//! It is a STANDALONE PRIMITIVE, deliberately decoupled from any one sink: +//! - The `fvi://` sink ([`crate::mux::fvi_sink`]) owns a `VideoMap`, appends +//! each video [`PesFrame`], and serializes it. +//! - The same `VideoMap` can later be populated as a side-channel during ANY +//! mux (e.g. `iso → mkv` while ALSO emitting a `.fvi` sidecar), and reused +//! for seek-indexing, recovery loss-mapping, and diagnostics. +//! +//! `VideoMap` is PURE DATA — it knows no output format. The on-disk shape is the +//! freemkv FVI format, whose normative spec is `docs/FVI_FORMAT.md` (ships +//! publicly with libfreemkv); the `fvi://` sink does the serialization. A +//! different output format would be a DIFFERENT sink reusing this same model, +//! not a pluggable encoder here. + +use crate::disc::{ColorSpace, DiscTitle, FrameRate, Stream as DiscStream, VideoStream}; +use crate::mux::codec::PictureInfo; +use crate::mux::codec::coding::{CodingType, FieldOrder}; +use crate::pes::{PesFrame, SourcePos}; + +// ── Format constants (cite docs/FVI_FORMAT.md) ─────────────────────────────── + +/// Value of the header `"format"` member — the FVI document signature +/// (`docs/FVI_FORMAT.md` §6). Identifies a stream as a freemkv video index. +pub const FVI_FORMAT: &str = "freemkv/video-index"; + +/// Value of the header `"fvi_version"` member — the FVI document format version +/// (`docs/FVI_FORMAT.md` §6, §11). This spec defines `1`. +pub const FVI_VERSION: u32 = 1; + +/// Producing tool tag for the header `"generator"` member +/// (`docs/FVI_FORMAT.md` §6). +pub const FVI_GENERATOR: &str = concat!("freemkv/", env!("CARGO_PKG_VERSION")); + +/// Header `"timescale"` for all `pts`/`dts` ticks (`docs/FVI_FORMAT.md` §10). +/// The highway carries presentation timestamps in nanoseconds, so the timescale +/// is `1_000_000_000` ticks per second. +pub const FVI_TIMESCALE: u64 = 1_000_000_000; + +/// Bytes per `src.sector` unit (`docs/FVI_FORMAT.md` §6.2, §9). The highway's +/// [`SourcePos`] counts 2048-byte logical sectors. +pub const FVI_SECTOR_SIZE: u32 = 2048; + +// ── Logical model (serialization-independent) ──────────────────────────────── + +/// Source-stream colour description (CICP code points), header-level +/// (`docs/FVI_FORMAT.md` §6.1 `colour`). +/// +/// Each field is the ITU-T H.273 / ISO 23091-2 code point for the title's +/// primary video, derived from the disc's [`ColorSpace`]. `full_range` is the +/// video-range flag (`false` = limited / TV range, the disc norm). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Colour { + pub primaries: u8, + pub transfer: u8, + pub matrix: u8, + pub full_range: bool, +} + +impl Colour { + /// Map the title's [`ColorSpace`] to CICP code points. Unknown colorimetry + /// maps to code point 2 ("unspecified"), the CICP convention. + pub fn from_color_space(cs: ColorSpace) -> Self { + // (primaries, transfer, matrix) per ITU-T H.273. + let (p, t, m) = match cs { + ColorSpace::Bt709 => (1, 1, 1), + ColorSpace::Bt2020 => (9, 14, 9), // BT.2020 NCL + ColorSpace::Bt470bg => (5, 5, 5), + ColorSpace::Smpte170m => (6, 6, 6), + ColorSpace::Unknown => (2, 2, 2), // unspecified + }; + Self { + primaries: p, + transfer: t, + matrix: m, + // Disc video is limited-range; full-range is not signalled at this + // layer, so report the disc norm. + full_range: false, + } + } +} + +/// Scan type for the header `stream.scan` member (`docs/FVI_FORMAT.md` §6.1). +/// `"mbaff"` is reachable only for codecs that signal it; MPEG-2 / disc video +/// resolves to `progressive` / `interlaced`. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Scan { + Progressive, + Interlaced, +} + +impl Scan { + pub fn as_str(self) -> &'static str { + match self { + Scan::Progressive => "progressive", + Scan::Interlaced => "interlaced", + } + } +} + +/// Source `medium` for the header `source.medium` member +/// (`docs/FVI_FORMAT.md` §6.2). Describes the physical/logical input the index +/// was built from. The bare resolver path has no input-URL context, so it +/// defaults to [`Medium::File`]; the CLI follow-up passes the real medium. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)] +pub enum Medium { + Disc, + Iso, + #[default] + File, + Stream, +} + +impl Medium { + pub fn as_str(self) -> &'static str { + match self { + Medium::Disc => "disc", + Medium::Iso => "iso", + Medium::File => "file", + Medium::Stream => "stream", + } + } +} + +/// Provenance root for the header (`docs/FVI_FORMAT.md` §6.2 `source`). +#[derive(Clone, Debug, PartialEq, Eq, Default)] +pub struct SourceInfo { + /// Input medium. + pub medium: Medium, + /// Source path / label (may be empty). + pub path: String, + /// 0-based title / program number the index was built from. + pub title: usize, + /// Playlist / PGC identifier, if known (empty → omitted). + pub playlist: String, + /// Disc volume identifier, if read (empty → omitted). + pub volume_id: String, +} + +/// Per-title video facts for the header `stream` object +/// (`docs/FVI_FORMAT.md` §6.1). +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct StreamInfo { + /// Registered codec id (Appendix B), e.g. `"mpeg2video"`, `"hevc"`. + pub codec: &'static str, + /// Coded luma dimensions in pixels. + pub width: u32, + pub height: u32, + /// Display aspect ratio as `(num, den)`. + pub dar: (u32, u32), + /// Nominal frame rate as an exact rational `(num, den)`. + pub frame_rate: (u32, u32), + /// Scan type. + pub scan: Scan, + /// Source colour (CICP code points). + pub colour: Colour, +} + +/// The header row: per-title facts (`docs/FVI_FORMAT.md` §6). +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct MapHeader { + /// The indexed elementary stream. + pub stream: StreamInfo, + /// Provenance root. + pub source: SourceInfo, + /// Total pictures if known at header time; `None` when streaming (omitted). + pub picture_count: Option, +} + +/// Map the disc's `Codec` to a registered FVI codec id +/// (`docs/FVI_FORMAT.md` Appendix B). The disc-info `Codec::id` strings differ +/// (`"mpeg2"`/`"mpeg1"`); FVI uses the bitstream names. +fn fvi_codec_id(codec: crate::disc::Codec) -> &'static str { + use crate::disc::Codec; + match codec { + Codec::Mpeg2 => "mpeg2video", + Codec::Mpeg1 => "mpeg1video", + Codec::H264 => "h264", + Codec::Hevc => "hevc", + Codec::Vc1 => "vc1", + // Not in the registry yet; carry the disc-info id so the field is still + // a stable, machine-readable token (readers ignore unknown codecs). + other => other.id(), + } +} + +impl MapHeader { + /// Assemble the header from the title's primary video stream + the supplied + /// provenance (`source`). Without a video stream there is nothing to index; + /// this returns neutral stream defaults so the header still serializes (the + /// record stream will be empty) — a malformed / audio-only title does not + /// panic. + pub fn from_title(title: &DiscTitle, source: SourceInfo) -> Self { + let video: Option<&VideoStream> = title.streams.iter().find_map(|s| match s { + DiscStream::Video(v) => Some(v), + _ => None, + }); + + let stream = match video { + Some(v) => { + let (width, height) = v.resolution.pixels(); + StreamInfo { + codec: fvi_codec_id(v.codec), + width, + height, + dar: display_aspect_ratio(v, width, height), + frame_rate: v.frame_rate.as_fraction(), + scan: if v.resolution.is_interlaced() { + Scan::Interlaced + } else { + Scan::Progressive + }, + colour: Colour::from_color_space(v.color_space), + } + } + None => StreamInfo { + codec: "unknown", + width: 0, + height: 0, + dar: (0, 1), + frame_rate: (0, 1), + scan: Scan::Progressive, + colour: Colour::from_color_space(ColorSpace::Unknown), + }, + }; + + Self { + stream, + source, + picture_count: None, + } + } +} + +/// Display aspect ratio as `(num, den)`. Anamorphic titles carry an explicit +/// `display_aspect`; square-pixel titles use the coded pixel dimensions. +fn display_aspect_ratio(v: &VideoStream, w: u32, h: u32) -> (u32, u32) { + match v.display_aspect { + Some((a, b)) if b != 0 => (a, b), + _ if h != 0 => (w, h), + _ => (0, 1), + } +} + +/// The title's nominal frame rate as a fraction — the single mapping site reused +/// by the header builder. (Retained as the canonical accessor.) +#[allow(dead_code)] +fn frame_rate_fraction(fr: FrameRate) -> (u32, u32) { + fr.as_fraction() +} + +/// One per-picture index record, distilled from a video [`PesFrame`] +/// (`docs/FVI_FORMAT.md` §7). +/// +/// `coding` is the codec-agnostic per-picture truth ([`PictureInfo`], set by +/// EVERY video parser that decodes coding — MPEG-2 fully, H.264/HEVC/VC-1 as +/// coding-type-only); `source` is the byte-exact provenance. Both are optional: +/// an audio / synthetic / provenance-absent frame yields a record whose +/// coding-derived members are omitted and whose `src` is the spec-defined null. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct PictureRecord { + /// Coded-order index, 0-based, contiguous. + pub n: u64, + /// Codec-agnostic per-picture coding info, if present. Set by every video + /// parser that decodes coding (MPEG-2 fully; H.264/HEVC/VC-1 carry + /// coding-type only); `None` for audio/subtitle/synthetic frames. + pub coding: Option, + /// Random-access / keyframe flag carried for EVERY codec on the frame + /// (`PesFrame::keyframe`): IDR/IRAP for HEVC/H.264, the I-picture flag for + /// MPEG-2/VC-1. Drives the codec-agnostic `key` member. + pub keyframe: bool, + /// Presentation timestamp in `timescale` ticks (nanoseconds). + pub pts_ns: Option, + /// Byte-exact source provenance, if present. + pub source: Option, +} + +/// Record `type` label (`docs/FVI_FORMAT.md` §7), codec-agnostic. +/// +/// When `coding` is present (any video codec — every parser now fills it), the +/// agnostic coding type is reported from [`PictureInfo::coding_type`]: +/// `CodingType::{I,P,B}` → `"I"`/`"P"`/`"B"`. When `coding` is absent +/// (audio / synthetic frames), the type degrades to the I-vs-non-I distinction +/// the frame's keyframe flag still carries: `keyframe` → "I", otherwise "P". +pub fn type_label(coding: Option, keyframe: bool) -> &'static str { + match coding { + Some(c) => match c.coding_type() { + CodingType::I => "I", + CodingType::P => "P", + CodingType::B => "B", + }, + // No PictureInfo: the highway still gives a keyframe flag. + None => { + if keyframe { + "I" + } else { + "P" + } + } + } +} + +/// Field-display-order label for the optional `field_order` member +/// (`docs/FVI_FORMAT.md` §7.1, Matroska element 0x9D), or `None` when the codec +/// did not measure it (signal absent / coding-type-only codec). `None` is an +/// HONEST absence — the writer OMITS the member rather than guessing a default. +pub fn field_order_label(coding: Option) -> Option<&'static str> { + match coding?.field_order()? { + FieldOrder::Tff => Some("tff"), + FieldOrder::Bff => Some("bff"), + FieldOrder::Progressive => Some("progressive"), + } +} + +/// Whether a picture is a random-access point for the `key` member +/// (`docs/FVI_FORMAT.md` §7), codec-agnostic. +/// +/// For EVERY codec the frame's own `keyframe` flag IS the random-access signal: +/// IDR/IRAP for HEVC/H.264, the I-picture flag for MPEG-2/VC-1 — authored by +/// each codec's parser through the highway. The codec-agnostic [`PictureInfo`] +/// carries NO GOP-closure (no `closed_gop`/`gop_start`), so we DO NOT claim the +/// stricter open-GOP clean-RAP precision; `key` is the parser-flagged +/// decode-restart point (an intra picture). This is the honest limitation +/// documented in `docs/FVI_FORMAT.md`. +pub fn is_random_access(coding: Option, keyframe: bool) -> bool { + // For a video frame `coding.keyframe()` == an intra (I) picture, which is + // exactly the highway's `frame.keyframe`; use the frame flag uniformly. + let _ = coding; + keyframe +} + +/// The reusable video index: a header plus an ordered list of per-picture +/// records. PURE DATA — serialization lives in the sink that consumes it. +#[derive(Clone, Debug)] +pub struct VideoMap { + header: MapHeader, + records: Vec, +} + +impl VideoMap { + /// Create an empty map with the header assembled from `title`'s primary + /// video stream + the supplied provenance. + pub fn new(title: &DiscTitle, source: SourceInfo) -> Self { + Self { + header: MapHeader::from_title(title, source), + records: Vec::new(), + } + } + + /// The header row. + pub fn header(&self) -> &MapHeader { + &self.header + } + + /// The per-picture records, in coded/arrival order. + pub fn records(&self) -> &[PictureRecord] { + &self.records + } + + /// Append one video frame as the next picture record, pulling the coding + /// truth from `frame.coding` and the provenance from `frame.source`. The + /// record index `n` is the current record count (coded order). Returns the + /// record just appended. + pub fn append_frame(&mut self, frame: &PesFrame) -> &PictureRecord { + let rec = PictureRecord { + n: self.records.len() as u64, + coding: frame.coding, + keyframe: frame.keyframe, + // pts is carried as ns; the highway always sets a presentation time + // (0 at start), so emit it. A future source genuinely lacking a PTS + // would set None and the writer omits the member. + pts_ns: Some(frame.pts), + source: frame.source, + }; + self.records.push(rec); + self.records.last().expect("just pushed") + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::disc::{ + Codec, ColorSpace, ContentFormat, FrameRate, HdrFormat, Resolution, VideoStream, + }; + + fn video_title(codec: Codec, res: Resolution, fr: FrameRate, cs: ColorSpace) -> DiscTitle { + let mut t = DiscTitle::empty(); + t.streams = vec![DiscStream::Video(VideoStream { + pid: 0x1011, + codec, + resolution: res, + frame_rate: fr, + hdr: HdrFormat::Sdr, + color_space: cs, + display_aspect: None, + secondary: false, + label: String::new(), + measured_cicp: None, + })]; + t.content_format = ContentFormat::BdTs; + t + } + + fn src(medium: Medium, path: &str, title: usize) -> SourceInfo { + SourceInfo { + medium, + path: path.to_string(), + title, + ..Default::default() + } + } + + fn vframe(coding: Option, pts: i64, source: Option) -> PesFrame { + let keyframe = coding.map(|c| c.keyframe()).unwrap_or(false); + PesFrame { + track: 0, + pts, + keyframe, + data: vec![0u8; 4], + duration_ns: None, + source, + coding, + } + } + + use crate::mux::codec::coding::Mpeg2Coding; + + /// An interlaced (tff) MPEG-2 frame picture of the given coding type. + fn mpeg2_pic(ct: CodingType) -> PictureInfo { + PictureInfo::mpeg2( + ct, + Mpeg2Coding { + top_field_first: true, + repeat_first_field: false, + progressive_frame: false, + progressive_sequence: false, + frame_picture: true, + }, + ) + } + + /// A canonical I-picture fixture (interlaced frame). + fn i_picture() -> PictureInfo { + mpeg2_pic(CodingType::I) + } + + #[test] + fn colour_maps_cicp_code_points() { + assert_eq!( + Colour::from_color_space(ColorSpace::Bt709), + Colour { + primaries: 1, + transfer: 1, + matrix: 1, + full_range: false + } + ); + assert_eq!( + Colour::from_color_space(ColorSpace::Bt2020), + Colour { + primaries: 9, + transfer: 14, + matrix: 9, + full_range: false + } + ); + assert_eq!(Colour::from_color_space(ColorSpace::Unknown).primaries, 2); + } + + #[test] + fn type_label_full_and_codec_agnostic_fallback() { + // coding present: full I/P/B from the agnostic coding_type(). + let mk = |ct| Some(mpeg2_pic(ct)); + assert_eq!(type_label(mk(CodingType::I), false), "I"); + assert_eq!(type_label(mk(CodingType::P), false), "P"); + assert_eq!(type_label(mk(CodingType::B), false), "B"); + // coding-type-only codec still reports its type. + assert_eq!( + type_label(Some(PictureInfo::coding_type_only(CodingType::B)), false), + "B" + ); + // No coding (audio/synthetic): degrade to I-vs-non-I from keyframe. + assert_eq!(type_label(None, true), "I"); + assert_eq!(type_label(None, false), "P"); + } + + #[test] + fn field_order_label_omitted_when_unmeasured() { + // MPEG-2 interlaced tff frame → "tff". + assert_eq!(field_order_label(Some(i_picture())), Some("tff")); + // Progressive frame → "progressive". + let prog = PictureInfo::mpeg2( + CodingType::I, + Mpeg2Coding { + top_field_first: true, + repeat_first_field: false, + progressive_frame: true, + progressive_sequence: false, + frame_picture: true, + }, + ); + assert_eq!(field_order_label(Some(prog)), Some("progressive")); + // Coding-type-only codec did not measure field order → None (omitted). + assert_eq!( + field_order_label(Some(PictureInfo::coding_type_only(CodingType::I))), + None + ); + // No coding at all → None. + assert_eq!(field_order_label(None), None); + } + + #[test] + fn is_random_access_codec_agnostic() { + // For EVERY codec the frame keyframe flag IS the random-access signal. + assert!(is_random_access(Some(i_picture()), true)); + // An I-picture whose frame flag is clear is NOT promoted — `key` follows + // the frame's keyframe flag, never fabricated GOP-closure. + assert!(!is_random_access(Some(i_picture()), false)); + // P/B with the flag clear → never. + assert!(!is_random_access(Some(mpeg2_pic(CodingType::P)), false)); + // No coding: the frame keyframe flag IS the RAP signal. + assert!(is_random_access(None, true)); + assert!(!is_random_access(None, false)); + } + + #[test] + fn fvi_codec_ids_use_bitstream_names() { + assert_eq!(fvi_codec_id(Codec::Mpeg2), "mpeg2video"); + assert_eq!(fvi_codec_id(Codec::Mpeg1), "mpeg1video"); + assert_eq!(fvi_codec_id(Codec::H264), "h264"); + assert_eq!(fvi_codec_id(Codec::Hevc), "hevc"); + assert_eq!(fvi_codec_id(Codec::Vc1), "vc1"); + } + + #[test] + fn header_from_title_pulls_video_facts() { + let t = video_title( + Codec::Mpeg2, + Resolution::R576i, + FrameRate::F25, + ColorSpace::Bt470bg, + ); + let h = MapHeader::from_title(&t, src(Medium::Iso, "iso://x.iso", 2)); + assert_eq!(h.stream.codec, "mpeg2video"); + assert_eq!((h.stream.width, h.stream.height), (720, 576)); + assert_eq!(h.stream.dar, (720, 576)); // square-pixel fallback + assert_eq!(h.stream.frame_rate, (25, 1)); + assert_eq!(h.stream.scan, Scan::Interlaced); + assert_eq!(h.stream.colour.matrix, 5); + assert_eq!(h.source.path, "iso://x.iso"); + assert_eq!(h.source.title, 2); + assert_eq!(h.source.medium, Medium::Iso); + } + + #[test] + fn header_audio_only_title_is_neutral_not_panic() { + let t = DiscTitle::empty(); + let h = MapHeader::from_title(&t, SourceInfo::default()); + assert_eq!(h.stream.codec, "unknown"); + assert_eq!((h.stream.width, h.stream.height), (0, 0)); + } + + #[test] + fn append_frame_numbers_records_in_order() { + let t = video_title( + Codec::Mpeg2, + Resolution::R1080p, + FrameRate::F23_976, + ColorSpace::Bt709, + ); + let mut map = VideoMap::new(&t, SourceInfo::default()); + map.append_frame(&vframe( + Some(i_picture()), + 0, + Some(SourcePos::at_byte(2048)), + )); + map.append_frame(&vframe( + Some(mpeg2_pic(CodingType::B)), + 42, + Some(SourcePos::at_byte(4096)), + )); + assert_eq!(map.records().len(), 2); + assert_eq!(map.records()[0].n, 0); + assert_eq!(map.records()[1].n, 1); + assert_eq!(map.records()[0].source.unwrap().sector, 1); + assert_eq!(map.records()[1].pts_ns, Some(42)); + } +} diff --git a/tests/fvi_pipeline.rs b/tests/fvi_pipeline.rs new file mode 100644 index 0000000..c6f3c31 --- /dev/null +++ b/tests/fvi_pipeline.rs @@ -0,0 +1,359 @@ +//! End-to-end `fvi://` tests: drive a REAL MPEG-2 Program-Stream image through +//! the public highway (`build_iso_pipeline`, MpegPs → PS demux → `Mpeg2Parser`) +//! and into the `fvi://` sink built by `output()`, then parse the `.fvi` back +//! and assert the per-picture index is correct. +//! +//! These tests deliberately use only the public API and the real parser / +//! pipeline — no stubbed frames that bypass the demuxer or the codec parse. + +use libfreemkv::disc::{ + Codec, ColorSpace, ContentFormat, DiscTitle, Extent, FrameRate, HdrFormat, Resolution, Stream, + VideoStream, +}; +use libfreemkv::pes::Stream as PesStream; +use libfreemkv::{DecryptKeys, SectorSource, build_iso_pipeline, output}; +use std::path::PathBuf; + +/// DVD video PES stream_id (0xE0). +const DVD_VIDEO_STREAM_ID: u8 = 0xE0; + +// ── MPEG-2 elementary-stream fixture builders (mirror the in-crate ones) ────── + +/// 720x480, 4:3, 29.97 (aspect_ratio_information=2, frame_rate_code=4). +fn m2_seq_header() -> Vec { + let (w, h, aspect, fr): (u16, u16, u8, u8) = (720, 480, 2, 4); + let mut hdr = vec![0x00, 0x00, 0x01, 0xB3u8]; + hdr.push((w >> 4) as u8); + hdr.push((((w & 0x0F) as u8) << 4) | (((h >> 8) & 0x0F) as u8)); + hdr.push((h & 0xFF) as u8); + hdr.push((aspect << 4) | (fr & 0x0F)); + hdr.extend_from_slice(&[0xFF, 0xFF, 0xFF, 0x00]); + hdr +} + +/// GOP header (00 00 01 B8) with a zeroed time-code / flags. +fn m2_gop() -> Vec { + vec![0x00, 0x00, 0x01, 0xB8u8, 0x00, 0x00, 0x00, 0x00] +} + +/// One coded picture: picture header (coding_type, temporal_reference) + a +/// picture coding extension carrying tff=1 (00 00 01 B5, ext-id 1000), + slice +/// padding. +fn m2_pic(coding_type: u8, tr: u16) -> Vec { + let b4 = ((tr >> 2) & 0xFF) as u8; + let b5 = (((tr & 0x03) as u8) << 6) | ((coding_type & 0x07) << 3); + let mut au = vec![0x00, 0x00, 0x01, 0x00u8, b4, b5, 0x00, 0x00]; + // Picture coding extension: e0=ext-id 1000, e2=0x03 (frame picture), + // e3 bit7 = top_field_first = 1. + au.extend_from_slice(&[0x00, 0x00, 0x01, 0xB5u8, 0x80, 0x00, 0x03, 0x80, 0x00]); + au.extend_from_slice(&[0xAA; 32]); + au +} + +// ── Program-Stream packing ──────────────────────────────────────────────────── + +/// A 14-byte MPEG-2 PS pack header (00 00 01 BA …) with no stuffing. +fn ps_pack_header() -> Vec { + let mut p = vec![0x00, 0x00, 0x01, 0xBAu8]; + // 9 bytes of SCR/mux-rate fields (content irrelevant to the demuxer's + // framing) + a final byte whose low 3 bits are pack_stuffing_length = 0. + p.extend_from_slice(&[0x44, 0x00, 0x04, 0x00, 0x04, 0x01, 0x01, 0x89, 0xC3]); + p.push(0xF8); // stuffing_length = 0 (low 3 bits) + p +} + +/// A video PES (stream_id 0xE0) carrying `es`, with a 33-bit PTS in 90 kHz +/// ticks and a bounded PES_packet_length. PTS prefix nibble is 0b0010. +fn video_pes(es: &[u8], pts: u64) -> Vec { + let mut pes = vec![0x00, 0x00, 0x01, DVD_VIDEO_STREAM_ID]; + // PES header: flags1=0x80, flags2=0x80 (PTS only), header_data_len=5. + let mut body = vec![0x80u8, 0x80, 0x05]; + // 5-byte PTS ('0010' marker + 33-bit value with marker bits). + let p = pts & 0x1_FFFF_FFFF; + body.push(0x21 | (((p >> 30) & 0x07) << 1) as u8); + body.push(((p >> 22) & 0xFF) as u8); + body.push((0x01 | (((p >> 15) & 0x7F) << 1)) as u8); + body.push(((p >> 7) & 0xFF) as u8); + body.push((0x01 | ((p & 0x7F) << 1)) as u8); + body.extend_from_slice(es); + let len = body.len() as u16; + pes.extend_from_slice(&len.to_be_bytes()); + pes.extend_from_slice(&body); + pes +} + +/// One GOP's worth of ES (seq header + GOP + I + P + B pictures). +fn gop_es() -> Vec { + let mut es = m2_seq_header(); + es.extend_from_slice(&m2_gop()); + es.extend_from_slice(&m2_pic(1, 0)); // I (keyframe, GOP opener) + es.extend_from_slice(&m2_pic(2, 2)); // P + es.extend_from_slice(&m2_pic(3, 1)); // B + es +} + +/// In-memory sector source serving a fixed byte image. +struct MemSource { + data: Vec, +} + +impl SectorSource for MemSource { + fn capacity_sectors(&self) -> u32 { + (self.data.len() / 2048) as u32 + } + fn read_sectors( + &mut self, + lba: u32, + count: u16, + buf: &mut [u8], + _recovery: bool, + ) -> libfreemkv::error::Result { + let start = lba as usize * 2048; + let want = count as usize * 2048; + for (i, b) in buf[..want].iter_mut().enumerate() { + *b = self.data.get(start + i).copied().unwrap_or(0); + } + Ok(want) + } +} + +fn mpeg2_dvd_title(extent_sectors: u32) -> DiscTitle { + let mut title = DiscTitle::empty(); + title.streams.push(Stream::Video(VideoStream { + pid: 0xE0, // DVD_VIDEO_PID + codec: Codec::Mpeg2, + resolution: Resolution::R480i, + frame_rate: FrameRate::F29_97, + hdr: HdrFormat::Sdr, + color_space: ColorSpace::Smpte170m, + display_aspect: Some((4, 3)), + secondary: false, + label: String::new(), + measured_cicp: None, + })); + title.content_format = ContentFormat::MpegPs; + title.extents = vec![Extent { + start_lba: 0, + sector_count: extent_sectors, + }]; + title +} + +/// Tiny unique temp dir helper. +fn tempdir() -> PathBuf { + use std::sync::atomic::{AtomicU64, Ordering}; + static N: AtomicU64 = AtomicU64::new(0); + let n = N.fetch_add(1, Ordering::Relaxed); + let p = std::env::temp_dir().join(format!("fmkv_fvi_pipe_{}_{}", std::process::id(), n)); + std::fs::create_dir_all(&p).unwrap(); + p +} + +/// Build a 6-sector PS image: GOP A in sector 0, GOP B in sector 3 — each in +/// its own 3-sector AACS-aligned region so the prefetcher's unit alignment is +/// satisfied and each batch carries a distinct, ascending source offset. +fn two_gop_image() -> Vec { + let mut data = vec![0u8; 6 * 2048]; + + let mut a = ps_pack_header(); + a.extend_from_slice(&video_pes(&gop_es(), 0)); + data[..a.len()].copy_from_slice(&a); + + let mut b = ps_pack_header(); + b.extend_from_slice(&video_pes(&gop_es(), 3003)); // ~0.1s later + let off = 3 * 2048; + data[off..off + b.len()].copy_from_slice(&b); + + data +} + +/// Drive the real highway and write every frame into the `fvi://` sink. +fn run_to_fvi(image: Vec, title: DiscTitle, path: &std::path::Path) { + let mut input = build_iso_pipeline( + MemSource { data: image }, + title.clone(), + DecryptKeys::None, + 3, // 3-sector (one AACS unit) batches → one source stamp per GOP region + ContentFormat::MpegPs, + None, + None, + ) + .expect("pipeline builds"); + + let url = format!("fvi://{}", path.display()); + let mut sink = output(&url, &title).expect("fvi sink opens"); + + while let Some(frame) = input.read().expect("read ok") { + sink.write(&frame).expect("sink write ok"); + } + sink.finish().expect("sink finish ok"); +} + +#[test] +fn fvi_sink_indexes_real_mpeg2_pipeline_output() { + let dir = tempdir(); + let path = dir.join("movie.fvi"); + run_to_fvi(two_gop_image(), mpeg2_dvd_title(6), &path); + + let text = std::fs::read_to_string(&path).unwrap(); + let mut lines = text.lines(); + + // ── Header line (docs/FVI_FORMAT.md v1 schema) ───────────────────────────── + let header: serde_json::Value = serde_json::from_str(lines.next().unwrap()).unwrap(); + assert_eq!(header["format"], "freemkv/video-index"); + assert_eq!(header["fvi_version"], 1); + assert_eq!(header["timescale"], 1_000_000_000u64); + let stream = &header["stream"]; + assert_eq!(stream["codec"], "mpeg2video"); + assert_eq!(stream["width"], 720); + assert_eq!(stream["height"], 480); + assert_eq!(stream["dar"], serde_json::json!([4, 3])); // anamorphic DVD + assert_eq!(stream["scan"], "interlaced"); // 480i + assert_eq!(stream["frame_rate"], serde_json::json!([30000, 1001])); + // SMPTE 170M → CICP (6,6,6), limited range. + assert_eq!(stream["colour"]["primaries"], 6); + assert_eq!(stream["colour"]["transfer"], 6); + assert_eq!(stream["colour"]["matrix"], 6); + assert_eq!(stream["colour"]["range"], "limited"); + // Provenance root: medium defaults to "file", sector_size present. + assert_eq!(header["source"]["sector_size"], 2048); + + // ── Records ─────────────────────────────────────────────────────────────── + let records: Vec = lines.map(|l| serde_json::from_str(l).unwrap()).collect(); + assert!( + records.len() >= 4, + "two GOPs of I/P/B → at least 4 pictures, got {}", + records.len() + ); + + // `n` is 0-based and contiguous in coded order. + for (i, r) in records.iter().enumerate() { + assert_eq!(r["n"], i as u64, "record n must be contiguous coded order"); + } + + // Every picture carries the codec-agnostic coding members from the REAL + // parser, derived through the `PictureInfo` accessors. + for r in &records { + assert!( + ["I", "P", "B"].contains(&r["type"].as_str().unwrap()), + "type must be a real coding type, got {}", + r["type"] + ); + // tff was set in the picture coding extension fixture, the frame is an + // interlaced (non-progressive) frame picture → field_order "tff", + // progressive false, 2 displayed fields. + assert_eq!( + r["field_order"], "tff", + "top_field_first survives the parse as field_order" + ); + assert_eq!(r["progressive"], false); + assert_eq!(r["nb_fields"], 2); + // No GOP-closure signal is carried by the codec-agnostic PictureInfo, so + // the `gop` member is honestly omitted (not fabricated). + assert!(r.get("gop").is_none(), "gop member omitted, never guessed"); + } + + // Exactly two I-pictures (one per GOP), each `type` I and a random-access + // point (`key` true — the parser-flagged intra/decode-restart point). The + // fixture's only intra pictures are the two GOP-opening I-frames. + let key_pics: Vec<&serde_json::Value> = records + .iter() + .filter(|r| r["key"] == serde_json::Value::Bool(true)) + .collect(); + assert_eq!(key_pics.len(), 2, "two intra/random-access pictures"); + for opener in &key_pics { + assert_eq!(opener["type"], "I", "a random-access point is an I-picture"); + } + + // The two stamped source sectors (sector 0 region and sector 3 region) reach + // the index, in ascending order — provenance carried, never reconstructed. + let src_sectors: Vec = records + .iter() + .filter_map(|r| r["src"]["sector"].as_u64()) + .collect(); + assert_eq!( + src_sectors, + vec![0, 3], + "stamped src sectors must reach the .fvi in arrival order; got {src_sectors:?}" + ); + // For each stamped record, sector == byte / 2048 (SourcePos::at_byte). The + // byte offset is exact (here 14 into each region, just past the pack + // header), so it is NOT sector-aligned — provenance is byte-exact. + for r in &records { + if let (Some(sector), Some(byte)) = (r["src"]["sector"].as_u64(), r["src"]["byte"].as_u64()) + { + assert_eq!(sector, byte / 2048, "src.sector must equal src.byte / 2048"); + } + } + + let _ = std::fs::remove_dir_all(&dir); +} + +/// Codec-agnostic path: a non-MPEG2 stream (HEVC/H.264/VC-1) emits frames with +/// `coding == None` but real `keyframe` + `source` + `pts`. The `.fvi` records +/// must still be USEFUL — `key`/`type` from the frame's keyframe flag, `src`/ +/// `pts` populated — NOT degraded to `type:"?"`/`src:null` just because +/// `PictureInfo` is MPEG-2-specific. The genuine null/"P"-fallback path only +/// fires when a field is truly absent (no provenance / non-key). No panic. +#[test] +fn fvi_sink_indexes_non_mpeg2_frames_codec_agnostically() { + use libfreemkv::pes::{PesFrame, SourcePos}; + + let dir = tempdir(); + let path = dir.join("nocoding.fvi"); + let title = mpeg2_dvd_title(0); + + let mk = |pts: i64, keyframe: bool, source: Option| PesFrame { + track: 0, + pts, + keyframe, + data: vec![0u8; 8], + duration_ns: None, + source, + coding: None, // non-MPEG2: no PictureInfo + }; + + let url = format!("fvi://{}", path.display()); + let mut sink = output(&url, &title).expect("fvi sink opens"); + // IDR (keyframe) with real provenance — must NOT be null/"?". + sink.write(&mk(1234, true, Some(SourcePos::at_byte(8192)))) + .unwrap(); + // Non-key with provenance. + sink.write(&mk(5678, false, Some(SourcePos::at_byte(16384)))) + .unwrap(); + // Keyframe with NO provenance — src genuinely null, but key/type still set. + sink.write(&mk(9012, true, None)).unwrap(); + sink.finish().unwrap(); + + let text = std::fs::read_to_string(&path).unwrap(); + let recs: Vec = text + .lines() + .skip(1) + .map(|l| serde_json::from_str(l).unwrap()) + .collect(); + assert_eq!(recs.len(), 3); + + // IDR: key true, type "I" (from keyframe), real src + pts, no mpeg2 fields. + assert_eq!(recs[0]["key"], true, "HEVC IDR → key from frame.keyframe"); + assert_eq!(recs[0]["type"], "I"); + assert_eq!(recs[0]["pts"], 1234); + assert_eq!(recs[0]["src"]["sector"], 4); // 8192 / 2048 + assert_eq!(recs[0]["src"]["byte"], 8192); + assert!( + recs[0].get("field_order").is_none() && recs[0].get("nb_fields").is_none(), + "coding-absent record omits field_order/nb_fields" + ); + assert!(recs[0].get("dts").is_none(), "no DTS on a frame → omitted"); + + // Non-key with provenance: key false, type "P", src still present. + assert_eq!(recs[1]["key"], false); + assert_eq!(recs[1]["type"], "P"); + assert_eq!(recs[1]["src"]["sector"], 8); // 16384 / 2048 + + // Keyframe without provenance: key/type still set, src genuinely null. + assert_eq!(recs[2]["key"], true); + assert_eq!(recs[2]["type"], "I"); + assert_eq!(recs[2]["src"], serde_json::Value::Null); + + let _ = std::fs::remove_dir_all(&dir); +}