diff --git a/FVI_FORMAT.md b/FVI_FORMAT.md deleted file mode 100644 index f12d4d12..00000000 --- a/FVI_FORMAT.md +++ /dev/null @@ -1,302 +0,0 @@ -# FVI — Freemkv Video Index Format - -**Specification version:** 1.0 (DRAFT)\ -**File extension:** `.fvi`\ -**Media type:** `application/vnd.freemkv.fvi+jsonl`\ -**Status:** Draft for review. This document is the normative reference for the FVI -format; implementations and downstream tools cite it by section. - ---- - -## 1. Scope and purpose - -FVI is an open, codec-agnostic, byte-exact **index of the coded pictures** in a -video bitstream, together with **provenance** back to the source medium. - -An FVI document answers, for every picture in a stream, three questions: - -1. **Where is it?** — the byte-exact offset of its first byte in the *source* - (the disc/ISO/file), so a reader can extract or seek to any picture without - re-parsing the whole bitstream. -2. **What is it?** — coding type, random-access capability, GOP boundary, and - (where the codec defines them) field/pulldown attributes. -3. **When is it?** — decode and presentation timestamps on a declared timescale. - -FVI is **not** a container, a codec, or a copy of the bitstream. It indexes; it -never stores coded samples. It is the serialized form of an indexer's per-picture -truth — carried from the demuxer, **never reconstructed** (§9). - -## 2. Conformance - -The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, -**SHOULD**, **SHOULD NOT**, **MAY**, and **OPTIONAL** are to be interpreted as -described in BCP 14 (RFC 2119, RFC 8174) when, and only when, they appear in all -capitals. - -A **conformant writer** MUST emit a document that satisfies §4–§10. A -**conformant reader** MUST accept any such document and MUST ignore unknown -object members (§11) so that forward-compatible extensions do not break it. - -## 3. Terminology - -- **Picture** — one coded video frame (or pair of fields coded as a frame). The - unit FVI indexes. -- **Access unit (AU)** — the set of bitstream bytes that decode to exactly one - picture (ISO/IEC 14496-10 §3; ISO/IEC 23008-2 §3). -- **Coded order** — the order pictures appear in the bitstream. FVI records are - emitted in coded order. -- **GOP / coded video sequence** — a self-contained run beginning at a - random-access point. -- **Provenance** — the mapping from an AU back to the exact bytes of the physical - source it was read from (§9). -- **Source position (`src`)** — `{ file, sector, byte }`, the provenance anchor of - an AU. - -## 4. Encoding - -An FVI document is a sequence of **UTF-8** text lines separated by a single LF -(`U+000A`). Each non-empty line is exactly one JSON value (RFC 8259), forming a -**JSON Lines / NDJSON** stream. A writer MUST NOT emit a UTF-8 BOM. A writer MUST -NOT pretty-print: each JSON value occupies exactly one line. - -The first line MUST be the **Header** object (§6). Each subsequent line is one -**Picture record** (§7), in coded order. - -Rationale: line-delimited JSON is streamable (a writer appends as it indexes; a -reader processes without loading the whole file), line-addressable (picture *n* -is near line *n+1*), append-safe, and parseable by every language without a -custom grammar — while remaining a precisely specified format, not an ad-hoc dump. - -A document MAY be concatenated for multiple elementary streams: each stream is its -own header line followed by its records. Readers MUST treat a Header line as the -start of a new stream section. - -## 5. Document structure - -``` -
line 1 (exactly one Header object) - line 2 .. N (one Picture record per picture, coded order) -[
…] (OPTIONAL further stream sections) -``` - -## 6. Header object - -| Member | JSON type | Req | Semantics / reference | -|---|---|---|---| -| `format` | string | MUST | Constant `"freemkv/video-index"`. Signature: a document begins with these bytes. | -| `fvi_version` | integer | MUST | Document format version. This spec defines `1`. | -| `generator` | string | SHOULD | Producing tool + version, e.g. `"freemkv/1.0.0-rc.6"`. | -| `stream` | object | MUST | The indexed elementary stream (§6.1). | -| `source` | object | MUST | Provenance root (§6.2). | -| `timescale` | integer | MUST | Ticks per second for all `pts`/`dts` (§10). E.g. `90000`. | -| `picture_count` | integer | MAY | Total pictures, if known at header time; OMITTED when streaming. | - -### 6.1 `stream` object - -| Member | JSON type | Req | Semantics / reference | -|---|---|---|---| -| `codec` | string | MUST | Registered codec id (Appendix B), e.g. `"mpeg2video"`, `"hevc"`. | -| `width`,`height` | integer | MUST | Coded luma dimensions in pixels. | -| `dar` | `[int,int]` | SHOULD | Display aspect ratio as `[num,den]`. | -| `frame_rate` | `[int,int]` | SHOULD | Nominal rate as exact rational `[num,den]` (e.g. `[24000,1001]`). | -| `scan` | string | MUST | `"progressive"`
`"interlaced"`
`"mbaff"` | -| `colour` | object | SHOULD | CICP per ITU-T H.273: `primaries`, `transfer`, `matrix` (integer CICP codes or registered names)
`range`: `"limited"` \| `"full"`
HDR: `mastering_display`, `max_cll`, `max_fall` per ITU-T H.273 / SMPTE ST 2086. | -| `language` | string | MAY | BCP 47 tag, if known. | - -### 6.2 `source` object - -| Member | JSON type | Req | Semantics / reference | -|---|---|---|---| -| `medium` | string | MUST | `"disc"`
`"iso"`
`"file"`
`"stream"` | -| `path` | string | MAY | Source path/label. | -| `title` | integer | MAY | Title/program number. | -| `playlist` | string | MAY | Playlist/PGC identifier. | -| `volume_id` | string | MAY | Disc volume identifier, if read. | -| `sector_size` | integer | SHOULD | Bytes per `src.sector` unit (e.g. `2048`). Lets readers convert `src` to an absolute byte offset. | - -## 7. Picture record - -One JSON object per coded picture, in coded order. - -| Member | JSON type | Req | Semantics / reference | -|---|---|---|---| -| `n` | integer | MUST | Coded-order index, 0-based, contiguous. | -| `src` | object | MUST | Provenance: `{ "file": int?, "sector": uint, "byte": uint }` — the offset of this AU's **first byte** in the source (§9). MUST be carried from demux, never reconstructed. | -| `type` | string | MUST | Coding type:
`"I"`
`"P"`
`"B"`
_ISO/IEC 13818-2 §6.3.9; H.264/H.265 slice types collapsed to frame type._ | -| `key` | boolean | MUST | `true` iff this picture is an intra (I) picture / parser-flagged decode-restart point (IDR / IRAP / I-picture).
_MPEG-2 open-GOP clean-RAP precision (`closed_gop`) is not currently distinguished — see note below._ | -| `gop` | boolean | SHOULD | `true` iff this picture begins a GOP / coded video sequence.
_Omitted when the implementation does not carry a distinct GOP-boundary signal._ | -| `pts` | integer\|null | SHOULD | Presentation timestamp in `timescale` ticks; `null` if unknown. | -| `dts` | integer\|null | MAY | Decode timestamp in `timescale` ticks. | -| `size` | integer | MAY | AU length in bytes; enables byte-range extraction with `src`. | -| `recovered` | boolean | MAY | `true` iff any byte of this AU came from a retried/marginal read (§9.1).
_Default `false`._ | -| codec ext | object | MAY | Codec-specific members under the codec's namespace (§8). | - -The `type` and `key` members are **codec-agnostic** and MUST be populated for -every codec. `type` is the I/P/B coding type the parser decoded (collapsing -H.264/H.265 slice types to a frame type); where no per-picture coding is carried -(audio / synthetic frames), `type` is `"I"` for a key picture else `"P"`. `key` -is the picture's random-access flag as the codec parser sets it (IDR / IRAP / -I-picture). A writer MUST NOT emit a degraded record (`type:"?"` or `src:null`) -merely because a codec lacks per-picture coding info — those fallbacks are -reserved for a field that is genuinely unavailable (e.g. provenance absent on a -synthetic source). - -> **Limitation (honest random-access).** `key` is set from the picture's -> intra / decode-restart flag. The per-picture coding model this index carries -> does **not** distinguish MPEG-2 open-GOP clean random-access points -> (`closed_gop`) from any other I-picture, so `key` is the parser-flagged -> decode-restart point, not a verified clean-RAP claim. A future revision MAY -> tighten `key` for codecs/profiles that carry that signal; readers MUST NOT -> assume present `key` precision beyond "intra / decode-restart point". - -### 7.1 Interlace / pulldown fields - -Codec-agnostic interlace/pulldown attributes, derived through the indexer's -per-picture coding accessors (MPEG-2: ISO/IEC 13818-2 §6.3.10). Emitted as -top-level members of the record, and ONLY when the codec actually measured the -signal — an OPTIONAL member that is omitted (not defaulted) when unknown: - -| Member | JSON type | Req | Semantics / reference | -|---|---|---|---| -| `field_order` | string | MAY | Display field order:
`"tff"` — top field first
`"bff"` — bottom field first
`"progressive"` — no field order applies
_Omitted when the codec did not signal it._ | -| `progressive` | boolean | MAY | `true` iff the picture is progressive.
_Omitted when the codec did not signal it._ | -| `nb_fields` | integer | MAY | Number of displayed field periods this picture occupies (the soft-telecine / 2:3 pulldown basis):
`1` for a single field picture
`2` for a normal frame
`3`/`4`/`6` for `repeat_first_field` pulldown per §6.3.10 | - -Codecs that carry only a coding type (e.g. H.264 / HEVC / VC-1 through this -pipeline) omit `field_order` and `progressive` rather than guessing a default. - -## 8. Codec model and extensibility - -Core record members (§7) are codec-agnostic and present for every codec. -Codec-specific data is either (a) promoted to top-level members for a small, -registered set per codec profile (e.g. MPEG-2 §7.1), or (b) placed under an -`ext` object keyed by codec id for richer/optional data: - -```json -{ - "n": 42, - "type": "P", - "key": false, - "src": { - "sector": 17, - "byte": 924 - }, - "ext": { - "hevc": { - "temporal_id": 0, - "nal_type": 1 - } - } -} -``` - -New codecs and members are added through Appendix B (codec registry) without a -breaking version bump, provided readers continue to ignore unknown members (§11). - -## 9. Provenance and recovery semantics - -`src` is **byte-exact** to the source as read. `src.sector` counts in -`source.sector_size`-byte units; `src.byte` is the offset within that sector of -the AU's first byte. For multi-file sources, `src.file` indexes a writer-declared -file list. Provenance MUST be the value observed at demux time; an implementation -MUST NOT recompute `src` by re-parsing — the point of FVI is to *carry* the truth. - -### 9.1 Recovery - -Because FVI is provenance-native, it can record reliability. A record with -`"recovered":true` indicates the AU's source bytes required retry/marginal-read -recovery. This lets downstream tools surface or quarantine pictures whose bytes -are not byte-identical to a clean read — a capability legacy index formats lack. - -## 10. Time model - -All `pts`/`dts` are integers in units of `1/timescale` seconds. `pts` is -presentation (display) time; `dts` is decode time. Records are in **coded** -(decode) order, so `pts` is not necessarily monotonic across records (B-pictures -reorder); `dts` is non-decreasing. Readers needing display order sort by `pts`. - -## 11. Versioning and forward compatibility - -- `fvi_version` is the document version; this spec defines `1`. -- **Additive** changes (new OPTIONAL members, new registered codecs) do NOT bump - `fvi_version`. Readers MUST ignore members they do not recognize. -- A change that alters the meaning of an existing member or makes a new member - REQUIRED bumps `fvi_version`. -- A reader encountering a higher `fvi_version` than it implements SHOULD process - the members it understands and MUST NOT reject the document solely for the - version being higher, unless a member it relies on is absent. - -## 12. Conformance requirements (summary) - -A conformant **writer** MUST: emit a Header first; emit records in coded order -with contiguous `n`; populate `src` from demux; use named/registered codec ids; -encode one JSON value per UTF-8 LF-terminated line. - -A conformant **reader** MUST: accept any §4–§10 document; ignore unknown members; -not assume `picture_count`, `pts`, or `size` are present unless required above. - ---- - -## Appendix A — JSON Schema (informative) - -Header: - -```json -{ - "$schema": "https://json-schema.org/draft/2020-12/schema", - "type": "object", - "required": ["format", "fvi_version", "stream", "source", "timescale"], - "properties": { - "format": { "const": "freemkv/video-index" }, - "fvi_version": { "type": "integer", "minimum": 1 }, - "timescale": { "type": "integer", "minimum": 1 }, - "stream": { "type": "object", "required": ["codec", "width", "height", "scan"] }, - "source": { "type": "object", "required": ["medium"] } - } -} -``` - -Record: - -```json -{ - "$schema": "https://json-schema.org/draft/2020-12/schema", - "type": "object", - "required": ["n", "src", "type", "key"], - "properties": { - "n": { "type": "integer", "minimum": 0 }, - "type": { "enum": ["I", "P", "B"] }, - "key": { "type": "boolean" }, - "src": { - "type": "object", - "required": ["sector", "byte"], - "properties": { - "file": { "type": "integer" }, - "sector": { "type": "integer", "minimum": 0 }, - "byte": { "type": "integer", "minimum": 0 } - } - } - } -} -``` - -## Appendix B — Registered codec identifiers - -| `codec` | Bitstream | Field profile | -|---|---|---| -| `mpeg2video` | ISO/IEC 13818-2 | §7.1 (field_order/progressive/nb_fields) | -| `mpeg1video` | ISO/IEC 11172-2 | §7.1 | -| `h264` | ISO/IEC 14496-10 | core + `ext.h264` | -| `hevc` | ISO/IEC 23008-2 | core + `ext.hevc` | -| `vc1` | SMPTE 421M | core | - -## Appendix C — Normative references - -- RFC 2119, RFC 8174 — Requirement keywords (BCP 14). -- RFC 8259 — JSON. -- ISO/IEC 13818-2 — MPEG-2 video (picture coding, §6.3.9–6.3.10). -- ISO/IEC 14496-10 — H.264/AVC. ISO/IEC 23008-2 — H.265/HEVC. -- ITU-T H.273 — Coding-independent code points (colour primaries/transfer/matrix). -- SMPTE ST 2086 — Mastering display colour volume (HDR). -- BCP 47 — Language tags. -- RFC 9559 — Matroska (alignment of colour/field-order semantics). diff --git a/README.md b/README.md index c833d045..6d761eb8 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,7 @@ Streams — unified PES pipeline ``` Build API documentation with `cargo doc --no-deps --open`. The public -[FVI format specification](FVI_FORMAT.md) describes exported video indexes. +[FVI format specification](https://freemkv.org/docs/fvi-format/) describes exported video indexes. ## Error Codes diff --git a/src/mux/codec/pgs.rs b/src/mux/codec/pgs.rs index 640bddd9..39ace908 100644 --- a/src/mux/codec/pgs.rs +++ b/src/mux/codec/pgs.rs @@ -294,7 +294,7 @@ impl CodecParser for PgsParser { match pcs_num_objects { // Every PCS starts a new set, including empty compositions. Keep // the clear's following WDS/END with it, rather than emitting orphan - // packets that lose timing in FFmpeg's pgs_frame_merge filter. + // packets that lose timing when display sets are merged. Some(_) => match pts { Some(start) => { out.extend(self.emit_pending(Some(start))); diff --git a/src/mux/codec/pgs_tests.rs b/src/mux/codec/pgs_tests.rs index 21aaaf27..39974272 100644 --- a/src/mux/codec/pgs_tests.rs +++ b/src/mux/codec/pgs_tests.rs @@ -625,7 +625,7 @@ fn ffmpeg_remux_keeps_all_subtitle_timestamps_without_warnings() { ); assert!(!packet["flags"].as_str().unwrap().contains('C'), "{packet}"); } - // The clear commands must survive FFmpeg's automatic pgs_frame_merge BSF. + // Remuxing must preserve both clear commands as empty compositions. let json = probe(&output, true); assert_eq!(json["frames"][1]["num_rects"], 0, "{json}"); assert_eq!(json["frames"][3]["num_rects"], 0, "{json}"); diff --git a/tests/fvi_pipeline.rs b/tests/fvi_pipeline.rs index 794c6293..b1c57a09 100644 --- a/tests/fvi_pipeline.rs +++ b/tests/fvi_pipeline.rs @@ -294,7 +294,7 @@ fn fvi_sink_indexes_real_mpeg2_pipeline_output() { vec![0, 3], "stamped src sectors must reach the .fvi in arrival order; got {src_sectors:?}" ); - // Per FVI_FORMAT.md §9, `src.byte` is the AU's offset WITHIN its 2048-byte + // Per https://freemkv.org/docs/fvi-format/#9-provenance-and-recovery-semantics, `src.byte` is the AU's offset WITHIN its 2048-byte // `src.sector` (always < 2048). These sources sit 14 bytes into their // sector (past the pack header), so byte must be exactly 14. for r in &records {