Skip to content

parse_schema: report when Formats/Latest was truncated at SCHEMA_SCAN_LIMIT #188

Description

@DrunkOnJava

Problem

rvt::formats::parse_schema (src/formats.rs, ~line 544) scans at most the first 64 KiB of the decompressed Formats/Latest stream:

const SCHEMA_SCAN_LIMIT: usize = 64 * 1024;
let data = if decompressed.len() > SCHEMA_SCAN_LIMIT { &decompressed[..SCHEMA_SCAN_LIMIT] } else { decompressed };

That cap is deliberate (the tail is binary object data that trips the class-name heuristic), but nothing records that it was hit. On real project files the stream is far larger than 64 KiB, and the Discussion #112 investigation (docs/disc-112-coordination.md) found this silent cap is the dominant boundary on how much of the schema we ever look at. Callers — rvt-schema, rvt-inspect, the formats_latest_integrity diagnostic added in #168 — cannot tell "the schema fit" from "we stopped looking".

What to do

  1. Add a field to SchemaTable (e.g. scan_truncated: bool, or scanned_bytes + total_bytes) set by parse_schema when the cap applies. Parsing results must not change.
  2. Surface it in at least one user-visible place: the Formats integrity row of rvt-inspect / export_diagnostics.formats_latest_integrity is the natural home (see src/compression.rs FormatsLatestIntegrity and how feat(diagnostics): Formats/Latest multipage integrity (A2/PARSE-001) #168 wired it through), or the rvt-schema summary line.
  3. Unit test: a synthetic buffer of >64 KiB (a tiny valid schema prefix followed by zeros) sets the flag; a small buffer does not.

Acceptance

Scope notes

Small Rust change + test; runs on the checked-in tier-1 fixtures (they are under 64 KiB, so the flag is false for them — the >64 KiB case is the synthetic buffer in the unit test). Ask on the issue if the diagnostic plumbing is unclear.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:readerWork in the reader area.good first issueGood for newcomershelp wantedExtra attention is neededtype:featureNew product or library capability.

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions