Skip to content

docs: complete D1.4a second-book evidence and extraction contract - #33

Merged
sunpole merged 15 commits into
mainfrom
docs/second-book-evidence-d1.4a
Jul 29, 2026
Merged

docs: complete D1.4a second-book evidence and extraction contract#33
sunpole merged 15 commits into
mainfrom
docs/second-book-evidence-d1.4a

Conversation

@sunpole

@sunpole sunpole commented Jul 24, 2026

Copy link
Copy Markdown
Owner

Цель

Завершить D1.4a: доказать техническую извлекаемость второй сохранённой книги и определить безопасный путь к отдельной базе без изменения текущего runtime, PWA и существующих данных.

Доказанный результат

  • PDF: _archive/source-files/Unlocking-Your-Dream-Student-Ma.pdf;
  • размер: 740193 bytes;
  • SHA-256: edb4127915bf720ee56c96612f825ea4e0ef2f5fc15192a56fb96e5c7ec4745a;
  • страниц: 55;
  • страниц с substantial direct text: 55;
  • searchable ratio: 1.0;
  • технические метрики: 85103 characters, 12479 words;
  • технический вывод: direct-text-extraction-viable.

Добавлено

  • docs/SECOND_BOOK_EVIDENCE.md;
  • docs/second-book-evidence.json;
  • docs/SECOND_BOOK_EXTRACTION_CONTRACT.md;
  • scripts/validate-second-book-evidence.mjs;
  • постоянная CI-проверка D1.4a;
  • baseline 23.8.12 в VERSION.md и ROADMAP.md;
  • factual uNews patchnote и новый real Chromium PNG exact GitHub evidence page.

Безопасный контракт

  • source PDF является immutable input;
  • полный извлечённый текст не коммитится в публичный репозиторий до rights review;
  • будущий extraction run сохраняет одну UTF-8 страницу на файл и provenance manifest;
  • второй dataset получает собственные local IDs и page/source references;
  • одинаковые numeric IDs между книгами не создают relation;
  • combined mode позже работает federated по отдельным indexes без destructive merge;
  • ru-current-v1 остаётся default и stable fallback.

Границы

Не изменялись:

  • существующие data/ файлы и 4 086 записей;
  • исходный PDF и _archive/;
  • runtime, PWA, Service Worker и package metadata;
  • versions/;
  • пользовательский интерфейс.

Временные analysis/capture workflows удалены из итогового diff. Полный извлечённый текст книги не сохранён в репозитории.

Проверки

GitHub Actions run №145 прошёл полностью:

  • regression tests;
  • project/data/handoff/screenshot/patchnote validation;
  • D1.2 dataset registry;
  • D1.3 deterministic data-quality reports;
  • D1.4 two-book architecture;
  • D1.4a evidence and extraction contract;
  • new patchnote/new image validation;
  • JavaScript syntax checks.

Следующий этап

D1.4b — private/non-public immutable raw-extraction pilot с page files, manifest и небольшой reviewed segmentation sample. Новый logical dataset в этом PR не создаётся.

Closes #32.

@sunpole sunpole changed the title docs: investigate second-book extractability D1.4a docs: complete D1.4a second-book evidence and extraction contract Jul 29, 2026
@sunpole
sunpole marked this pull request as ready for review July 29, 2026 04:40
@sunpole
sunpole merged commit 24cf1cf into main Jul 29, 2026
1 check passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 766ec19126

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"warnings": []
}
],
"full_output_sha256": "deterministic aggregate hash",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Define the aggregate-hash algorithm

Specify exactly which bytes feed full_output_sha256—including page order, separators, filename inclusion, and encoding. In the D1.4b pilot, concatenating page contents, concatenating page hashes, or hashing serialized manifest data would all satisfy the current wording while producing different values for the same extraction, so the raw-extraction gate cannot actually require a reproducible aggregate hash.

AGENTS.md reference: AGENTS.md:L145-L149

Useful? React with 👍 / 👎.

]
},
"text_layer": {
"tool": "pdftotext -layout (poppler-utils)",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Record the exact extraction-tool version

Replace this generic tool label with the exact pdftotext/Poppler version and execution platform used by the one-time workflow. Because that workflow installed whatever ubuntu-latest provided and the evidence asserts exact layout-derived character, word, and line counts, a future reproduction with a different Poppler release cannot distinguish tool-output drift from incorrect evidence.

Useful? React with 👍 / 👎.

Comment thread VERSION.md
Comment on lines +15 to +17
## Documentation, data and extraction baseline

`v23.8.12`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Synchronize the v23.8.12 documentation baseline

Add the v23.8.12 milestone to CHANGELOG.md and map the new evidence, contract, and validator in docs/FILE_MAP.md. This line advances the maintained baseline to v23.8.12, but the changelog still ends at v23.8.10 and the file map omits every newly introduced D1.4a artifact, leaving the repository's maintained history and file inventory inconsistent with VERSION.md and ROADMAP.md.

AGENTS.md reference: AGENTS.md:L199-L204

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

D1.4a: подготовить доказательства второй книги и extraction contract

1 participant