Skip to content

docs: deep-clean pass — stale names, dead links, UI terminology, nav, orphaned images - #5282

Merged
QuiltSimon merged 8 commits into
masterfrom
docs/deep-clean
Sep 4, 2026
Merged

docs: deep-clean pass — stale names, dead links, UI terminology, nav, orphaned images#5282
QuiltSimon merged 8 commits into
masterfrom
docs/deep-clean

Conversation

@QuiltSimon

@QuiltSimon QuiltSimon commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Deep-clean editing pass over docs/ (readability, correctness, consistency). No changes outside docs/. Follows up on today's screenshot refresh (#5281).

What changed, by category

1. Stale product names and dead links (bd0515e)

  • advanced-features/good-practice.md: dropped the ancient "Quilt T4 Role" product name from the Scripts.CreateAdmin event description (now "the Quilt role").
  • advanced-features/s3-bucket-organization.md: removed the link to blog.quiltdata.com (DNS still resolves to Medium's IPs but all connections time out — the blog is gone); replaced with a plain lead-in to the existing data-lifecycle diagram. Also fixed the "on or more" typo.
  • Fixed 4 dead docs.quilt.bio deep links (GitBook changed its URL structure) by converting them to relative links: .quiltignore and Voila and cross-account bucket-policy links in FAQ.md, Package.set_dir reference in Troubleshooting.md.
  • advanced-features/s3-prefix-permissions.md: the "hide the Bucket tab" link pointed at a nonexistent Admin.md#show-and-hide-features-in-the-quilt-catalog anchor; it now points at Preferences.md, where that heading actually lives.

2. Terminology and UI-language consistency (bd0515e)

  • ElasticSearch / Elastic SearchElasticsearch (7 spots: Architecture, Qurator, Catalog Troubleshooting, Metadata, Catalog Installation, benchling example).
  • CloudformationCloudFormation (Catalog/Installation.md).
  • Quilt3quilt3 when referring to the Python package (working-with-a-bucket.md).
  • Old "Bucket tab" UI language → Files tab (s3-prefix-permissions.md, Preferences.md, FAQ.md "Last Modified" answer).
  • Catalog/Admin.md: the admin panel is now reached via the Admin entry in the left sidebar, not a navbar dropdown; removed the stale admin-dropdown.png screenshot reference.

3. Readability and typo fixes on high-traffic pages (dd4b791)

  • README.md: untangled the grammar of the three-component overview list (double "which is", "which provide extension of the core Quilt Capabilities").
  • FAQ.md: the "Do I have to login via quilt3 to use the Quilt APIs?" section was completely empty — it now has a short answer pointing to the headless-credentials question below it; fixed "a collection files" → "a collection of files".
  • Catalog/Installation.md: "find, understand, and file discoveries" → "make discoveries".
  • Walkthroughs: "There is a one local registry", "Packages names", "authenticate against, if" → "authenticate against it, if", "Updated version my package" → "Updated version of my package", "The message needs to add notes on a new revision" → commit-style-note phrasing.

4. SUMMARY.md nav cleanup (749d3c5)

  • Moved product-wide FAQ and Troubleshooting out of the Python SDK "More" subsection into the top-level intro block (they apply to the whole product, not just the SDK). Changelog and Contributing stay under SDK "More" since the changelog is SDK-centric.
  • Moved Metadata Management and Metadata Workflows from the top level (where they sat oddly above the section headers) into the "Quilt Platform" section.
  • No new top-level sections; all entries verified to point at real files.

5. Orphaned image cleanup (eb8a221)

Deleted 18 images verified unreferenced by any file in the whole repo (including api-reference/ and catalog source): active-directory.png, admin-bucket.png, admin-dropdown.png, athena-ui.png, buckets_dropdown.png, catalog_bucket_tab.png, catalog_homepage.png, catalog_packages_tab.png, catalog_search.png, create-stack.png, google_auth.png, okta-sso-general.png, params.jpg, quilt.gif, qurator-icon.png, skip.png, start.png, three-bucket-flow.png.

Kept certmgr-windows.png (used by api-reference) and the currently unreferenced but freshly captured catalog-filesbrowser-tab.png / catalog-home-search.png from today's screenshot batch.

Verification

  • npx --package=markdownlint-cli markdownlint . (exactly what lint-docs runs): clean.
  • uv run poe testdocs from api/python (exactly what test-testdocs runs): 42 passed, 70 skipped. All pytest.mark.skip / skipfile markers preserved.
  • Scripted audit of every relative link and image reference in docs/**/*.md: all resolve.
  • All SUMMARY.md entries point at real files.

Flagged but NOT changed

  • technical-reference.md still lists T4BucketReadRole / T4BucketWriteRole — these look like real CloudFormation resource/logical names, so I left them; someone who knows the current template should confirm.
  • Architecture.md baseline cost table ("$620/month", Elasticsearch $258.42, etc.) is likely stale but I can't verify current pricing.
  • Catalog/Admin.md "Users can switch between assigned roles via the dropdown menu in the navbar" — left as-is (the switch-role screenshots still show this flow) but worth re-verifying against v26.8 UI.
  • Catalog/Admin.md "Settings" section says custom links go "in the navbar" — may need re-wording for the sidebar UI, but the settings semantics are unclear to me so I didn't touch it.
  • FAQ "requires at least 256 IPs / minimum CIDR /24" vs Architecture.md's "recommend a CIDR block no smaller than /24" — these read as slightly different claims (hard requirement vs recommendation); left both.
  • walkthrough/*.md files are paired with .ipynb notebooks; my typo fixes touched only the .md side. If these are ever regenerated from the notebooks, the same typos should be fixed there.
  • External link rot found but deliberately not "fixed": medium.com and prestodb.io return 403 to curl (bot-blocking, pages load fine in a browser); legacy.quiltdata.com returns 200 and stays.

Correctness fixes from code audit (second batch)

A code-vs-docs audit produced a prioritized gap report; the high-confidence items were verified against origin/master and implemented (4 more commits).

Broken / incorrect examples

  • Quickstart: the "Access Your First File" example printed PackageEntry.get() as if it returned file contents; it returns the physical-key URI (quilt3/packages.py:274). The example now shows get() (location) vs get_as_string() (contents), and executes in testdocs.
  • uploading-a-package: new "Pushing again: revision conflicts" section documenting the top-hash conflict guard (QuiltConflictException, packages.py:1655-1668), the browse-then-push pattern, and force / dedupe. Note: this and other walkthrough pages are generated from .ipynb by test-gendocs; all walkthrough edits (including the batch-1 typo fixes, which had broken test-gendocs on this PR) are now made in the notebooks and the .md regenerated via poe gendocs.
  • installing-a-package: browse() with no registry argument reads the local registry, not the "default remote registry" the doc claimed (quilt3/backends/__init__.py:14-15).
  • authentication.md (verified hand-written, not in gendocs/pydocmd.yml): key.emailkey.user_email ×3 (quilt3/api_keys.py:25).
  • FAQ: manifest deletion is quilt3.delete_package, not quilt3.Package.delete (which edits the in-memory package); the R/CLI example misused push --dir as a registry path (it is the local directory; the registry is --registry); the notebook-sync recipe now warns about the conflict guard and about set_dir(".", ...) resetting package metadata.
  • Troubleshooting: new QuiltConflictException section.

Enforcement rules and stale lists

  • tabulator.md: column-name regex corrected in both places to ^[A-Za-z][A-Za-z0-9_-]*$, max 255 (shared/schemas/tabulatorTable.yml.json:47-49) — the doc claimed ^[a-z_][a-z0-9_]*$, which wrongly forbids uppercase/hyphens and wrongly allows a leading underscore.
  • Packaging.md: new Limits section listing the seven pkgpush error codes (ManifestTooLarge, PackageTooLargeToCopy, TooManyFilesToCopy, PackageTooLargeToHash, TooManyFilesToHash, FileTooLargeForHashing, RequestTooLarge) and what triggers each (values are stack-configured env vars); documented the three-way workflow semantics (omitted → default workflow applied, "" → none, name → that one) and the ^[\w-]+/[\w-]+$ package-name rule.
  • Metadata.md: dropped the S3 Select sizing guideline (service closed to new customers; Quilt no longer calls it).
  • Preview.md: media extension list fixed (.tsv removed — it renders as tabular; duplicate .mp4 removed; .m2ts added); bzip2 (.bz2) decompression documented alongside gzip.
  • VisualizationDashboards.md: quilt_summarize.json types list completed with json, jupyter, vega (schema enum has 9, doc listed 6).
  • Preferences.md: documented the missing ui.actions.openInDesktop (QuiltSync button; hidden from the visual editor but works via YAML) and ui.blocks.qurator keys.

New documentation

  • Admin.md: "Beta and preview features" section — the global Enable beta features switch (gates bucket Overview v2, the bucket header card, and the Athena Tabulator-tables panel; there is no ui.blocks.overviewV2 preference key) and the three Preview features toggles (New front door, ElasticSearch query console, Data products), per catalog/app/utils/features.ts; plus a short Support Diagnostics paragraph.
  • Search.md: new user-facing sections on the faceted search page: package vs object result types, bucket scope, the 7 package + 6 object system filters, typed user-metadata facets, the Include historical versions toggle (default latest-only), list/table views with per-column sorting, sort presets, and the automatic trailing-* substring rewrite for plain queries.
  • API reference: quilt3.admin.policies (9 public functions) added to gendocs/pydocmd.yml and api-reference/Admin.md regenerated. Regeneration was contained: only the policies section changed; cli.md and walkthroughs byte-identical. (This is the one edit outside docs/.)

CI

  • markdownlint, poe testdocs (42 passed / 70 skipped), and poe gendocs-check all pass locally. The earlier test-gendocs failure on this PR (batch-1 typo fixes made in generated .md only) is fixed by moving those edits into the .ipynb sources.

Flagged for maintainers (not changed)

  • sso-permissions.md store_last_login_context: the doc instructs setting this key but its own linked schema (shared/schemas/sso-config-1.0.json, additionalProperties: false) forbids it. Needs reconciling against the enterprise-side schema.
  • technical-reference.md PkgSelectLambdaRole (and possibly SearchHandlerRole): the pkgselect lambda no longer exists in the repo; KMS instructions likely stale, but confirming requires the deployment repo.
  • materialization.md: still describes the pre-6.4 copy-everything push default; current default copies only cross-bucket entries (selector_fn). A rewrite needs product wording decisions.
  • working-with-manifests.md: documents only the legacy SHA256 hash type; write default is sha2-256-chunked and CRC64NVME is supported. Notebook-generated file; left for a dedicated pass.
  • working-with-a-bucket.md search section: stale prerequisite (navigator_url alone doesn't set registryUrl) and stale return shape; needs regenerated example outputs against a live catalog.
  • Qurator docs: "always-available" overstates (requires authentication); search-bar → Qurator routing ships behind the front-door preview flag — worth documenting when that flag graduates.
  • pkgpush limit values: only the error codes are documented; the actual limit values are deployment-repo env vars, unverifiable from the public repo.

Greptile Summary

This PR performs a broad documentation cleanup and expands several behavior-focused reference sections.

  • Corrects stale product terminology, links, navigation, screenshots, and SDK examples.
  • Documents Catalog feature gates, search capabilities, package-operation limits, preferences, preview formats, and policy APIs.
  • Keeps generated walkthrough sources and rendered Markdown synchronized and removes unreferenced image assets.
  • Two Search-page descriptions remain broader than the implemented behavior: NOT queries are rewritten, and table view is package-only.

Confidence Score: 4/5

The PR appears safe to merge after optional documentation corrections for the Search page’s query-rewrite and table-view descriptions.

The underlying code and interfaces are unchanged, and the reviewed documentation largely matches current behavior; the remaining findings are non-blocking documentation inaccuracies affecting NOT queries and object-result view expectations.

Files Needing Attention: docs/Catalog/Search.md

Important Files Changed

Filename Overview
docs/Catalog/Search.md Adds detailed search filters, revision handling, views, sorting, and query-rewrite documentation, with two capability descriptions that need narrower wording.
docs/Catalog/Admin.md Updates navigation terminology and accurately documents current beta, preview, and support-diagnostics controls.
docs/Catalog/Packaging.md Documents Packaging Engine request semantics and server-side operation limits consistently with the implementation.
docs/Catalog/Preferences.md Adds valid QuiltSync and Qurator preference keys and updates Files-tab terminology.
docs/Quickstart.md Correctly distinguishes PackageEntry location retrieval from reading file contents.
docs/walkthrough/uploading-a-package.md Adds accurate package revision-conflict guidance synchronized with its notebook source.
docs/api-reference/Admin.md Adds generated documentation for the nine public policy-management functions.
gendocs/pydocmd.yml Adds the public quilt3.admin.policies module to generated API documentation.

Reviews (1): Last reviewed commit: "docs: add quilt3.admin.policies to the A..." | Re-trigger Greptile

Greptile also left 2 inline comments on this PR.

smkohnstamm and others added 4 commits September 3, 2026 21:00
- good-practice.md: drop ancient 'T4' product name from role description
- s3-bucket-organization.md: remove dead blog.quiltdata.com link (domain no longer serves); fix typo
- Normalize Elasticsearch / CloudFormation / quilt3 spelling
- Bucket tab -> Files tab (current catalog UI); Admin panel reached via left sidebar
- Fix 4 dead docs.quilt.bio deep links by converting to relative links
- s3-prefix-permissions.md: fix hide-Files-tab link that pointed at a nonexistent Admin.md anchor

Co-authored-by: Cursor <cursoragent@cursor.com>
- README: untangle grammar in the three-component overview list
- FAQ: answer the previously empty 'Do I have to log in via quilt3' section;
  fix 'collection [of] files'; Files view naming
- Catalog/Installation: 'file discoveries' -> 'make discoveries'
- walkthroughs: fix typos (package names, 'a one local registry',
  authenticate-against-it, commit message example)

Co-authored-by: Cursor <cursoragent@cursor.com>
- Move product-wide FAQ and Troubleshooting from the Python SDK 'More'
  subsection to the top-level intro block
- Move Metadata Management and Metadata Workflows from top level into the
  Quilt Platform section where they belong

Co-authored-by: Cursor <cursoragent@cursor.com>
Each verified unreferenced by any file in the repo (including api-reference
and catalog source). Mostly 2019-era catalog screenshots and CloudFormation
console shots replaced by today's refresh; admin-dropdown.png orphaned by
the Admin.md sidebar-UI fix.

Co-authored-by: Cursor <cursoragent@cursor.com>
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 34.92%. Comparing base (5e29a8f) to head (bd8e257).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #5282   +/-   ##
=======================================
  Coverage   34.92%   34.92%           
=======================================
  Files         741      741           
  Lines       23804    23804           
  Branches     6429     6429           
=======================================
  Hits         8314     8314           
  Misses      13746    13746           
  Partials     1744     1744           
Flag Coverage Δ
catalog 34.92% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

smkohnstamm and others added 4 commits September 3, 2026 21:26
- Quickstart: PackageEntry.get() returns the physical-key URI, not file
  contents; example now shows get() vs get_as_string() honestly
  [quilt3/packages.py:274-278,310-317]
- uploading-a-package: document the push top-hash conflict guard
  (QuiltConflictException), browse-then-push, force, dedupe
  [quilt3/packages.py:1655-1668,1455-1456]; edits made in the .ipynb source
  and .md regenerated via gendocs so test-gendocs stays green (also syncs the
  typo fixes from the previous commit into the notebooks)
- installing-a-package: browse() with no registry reads the *local* registry,
  not the default remote [quilt3/backends/__init__.py:14-15]
- authentication.md (hand-written, not autogenerated): key.email ->
  key.user_email, the actual APIKey field [quilt3/api_keys.py:25]
- FAQ: manifest deletion API is quilt3.delete_package (not Package.delete);
  fix the R/CLI push example (--dir is the local dir, --registry is the
  registry) [quilt3/main.py:461-490]; note conflict guard + metadata reset
  caveats on the sync recipe
- Troubleshooting: new QuiltConflictException section

Co-authored-by: Cursor <cursoragent@cursor.com>
- tabulator: column-name regex is ^[A-Za-z][A-Za-z0-9_-]*$ max 255, not
  ^[a-z_][a-z0-9_]*$ [shared/schemas/tabulatorTable.yml.json:47-49]
- Packaging: new Limits section (7 pkgpush error codes), workflow
  omitted/empty/named semantics, package-name regex
  [lambdas/pkgpush/__init__.py:62-73,447-455,646-656,788-795,895-911,976-986]
- Metadata: drop S3 Select sizing guideline (service sunset; Quilt no longer
  calls it)
- Preview: media list fixed (.tsv is tabular, .mp4 deduped, .m2ts added);
  bzip2 (.bz2) decompression documented [Preview loaders]
- VisualizationDashboards: quilt_summarize types list completed with json,
  jupyter, vega [shared/schemas/quilt_summarize.json]
- Preferences: document ui.actions.openInDesktop and ui.blocks.qurator
  [utils/BucketPreferences/BucketPreferences.ts:21,64,160,172]

Co-authored-by: Cursor <cursoragent@cursor.com>
- Admin: new 'Beta and preview features' section — the global beta switch
  (gates Overview v2, bucket header card, Athena Tabulator panel; no
  ui.blocks key exists for Overview v2) and the three preview toggles
  (front-door, elasticsearch-queries, data-products)
  [catalog/app/utils/features.ts:30-46, Admin/Settings/Settings.tsx:395-442];
  plus a Support Diagnostics paragraph
- Search: new sections covering result types, bucket scope, the 7 package +
  6 object system filters, typed user-metadata facets, the Include
  historical versions toggle, list/table views, sort presets, and the
  trailing-* query rewrite [containers/Search/model.ts:627-644,930-943,
  Search/Layout/PackageFilters.tsx:450-453]

Co-authored-by: Cursor <cursoragent@cursor.com>
The module (9 public functions) was missing from gendocs/pydocmd.yml, so the
generated Admin.md never included it. Added the module and regenerated via
'poe gendocs' (regeneration was contained: only this section changed; cli.md
and walkthroughs byte-identical).

Co-authored-by: Cursor <cursoragent@cursor.com>
@QuiltSimon
QuiltSimon marked this pull request as ready for review September 4, 2026 12:25
@QuiltSimon
QuiltSimon merged commit afb16dc into master Sep 4, 2026
47 checks passed
@QuiltSimon
QuiltSimon deleted the docs/deep-clean branch September 4, 2026 12:25
QuiltSimon added a commit that referenced this pull request Sep 4, 2026
… orphaned images (#5282)

Co-authored-by: Simon Kohnstamm <kohnsts@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread docs/Catalog/Search.md
Comment on lines +76 to +80
If your query contains none of the Elasticsearch special syntax below (field
selectors, wildcards, quotes, logical operators, grouping, fuzzy or negation
operators), the catalog automatically appends a trailing `*` so that plain
terms match as substrings. Queries that use any special syntax are passed
through unchanged.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 NOT Queries Are Rewritten

This says queries containing logical or negation operators pass through unchanged. However, the search rewrite recognizes AND, OR, and -, but not NOT. A documented query such as NOT foo is therefore sent as NOT foo*, changing exact-term negation to prefix negation and potentially producing unexpected results. Please document this exception or include NOT in the rewrite guard.

Comment thread docs/Catalog/Search.md
Comment on lines +65 to +69
#### Views and sorting

Results display as a list or as a **table** with togglable columns and
per-column sorting. The sort dropdown offers: Best match, Most recent first,
Least recent first, A → Z, and Z → A.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Table View Is Package-Only

This presents table view and per-column sorting as available for all search results, but the Catalog enables table view only for package results. Object searches always use the list, so please scope this paragraph to packages to avoid directing users toward an unavailable view.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants