Skip to content

feat(sdk): implement Rust SDK workflows and qualification gates - #160

Draft
forhappy wants to merge 69 commits into
mainfrom
codex/crab-sdk-remote-foundation
Draft

feat(sdk): implement Rust SDK workflows and qualification gates#160
forhappy wants to merge 69 commits into
mainfrom
codex/crab-sdk-remote-foundation

Conversation

@forhappy

@forhappy forhappy commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Objective and status

Deliver the complete Rust SDK defined by crab/docs/architecture/crab-sdk.md. The implementation milestone provides the public remote, write, local, managed, and native smart-HTTP surfaces plus permanent release gates. The PR remains a draft until every mandatory backend, managed-service, platform, fault, package, and performance cell has retained current-head evidence; registry publication is a separate authorized release action.

Implemented

  • Pinned remote Git reads, raw and hydrated content streams, archives, limits, progress, cancellation, read-only LFS verification, and explicit S3/GCS/Azure configuration.
  • Shared publication ownership across SDK, CLI, and HTTP: durable plans and receipts, renewable operation/ref leases, GC fences, atomic ref batches, historical reconciliation, retry-safe resumption, and preserved terminal outcomes.
  • Remote initialization plus streamed commit and file-edit preparation, including large-file artifacts, dependency validation, exact request binding, and recovery without a Git executable.
  • Local configuration/tool handshake, open/clone/fetch, shallow/deepen/unshallow, tags, prune, linked worktrees, durable fetch recovery, and native smart-HTTP Git transport.
  • Local status, stage, commit, checkout, pull with FF/merge/rebase and conflict continue/abort, hydration/dehydration, content prefetch, prepared push, force-with-lease, atomic refspecs, dry-run, and push recovery.
  • Managed repository list/create/rename/archive/restore with service revision tokens, scoped grant refresh, protected publication/finalization recovery, placement isolation, and typed authorization failures.
  • Public SDK guide and compiling examples, feature/capability inventory, API snapshot, MSRV metadata, publishable dependency manifests, tarball-based external-consumer builds, and a dedicated SDK CI workflow.

Milestone evidence through 447be2d7ec4

  • cargo +1.91.1 test -p crab-sdk --all-features --locked -j1: all 56 library tests and every non-live integration test pass after the current-head fixes; provider/service tests remain ignored locally because they require dedicated infrastructure.
  • cargo +1.91.1 clippy -p crab-sdk -p crab-remote --all-targets --all-features --locked -j1 -- -D warnings: pass.
  • actionlint, architecture gates, Rust formatting, SDK Python tests, capability inventory, and feature-closure checks pass locally. The architecture policy now classifies every workspace package and admits the exact 18-crate publishable SDK dependency closure.
  • SDK inventory: 48 qualification cells resolve to behavioral tests. Direct partial-clone and sparse-checkout mutations return UnsupportedCapability before changing repository bytes.
  • Packaging: all 18 crates package without path dependencies, then four clean external consumers build from generated tarballs for minimal, remote-content, local, and managed profiles.
  • Public API generation reproduces the 536-line snapshot exactly with cargo-public-api 0.52.0 and pinned nightly-2026-06-19 while supported compilation remains on Rust 1.91.1.
  • CI run 34273648421 passed current stable, Ubuntu 24.04, the contracts/docs/packages job, and the controlled 1 GiB read benchmark. The benchmark completed 20 cold/warm core/SDK samples, observed equal origin request and response-byte counts, recorded zero writes and failures, kept SDK peak RSS at 511,160,320 bytes under the 512 MiB ceiling, and measured median SDK/core ratios of 0.995 cold and 0.868 warm.
  • The same run passed every RustFS read, remote write/recovery, and 1 GiB local round-trip assertion. Its final cleanup failure was shared CAS and GC-fence data outside repository prefixes; the isolated workflow now removes its owned fixtures and still requires an empty bucket afterward.
  • Windows diagnosed the two hydrated-preparation failures as a Tokio file handle surviving until the immediately following staging reopen. Preparation now awaits and closes that handle on success, cancellation, and error. macOS cancellation now begins only after the first bounded stream frame, eliminating the race with a producer that could finish before cancellation.
  • Current-head CI run 34276014605 is validating the Windows, macOS, RustFS, package, performance, and affected workspace gates.

Release blockers before finalization

  • Retain passing current-head Windows, macOS, RustFS, contracts/package, performance, and affected-workspace artifacts.
  • Complete live GCS and Azure qualification on dedicated buckets.
  • Complete the real managed-service lifecycle/fault suite: grant expiry, revocation, lost finalize response, placement changes, cross-principal cache isolation, and capability rejection.
  • Complete the shared publication fault matrix for CLI, HTTP, compaction, scoped GC, lease loss, cancellation, and process termination, with current-head CI evidence.
  • Require every dedicated SDK and affected workspace check to pass on the rebased head.

No skipped live test, local diagnostic, or generated inventory is treated as release evidence. No registry publication is included in this PR.

@forhappy forhappy changed the title feat(sdk): add remote read foundation and owned cleanup feat(sdk): add remote reads and shared publication foundations Sep 7, 2026
@forhappy
forhappy force-pushed the codex/crab-sdk-remote-foundation branch from 4c88214 to 04c2cf5 Compare September 8, 2026 00:09
@forhappy forhappy changed the title feat(sdk): add remote reads and shared publication foundations feat(sdk): add remote reads and recoverable ref mutations Sep 8, 2026
@forhappy
forhappy force-pushed the codex/crab-sdk-remote-foundation branch from cc06501 to d26d327 Compare September 8, 2026 03:11
@forhappy
forhappy marked this pull request as ready for review September 8, 2026 13:50
@forhappy forhappy changed the title feat(sdk): add remote reads and recoverable ref mutations feat(sdk): implement Rust SDK workflows and qualification gates Sep 8, 2026
@forhappy
forhappy marked this pull request as draft September 8, 2026 18:42
@forhappy
forhappy force-pushed the codex/crab-sdk-remote-foundation branch from 4f7e418 to 21d5d7e Compare September 8, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant