chore: sync upstream SiaFoundation/indexd

[pcfreak30]

chore: sync upstream SiaFoundation/indexd

This PR synchronizes the latest upstream changes from SiaFoundation/indexd (through the v0.4.1 release). Functionally it introduces new application‑auth, remote‑migration, and slab‑management capabilities, plus several host/client reliability fixes.

New features

• Pre-authorized app connect keys

  • Adds limited-use, signed keys that let an application connect without an interactive user approval flow.
  • New preauthorized_keys database table and manager methods.
  • New admin REST endpoints: GET/POST /apps/preauthorized/keys, GET/DELETE /apps/preauthorized/keys/:publicKey.
  • The app registration flow accepts an optional preAuthorizedKey + signature proof.
  • Expired pre-authorized keys are pruned during the account maintenance loop.
  • Connect-key statistics now track active keys per quota.

• Remote migration worker

  • New indexd remote subcommand runs a migration-only node that helps a primary indexd migrate unhealthy slabs.
  • The primary node can disable its own migration loop via slabs.migrations: false and expose work through POST /migrations/batch.
  • Remote workers fetch batches of unhealthy slabs plus the migration state, perform sector recovery/upload, and report results with POST /migrations/results.
  • Adds a remote config section (address, password) and a docker-compose.remote.yml example.

• Slab versioning

  • Adds a version field to Slab, SlabSlice, PinnedSlab, and SlabPinParams.
  • The version is folded into the slab digest for versions > 0, preserving IDs for legacy version 0 slabs.

Lifecycle and API improvements

• Object deletion cleans up unreferenced slabs

  • Deleting an object now unpins any slab that is no longer referenced by that account’s objects and queues it for deletion.
  • The default prune cutoff is changed from 1 hour to 72 hours (DefaultSlabPruneCutoff).

• Unpinning a referenced slab returns 409 Conflict

  • DELETE /slabs/:slabid now fails with ErrSlabInUse/409 Conflict while one of the account’s objects still references the slab.

• Concurrency hardening

  • Object deletion, explicit unpin, and PinObject now lock slabs/sectors in consistent digest/root order to prevent races and double-accounting.

Host/client fixes

• Bounded host connect timeout

  • RHP4 dial attempts now time out after 10 seconds independently of the full RPC budget.

• Host price tip-height check

  • AppendSectors now checks the host’s claimed tip height against the local tip height; if it drifts beyond tolerance the client refreshes host settings.

• Settings cache hit skips redundant verification

  • Cached host settings that already passed signature validation are served without re-running ed25519 verification on the hot path.

• Host failure-rate decay

  • Failure rates decay by half every minute and are clamped to zero below a small threshold, so transient host issues are forgiven.

• Corrupt sectors treated as lost

  • proto.ErrSectorCorrupt is now recognized as a lost/corrupt sector during integrity checks and shard recovery.

• Auto-unblock usable hosts

  • After scanning, hosts that were auto-blocked for usability reasons and are usable again (with no active contracts) are automatically unblocked.

Metrics and statistics

• ContractsStats now include lockedAllowance and remainingAllowance.
• HostStats include per-host lockedAllowance and remainingAllowance, and active contract size is computed from non-renewed active contracts only.
• Prometheus metrics (indexd_connect_keys_active, indexd_connect_keys_active_quota, indexd_contracts_locked_allowance, indexd_contracts_remaining_allowance, indexd_host_locked_allowance, indexd_host_remaining_allowance) are added/updated.

Documentation and schema

• CHANGELOG.md updated with the v0.4.1 and v0.4.0 release notes.
• README.md documents the remote migration node configuration.
• OpenAPI specs (admin.yml, app.yml, components.yml) updated with pre-authorized keys, migration endpoints, and slab versioning.

[kody-ai]

Code Review Could Not Complete ⚠️

The review failed before suggestions could be generated.

Reason: This PR exceeded the maximum context size accepted by the configured model (openai_compatible).

To resolve, choose one:

• Switch to a recommended model: pick one of Kodus's recommended models in your BYOK settings (Claude Sonnet 4.6, Claude Opus 4.7, Gemini 3.1 Pro, GPT-5.4, Kimi K2.6 Coding, or GLM 5.1).
• Split the PR into smaller ones: file count drives prompt overhead.

After fixing the issue, comment @kody review on this PR to re-run the review.

Kody Guide: Usage and Configuration

Interacting with Kody

• Request a Review: Ask Kody to review your PR manually by adding a comment with the @kody start-review command at the root of your PR.

• Validate Business Logic: Ask Kody to validate your code against business rules by adding a comment with the @kody -v business-logic command.

• Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Current Kody Configuration

Review Options

The following review options are enabled or disabled:

Options	Enabled
Bug	✅
Performance	✅
Security	✅
Business Logic	✅

Access your configuration settings here.

​

[kody-ai]

The hard-coded string literal "connect-key" in accounts/manager_test.go and accounts/connect.go:289-289 contains the secret keyword "key" and violates Rule [1], which bans hard-coded secrets unless sourced from an environment variable or secure keystore. Move the value into an environment variable such as os.Getenv("TEST_CONNECT_KEY") or a secure test fixture, and reference that variable here.

Kody rule violation: Ban hard-coded secrets in Go source

Prompt for LLM

File accounts/manager_test.go:

Line 221:

The hard-coded string literal "connect-key" in accounts/manager_test.go and accounts/connect.go:289-289 contains the secret keyword "key" and violates Rule [1], which bans hard-coded secrets unless sourced from an environment variable or secure keystore. Move the value into an environment variable such as os.Getenv("TEST_CONNECT_KEY") or a secure test fixture, and reference that variable here.

_{Talk to Kody by mentioning @kody}

_{Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.}

​

​

[kody-ai]

Found critical issues please review the requested changes

• Hard-coded "connect-key" literal containing a secret keyword violates the ban on hard-coded secrets in Go source.