Skip to content

Migrate DB ledger read path to raw ledger bytes/view - #945

Open
cjonas9 wants to merge 7 commits into
mainfrom
optimize-getHealth
Open

Migrate DB ledger read path to raw ledger bytes/view#945
cjonas9 wants to merge 7 commits into
mainfrom
optimize-getHealth

Conversation

@cjonas9

@cjonas9 cjonas9 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What

This PR covers some lingering ledger read path migrations to use ledger bytes/views where applicable. This has the immediate affect of optimizing getHealth and getFeeStats, though no changes to these handlers was required. Here are the functions modified:

  • getLedgerRange/getLedgerRangeWithCache/getLedgerRangeWithoutCache
  • StreamLedgerRange (takes a StreamLedgerViewFn parameter instead of a StreamLedgerFn now)
    • some caller changes: daemon.mustInitializeStorage -> db.MultiMigration.Apply handle views instead of full LCMs. This is converted to an LCM in db.MultiMigration.Apply

An adjacent optimization to the getLedgerRange* functions was also made alongside these changes. In particular, these functions return only sequence numbers, but fetch entire LCM blobs (>1Mb). This is on the hot-path. To mitigate this, now, this behavior is a fallback -- we fetch the first 1Kb prefix of an LCM and use a view over it to fetch our sequences. This saves us some heavier fetching work and is relevant here because we no longer need to decode anything but the data that's relevant to the function. :-)

Finally, the go SDK version included was bumped, and accordingly, the abstraction of a RawLedger (a []byte overlay) was removed. This is because the SDK now includes a LedgerHeader() accessor, making the RawLedger obsolete.

Why

See #732. This is a component of adopting ledger views in RPC for query-path performance purposes.

Known limitations

N/A

@socket-security

socket-security Bot commented Aug 24, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatedgolang/​github.com/​stellar/​go-stellar-sdk@​v0.7.2 ⏵ v0.7.375 +1100100100100

View full report

@cjonas9 cjonas9 linked an issue Aug 24, 2026 that may be closed by this pull request
@cjonas9
cjonas9 marked this pull request as ready for review August 25, 2026 15:26
Copilot AI balanced review requested due to automatic review settings August 25, 2026 15:26
@cjonas9 cjonas9 self-assigned this Aug 25, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates ledger reads toward zero-copy XDR views for issue #732, reducing unnecessary decoding of large ledger metadata.

Changes:

  • Adds view-based ledger retrieval and streaming.
  • Optimizes ledger-range and latest-ledger field extraction.
  • Updates migrations, callers, mocks, tests, and SDK dependency.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
go.mod Bumps the Stellar SDK dependency.
go.sum Updates SDK checksums.
cmd/stellar-rpc/internal/db/ledger.go Implements view-based reads and prefix range queries.
cmd/stellar-rpc/internal/db/ledger_test.go Tests view streaming and prefix fallback.
cmd/stellar-rpc/internal/db/migration.go Lazily decodes views for migrations.
cmd/stellar-rpc/internal/db/migration_test.go Adapts migration tests to views.
cmd/stellar-rpc/internal/db/mocks.go Updates database reader mocks.
cmd/stellar-rpc/internal/daemon/daemon.go Processes streamed ledger views during initialization.
cmd/stellar-rpc/internal/methods/get_latest_ledger.go Builds responses directly from ledger views.
cmd/stellar-rpc/internal/methods/get_latest_ledger_test.go Updates latest-ledger test reader.
cmd/stellar-rpc/internal/methods/mocks.go Updates method-layer mocks for view APIs.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +265 to +267
// ledgerCloseTimePrefixBytes is the fast-path meta prefix fetched for range
// endpoints. Parsing falls back to the full blob if the header extends past it.
const ledgerCloseTimePrefixBytes = 1024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a micro-optimization that isn't worth the add'l double-fetch complexity. The bulk of the time should be on encodes and decodes rather than the DB fetches themselves, which we mitigate via views already. Can you justify that this is worthwhile via some numbers?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, good catch, I'll try to get some profiling on this later.

I'd intended to essentially eliminate the double-fetch path and just left that in for safety. I had figured that it was some wasted effort to fetch the entire DB when almost assuredly, only a small number of prefix bytes of the LCM were needed. But that can be discovered with numbers, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: To Do

Development

Successfully merging this pull request may close these issues.

Adopt XDR views: Migrate DB ledger read path to bytes / view

3 participants