Skip to content

getLedgerEntries: optionally use high-performance Core server#353

Merged
Shaptic merged 30 commits intoprotocol-23from
migrate-getledgerentries
Feb 7, 2025
Merged

getLedgerEntries: optionally use high-performance Core server#353
Shaptic merged 30 commits intoprotocol-23from
migrate-getledgerentries

Conversation

@Shaptic
Copy link
Contributor

@Shaptic Shaptic commented Jan 28, 2025

What

This introduces STELLAR_CAPTIVE_CORE_HTTP_QUERY_PORT, enabling a high-performance HTTP server on the Captive Core subprocess for fetching ledger entries.

This new endpoint is plumbed all the way through the getLedgerEntries handler, removing those database queries entirely.

This also updates the stellar/go monorepo reference and its cascading dependencies.

If STELLAR_CAPTIVE_CORE_HTTP_QUERY_PORT is set to 0 (the default value) RPC will use it's old and proven DB implementation of getLedgerEntries instead of querying Core.

Why

With CAP-66 arriving in Protocol 23 and its high-performance /getledgerentry endpoint (whose client was implemented in https://github.com/stellar/go/pull/5542), we don't need to store ledger entries in RPC anymore.

Today, we still need to provide snapshots to the simulation library, but tomorrow (in the opaque sense) we won't need to, so this is part of the transition work.

Related: #269.

Known limitations

TODO:

@Shaptic
Copy link
Contributor Author

Shaptic commented Jan 28, 2025

@2opremio this should probably target a new feature branch e.g. protocol-23 🙏

@2opremio 2opremio force-pushed the migrate-getledgerentries branch from c1923f2 to 41dd850 Compare February 5, 2025 17:48
@2opremio 2opremio force-pushed the migrate-getledgerentries branch from 0e0847c to a8c4b29 Compare February 6, 2025 02:22
@2opremio 2opremio force-pushed the migrate-getledgerentries branch from a8c4b29 to 372638d Compare February 6, 2025 02:24
@2opremio 2opremio changed the title [Draft] Replace database-backed getLedgerEntries with high-performance Core server Replace database-backed getLedgerEntries with high-performance Core server Feb 6, 2025
@2opremio 2opremio changed the title Replace database-backed getLedgerEntries with high-performance Core server getLedgerEntries: optionally use high-performance Core server Feb 6, 2025
@2opremio 2opremio marked this pull request as ready for review February 6, 2025 02:43
@2opremio 2opremio force-pushed the migrate-getledgerentries branch from 63b0d98 to 13d28da Compare February 6, 2025 02:52
@2opremio 2opremio force-pushed the migrate-getledgerentries branch from b78154c to 1a9e7cb Compare February 6, 2025 03:26
Copy link
Contributor Author

@Shaptic Shaptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really solid! Coupla questions but otherwise pretty sure this can get a ✔️

@Shaptic Shaptic changed the base branch from main to protocol-23 February 7, 2025 21:52
@Shaptic Shaptic merged commit 55c706c into protocol-23 Feb 7, 2025
19 checks passed
@Shaptic Shaptic deleted the migrate-getledgerentries branch February 7, 2025 21:52
@Shaptic Shaptic mentioned this pull request Feb 7, 2025
captiveCorePort uint16

// These only need to be unconflicting ports
captiveCorePort uint16
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@2opremio @Shaptic do you still need this captiveCorePort field given that we have introduced captiveCoreHTTPPort? It seems like captiveCorePort and captiveCoreHTTPPort are both trying to configure the same port

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

captiveCorePort is the peer port and not the http port. It should probably be tagged as such for clarity.

2opremio added a commit to 2opremio/soroban-rpc that referenced this pull request Mar 25, 2025
…lar#353)

* Replace getLedgerEntries DB queries with Core fetches
* Add infrastructure for testing the new Core http query server
* Sort entries in response according to request order
* Only test the query server from protocol 23 onwards
* Enable debug printouts for integration tests
* Make sure all ports are allocated at once to minimize clashes

---------

Co-authored-by: Alfonso Acosta <alfonso@stellar.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants