downloader: decentralized snapshot distribution via chain.toml P2P discovery#20526
Merged
Conversation
…OC (#19657) Add core library for decentralized snapshot info-hash distribution via discv5 ENR metadata and BitTorrent. This runs parallel to the existing preverified.toml system with no modifications to existing code. New components: - ChainToml ENR entry type (FrozenTx + InfoHash, 28 bytes) - chain.toml generation from local .torrent files (deterministic) - chain.toml torrent creation and publish orchestration - P2P discovery via NodeSource interface (highest FrozenTx wins) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…eline (#19657) - Add enrUpdater field and SetENRUpdater/PublishLocalChainToml methods to Downloader - Add public GetP2PServer() to sentry GrpcServer for ENR access - Wire ENR updater callback in backend.go connecting sentry servers to downloader - Generate initial chain.toml from existing torrents on startup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add --snap.p2p-manifest flag that enables discovering snapshot manifests (chain.toml) from P2P peers via ENR instead of using centralized preverified.toml. Consumer-side implementation: - New flag --snap.p2p-manifest: skips centralized preverified.toml, starts with empty registry, populates from P2P discovery - Two-phase discovery loop: acquiring mode (merge discovered entries into registry) before initial sync, verify mode (compare and log mismatches) after SaveSnapshotHashes writes preverified.toml - CompositeNodeSource + ResolvingPeerNodeSource: combines discv5 routing table and devp2p connected peers, resolving stale handshake ENR records via discv5 Resolve() - Delayed ENR re-publish (30s) for existing chain.toml on startup, since P2P servers start lazily via SetStatus() - Chain.toml torrent seeding after PublishLocalChainToml Testing confirmed: - ENR discovery works: Node C (consumer) found chain-toml entries from Nodes A/B (producers) via resolved devp2p peer records - chain.toml info-hash successfully discovered from peers Known issues for next phase: - Torrent peer discovery: DownloadChainTomlByInfoHash times out because DHT is disabled (NoDHT=true) and public trackers can't route localhost. Need to wire torrent DHT and/or announce properly for small numbers of torrent sources. - Timing: ENR re-publish requires 30s delay for P2P server startup; peer connections before this have stale ENR records requiring discv5 Resolve() to get updated entries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tadata (#19657) Replace the single FrozenTx field in the chain-toml ENR entry with two fields that distinguish entries the node can vouch for (AuthoritativeTx) from entries heard from peers (KnownTx). For the initial implementation both values are equal, derived from the preverified registry's ExpectBlocks. The receiver decides its trust policy. Also removes the frozenTx parameter from PublishChainToml — values are now computed internally from the preverified registry. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3 tasks
AskAlexSharov
approved these changes
Apr 14, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR rebases and integrates decentralized snapshot manifest distribution by advertising a chain.toml torrent info-hash (plus BitTorrent port) via DevP2P ENR, allowing new nodes to discover snapshot availability from peers instead of relying on a centralized preverified.toml.
Changes:
- Add ENR entries (
chain-toml,bt) plus wiring in the backend to publish local manifests and discover manifests from peers. - Add downloader-side chain.toml generation/publishing, P2P discovery/merge loop, and a torrent peer manager to inject DevP2P peers into BitTorrent.
- Make the snapshot stage optionally wait for P2P manifest readiness (
--snap.p2p-manifest) before building download requests.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| p2p/server.go | Exposes discovery v5 instance to consumers. |
| p2p/sentry/sentry_grpc_server.go | Exposes started P2P server to backend wiring. |
| p2p/enr/chain_toml.go | Adds chain-toml ENR entry type and RLP encoding. |
| p2p/enr/chain_toml_test.go | Tests ENR encoding/size behavior for chain-toml. |
| p2p/enr/bt.go | Adds bt ENR entry type (BitTorrent port). |
| p2p/enr/bt_test.go | Tests ENR encoding/size behavior for bt. |
| node/ethconfig/config.go | Adds config flags/channel for P2P manifest mode readiness. |
| cmd/utils/flags.go | Adds --snap.p2p-manifest CLI flag and config plumbing. |
| node/cli/default_flags.go | Includes the new CLI flag in defaults. |
| node/eth/backend.go | Wires ENR publishing, node source discovery, starts discovery loop and peer manager. |
| db/downloader/downloader.go | Adds ENR updater hooks, manifest discovery controls, and post-download republish. |
| db/downloader/chaintoml.go | Generates/saves/builds/publishes chain.toml + its torrent. |
| db/downloader/chaintoml_test.go | Unit tests for chain.toml generation/publishing helpers. |
| db/downloader/chaintoml_consumer.go | Implements TOML parse/merge + info-hash download and discovery loop logic. |
| db/downloader/chaintoml_consumer_test.go | Tests TOML parsing/merge/compare utilities. |
| db/downloader/p2p_chaintoml.go | NodeSource abstractions and “best peer” selection for chain.toml. |
| db/downloader/p2p_chaintoml_test.go | Tests peer selection and ENR extraction behavior. |
| db/downloader/torrent_peer_manager.go | Keeps BitTorrent peer set in sync with DevP2P peers. |
| db/downloader/torrent_peer_manager_test.go | Tests add/remove behavior of the peer manager. |
| db/downloader/bt_peer_discovery_test.go | End-to-end test for direct peer injection from ENR BT port. |
| execution/stagedsync/stage_snapshots.go | Adds manifestReady gating before snapshot download request construction. |
| execution/stagedsync/stageloop/stageloop.go | Threads manifestReady through stage construction. |
| execution/execmodule/execmoduletester/exec_module_tester.go | Updates snapshots stage cfg call signature. |
| execution/commitment/qmtree/PRUNE_POLICY.md | Adds QMTree prune policy design doc (includes a time-specific milestone). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ) | ||
|
|
||
| func TestChainTomlPath(t *testing.T) { | ||
| assert.Equal(t, "/data/snapshots/chain.toml", ChainTomlPath("/data/snapshots")) |
There was a problem hiding this comment.
TestChainTomlPath hard-codes a POSIX path ("/data/snapshots/chain.toml"), but ChainTomlPath uses filepath.Join, which will produce platform-specific separators. This test will fail on Windows. Build the expected path with filepath.Join (or filepath.FromSlash) instead of a hard-coded string.
Suggested change
| assert.Equal(t, "/data/snapshots/chain.toml", ChainTomlPath("/data/snapshots")) | |
| assert.Equal(t, filepath.Join("/data/snapshots", "chain.toml"), ChainTomlPath("/data/snapshots")) |
Comment on lines
+118
to
+123
| // Read the downloaded chain.toml file | ||
| filePath := ChainTomlPath(snapDir) | ||
| data, err := os.ReadFile(filePath) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("reading downloaded chain.toml: %w", err) | ||
| } |
There was a problem hiding this comment.
DownloadChainTomlByInfoHash assumes the downloaded payload will end up at ChainTomlPath(snapDir) and reads that file directly. A peer can advertise an info-hash whose metainfo names a different file, in which case this reads stale/incorrect data (or nothing) even though the torrent completed. Consider validating the metainfo (single file, expected name) and reading the file(s) from the torrent's info (or storage) rather than assuming a fixed on-disk path.
Comment on lines
+162
to
+173
| // If P2P manifest mode is enabled, wait for chain.toml discovery before | ||
| // building download requests. Without this, the preverified registry is | ||
| // empty and OtterSync would complete instantly with nothing to download. | ||
| if cfg.manifestReady != nil { | ||
| log.Info(fmt.Sprintf("[%s] Waiting for P2P manifest discovery...", s.LogPrefix())) | ||
| select { | ||
| case <-cfg.manifestReady: | ||
| log.Info(fmt.Sprintf("[%s] P2P manifest ready, proceeding with download", s.LogPrefix())) | ||
| case <-ctx.Done(): | ||
| return ctx.Err() | ||
| } | ||
| } |
There was a problem hiding this comment.
Waiting on cfg.manifestReady has no timeout/fallback, so a node started with --snap.p2p-manifest can hang forever at the snapshot stage if discovery never succeeds (no peers, empty manifests, download failures, etc.). Consider adding a bounded wait (configurable timeout) and a clear failure mode (e.g., proceed with centralized preverified, or abort with an explicit error) so the sync loop can't stall indefinitely.
Comment on lines
+93
to
+105
| func DiscoverChainToml(nodes NodeSource) *ChainTomlPeer { | ||
| var best *ChainTomlPeer | ||
|
|
||
| for _, node := range nodes.AllNodes() { | ||
| var ct enr.ChainToml | ||
| if err := node.Record().Load(&ct); err != nil { | ||
| continue // peer doesn't have chain-toml entry | ||
| } | ||
|
|
||
| if best == nil || ct.KnownTx > best.ChainToml.KnownTx { | ||
| best = &ChainTomlPeer{ChainToml: ct, Node: node} | ||
| } | ||
| } |
There was a problem hiding this comment.
DiscoverChainToml currently picks the highest KnownTx without any sanity checks. It should likely skip invalid/malformed entries (e.g., KnownTx < AuthoritativeTx, or an all-zero InfoHash), otherwise the downloader can attempt to fetch a zero/invalid info-hash or prefer obviously-bad ENRs.
Comment on lines
+42
to
+55
| for _, e := range entries { | ||
| if e.IsDir() || !strings.HasSuffix(e.Name(), ".torrent") { | ||
| continue | ||
| } | ||
|
|
||
| fPath := filepath.Join(torrentDir, e.Name()) | ||
| mi, err := metainfo.LoadFromFile(fPath) | ||
| if err != nil { | ||
| continue // skip unreadable torrent files | ||
| } | ||
|
|
||
| // The data file name is the torrent file name without .torrent suffix. | ||
| dataName := strings.TrimSuffix(e.Name(), ".torrent") | ||
| infoHash := mi.HashInfoBytes() |
There was a problem hiding this comment.
GenerateChainToml currently includes every ".torrent" file in the snapshots dir, which will also pick up the generated "chain.toml.torrent". That makes the manifest self-referential and unstable (the chain.toml content can change just because its own torrent hash changed), and may publish an entry for "chain.toml" that consumers might treat like a snapshot file. Exclude ChainTomlFileName+".torrent" (and possibly other internal torrents) from the scan so the manifest only lists actual snapshot data torrents.
| Port int | ||
| } | ||
|
|
||
| func (a ipPortAddr) Network() string { return "" } |
There was a problem hiding this comment.
ipPortAddr implements net.Addr but Network() currently returns an empty string. net.Addr.Network is expected to return a non-empty network name (e.g. "tcp"), and some consumers use it for logging/behavior. Returning "tcp" (or whatever the torrent client expects) would be safer and more conventional.
Suggested change
| func (a ipPortAddr) Network() string { return "" } | |
| func (a ipPortAddr) Network() string { return "tcp" } |
Comment on lines
+290
to
+301
| // Signal that the P2P manifest is ready so the snapshot stage can proceed. | ||
| if d.manifestReady != nil { | ||
| select { | ||
| case <-d.manifestReady: | ||
| // already closed | ||
| default: | ||
| close(d.manifestReady) | ||
| } | ||
| } | ||
| } else { | ||
| d.logger.Debug("[chaintoml] no new entries from peer") | ||
| } |
There was a problem hiding this comment.
manifestReady is only closed when ApplyDiscoveredChainToml reports newCount > 0. If the discovered manifest is empty (or if the local registry already contains all entries), manifestReady is never closed and StageSnapshots will block indefinitely in P2P manifest mode. Close/signal readiness after the first successful discovery+download+parse attempt (even if it adds 0 entries), and/or add a bounded wait + fallback policy.
Suggested change
| // Signal that the P2P manifest is ready so the snapshot stage can proceed. | |
| if d.manifestReady != nil { | |
| select { | |
| case <-d.manifestReady: | |
| // already closed | |
| default: | |
| close(d.manifestReady) | |
| } | |
| } | |
| } else { | |
| d.logger.Debug("[chaintoml] no new entries from peer") | |
| } | |
| } else { | |
| d.logger.Debug("[chaintoml] no new entries from peer") | |
| } | |
| // Signal that the P2P manifest is ready so the snapshot stage can proceed. | |
| // Readiness depends on a successful discovery+download+apply cycle, even if it | |
| // adds zero new entries because the manifest is empty or already fully present. | |
| if d.manifestReady != nil { | |
| select { | |
| case <-d.manifestReady: | |
| // already closed | |
| default: | |
| close(d.manifestReady) | |
| } | |
| } |
Comment on lines
+14
to
+23
| // AuthoritativeTx and KnownTx distinguish two classes of entries: | ||
| // - Authoritative: entries the node can vouch for (local disk + preverified.toml) | ||
| // - Known: all entries including those heard from peers (≥ AuthoritativeTx) | ||
| // | ||
| // These ranges are non-interleaved: authoritative entries cover tx 0..AuthoritativeTx, | ||
| // known entries extend to KnownTx. The receiver decides its trust policy. | ||
| type ChainToml struct { | ||
| AuthoritativeTx uint64 // max tx for entries from local disk + preverified.toml | ||
| KnownTx uint64 // max tx for all entries (≥ AuthoritativeTx) | ||
| InfoHash [20]byte // BitTorrent V1 info-hash (SHA1) of the chain.toml torrent |
There was a problem hiding this comment.
ChainToml's field names/comments describe transaction-number ranges ("max tx"), but the current publisher sets these values from snapcfg.Cfg.ExpectBlocks (a block height), not a txNum. This mismatch makes the ENR metadata ambiguous and can lead to wrong peer selection/policy decisions. Either rename the fields to reflect blocks (e.g., AuthoritativeBlock/KnownBlock) or update the comments and ensure the values are computed in the same units everywhere (and validate invariants like Known >= Authoritative).
Comment on lines
+541
to
+546
| btPort := enr.BT(backend.downloader.TorrentPort()) | ||
| backend.downloader.SetENRUpdater(func(ct enr.ChainToml) { | ||
| for _, srv := range backend.sentryServers { | ||
| if p2p := srv.GetP2PServer(); p2p != nil { | ||
| p2p.LocalNode().Set(ct) | ||
| p2p.LocalNode().Set(btPort) |
There was a problem hiding this comment.
btPort is derived from downloader.TorrentPort() (int) and cast directly to enr.BT (uint16) and then always set into the ENR. If TorrentPort() is 0 (not listening yet / disabled) this advertises an unusable port, and a direct cast can also silently truncate if something unexpected is returned. Consider only setting the "bt" ENR key when the port is in [1..65535], and log/debug otherwise.
Suggested change
| btPort := enr.BT(backend.downloader.TorrentPort()) | |
| backend.downloader.SetENRUpdater(func(ct enr.ChainToml) { | |
| for _, srv := range backend.sentryServers { | |
| if p2p := srv.GetP2PServer(); p2p != nil { | |
| p2p.LocalNode().Set(ct) | |
| p2p.LocalNode().Set(btPort) | |
| backend.downloader.SetENRUpdater(func(ct enr.ChainToml) { | |
| torrentPort := backend.downloader.TorrentPort() | |
| validTorrentPort := torrentPort > 0 && torrentPort <= 65535 | |
| if !validTorrentPort { | |
| logger.Debug("[chaintoml] skipping bt ENR entry update", "torrentPort", torrentPort) | |
| } | |
| for _, srv := range backend.sentryServers { | |
| if p2p := srv.GetP2PServer(); p2p != nil { | |
| p2p.LocalNode().Set(ct) | |
| if validTorrentPort { | |
| p2p.LocalNode().Set(enr.BT(torrentPort)) | |
| } |
Nine issues from the automated review on #20526: 1. chaintoml.go: exclude chain.toml.torrent from generated manifest — previously the manifest was self-referential (its own torrent hash changed every time the manifest was rebuilt, causing a feedback loop). Consumers find chain.toml via the ENR info-hash, not the manifest. 2. chaintoml_consumer.go: close manifestReady after any successful discovery+apply cycle, not only when new entries were added. An already-fully-merged manifest was never producing the ready signal and --snap.p2p-manifest nodes would hang indefinitely. 3. stage_snapshots.go: bound the wait on manifestReady with a 5-minute timeout and fall back to the preverified registry. Avoids indefinite stalls when P2P discovery never succeeds (no peers, unreachable info-hash, etc.). 4. p2p_chaintoml.go: DiscoverChainToml now rejects ENR entries with a zero InfoHash or KnownBlocks < AuthoritativeBlocks. These would otherwise lead the downloader to chase unusable info-hashes. 5. backend.go: resolve torrent port inside the ENR updater callback (was captured at wiring time when the torrent client wasn't yet listening) and only set the "bt" ENR key when the port is in [1..65535]. Previously a zero port could be advertised. 6. enr/chain_toml.go: rename AuthoritativeTx/KnownTx → AuthoritativeBlocks/KnownBlocks. The fields were always populated from snapcfg.Cfg.ExpectBlocks (a block count) but named as if they were txNums. RLP encoding is positional so the rename is wire-compat. 7. chaintoml_consumer.go: validate the downloaded torrent's metainfo before reading from disk. Reject multi-file torrents and torrents whose single file is not named chain.toml, closing the attack surface where a peer could advertise an info-hash pointing at a bundle or misnamed file. 8. chaintoml_test.go: portable path test using filepath.Join on both sides so TestChainTomlPath no longer fails on Windows. 9. chaintoml_consumer.go: ipPortAddr.Network() returns "tcp" instead of "" — net.Addr.Network is expected to be non-empty and some torrent-client consumers use it for logging or dialer selection.
mh0lt
added a commit
that referenced
this pull request
Apr 15, 2026
… commit
The glamsterdam suite was tracking upstream
ethpandaops/ethereum-package@main unpinned, while other suites
(regular, pectra) pin to a specific version. Upstream commit 835dd9b
("feat: support gpu ere prover in zkboost", 2026-04-15) introduced an
undefined GpuConfig reference in zkboost_launcher.star:272, breaking
every Erigon PR that ran glamsterdam from that point on — regardless
of what the PR changed. Six unrelated PRs (#20471, #20472, #20526,
#20583, #20584, #20585) all failed identically.
Pin to e07503d16b (2026-04-13, the commit before the break) to stop
the bleeding. This was the last state under which our main successfully
ran glamsterdam. Revisit once upstream stabilises, or switch to a
tagged release that supports gloas_fork_epoch / fulu_fork_epoch which
our glamsterdam.io config requires.
2 tasks
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Apr 16, 2026
…nge arithmetic (#20527) ## Summary Foundation for decentralized snapshot distribution (#19660) and sparse snapshot loading. Adds the state model that storage uses to decide what to download, what to seed, what to advertise, and what trust level to require. ### New package: `node/components/storage/snapshot/` **Trust model** (`trust.go`): - `TrustLevel`: none → consensus → verified (incremental, from #19657/#19658/#19659) - Files carry trust indicating how integrity was established - Promotion only (never demotion): downloaded → consensus agreed → UCAN verified - `Satisfies(required)` for consumer-side filtering **Range arithmetic** (`ranges.go`): - `StepRange`: half-open interval [From, To) of aggregator steps - `StepRanges`: sorted, non-overlapping range set with: - Normalize, Coverage, Contains, IsComplete - Gaps (find uncovered ranges within a window) - GapsAgainst (what does a peer have that I'm missing?) - Union (combine two range sets) **File inventory** (`inventory.go`): - `FileEntry`: snapshot file with domain, step range, torrent hash, trust level, local/remote/seeding flags - `Inventory`: thread-safe per-domain file registry - Coverage queries (local-only, full, or filtered by trust level) - Gap analysis against peer manifests - Atomic `ReplaceWithMerge` for merge-safe file rotation - `PromoteTrust` for trust ladder progression ### How it connects - **Decentralized snapshots**: inventory is the source of truth for chain.toml generation and consumer-side gap fill - **Sparse snapshots**: inventory tracks which step ranges are locally available for partial loading - **Storage component**: inventory will be wired into the storage service for download/seed coordination - **Downloader**: reads inventory to know what to seed, writes when downloads complete ## Test plan - [x] 14 tests covering range arithmetic, inventory CRUD, trust promotion, merge rotation, local vs remote - [x] `make lint` passes - [x] `make erigon` builds **Depends on:** #20526 (decentralized snapshots rebase)
mh0lt
added a commit
that referenced
this pull request
Apr 17, 2026
…r refs to Provider After the merge with main (PR #20526 added chain.toml P2P discovery), two cleanup items were missed: 1. setUpSnapDownloader and initDownloader functions remained in backend.go but were never called — their work is done by node/components/downloader/Provider since PR #20471. 2. The new P2P/chain.toml wiring in backend.go (ENR updater, node source, chain.toml publish, peer manager) referenced the deleted backend.downloader field. These calls now go through backend.downloaderProvider.Downloader. All 7 method accesses are hoisted via a local alias to reduce line noise, and guarded by a nil check on both the Provider and its embedded *Downloader. Remote-downloader mode has a nil Downloader field, so the entire chain.toml block is skipped — remote downloaders run their own P2P stack. Lint: clean. Build: erigon + integration OK.
bloxster
added a commit
that referenced
this pull request
Jun 12, 2026
…21687) Addresses code→docs gaps found in weekly maintenance run w24, plus some Fundamentals housekeeping. **Scope: release/3.5 only.** ### Flags & env vars (`configuring-erigon`) - Add `--snap.chaintoml-url` flag (PR #21584), including `ERIGON_REMOTE_PREVERIFIED` override precedence - Add `--snap.p2p-manifest` flag (PR #20526), tagged *(New in v3.5)* - Update `EXEC3_PARALLEL` default to `true` (PR #21591) - Add `--exec.serial` and `--exec.no-prune` flags (new in v3.5); also document `--exec.batched-io` and `--exec.state-cache` - Document `--exec.workers`. Its effective default is the **full CPU-core count** (inherited from the `EXEC3_WORKERS` fallback when the flag is unset); the flag's own `--help` text says "half the CPU cores", which is a known inconsistency in the binary ### Prune modes / snapshots - Update `pruning-modes.md` with the EIP-8252 retention-window breaking change (v3.5): full mode now prunes block bodies/receipts to the last 262,144 blocks (previously kept all post-merge blocks) and will stop serving older block/receipt data; state-history window grows 100k→262k - Add new "Snapshots Management" page under Fundamentals (`seg du`, snapshot categories, node-type estimates, EIP-8252 retention window). The `seg du` example uses an archive datadir, since the estimator only sums on-disk files and the archive row therefore equals the current total - Fix the `erigon snapshots …` ver-format upgrade/downgrade commands in `get-started/installation/upgrading.md` ### Fundamentals section housekeeping - Fix a Mermaid parse error on the **Architecture** page — the Caplin→Execution edge label had unquoted parentheses (`|new blocks<br/>(Engine API)|`), which the flowchart parser rejects; now quoted - Reorder the Fundamentals sidebar into a clean integer reading order. It had grown to ~23 entries with colliding `sidebar_position` values and scattered related pages. New order: concepts first (Architecture, Database, Pruning Modes, Snapshots Management, Caplin) → configuration → operations/tuning → security → integrations - **NAT** moved out of the CLI Reference subfolder to a top-level Fundamentals page (next to Default Ports): `/fundamentals/configuring-erigon/nat` → `/fundamentals/nat`. The old URL is preserved via a client-side redirect (`@docusaurus/plugin-client-redirects`, pinned to 3.10.0); the site is on GitHub Pages, which can't do host-level 301s - The **CLI Reference** page was flattened (`configuring-erigon/index.mdx` → `configuring-erigon.mdx`); its `/fundamentals/configuring-erigon` URL is unchanged ### Mobile UI - Adds a theme-color meta tag to the docs site so mobile browsers tint the address/status bar with the Erigon brand orange — the same behavior the main website, Cocoon and Zilkworm docs already have. Regenerated `llms.txt` / `llms-full.txt` artifacts. --- _Updated after Copilot + @yperbasis review: replaced the `seg du` example with **real mainnet-archive output** (correct ByteCount renderings, `extensions:` line, byte-exact estimates table); documented `other_extensions`; made `--datadir` optional and chain-agnostic; clarified full-mode block-pruning impact; corrected the `--exec.workers` default and tagged all `--exec.*` flags as new in v3.5; added `--exec.batched-io` / `--exec.state-cache`; fixed the RPC cert filename typos in **TLS Authentication** (`RPC key.pem` → `RPC-key.pem`, `RPC.crtv` → `RPC.crt`); gave NAT an integer sidebar position and moved the **Modules** overview first; removed a stray editorial note in the CLI reference; typed the `seg du` code fences as `text`; de-duplicated pruning-mode concepts on the Snapshots page (now deferring to `pruning-modes.md`); and fixed `sidebar_position` collisions._ --------- Co-authored-by: Andy (NanoClaw) <andy@nanoclaw.ai> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Bloxster <gianni.morselli@erigon.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebase of
feat/decentralized-snapshotsonto current main. Adds decentralized snapshot distribution where nodes advertise their chain.toml (snapshot manifest) via DevP2P ENR entries and new nodes discover snapshot availability from peers without relying on a centralized registry.Architecture
ENR entries:
chain-toml: advertises chain.toml torrent info-hash withAuthoritativeTx/KnownTxmetadatabt: advertises BitTorrent listen port for direct peer injectionFlow (new node with
--snap.p2p-manifest):chain-tomlENR entriesFlow (seeding node):
New files
db/downloader/chaintoml.godb/downloader/chaintoml_consumer.godb/downloader/p2p_chaintoml.godb/downloader/torrent_peer_manager.gop2p/enr/bt.gop2p/enr/chain_toml.goChanges to existing files
db/downloader/downloader.go— new P2P methods (SetENRUpdater, PublishLocalChainToml, etc.)node/eth/backend.go— wire ENR updater, P2P discovery, peer manager after sentry initexecution/stagedsync/stage_snapshots.go— wait formanifestReadybefore building download requestsnode/ethconfig/config.go—P2PManifestandManifestReadyconfig fieldsPreviously tested
3-node Hoodi test (seeder A, seeder B, leecher C with empty datadir):
Test plan
make lintpassesmake erigon integrationbuildsgo test ./db/downloader/...passes