[r3.5] docs: add --snap.chaintoml-url, update EXEC3_PARALLEL default#21687
Conversation
…ALLEL default for release/3.5
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
…oved to dedicated PR)
yperbasis
left a comment
There was a problem hiding this comment.
Verified the claims against release/3.5 code; requesting changes for these outstanding items:
snapshots-management.mdx
- The example output is impossible:
duComputeEstimatesonly sums files already on disk (archiveTotal += f.Sizeunconditionally), so the archive estimate always equals thetotal:line — afull-mode node can't showtotal: 1.2TBwith archive1.4TB. Rows also print in fixed order archive→full→blocks→minimal (the example shows blocks before full), andsteps 0–18.500is ~2.6× too high for mainnet@22M blocks at 390,625 txns/step. Suggest making the example an archive datadir and adding a sentence that estimates for less-pruned modes can't exceed what's on disk. - "one step ≈ a fixed number of blocks, depending on chain activity" is self-contradictory — a step is a fixed number of transactions (
config3.DefaultStepSize= 390,625 txNums), so the block span is what varies. Please reword. --datadir"defaults to the standard per-network data directory" — the default ispaths.DefaultDataDir()(OS-specific, chain-agnostic;seg duhas no--chain). Drop "per-network".
NAT page URL
configuring-erigon/nat.md→nat.mdchanges the route/fundamentals/configuring-erigon/nat→/fundamentals/nat, and no redirects plugin is configured, so external links/bookmarks 404. Add a@docusaurus/plugin-client-redirectsentry (or consciously accept the break) — and correct the PR body's "no files move, all page URLs unchanged" either way.
PR body staleness (it becomes the merge commit message)
- Body still says
--exec.workersdefaults to "half the CPU cores"; the diff correctly documents the full CPU count. - "Minor formatting fix in the glossary 'Pruning' entry" — no glossary change is in this diff (
docs/site/help-center/glossary-of-key-terms.mdis untouched; the related fix merged via #21689).
Nits
- The new Execution section omits
--exec.batched-ioand--exec.state-cache, both registered indefault_flags.goand visible in--help. nat.mdatsidebar_position: 10.5undercuts the "clean 1–23" renumbering — make it an integer.--snap.p2p-manifestis also absent on release/3.4, so it deserves the same (New in v3.5) tag as--snap.chaintoml-url.
A leftover LLM/editorial line ("Here is the rewritten and merged
Downloader section...") sat above the "Downloader and Synchronization"
heading in the CLI reference and leaked into the generated llms-full.txt
bundles. Remove it and regenerate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s, sidebar snapshots-management.mdx: - Rewrite the seg du example as an archive datadir: the estimator sums only on-disk files, so the archive row always equals the current total and no node-type estimate can exceed it. Fix row order (archive→full→blocks→minimal) and the step count (~7,000 for mainnet@22M blocks, not 18,500). Add a note explaining the estimator can't exceed what's on disk. - Reword the "step" definition: a step is a fixed number of transactions (config3.DefaultStepSize = 390,625 txNums), so blocks-per-step varies. - Drop "per-network" from the --datadir default (seg du has no --chain; default is paths.DefaultDataDir(), chain-agnostic). configuring-erigon.mdx: - Document --exec.batched-io and --exec.state-cache (registered, shown in --help). - Tag --snap.p2p-manifest as (New in v3.5), like --snap.chaintoml-url. Fundamentals sidebar: give NAT an integer position (11, after Default Ports) and renumber the following pages so the order stays clean 1–24. Regenerate llms.txt / llms-full.txt (+ static copies). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NAT moved from /fundamentals/configuring-erigon/nat to /fundamentals/nat. The docs site is served as static files on GitHub Pages, which can't do host-level 301s, so use the official @docusaurus/plugin-client-redirects (pinned to 3.10.0, matching core) to emit a client-side redirect from the old URL. Verified the stub builds at the old path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@yperbasis thanks for the thorough review — all points addressed. Pushed in
NAT page URL → kept the move but added a client-side redirect via PR body staleness
Nits
Also removed a stray editorial line ("Here is the rewritten and merged Downloader section…") that Copilot caught leaking into the CLI reference. Ready for another look when you have a moment. |
yperbasis
left a comment
There was a problem hiding this comment.
Verified the flag/prune/seg du claims against the code on release/3.5 — three remaining issues:
-
snapshots-management.mdx— example file counts don't add up. The breakdown rows sum to 40,008 files but the total line says 42,810.doDUcomputes the total as the exact sum of the category rows, so real output can't disagree. (Bytes/percentages are consistent.) -
snapshots-management.mdx—minimal 0.9TBis not a possibleByteCountrendering. 1254.1 GB − 300.0 GB = 954.1 GB < 1024 GB, so the binary prints954.1GB(the page itself says sizes use theByteCountformat).full 1.0TB/blocks 1.1TBare fine. -
configuring-erigon.mdx— "(New in v3.5)" tagging inconsistent. Only--exec.serialand--exec.no-pruneare tagged, but all seven--exec.*flags are new to the erigon binary in v3.5 (none exist in release/3.4'scmd/utils/flags.go; only theintegrationtool had--exec.workers). Tag the section once, e.g. "all--exec.*flags are new in v3.5; their env-var equivalents existed earlier".
…* tagging snapshots-management.mdx: - Fix the seg du example total file count: rows sum to 40,008, not 42,810 (doDU prints the total as the exact sum of the category rows). - Fix the minimal estimate rendering: 1254.1GB − 300.0GB = 954.1GB, which is < 1024GB so ByteCount prints "954.1GB", not "0.9TB". configuring-erigon.mdx: - All seven --exec.* flags are new to the erigon binary in v3.5 (none exist in release/3.4's cmd/utils/flags.go). Tag the section once instead of marking only --exec.serial / --exec.no-prune, and note the env-var equivalents predate the flags. Regenerate llms bundles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
….5-flag-updates-20260609
|
@yperbasis all three verified and fixed in
|
yperbasis
left a comment
There was a problem hiding this comment.
The three items from my previous review are fixed (verified against release/3.5 code). One remaining blocker, all within the seg du example block in snapshots-management.mdx — it is still not output the binary could produce:
other 0.5GBis not a possibleByteCountrendering. Sub-GiB sizes print in MB (common/bytes.go:26), so a 0.5 GiB category renders as512.0MB.- Missing
extensions:line. With a non-emptyothercategory,duFormatHumanalways printsextensions: <sorted list>directly under theotherrow — the example shows 8otherfiles but no such line (the page's own table even says extensions are "listed inline"). - The sizes can't belong to
mainnet | archiveat block 22M and contradict the Database page in these same docs (real Nov 2024 mainnet archive @ ~21.2M: snapshots total 2.3TB, ~1.2TB of it block/tx segments, idx 430GB > domain 300GB > history 280GB). The example shows total 1.2TB withdomainsas the largest category (612GB, 48.8%) and only 410GB of block segments — on mainnet archive, block segments dominate.
Simplest fix that kills the whole class: run erigon seg du against a real mainnet archive datadir and paste the actual output, lightly trimmed. That also fixes the estimates-table column padding, which currently doesn't match the real %-13s %10s %10s %-15s %s format (the breakdown rows, by contrast, are already byte-exact). Remember to regenerate the four llms*.txt artifacts afterwards.
Optional nit: the category table's "domains … usually the largest category" holds for pruned (full/minimal) nodes but not for mainnet archive — consider "typically the largest category on pruned nodes".
…y table) Address yperbasis's optional nit: "domains … usually the largest category" holds for pruned (full/minimal) nodes, not mainnet archive where block segments dominate. The main seg du example block rewrite (real output) will follow separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks @yperbasis — all three confirmed against the code, you're right on each:
Rather than hand-fabricate the block (and risk the estimate rows being subtly off again), I'm taking your recommended fix: paste real, lightly-trimmed In the meantime I've pushed the optional nit ( |
Captured from a live mainnet archive node (`--prune.mode archive`, block 25,296,977) via `erigon seg du` / `-v`. Replaces the fabricated example, resolving yperbasis's remaining blocker: - real ByteCount renderings (e.g. other = 9.8MB, not 0.5GB) - the `extensions:` line under `other` is present - realistic archive proportions (block segments dominant at 48.3%) - byte-exact estimates-table column padding (it's actual binary output) Verbose example now shows real block-segments + domains subcategory trees. Regenerate llms bundles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@yperbasis done in
The verbose example shows the real Two sanity notes:
|
snapshots-management.mdx repeated content that belongs to pruning-modes.md. Keep pruning-modes.md as the single source of truth and reduce the snapshots page to its snapshot-specific angle + pointers: - Drop the Archive/Full/Blocks/Minimal mode table (duplicated the pruning-modes table) — keep the seg-du mode-detection + estimates note. - Collapse the "EIP-8252 retention window change" re-explanation to a brief "what changes on disk" framing + a pointer to the pruning-modes breaking-change note; keep the "run seg du after the first prune pass" bit. - "Manual snapshot operations" (snapshot file-format upgrade/downgrade) is unchanged — it's snapshot-specific, not duplicated. Regenerate llms bundles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot review: the two seg du example blocks used bare ``` fences. Type them as ```text (plain terminal output) to match the repo convention (logs.md and 14 other files) and the docs-routine §7.3 rule, and to stop Prism from auto-highlighting the box-drawing/tree characters. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
….5-flag-updates-20260609
) Follow-up to #21687. Adds one cross-link per section of the **CLI Reference** (`configuring-erigon`) to its dedicated page — placed in the section explainer where possible, or on the most relevant flag. One link per target page, no duplicates of the links already present (nat / supported-networks / pruning-modes / eth_getProof). **Explainer links:** Database, Snapshots Management, TxPool, Default Ports, Interacting with Erigon (RPC), MCP, Logs, Sentry, Downloader, Caplin, Shutter Network, Docker Compose. **Targeted flag links:** JWT (on `--authrpc.jwtsecret`), Creating a dashboard (on `--metrics`). Build passes the `onBrokenLinks: 'throw'` check; `llms.txt` / `llms-full.txt` regenerated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bloxster <gianni.morselli@erigon.tech> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…amentals docs) (erigontech#21765) Forward-ports the docs changes merged to `release/3.5` in erigontech#21687 onto `main`. Verified against `origin/main` source per the dual-commit rule — all documented flags/defaults exist on main: `snap.chaintoml-url`, `snap.p2p-manifest`, the seven `exec.*` flags, `EXEC3_PARALLEL=true`, and `exec.workers` Value `NumCPU()/2` (same "half" `--help` text, so the doc's effective-default note applies identically). **Includes:** - New **Snapshots Management** page (`seg du`) with real mainnet-archive output - EIP-8252 retention-window breaking-change note in `pruning-modes.md` - CLI Reference: `--snap.chaintoml-url`, `--snap.p2p-manifest`, and the `--exec.*` flags (serial / no-prune / batched-io / state-cache), with the `--exec.workers` effective-default clarification - CLI Reference **flattened** (`configuring-erigon/index.mdx` → `configuring-erigon.mdx`; `/fundamentals/configuring-erigon` URL unchanged) - **NAT** moved to a top-level page; old URL preserved via `@docusaurus/plugin-client-redirects` - TLS Authentication cert-filename typo fix; Fundamentals sidebar reorder; mobile theme-color - `llms.txt` / `llms-full.txt` regenerated; build passes `onBrokenLinks: 'throw'` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Bloxster <gianni.morselli@erigon.tech>
…gontech#21772) Main counterpart of erigontech#21763 (which added the same cross-links on release/3.5). Now that erigontech#21765 (the erigontech#21687 forward-port) has merged, the flattened CLI Reference page and its link targets exist on `main`, so this applies cleanly. Adds one cross-link per section of the **CLI Reference** (`configuring-erigon`) to its dedicated page — in the section explainer where possible, or on the most relevant flag. One link per target, no duplicates of the links already present (nat / supported-networks / pruning-modes / eth_getProof). **Explainer links:** Database, Snapshots Management, TxPool, Default Ports, Interacting with Erigon (RPC), MCP, Logs, Sentry, Downloader, Caplin, Shutter Network, Docker Compose. **Targeted flag links:** JWT (on `--authrpc.jwtsecret`), Creating a dashboard (on `--metrics`). Build passes the `onBrokenLinks: 'throw'` check; `llms.txt` / `llms-full.txt` regenerated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Bloxster <gianni.morselli@erigon.tech> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses code→docs gaps found in weekly maintenance run w24, plus some Fundamentals housekeeping. Scope: release/3.5 only.
Flags & env vars (
configuring-erigon)--snap.chaintoml-urlflag (PR db/snapcfg, cmd/utils, node: add --snap.chaintoml-url override #21584), includingERIGON_REMOTE_PREVERIFIEDoverride precedence--snap.p2p-manifestflag (PR downloader: decentralized snapshot distribution via chain.toml P2P discovery #20526), tagged (New in v3.5)EXEC3_PARALLELdefault totrue(PR common/dbg: default EXEC3_PARALLEL=true #21591)--exec.serialand--exec.no-pruneflags (new in v3.5); also document--exec.batched-ioand--exec.state-cache--exec.workers. Its effective default is the full CPU-core count (inherited from theEXEC3_WORKERSfallback when the flag is unset); the flag's own--helptext says "half the CPU cores", which is a known inconsistency in the binaryPrune modes / snapshots
pruning-modes.mdwith the EIP-8252 retention-window breaking change (v3.5): full mode now prunes block bodies/receipts to the last 262,144 blocks (previously kept all post-merge blocks) and will stop serving older block/receipt data; state-history window grows 100k→262kseg du, snapshot categories, node-type estimates, EIP-8252 retention window). Theseg duexample uses an archive datadir, since the estimator only sums on-disk files and the archive row therefore equals the current totalerigon snapshots …ver-format upgrade/downgrade commands inget-started/installation/upgrading.mdFundamentals section housekeeping
|new blocks<br/>(Engine API)|), which the flowchart parser rejects; now quotedsidebar_positionvalues and scattered related pages. New order: concepts first (Architecture, Database, Pruning Modes, Snapshots Management, Caplin) → configuration → operations/tuning → security → integrations/fundamentals/configuring-erigon/nat→/fundamentals/nat. The old URL is preserved via a client-side redirect (@docusaurus/plugin-client-redirects, pinned to 3.10.0); the site is on GitHub Pages, which can't do host-level 301sconfiguring-erigon/index.mdx→configuring-erigon.mdx); its/fundamentals/configuring-erigonURL is unchangedMobile UI
Regenerated
llms.txt/llms-full.txtartifacts.Updated after Copilot + @yperbasis review: replaced the
seg duexample with real mainnet-archive output (correct ByteCount renderings,extensions:line, byte-exact estimates table); documentedother_extensions; made--datadiroptional and chain-agnostic; clarified full-mode block-pruning impact; corrected the--exec.workersdefault and tagged all--exec.*flags as new in v3.5; added--exec.batched-io/--exec.state-cache; fixed the RPC cert filename typos in TLS Authentication (RPC key.pem→RPC-key.pem,RPC.crtv→RPC.crt); gave NAT an integer sidebar position and moved the Modules overview first; removed a stray editorial note in the CLI reference; typed theseg ducode fences astext; de-duplicated pruning-mode concepts on the Snapshots page (now deferring topruning-modes.md); and fixedsidebar_positioncollisions.