Skip to content

docs: bring v3.3 documentation updates to main#20264

Closed
bloxster wants to merge 111 commits into
mainfrom
gitbook_docs_v3.3
Closed

docs: bring v3.3 documentation updates to main#20264
bloxster wants to merge 111 commits into
mainfrom
gitbook_docs_v3.3

Conversation

@bloxster

@bloxster bloxster commented Apr 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Ports all v3.3.x documentation work from gitbook_docs_v3.3 to main, per the new convention that docs changes must land on both the release branch and main.

Same set of commits as PR #20263 (gitbook_docs_v3.3 → release/3.3).

Commits included:

  • docs: March 2026 w13/w14 — v3.3.10 updates
  • docs: Sync Times, Datadir Structure, Dev Chain pages
  • docs: overlay, parity, graphql namespace pages
  • docs: coverage audit fixes (security, ports, logs, caplin, configuring-erigon, etc.)
  • docs(gitbook-help): FAQs, best practices, known issues, troubleshooting gaps filled
  • docs: creating-a-dashboard accuracy fixes

🤖 Generated with Claude Code

mh0lt and others added 30 commits November 17, 2025 13:51
This potentially fixes:
#17590

Or at leaste it does not make it worse. From the goroutine trace it
seemed like the timers here may be deadlocked. This is based on the fact
that they are all asleep in the dump, and looking at the logs at leaste
one of them ought to have fired. As this is a guess and not a definitae
fix I have run qa many times in repetition just to make sure it doesnt
make the situation worse.

I think it also makes the code slightly more idiomatic go.
Fixes panic here:
https://github.com/erigontech/erigon/actions/runs/19416027636/job/55544466840

* Announce dispatcher torrent input desync fixes
* indexed.GetFirst memory allocation improvement

Reported in
https://discord.com/channels/1133875232453693480/1161378785908772965/1439803663588659333.

Needs specific CI run to verify.
also related to `no space left` on runner
… is present (#17963)

Cherry-pick #17935

Co-authored-by: lupin012 <58134934+lupin012@users.noreply.github.com>
Should fix #17855 (definitely
does for WSL), and possibly
#16012 pending test for raw
drives, or running native Windows but that wouldn't invalidate this fix.
Change Docker base and builder images in release.yml.
For better compatibility build and base images switched to Debian
Bookworm (12).

Mentioned in  #17989
pick #18006

---------

Co-authored-by: Giulio Rebuffo <giulio.rebuffo@gmail.com>
seems #18049 is important - but
i think it's big and controversial - likely will take time to
think/merge it. So, here is minimalistic version of that PR.
We actually never use metadata, so let's do one less network call

Co-authored-by: Giulio rebuffo <giulio.rebuffo@gmail.com>
pick #18132

Co-authored-by: Bilog WEB3 <155262265+Bilogweb3@users.noreply.github.com>
Pulls minimal fix for thrashing in the announce code caused when there's
a lot of already completed torrents added.

Should fix erigontech/erigon-qa#320 and
#18162.

Also fixes a rate limiter bug I mentioned to @AskAlexSharov, and an
ancient spammy log line.

I minimized the fix identified on the development branch of
anacrolix/torrent, if this isn't enough I can just pull the full fix.
The 3.3 changes:
https://github.com/anacrolix/torrent/compare/downstreams/erigon/release/3.3...erigon/release/3.3?expand=1.
closes #17990

Co-authored-by: awskii <awskii@users.noreply.github.com>
to better understand 
```
[WARN] [12-09|17:49:12.381] Could not to increment beacon state, trying again later app=caplin err="idx: index panic: at=v1.1-013050-013100-ValidatorEffectiveBalance.seg, runtime error: invalid memory address or nil pointer dereference, [type.go:540 panic.go:783 panic.go:262 signal_unix.go:925 type.go:556 caplin_state_snapshots.go:661 caplin_state_snapshots.go:648 caplin_state_snapshots.go:689 state_antiquary.go:556 state_antiquary.go:91 asm_amd64.s:1693]" slot=13206430
```
This fixes excessive updates in the announcer work queue due to an
excessively triggered event in the torrent client
[here](anacrolix/torrent@f946a94#diff-449c382592420ab0e94eb0707f863e9fdaeceea483a1f39ebce93c42b57ba04dL52-L1726).

This caused extremely high CPU when torrents exceeded around 13k in
count and caused a bottleneck on the client it couldn't recover from. It
didn't stop anything from operating it just got very slow, without any
scheduler issues or thrashing.

There's still more aggressive improvements on main but I tried to keep
it minimal.

The behaviour matches the profile in
#18162 so it should be an
exact fix for that. I think it also finally fixes
erigontech/erigon-qa#320.
Accidentally lost the original thrashing fix from
#18179 while cleaning up
branches for #18256.
pick #18267

Co-authored-by: Ilya Mikheev <54912776+JkLondon@users.noreply.github.com>
Co-authored-by: JkLondon <me@ilyamikheev.com>
This PR fixes the Beacon API events endpoint to publish
SingleAttestation objects (introduced in Electra) under the
`single_attestation` topic instead of `attestation`.

Currently, Caplin publishes all attestations under the `attestation`
topic, regardless of whether they are pre-Electra `Attestation` objects
or post-Electra `SingleAttestation` objects.

**References:**
-

https://github.com/ethereum/consensus-specs/blob/dev/specs/electra/p2p-interface.md#beacon_attestation_subnet_id
-

https://github.com/ethereum/beacon-APIs/blob/master/apis/eventstream/index.yaml#L71
-

https://github.com/ethereum/consensus-specs/blob/dev/specs/electra/beacon-chain.md#singleattestation

**For context:**

We'd love to see this added. We have some contributoors pushing to the
ethPandaOps Xatu dataset using Caplin - but we're not capturing
attestations because of it. **For example:**

https://lab.ethpandaops.io/xatu/contributors/abstractdrank74#attestation-latency

Co-authored-by: Matty Evans <mattevansnz@gmail.com>
lystopad and others added 12 commits March 25, 2026 14:59
…build (#20150)

## Summary

Backport of [#19853](#19853)
(originally cherry-picked to `release/3.4` as `38aca05899`) to
`release/3.3`.

- Improve release workflow robustness (early release existence check,
artifact verification, non-fatal skopeo delete, GitHub App token for
publish step)
- Inline debian package build — removes the separate
`reusable-release-build-debian-pkg.yml` workflow file
- Fix debian control file heredoc inside for loop
- Update docker actions to v4.0.0 (Node.js 24)
- Pin `actions/create-github-app-token` to v2.2.1 SHA (Node.js 24)
- Add `disable_version_check` input flag
- Bump `actions/checkout`, `upload-artifact`, `download-artifact` to
latest versions
- Change `checkout_ref` default to `'release/x.x'` — **operator must set
this manually** (acts as a safety check to prevent accidentally building
from wrong branch)

## 3.3-specific adjustments (second commit)

1. **`mcp` excluded from `BINARIES`** — `cmd/mcp` does not exist in
`release/3.3`; it was intentionally kept out of this branch
2. **Re-applied changes skipped by git auto-merge** — several hunks were
silently resolved to the 3.3 version due to context differences;
manually re-applied: `checkout_ref` default, `disable_version_check`
input, version validation step, and all action version bumps

## Test plan

- [ ] Trigger the release workflow on `release/3.3` with
`perform_release=false` and verify the build-release + build-debian-pkg
jobs complete successfully
- [ ] Verify `checkout_ref` default shows `'release/x.x'` in the
workflow dispatch UI (not `'main'`)
- [ ] Verify `reusable-release-build-debian-pkg.yml` is gone and debian
build runs inline
…lback (#20164)

Cherry-pick of #20157 and #20162 to `release/3.3`.

## Summary

- `build-release`: use GitHub App token for checkout so `git push` of
the release tag has `workflows` scope (#20157)
- `In-case-of-failure`: same App token treatment for the rollback `git
push -d` that deletes the tag on failure (#20162)

Both fixes require `RELEASE_BOT` App to have `Workflows: Write`
permission (already granted).

Co-Authored-By: Claude
- #Options section updated to verbatim erigon --help output from v3.3.10
  (includes new --rpc.blockrange.limit flag)
- Fixed broken Lighthouse link: mainnet-validator.html → validator_management.html

Co-Authored-By: Andy NanoClaw <noreply@nanoclaw.bot>
- hardware-requirements.md: add Sync Times section with table
  (Ethereum/Gnosis × Archive/Full/Minimal; Polygon noted as 3.1 only)
- new page: fundamentals/datadir-structure.md — directory layout with
  explanation of each folder + History on cheap disk symlink strategy
- new page: fundamentals/dev-chain.md — quick start, key flags, and
  link to full DEV_CHAIN.md tutorial
- SUMMARY.md: insert Datadir Structure after Optimizing Storage;
  insert Dev Chain at end of FUNDAMENTALS

Co-Authored-By: Andy NanoClaw <noreply@nanoclaw.bot>
- FAQs: add items 21 (pruned history / no earlier blocks) and 22 (RPi / ARM)
- Best practices: add Public RPC hardening row to Security table;
  add new section 6 "Advanced Performance Tricks" with shell commands
  for sync.loop.block.limit, vmtouch, SNAPSHOT_MADV_RND, db.pagesize
- Common errors: add items 8 (Docker UID/GID 1000 permission denied)
  and 9 (libsilkworm_capi.so missing shared library)
- Troubleshooting: add "Collecting Diagnostics" section (SIGUSR1 + pprof)
  and Hetzner firewall note for P2P ports
- Known issues: add db.pagesize section explaining the flag is set-once
  with link to integration README

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Drop deprecated DOCKER_BUILDKIT=1 and COMPOSE_DOCKER_CLI_BUILD=1 env
  vars from the docker-compose BuildKit workaround (BuildKit is default
  in Docker Engine 23.0+ and Compose V2)
- Label vmmap as macOS-only; add Linux equivalents (pmap -x PID and
  /proc/<PID>/smaps)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DOCS→CODE (stale/wrong):
- security.md: remove --txpool.nolocals (flag does not exist in Erigon)
- default-ports.md: remove --diagnostics.endpoint.port 6062 (flag not in binary)

Port fixes (default-ports.md):
- 30304: correct label from eth/67 to eth/69
- 8545: remove WebSocket from row; add separate 8546 row for WebSocket

Inaccuracies:
- logs.md: fix --verbosity default from 2 to 3/info
- configuring-erigon.md: --prune.include-commitment-history is the primary
  flag name; the experimental.* variants are aliases

CODE→DOCS gaps — new flag documentation:
- security.md: add --ws.origins CORS note, --ipcdisable/ipcpath
- logs.md: add --vmodule per-module verbosity flag
- caplin.md: add Beacon API configuration section (addr, cors, timeouts, groups)
- dev-chain.md: add --miner.etherbase, --miner.gaslimit, --miner.extradata
- layer-2-networks.md: add --polygon.wit-protocol note
- configuring-erigon.md: Network (+5 flags), RPC (+14 flags), DB (+etl.bufferSize),
  Sync (+fcu.timeout/prune/snap/experimental), GPO (+maxprice), Logging (+vmodule),
  new Silkworm section (10 flags), new Miscellaneous section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- overlay.md: full page written from source — two methods
  (overlay_getLogs with state override, overlay_callConstructor),
  archive node requirement, timeout flags, curl examples
- parity.md: minimal stub — only parity_listStorageKeys implemented,
  link to OpenEthereum archived spec, full parameter table
- graphql.md: minimal stub — --graphql flag, /graphql and /graphql/ui
  endpoints, link to Geth docs (EIP-1767 reference)
- interacting-with-erigon/README.md: add all three to namespace index
- SUMMARY.md: add all three pages

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add --metrics.port flag (default 6060); remove vague "default port varies"
- Fix erigon.json panel sections: replace fictional Storage/Network/State
  sections with actual Blockchain, Block consume delay, RPC, Private api
- Add warning that erigon_internals.json is Erigon's internal Grafana Cloud
  dashboard (requires pre-release Grafana, not for end users)
- Add missing ERIGON_GRAFANA_DASHBOARD env var to config table
- Strip all broken LaTeX $$ $\text{...}$ $$ formatting throughout
- Fix readme.md -> Readme.md casing; fix dashboards path reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix broken link: execution-apis/api-documentation/ → execution-apis/ (404) in interacting-with-erigon/README.md and two .gitbook includes
- Fix broken internal link to non-existent help-center/common-errors-and-solutions.md in configuring-erigon.md
- Update stale erigon.gitbook.io URLs to docs.erigon.tech (supported-networks, jwt)
- Fix URL typo in how-to-run-a-polygon-node.md (technologyspecifies → technology)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@awskii

awskii commented Apr 1, 2026

Copy link
Copy Markdown
Member

why it also comments most of pipelines, producing not only documentation changes?

@bloxster bloxster closed this Apr 1, 2026
@bloxster

bloxster commented Apr 1, 2026

Copy link
Copy Markdown
Collaborator Author

Closed because it contained unnecessary modifications to code.

pull Bot pushed a commit to Dustin4444/erigon that referenced this pull request May 13, 2026
## Summary

Brings the full `docs/site/` Docusaurus tree into `main`, incorporating
all documentation improvements developed against `release/3.4`.

**Scope:** `docs/site/**` + root `llms.txt` / `llms-full.txt` + removal
of superseded `docs/gitbook/` and `docs/gitbook-help/`. No changes to
Go, proto, or any non-docs source files.

### Included — merged to release/3.4

| PR | What |
|----|------|
| [erigontech#20883](erigontech#20883) | Docusaurus
v3 migration — full `docs/site/` tree, Docusaurus config, versioned v3.3
snapshot |
| [erigontech#20263](erigontech#20263) /
[erigontech#20264](erigontech#20264) | All v3.3
docs ported; branch/versioning convention established |
| [erigontech#20978](erigontech#20978) | Mobile
footer fix, SEO meta tags, OG image |
| [erigontech#20991](erigontech#20991) | Self-host
brand fonts (remove Google Fonts / CDN) |
| [erigontech#21000](erigontech#21000) | `llms.txt`
/ `llms-full.txt` generator script + root artifacts |
| [erigontech#21018](erigontech#21018) | May 2026
w19 maintenance — stale flags, broken links, accuracy fixes |
| [erigontech#21045](erigontech#21045) | CI:
docs-only path filter (skip Go jobs, run docs-site build) |
| [erigontech#21063](erigontech#21063) | `trace`
response fields reference + sync-monitoring guidance |
| [erigontech#21074](erigontech#21074) | Regenerate
`llms.txt` after sync-modes update |
| [erigontech#20997](erigontech#20997) | Brand font
consistency fix, installation page UX |

### Included — pending review on release/3.4

| PR | What |
|----|------|
| [erigontech#21030](erigontech#21030) | Automated
disk size pipeline: `update-disk-sizes.py`, `disk-sizes.json`,
`hardware-requirements.mdx` JSX fix, `generate-llms.py` `—` fallback,
unit tests |
| [erigontech#21129](erigontech#21129) | May 2026
w20 maintenance — `--caplin.nat`, `--caplin.columns-keep-slots`, RPC
subscription defaults, `nat.md` Caplin section, log.dir.verbosity
default |

### What changes on `main`

- `docs/site/` added (full Docusaurus tree, current v3.4 + frozen v3.3
snapshot)
- `docs/gitbook/` and `docs/gitbook-help/` removed (superseded by
Docusaurus)
- Root `llms.txt` and `llms-full.txt` updated to Docusaurus-generated
versions

> ⚠️ This PR does **not** remove `docs/gitbook/` yet — that cleanup will
be a separate commit once this PR is reviewed and approved.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Bloxster <bloxster@proton.me>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: lupin012 <58134934+lupin012@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.