Holesky Rescue - Feb 2025

This issue acts as a "front page" for the rescue efforts on Holesky. Holesky failed the Electra upgrade due to a config issue in some ELs.

The config issue in Nethermind, Geth & Besu resulted in an invalid block being justified. This means that for some users it will be impossible to start attesting on the valid chain without being slashed. Lighthouse will natively protect against this slashing; if you are affected your VC will log errors about slashing protection but you will not be slashed.

The community is still developing a unified approach to solving the slashing issue, but it is a strong possibility that these validators are destined to be slashed or leaked out. Even if your validators cannot attest, they can still produce blocks which is very valuable. We encourage all Holesky validators to try and get their setups online and functioning.

Lighthouse Advice

If you're a Holesky validator, we recommend you use the v7.0.0-beta.1 release.

There are some VC flags that may be relevant to your scenario:

• --beacon-nodes-sync-tolerances 1000,2000,3000
  • This flag may no longer be necessary due to recent improvements in block frequency.
• --disable-attesting
  • Only run this flag if you are struggling to sync a CL and EL pair and/or running out of memory. Once you are synced, remove this flag to allow your validator to attest to the canonical chain.

No additional flags are required for the BN.

Disabling Slashing Protection

DO NOT DO THIS UNTIL INSTRUCTED TO DO SO

If your validators won't attest due to CRIT errors from the slashing protection database, you should disable slashing protection in coordination with the rest of the validator set (monitor Discord/Telegram). To disable slashing protection in Lighthouse:

1. Stop the VC. Delete the slashing_protection.sqlite file located in $datadir/validators. You should also delete slashing_protection.sqlite-journal.
2. Start the VC again, using the flag --init-slashing-protection. The flag is required to force Lighthouse to start in the absence of a slashing protection DB. A new slashing protection DB will be created, but will not contain any of the historic information that would block your validators from signing.

Once slashing protection is disabled, your validators will most likely get slashed. Our hope is that they contribute to finalizing Holesky before doing so (hence the importance of a coordinated slashing).

EL Client Advice

• Make sure your EL is running a patched version (not relevant for Reth & Erigon, which do not have a config bug).
• If your EL was affected by the config issue, you may need to drop the EL database and resync.
• Reach out to your EL team for specific info.