Skip to content

test: run against stable lighthouse release in sim tests #7973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
nflaig merged 2 commits into from
Jun 18, 2025
Merged

test: run against stable lighthouse release in sim tests #7973

nflaig merged 2 commits into from
Jun 18, 2025

Conversation

@nflaig
Copy link
Member

@nflaig nflaig commented Jun 18, 2025

Motivation

Sim tests are failing likely due to sigp/lighthouse#7444. It might be that Lighthouse drops support for pre-electra altogether so we need to see how to make that work with our multi fork sim tests.

Description

Run against stable lighthouse release in sim tests

@codecov
Copy link

codecov bot commented Jun 18, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 55.74%. Comparing base (1b4cda3) to head (3e311a7).
Report is 1 commits behind head on unstable.

Additional details and impacted files 
@@            Coverage Diff            @@
##           unstable    #7973   +/-   ##
=========================================
  Coverage     55.74%   55.74%           
=========================================
  Files           833      833           
  Lines         59383    59383           
  Branches       4581     4581           
=========================================
+ Hits          33102    33105    +3     
+ Misses        26214    26211    -3     
  Partials         67       67
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nflaig nflaig marked this pull request as ready for review June 18, 2025 10:32
@nflaig nflaig requested a review from a team as a code owner June 18, 2025 10:32
nflaig
nflaig commented Jun 18, 2025
View reviewed changes
@nflaig nflaig mentioned this pull request Jun 18, 2025
Merged
ensi321
ensi321 reviewed Jun 18, 2025
View reviewed changes
.env.test
# Use either image or local binary for the testing
GETH_BINARY_DIR=
LIGHTHOUSE_DOCKER_IMAGE=sigp/lighthouse:latest-amd64-unstable
LIGHTHOUSE_DOCKER_IMAGE=sigp/lighthouse:v7.0.1
Copy link
Contributor

@ensi321 ensi321 Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want to do v7.0.1-amd64?

Copy link
Member Author

@nflaig nflaig Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

their image is multi arch, I am not exactly sure why we specifically used amd64 but this allows to run sim tests locally on any platform

@github-actions
Copy link
Contributor

github-actions bot commented Jun 18, 2025

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 05dde62 Previous: 1b4cda3 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0611 ms/op 1.1698 ms/op 0.91
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 37.807 us/op 41.347 us/op 0.91
BLS verify - blst 751.35 us/op 999.25 us/op 0.75
BLS verifyMultipleSignatures 3 - blst 1.2480 ms/op 1.6162 ms/op 0.77
BLS verifyMultipleSignatures 8 - blst 1.7253 ms/op 2.2139 ms/op 0.78
BLS verifyMultipleSignatures 32 - blst 4.9996 ms/op 7.0534 ms/op 0.71
BLS verifyMultipleSignatures 64 - blst 9.4442 ms/op 12.901 ms/op 0.73
BLS verifyMultipleSignatures 128 - blst 17.976 ms/op 21.845 ms/op 0.82
BLS deserializing 10000 signatures 724.71 ms/op 812.12 ms/op 0.89
BLS deserializing 100000 signatures 7.2499 s/op 7.6013 s/op 0.95
BLS verifyMultipleSignatures - same message - 3 - blst 930.84 us/op 930.17 us/op 1.00
BLS verifyMultipleSignatures - same message - 8 - blst 1.1356 ms/op 1.0484 ms/op 1.08
BLS verifyMultipleSignatures - same message - 32 - blst 1.8032 ms/op 1.9225 ms/op 0.94
BLS verifyMultipleSignatures - same message - 64 - blst 2.7256 ms/op 2.6163 ms/op 1.04
BLS verifyMultipleSignatures - same message - 128 - blst 4.8061 ms/op 4.4612 ms/op 1.08
BLS aggregatePubkeys 32 - blst 19.670 us/op 20.190 us/op 0.97
BLS aggregatePubkeys 128 - blst 72.912 us/op 72.447 us/op 1.01
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 52.524 ms/op 51.250 ms/op 1.02
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 48.438 ms/op 56.346 ms/op 0.86
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 39.128 ms/op 49.561 ms/op 0.79
getSlashingsAndExits - default max 76.560 us/op 86.761 us/op 0.88
getSlashingsAndExits - 2k 361.03 us/op 343.69 us/op 1.05
proposeBlockBody type=full, size=empty 8.0390 ms/op 5.7128 ms/op 1.41
isKnown best case - 1 super set check 228.00 ns/op 242.00 ns/op 0.94
isKnown normal case - 2 super set checks 226.00 ns/op 232.00 ns/op 0.97
isKnown worse case - 16 super set checks 230.00 ns/op 231.00 ns/op 1.00
InMemoryCheckpointStateCache - add get delete 2.6320 us/op 2.5160 us/op 1.05
validate api signedAggregateAndProof - struct 1.4592 ms/op 1.4888 ms/op 0.98
validate gossip signedAggregateAndProof - struct 1.4882 ms/op 1.6054 ms/op 0.93
batch validate gossip attestation - vc 640000 - chunk 32 122.22 us/op 121.27 us/op 1.01
batch validate gossip attestation - vc 640000 - chunk 64 105.39 us/op 105.85 us/op 1.00
batch validate gossip attestation - vc 640000 - chunk 128 95.576 us/op 100.63 us/op 0.95
batch validate gossip attestation - vc 640000 - chunk 256 99.173 us/op 107.91 us/op 0.92
pickEth1Vote - no votes 997.42 us/op 1.0363 ms/op 0.96
pickEth1Vote - max votes 5.6860 ms/op 6.4579 ms/op 0.88
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.058 ms/op 13.833 ms/op 0.80
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 16.059 ms/op 23.074 ms/op 0.70
pickEth1Vote - Eth1Data fastSerialize value x2048 452.74 us/op 457.30 us/op 0.99
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.5274 ms/op 3.7431 ms/op 0.94
bytes32 toHexString 380.00 ns/op 430.00 ns/op 0.88
bytes32 Buffer.toString(hex) 241.00 ns/op 305.00 ns/op 0.79
bytes32 Buffer.toString(hex) from Uint8Array 329.00 ns/op 360.00 ns/op 0.91
bytes32 Buffer.toString(hex) + 0x 239.00 ns/op 273.00 ns/op 0.88
Object access 1 prop 0.12700 ns/op 0.13900 ns/op 0.91
Map access 1 prop 0.13300 ns/op 0.14000 ns/op 0.95
Object get x1000 5.9740 ns/op 6.7020 ns/op 0.89
Map get x1000 6.6200 ns/op 7.0610 ns/op 0.94
Object set x1000 31.465 ns/op 33.501 ns/op 0.94
Map set x1000 20.536 ns/op 21.943 ns/op 0.94
Return object 10000 times 0.29080 ns/op 0.30990 ns/op 0.94
Throw Error 10000 times 4.3262 us/op 4.7155 us/op 0.92
toHex 151.90 ns/op 144.13 ns/op 1.05
Buffer.from 122.32 ns/op 127.47 ns/op 0.96
shared Buffer 99.248 ns/op 85.033 ns/op 1.17
fastMsgIdFn sha256 / 200 bytes 2.2330 us/op 2.4560 us/op 0.91
fastMsgIdFn h32 xxhash / 200 bytes 220.00 ns/op 248.00 ns/op 0.89
fastMsgIdFn h64 xxhash / 200 bytes 271.00 ns/op 275.00 ns/op 0.99
fastMsgIdFn sha256 / 1000 bytes 7.2680 us/op 7.6700 us/op 0.95
fastMsgIdFn h32 xxhash / 1000 bytes 351.00 ns/op 385.00 ns/op 0.91
fastMsgIdFn h64 xxhash / 1000 bytes 344.00 ns/op 396.00 ns/op 0.87
fastMsgIdFn sha256 / 10000 bytes 64.470 us/op 68.205 us/op 0.95
fastMsgIdFn h32 xxhash / 10000 bytes 1.8070 us/op 2.0620 us/op 0.88
fastMsgIdFn h64 xxhash / 10000 bytes 1.2280 us/op 1.2990 us/op 0.95
send data - 1000 256B messages 13.639 ms/op 15.458 ms/op 0.88
send data - 1000 512B messages 16.787 ms/op 20.520 ms/op 0.82
send data - 1000 1024B messages 26.297 ms/op 28.255 ms/op 0.93
send data - 1000 1200B messages 24.032 ms/op 28.306 ms/op 0.85
send data - 1000 2048B messages 22.589 ms/op 27.055 ms/op 0.83
send data - 1000 4096B messages 26.270 ms/op 31.233 ms/op 0.84
send data - 1000 16384B messages 77.755 ms/op 90.447 ms/op 0.86
send data - 1000 65536B messages 213.57 ms/op 229.14 ms/op 0.93
enrSubnets - fastDeserialize 64 bits 876.00 ns/op 1.8250 us/op 0.48
enrSubnets - ssz BitVector 64 bits 322.00 ns/op 364.00 ns/op 0.88
enrSubnets - fastDeserialize 4 bits 128.00 ns/op 137.00 ns/op 0.93
enrSubnets - ssz BitVector 4 bits 329.00 ns/op 361.00 ns/op 0.91
prioritizePeers score -10:0 att 32-0.1 sync 2-0 118.60 us/op 132.64 us/op 0.89
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 141.29 us/op 160.62 us/op 0.88
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 206.40 us/op 231.57 us/op 0.89
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 383.36 us/op 423.79 us/op 0.90
prioritizePeers score 0:0 att 64-1 sync 4-1 468.97 us/op 518.20 us/op 0.90
array of 16000 items push then shift 1.6372 us/op 1.8524 us/op 0.88
LinkedList of 16000 items push then shift 7.1940 ns/op 8.0390 ns/op 0.89
array of 16000 items push then pop 83.152 ns/op 82.239 ns/op 1.01
LinkedList of 16000 items push then pop 7.1520 ns/op 7.4010 ns/op 0.97
array of 24000 items push then shift 2.4619 us/op 2.4610 us/op 1.00
LinkedList of 24000 items push then shift 7.1270 ns/op 7.2830 ns/op 0.98
array of 24000 items push then pop 104.10 ns/op 106.35 ns/op 0.98
LinkedList of 24000 items push then pop 7.1070 ns/op 7.2900 ns/op 0.97
intersect bitArray bitLen 8 6.5070 ns/op 6.8430 ns/op 0.95
intersect array and set length 8 38.672 ns/op 39.665 ns/op 0.97
intersect bitArray bitLen 128 30.408 ns/op 33.018 ns/op 0.92
intersect array and set length 128 633.93 ns/op 713.99 ns/op 0.89
bitArray.getTrueBitIndexes() bitLen 128 1.0430 us/op 1.1360 us/op 0.92
bitArray.getTrueBitIndexes() bitLen 248 1.8260 us/op 1.9460 us/op 0.94
bitArray.getTrueBitIndexes() bitLen 512 3.8010 us/op 3.9460 us/op 0.96
Buffer.concat 32 items 622.00 ns/op 766.00 ns/op 0.81
Uint8Array.set 32 items 962.00 ns/op 1.2540 us/op 0.77
Buffer.copy 2.1340 us/op 3.1700 us/op 0.67
Uint8Array.set - with subarray 1.8890 us/op 2.4840 us/op 0.76
Uint8Array.set - without subarray 1.0810 us/op 1.8340 us/op 0.59
getUint32 - dataview 202.00 ns/op 236.00 ns/op 0.86
getUint32 - manual 126.00 ns/op 135.00 ns/op 0.93
Set add up to 64 items then delete first 2.1872 us/op 2.3054 us/op 0.95
OrderedSet add up to 64 items then delete first 3.2444 us/op 3.3729 us/op 0.96
Set add up to 64 items then delete last 2.5675 us/op 2.4006 us/op 1.07
OrderedSet add up to 64 items then delete last 3.9588 us/op 4.1495 us/op 0.95
Set add up to 64 items then delete middle 2.5476 us/op 2.8192 us/op 0.90
OrderedSet add up to 64 items then delete middle 5.4352 us/op 5.8468 us/op 0.93
Set add up to 128 items then delete first 5.8314 us/op 5.9390 us/op 0.98
OrderedSet add up to 128 items then delete first 7.7810 us/op 8.4486 us/op 0.92
Set add up to 128 items then delete last 5.4695 us/op 5.3071 us/op 1.03
OrderedSet add up to 128 items then delete last 8.3331 us/op 8.4176 us/op 0.99
Set add up to 128 items then delete middle 5.2962 us/op 5.3031 us/op 1.00
OrderedSet add up to 128 items then delete middle 16.050 us/op 18.231 us/op 0.88
Set add up to 256 items then delete first 10.729 us/op 11.127 us/op 0.96
OrderedSet add up to 256 items then delete first 17.019 us/op 16.376 us/op 1.04
Set add up to 256 items then delete last 10.835 us/op 10.688 us/op 1.01
OrderedSet add up to 256 items then delete last 15.749 us/op 17.033 us/op 0.92
Set add up to 256 items then delete middle 12.078 us/op 10.713 us/op 1.13
OrderedSet add up to 256 items then delete middle 45.560 us/op 44.049 us/op 1.03
transfer serialized Status (84 B) 2.3360 us/op 2.4910 us/op 0.94
copy serialized Status (84 B) 1.3250 us/op 1.3000 us/op 1.02
transfer serialized SignedVoluntaryExit (112 B) 2.3560 us/op 2.4840 us/op 0.95
copy serialized SignedVoluntaryExit (112 B) 1.2840 us/op 1.3400 us/op 0.96
transfer serialized ProposerSlashing (416 B) 2.4720 us/op 2.5990 us/op 0.95
copy serialized ProposerSlashing (416 B) 1.4430 us/op 2.6190 us/op 0.55
transfer serialized Attestation (485 B) 2.5150 us/op 2.5850 us/op 0.97
copy serialized Attestation (485 B) 1.8910 us/op 1.4870 us/op 1.27
transfer serialized AttesterSlashing (33232 B) 2.6600 us/op 2.8540 us/op 0.93
copy serialized AttesterSlashing (33232 B) 4.6300 us/op 3.9310 us/op 1.18
transfer serialized Small SignedBeaconBlock (128000 B) 3.1620 us/op 3.5480 us/op 0.89
copy serialized Small SignedBeaconBlock (128000 B) 12.572 us/op 11.393 us/op 1.10
transfer serialized Avg SignedBeaconBlock (200000 B) 3.8110 us/op 4.0480 us/op 0.94
copy serialized Avg SignedBeaconBlock (200000 B) 15.023 us/op 14.554 us/op 1.03
transfer serialized BlobsSidecar (524380 B) 3.9520 us/op 4.0800 us/op 0.97
copy serialized BlobsSidecar (524380 B) 60.891 us/op 166.16 us/op 0.37
transfer serialized Big SignedBeaconBlock (1000000 B) 3.8690 us/op 4.9750 us/op 0.78
copy serialized Big SignedBeaconBlock (1000000 B) 113.49 us/op 138.09 us/op 0.82
pass gossip attestations to forkchoice per slot 2.7774 ms/op 3.3042 ms/op 0.84
forkChoice updateHead vc 100000 bc 64 eq 0 463.93 us/op 496.18 us/op 0.94
forkChoice updateHead vc 600000 bc 64 eq 0 3.1363 ms/op 3.0869 ms/op 1.02
forkChoice updateHead vc 1000000 bc 64 eq 0 4.9831 ms/op 5.8819 ms/op 0.85
forkChoice updateHead vc 600000 bc 320 eq 0 2.9251 ms/op 2.9951 ms/op 0.98
forkChoice updateHead vc 600000 bc 1200 eq 0 3.0093 ms/op 2.9447 ms/op 1.02
forkChoice updateHead vc 600000 bc 7200 eq 0 3.3435 ms/op 3.3153 ms/op 1.01
forkChoice updateHead vc 600000 bc 64 eq 1000 10.669 ms/op 12.765 ms/op 0.84
forkChoice updateHead vc 600000 bc 64 eq 10000 10.620 ms/op 11.987 ms/op 0.89
forkChoice updateHead vc 600000 bc 64 eq 300000 19.910 ms/op 16.547 ms/op 1.20
computeDeltas 500000 validators 300 proto nodes 4.1574 ms/op 4.2933 ms/op 0.97
computeDeltas 500000 validators 1200 proto nodes 4.2708 ms/op 4.3086 ms/op 0.99
computeDeltas 500000 validators 7200 proto nodes 4.2598 ms/op 4.2026 ms/op 1.01
computeDeltas 750000 validators 300 proto nodes 6.1388 ms/op 6.5601 ms/op 0.94
computeDeltas 750000 validators 1200 proto nodes 6.4580 ms/op 6.7036 ms/op 0.96
computeDeltas 750000 validators 7200 proto nodes 6.7466 ms/op 6.8257 ms/op 0.99
computeDeltas 1400000 validators 300 proto nodes 12.712 ms/op 12.240 ms/op 1.04
computeDeltas 1400000 validators 1200 proto nodes 13.100 ms/op 11.615 ms/op 1.13
computeDeltas 1400000 validators 7200 proto nodes 12.821 ms/op 11.652 ms/op 1.10
computeDeltas 2100000 validators 300 proto nodes 21.447 ms/op 17.654 ms/op 1.21
computeDeltas 2100000 validators 1200 proto nodes 19.484 ms/op 17.845 ms/op 1.09
computeDeltas 2100000 validators 7200 proto nodes 19.902 ms/op 18.110 ms/op 1.10
altair processAttestation - 250000 vs - 7PWei normalcase 3.4352 ms/op 3.1152 ms/op 1.10
altair processAttestation - 250000 vs - 7PWei worstcase 5.2302 ms/op 3.9767 ms/op 1.32
altair processAttestation - setStatus - 1/6 committees join 169.04 us/op 158.69 us/op 1.07
altair processAttestation - setStatus - 1/3 committees join 304.45 us/op 307.64 us/op 0.99
altair processAttestation - setStatus - 1/2 committees join 423.34 us/op 405.17 us/op 1.04
altair processAttestation - setStatus - 2/3 committees join 526.48 us/op 534.15 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 704.27 us/op 728.01 us/op 0.97
altair processAttestation - setStatus - 100% committees join 834.36 us/op 865.05 us/op 0.96
altair processBlock - 250000 vs - 7PWei normalcase 5.2905 ms/op 5.7799 ms/op 0.92
altair processBlock - 250000 vs - 7PWei normalcase hashState 33.324 ms/op 35.812 ms/op 0.93
altair processBlock - 250000 vs - 7PWei worstcase 50.306 ms/op 43.158 ms/op 1.17
altair processBlock - 250000 vs - 7PWei worstcase hashState 98.827 ms/op 90.939 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 2.4592 ms/op 2.0916 ms/op 1.18
phase0 processBlock - 250000 vs - 7PWei worstcase 26.292 ms/op 25.551 ms/op 1.03
altair processEth1Data - 250000 vs - 7PWei normalcase 385.83 us/op 385.40 us/op 1.00
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 10.588 us/op 6.5640 us/op 1.61
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 57.338 us/op 46.450 us/op 1.23
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 20.292 us/op 11.944 us/op 1.70
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 8.6690 us/op 7.7470 us/op 1.12
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 192.38 us/op 190.91 us/op 1.01
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 2.1532 ms/op 2.0803 ms/op 1.04
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.8476 ms/op 3.0804 ms/op 0.92
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.9148 ms/op 2.8596 ms/op 1.02
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 6.6286 ms/op 6.6052 ms/op 1.00
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.8816 ms/op 3.0445 ms/op 0.95
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 6.6851 ms/op 6.4873 ms/op 1.03
Tree 40 250000 create 735.17 ms/op 642.99 ms/op 1.14
Tree 40 250000 get(125000) 165.41 ns/op 165.72 ns/op 1.00
Tree 40 250000 set(125000) 2.4155 us/op 2.0108 us/op 1.20
Tree 40 250000 toArray() 23.415 ms/op 21.467 ms/op 1.09
Tree 40 250000 iterate all - toArray() + loop 23.851 ms/op 21.810 ms/op 1.09
Tree 40 250000 iterate all - get(i) 66.919 ms/op 67.079 ms/op 1.00
Array 250000 create 3.0125 ms/op 3.9084 ms/op 0.77
Array 250000 clone - spread 1.9360 ms/op 1.6517 ms/op 1.17
Array 250000 get(125000) 0.50300 ns/op 0.46700 ns/op 1.08
Array 250000 set(125000) 0.48400 ns/op 0.49300 ns/op 0.98
Array 250000 iterate all - loop 109.90 us/op 121.74 us/op 0.90
phase0 afterProcessEpoch - 250000 vs - 7PWei 49.228 ms/op 48.522 ms/op 1.01
Array.fill - length 1000000 6.5755 ms/op 3.9653 ms/op 1.66
Array push - length 1000000 19.786 ms/op 15.317 ms/op 1.29
Array.get 0.29457 ns/op 0.31630 ns/op 0.93
Uint8Array.get 0.48779 ns/op 0.49134 ns/op 0.99
phase0 beforeProcessEpoch - 250000 vs - 7PWei 20.436 ms/op 20.637 ms/op 0.99
altair processEpoch - mainnet_e81889 315.89 ms/op 344.14 ms/op 0.92
mainnet_e81889 - altair beforeProcessEpoch 18.888 ms/op 18.474 ms/op 1.02
mainnet_e81889 - altair processJustificationAndFinalization 5.8790 us/op 6.2170 us/op 0.95
mainnet_e81889 - altair processInactivityUpdates 5.0752 ms/op 4.9315 ms/op 1.03
mainnet_e81889 - altair processRewardsAndPenalties 47.331 ms/op 38.959 ms/op 1.21
mainnet_e81889 - altair processRegistryUpdates 1.1720 us/op 763.00 ns/op 1.54
mainnet_e81889 - altair processSlashings 204.00 ns/op 217.00 ns/op 0.94
mainnet_e81889 - altair processEth1DataReset 192.00 ns/op 187.00 ns/op 1.03
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4202 ms/op 1.4412 ms/op 0.99
mainnet_e81889 - altair processSlashingsReset 1.2220 us/op 1.0590 us/op 1.15
mainnet_e81889 - altair processRandaoMixesReset 2.0290 us/op 1.2080 us/op 1.68
mainnet_e81889 - altair processHistoricalRootsUpdate 209.00 ns/op 189.00 ns/op 1.11
mainnet_e81889 - altair processParticipationFlagUpdates 681.00 ns/op 600.00 ns/op 1.14
mainnet_e81889 - altair processSyncCommitteeUpdates 154.00 ns/op 167.00 ns/op 0.92
mainnet_e81889 - altair afterProcessEpoch 49.920 ms/op 47.362 ms/op 1.05
capella processEpoch - mainnet_e217614 1.0106 s/op 957.78 ms/op 1.06
mainnet_e217614 - capella beforeProcessEpoch 76.501 ms/op 64.987 ms/op 1.18
mainnet_e217614 - capella processJustificationAndFinalization 6.2180 us/op 6.2560 us/op 0.99
mainnet_e217614 - capella processInactivityUpdates 15.970 ms/op 17.002 ms/op 0.94
mainnet_e217614 - capella processRewardsAndPenalties 208.08 ms/op 197.76 ms/op 1.05
mainnet_e217614 - capella processRegistryUpdates 8.5380 us/op 7.7230 us/op 1.11
mainnet_e217614 - capella processSlashings 252.00 ns/op 223.00 ns/op 1.13
mainnet_e217614 - capella processEth1DataReset 198.00 ns/op 211.00 ns/op 0.94
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.7243 ms/op 4.7031 ms/op 1.00
mainnet_e217614 - capella processSlashingsReset 1.0860 us/op 1.0740 us/op 1.01
mainnet_e217614 - capella processRandaoMixesReset 1.6430 us/op 1.4550 us/op 1.13
mainnet_e217614 - capella processHistoricalRootsUpdate 300.00 ns/op 219.00 ns/op 1.37
mainnet_e217614 - capella processParticipationFlagUpdates 594.00 ns/op 598.00 ns/op 0.99
mainnet_e217614 - capella afterProcessEpoch 129.01 ms/op 128.12 ms/op 1.01
phase0 processEpoch - mainnet_e58758 373.55 ms/op 336.23 ms/op 1.11
mainnet_e58758 - phase0 beforeProcessEpoch 95.790 ms/op 86.563 ms/op 1.11
mainnet_e58758 - phase0 processJustificationAndFinalization 6.5900 us/op 9.5840 us/op 0.69
mainnet_e58758 - phase0 processRewardsAndPenalties 43.488 ms/op 44.658 ms/op 0.97
mainnet_e58758 - phase0 processRegistryUpdates 3.6820 us/op 3.8450 us/op 0.96
mainnet_e58758 - phase0 processSlashings 197.00 ns/op 225.00 ns/op 0.88
mainnet_e58758 - phase0 processEth1DataReset 198.00 ns/op 215.00 ns/op 0.92
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.4331 ms/op 1.4456 ms/op 0.99
mainnet_e58758 - phase0 processSlashingsReset 1.1180 us/op 1.2810 us/op 0.87
mainnet_e58758 - phase0 processRandaoMixesReset 1.4690 us/op 1.5490 us/op 0.95
mainnet_e58758 - phase0 processHistoricalRootsUpdate 198.00 ns/op 220.00 ns/op 0.90
mainnet_e58758 - phase0 processParticipationRecordUpdates 974.00 ns/op 997.00 ns/op 0.98
mainnet_e58758 - phase0 afterProcessEpoch 40.308 ms/op 43.270 ms/op 0.93
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.5278 ms/op 1.8805 ms/op 0.81
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.8937 ms/op 6.4469 ms/op 0.45
altair processInactivityUpdates - 250000 normalcase 19.564 ms/op 21.440 ms/op 0.91
altair processInactivityUpdates - 250000 worstcase 21.745 ms/op 21.799 ms/op 1.00
phase0 processRegistryUpdates - 250000 normalcase 10.537 us/op 7.3730 us/op 1.43
phase0 processRegistryUpdates - 250000 badcase_full_deposits 290.66 us/op 339.48 us/op 0.86
phase0 processRegistryUpdates - 250000 worstcase 0.5 114.51 ms/op 126.88 ms/op 0.90
altair processRewardsAndPenalties - 250000 normalcase 26.237 ms/op 33.430 ms/op 0.78
altair processRewardsAndPenalties - 250000 worstcase 27.407 ms/op 31.883 ms/op 0.86
phase0 getAttestationDeltas - 250000 normalcase 18.326 ms/op 6.9168 ms/op 2.65
phase0 getAttestationDeltas - 250000 worstcase 7.8096 ms/op 11.285 ms/op 0.69
phase0 processSlashings - 250000 worstcase 93.713 us/op 92.067 us/op 1.02
altair processSyncCommitteeUpdates - 250000 11.340 ms/op 15.571 ms/op 0.73
BeaconState.hashTreeRoot - No change 219.00 ns/op 353.00 ns/op 0.62
BeaconState.hashTreeRoot - 1 full validator 83.868 us/op 122.12 us/op 0.69
BeaconState.hashTreeRoot - 32 full validator 1.0645 ms/op 1.6679 ms/op 0.64
BeaconState.hashTreeRoot - 512 full validator 11.461 ms/op 15.995 ms/op 0.72
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 103.34 us/op 172.81 us/op 0.60
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.7038 ms/op 2.3399 ms/op 0.73
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 23.882 ms/op 33.097 ms/op 0.72
BeaconState.hashTreeRoot - 1 balances 80.467 us/op 100.66 us/op 0.80
BeaconState.hashTreeRoot - 32 balances 1.0025 ms/op 967.00 us/op 1.04
BeaconState.hashTreeRoot - 512 balances 7.5256 ms/op 10.747 ms/op 0.70
BeaconState.hashTreeRoot - 250000 balances 186.30 ms/op 209.68 ms/op 0.89
aggregationBits - 2048 els - zipIndexesInBitList 21.636 us/op 25.710 us/op 0.84
byteArrayEquals 32 54.063 ns/op 59.277 ns/op 0.91
Buffer.compare 32 19.532 ns/op 18.673 ns/op 1.05
byteArrayEquals 1024 1.6001 us/op 1.7467 us/op 0.92
Buffer.compare 1024 26.279 ns/op 26.949 ns/op 0.98
byteArrayEquals 16384 25.484 us/op 27.981 us/op 0.91
Buffer.compare 16384 207.05 ns/op 220.28 ns/op 0.94
byteArrayEquals 123687377 189.66 ms/op 223.15 ms/op 0.85
Buffer.compare 123687377 6.0764 ms/op 9.1346 ms/op 0.67
byteArrayEquals 32 - diff last byte 52.243 ns/op 58.334 ns/op 0.90
Buffer.compare 32 - diff last byte 17.466 ns/op 19.395 ns/op 0.90
byteArrayEquals 1024 - diff last byte 1.5752 us/op 1.7628 us/op 0.89
Buffer.compare 1024 - diff last byte 25.882 ns/op 28.821 ns/op 0.90
byteArrayEquals 16384 - diff last byte 25.078 us/op 28.423 us/op 0.88
Buffer.compare 16384 - diff last byte 240.07 ns/op 224.50 ns/op 1.07
byteArrayEquals 123687377 - diff last byte 190.92 ms/op 216.13 ms/op 0.88
Buffer.compare 123687377 - diff last byte 7.4538 ms/op 9.3343 ms/op 0.80
byteArrayEquals 32 - random bytes 5.2630 ns/op 5.9890 ns/op 0.88
Buffer.compare 32 - random bytes 18.471 ns/op 20.913 ns/op 0.88
byteArrayEquals 1024 - random bytes 5.2010 ns/op 5.8830 ns/op 0.88
Buffer.compare 1024 - random bytes 17.611 ns/op 20.469 ns/op 0.86
byteArrayEquals 16384 - random bytes 5.2100 ns/op 5.7990 ns/op 0.90
Buffer.compare 16384 - random bytes 17.618 ns/op 18.786 ns/op 0.94
byteArrayEquals 123687377 - random bytes 6.6500 ns/op 7.1600 ns/op 0.93
Buffer.compare 123687377 - random bytes 19.700 ns/op 20.550 ns/op 0.96
regular array get 100000 times 40.512 us/op 51.783 us/op 0.78
wrappedArray get 100000 times 33.915 us/op 48.084 us/op 0.71
arrayWithProxy get 100000 times 13.701 ms/op 15.151 ms/op 0.90
ssz.Root.equals 47.630 ns/op 50.987 ns/op 0.93
byteArrayEquals 46.675 ns/op 49.760 ns/op 0.94
Buffer.compare 10.661 ns/op 11.266 ns/op 0.95
processSlot - 1 slots 10.420 us/op 12.712 us/op 0.82
processSlot - 32 slots 2.0352 ms/op 3.2491 ms/op 0.63
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 3.1815 ms/op 3.9502 ms/op 0.81
getCommitteeAssignments - req 1 vs - 250000 vc 2.2007 ms/op 2.3135 ms/op 0.95
getCommitteeAssignments - req 100 vs - 250000 vc 4.2278 ms/op 4.4918 ms/op 0.94
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4919 ms/op 5.1634 ms/op 0.87
findModifiedValidators - 10000 modified validators 742.09 ms/op 803.50 ms/op 0.92
findModifiedValidators - 1000 modified validators 734.76 ms/op 801.57 ms/op 0.92
findModifiedValidators - 100 modified validators 193.36 ms/op 283.17 ms/op 0.68
findModifiedValidators - 10 modified validators 277.54 ms/op 184.61 ms/op 1.50
findModifiedValidators - 1 modified validators 227.76 ms/op 162.97 ms/op 1.40
findModifiedValidators - no difference 200.54 ms/op 203.27 ms/op 0.99
compare ViewDUs 7.6344 s/op 7.4244 s/op 1.03
compare each validator Uint8Array 1.8598 s/op 1.9977 s/op 0.93
compare ViewDU to Uint8Array 1.3032 s/op 2.2797 s/op 0.57
migrate state 1000000 validators, 24 modified, 0 new 1.0384 s/op 1.0902 s/op 0.95
migrate state 1000000 validators, 1700 modified, 1000 new 1.3563 s/op 1.4902 s/op 0.91
migrate state 1000000 validators, 3400 modified, 2000 new 1.5061 s/op 1.6113 s/op 0.93
migrate state 1500000 validators, 24 modified, 0 new 1.0134 s/op 1.0765 s/op 0.94
migrate state 1500000 validators, 1700 modified, 1000 new 1.1169 s/op 1.2564 s/op 0.89
migrate state 1500000 validators, 3400 modified, 2000 new 1.3271 s/op 1.3542 s/op 0.98
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.4800 ns/op 4.8400 ns/op 0.93
state getBlockRootAtSlot - 250000 vs - 7PWei 501.05 ns/op 913.20 ns/op 0.55
naive computeProposerIndex 100000 validators 55.266 ms/op 65.528 ms/op 0.84
computeProposerIndex 100000 validators 1.5507 ms/op 1.7016 ms/op 0.91
naiveGetNextSyncCommitteeIndices 1000 validators 8.4301 s/op 9.9651 s/op 0.85
getNextSyncCommitteeIndices 1000 validators 112.04 ms/op 123.11 ms/op 0.91
naiveGetNextSyncCommitteeIndices 10000 validators 7.2945 s/op 11.075 s/op 0.66
getNextSyncCommitteeIndices 10000 validators 109.85 ms/op 129.35 ms/op 0.85
naiveGetNextSyncCommitteeIndices 100000 validators 8.1575 s/op 8.2897 s/op 0.98
getNextSyncCommitteeIndices 100000 validators 112.55 ms/op 128.52 ms/op 0.88
naive computeShuffledIndex 100000 validators 25.559 s/op 27.622 s/op 0.93
cached computeShuffledIndex 100000 validators 599.71 ms/op 634.13 ms/op 0.95
naive computeShuffledIndex 2000000 validators 546.29 s/op 506.17 s/op 1.08
cached computeShuffledIndex 2000000 validators 72.759 s/op 51.328 s/op 1.42
computeProposers - vc 250000 672.89 us/op 719.02 us/op 0.94
computeEpochShuffling - vc 250000 46.357 ms/op 48.216 ms/op 0.96
getNextSyncCommittee - vc 250000 11.176 ms/op 11.869 ms/op 0.94
computeSigningRoot for AttestationData 37.595 us/op 25.254 us/op 1.49
hash AttestationData serialized data then Buffer.toString(base64) 1.8685 us/op 1.7611 us/op 1.06
toHexString serialized data 1.4879 us/op 1.2547 us/op 1.19
Buffer.toString(base64) 168.95 ns/op 171.86 ns/op 0.98
nodejs block root to RootHex using toHex 165.46 ns/op 172.02 ns/op 0.96
nodejs block root to RootHex using toRootHex 95.937 ns/op 105.28 ns/op 0.91
browser block root to RootHex using the deprecated toHexString 250.26 ns/op 227.44 ns/op 1.10
browser block root to RootHex using toHex 202.60 ns/op 183.39 ns/op 1.10
browser block root to RootHex using toRootHex 175.22 ns/op 168.74 ns/op 1.04

by benchmarkbot/action

@nflaig nflaig enabled auto-merge (squash) June 18, 2025 12:03
@nflaig nflaig merged commit b93e9af into unstable Jun 18, 2025
29 of 31 checks passed
@nflaig nflaig deleted the nflaig/lh-version branch June 18, 2025 12:14
KatyaRyazantseva pushed a commit to KatyaRyazantseva/lodestar that referenced this pull request Jun 19, 2025
@nflaig @KatyaRyazantseva
test: run against stable lighthouse release in sim tests (ChainSafe#7973
9ce5cd9 
)

**Motivation**

[Sim tests are
failing](https://github.com/ChainSafe/lodestar/actions/runs/15706086864/job/44325974978)
likely due to sigp/lighthouse#7444. It might be
that Lighthouse drops support for pre-electra altogether so we need to
see how to make that work with our multi fork sim tests.

**Description**

Run against stable lighthouse release in sim tests
@wemeetagain
Copy link
Member

wemeetagain commented Jul 15, 2025

🎉 This PR is included in v1.32.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ensi321 ensi321 ensi321 left review comments

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@nflaig @wemeetagain @ensi321