-
Notifications
You must be signed in to change notification settings - Fork 1.8k
The fast node reports triedb layer & Head state missing when stoppping & starting #2829
Copy link
Copy link
Closed
Description
System information
Geth version: v1.5.2-alpha & v1.4.15
OS & Version: Windows/Linux/OSX
Commit hash : 3a6ef6a & ec318b9
Expected behaviour
The fast node restarts as normal.
Actual behaviour
Steps to reproduce the behaviour
enable the fast node feature with --tries-verify-mode none, and restart the node.
Backtrace
t=2024-12-26T10:13:02+0000 lvl=error msg="Bsc extension Handshake failed" peer=4aae98d0 err=EOF
t=2024-12-26T10:13:03+0000 lvl=info msg="Imported new chain segment" number=45208967 hash=0xfeead671252756c4545f21fa3a5094eac70510728a7df8ed33fa0bb6cfd76497 miner=0xF8de5e61322302b2c6e0a525cC842F10332811bf blocks=1 txs=161 blobs=0 mgas=33.202115 elapsed=119.459ms mgasps=277.9350712873532 snapdiffs="4.06 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B"
t=2024-12-26T10:13:03+0000 lvl=info msg="Got interrupt, shutting down..."
t=2024-12-26T10:13:03+0000 lvl=info msg="HTTP server stopped" endpoint=[::]:8545
t=2024-12-26T10:13:03+0000 lvl=info msg="IPC endpoint closed" url=/server/data-seed/geth.ipc
t=2024-12-26T10:13:04+0000 lvl=info msg="Ethereum protocol stopped"
t=2024-12-26T10:13:04+0000 lvl=info msg="Transaction pool stopped"
t=2024-12-26T10:13:04+0000 lvl=info msg="Failed to journal in-memory trie nodes" err="triedb layer [0xf08e7cf30ae1b3c439ed4068e4ac16259213eb563b64868520511832ee804bcd] missing"
t=2024-12-26T10:13:04+0000 lvl=info msg="Blockchain stopped"
t=2024-12-26T10:13:23+0000 lvl=warn msg="Using deprecated resource file, please move this file to the 'geth' subdirectory of datadir." file=/server/data-seed/nodekey
t=2024-12-26T10:13:23+0000 lvl=warn msg="The light server has been deprecated, please remove this flag" flag=light.serve
t=2024-12-26T10:13:23+0000 lvl=info msg="Automatically disables snap protocol due to verify mode" mode=none
t=2024-12-26T10:13:23+0000 lvl=warn msg="Only local TriesVerifyMode can support snap sync, resetting to full sync" mode=none
t=2024-12-26T10:13:23+0000 lvl=info msg="Set global gas cap" cap=50000000
t=2024-12-26T10:13:23+0000 lvl=info msg="Initializing the KZG library" backend=gokzg
t=2024-12-26T10:13:23+0000 lvl=info msg="Using pebble as the backing database"
t=2024-12-26T10:13:23+0000 lvl=info msg="Allocated cache and file handles" database=/server/data-seed/geth/chaindata cache="1.95 GiB" handles=5000 "memory table"="500.00 MiB"
t=2024-12-26T10:13:24+0000 lvl=info msg="Opened ancient database" database=/server/data-seed/geth/chaindata/ancient/chain readonly=false frozen=45118956
t=2024-12-26T10:13:24+0000 lvl=info msg="State scheme set to already existing disk db" scheme=path
t=2024-12-26T10:13:24+0000 lvl=info msg="Capped dirty cache size" provided="1.22 GiB" adjusted="256.00 MiB"
t=2024-12-26T10:13:24+0000 lvl=info msg="Clean cache size" provided="750.00 MiB" adjusted="1.70 GiB"
t=2024-12-26T10:13:24+0000 lvl=info msg="Allocated memory caches" state_scheme=path trie_clean_cache="1.70 GiB" trie_dirty_cache="256.00 MiB" snapshot_cache="1000.00 MiB"
t=2024-12-26T10:13:24+0000 lvl=info msg="Unprotected transactions allowed"
t=2024-12-26T10:13:24+0000 lvl=info msg=Parlia chainConfig="{ChainID: 56 Homestead: 0 DAO: <nil> DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: 0, Muir Glacier: 0, Ramanujan: 0, Niels: 0, MirrorSync: 5184000, Bruno: 13082000, Berlin: 31302048, YOLO v3: <nil>, CatalystBlock: <nil>, London: 31302048, ArrowGlacier: <nil>, MergeFork:<nil>, Euler: 18907621, Gibbs: 23846001, Nano: 21962149, Moran: 22107423, Planck: 27281024,Luban: 29020050, Plato: 30720096, Hertz: 31302048, Hertzfix: 34140700, ShanghaiTime: 1705996800, KeplerTime: 1705996800, FeynmanTime: 1713419340, FeynmanFixTime: 1713419340, CancunTime: 1718863500, HaberTime: 1718863500, HaberFixTime: 1727316120, BohrTime: 1727317200, Engine: parlia}"
t=2024-12-26T10:13:25+0000 lvl=info msg="Initialising Ethereum protocol" network=56 dbversion=8
t=2024-12-26T10:13:25+0000 lvl=error msg="Zero trie root hash!"
t=2024-12-26T10:13:25+0000 lvl=info msg="New journal reader for journal kv"
t=2024-12-26T10:13:25+0000 lvl=info msg="Failed to load journal, discard it" err="unmatched journal want 51a6371b7ba6bf72f5e2dac9c1caa42e3957cda534f93bcde53487050af55cb2 got 56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421"
t=2024-12-26T10:13:25+0000 lvl=info msg="New async node buffer" limit="256.00 MiB" layers=0
t=2024-12-26T10:13:25+0000 lvl=info msg="Initialised chain configuration" config="{ChainID: 56 Homestead: 0 DAO: <nil> DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: 0, Muir Glacier: 0, Ramanujan: 0, Niels: 0, MirrorSync: 5184000, Bruno: 13082000, Berlin: 31302048, YOLO v3: <nil>, CatalystBlock: <nil>, London: 31302048, ArrowGlacier: <nil>, MergeFork:<nil>, Euler: 18907621, Gibbs: 23846001, Nano: 21962149, Moran: 22107423, Planck: 27281024,Luban: 29020050, Plato: 30720096, Hertz: 31302048, Hertzfix: 34140700, ShanghaiTime: 1705996800, KeplerTime: 1705996800, FeynmanTime: 1713419340, FeynmanFixTime: 1713419340, CancunTime: 1718863500, HaberTime: 1718863500, HaberFixTime: 1727316120, BohrTime: 1727317200, Engine: parlia}"
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded most recent local block" number=45208967 hash=0xfeead671252756c4545f21fa3a5094eac70510728a7df8ed33fa0bb6cfd76497 root=0xf08e7cf30ae1b3c439ed4068e4ac16259213eb563b64868520511832ee804bcd td=89840182 age=22s
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded most recent local finalized block" number=45208965 hash=0xc0321316d4f8a7109afdcff773d0d501841363f14d5bbbbd60bf0c66659e3bc0 root=0x613237e08b1a2294e41621aacfe9b62cc81bd4bec1c599f042292996df5a42fe td=89840178 age=28s
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded last snap-sync pivot marker" number=6920578
t=2024-12-26T10:13:25+0000 lvl=warn msg="Head state missing, repairing" number=45208967 hash=0xfeead671252756c4545f21fa3a5094eac70510728a7df8ed33fa0bb6cfd76497 diskRoot=0x6922d0fd3f8e6126c8db5eb8ae829de018815ddca27a381e185e24661c34997c
t=2024-12-26T10:13:25+0000 lvl=info msg="Snapshot loaded" diskRoot=0x6922d0fd3f8e6126c8db5eb8ae829de018815ddca27a381e185e24661c34997c root=0xf08e7cf30ae1b3c439ed4068e4ac16259213eb563b64868520511832ee804bcd
t=2024-12-26T10:13:25+0000 lvl=info msg="Rewound to block with state" number=45208879 hash=0x488bb3e5bf8b8c9c61a1c4686dfd85455e0e85a3f48ffce22f19aa473aff23b7
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded most recent local header" number=45208967 hash=0xfeead671252756c4545f21fa3a5094eac70510728a7df8ed33fa0bb6cfd76497 hash=0xf08e7cf30ae1b3c439ed4068e4ac16259213eb563b64868520511832ee804bcd td=89840182 age=22s
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded most recent local block" number=45208879 hash=0x488bb3e5bf8b8c9c61a1c4686dfd85455e0e85a3f48ffce22f19aa473aff23b7 root=0x6922d0fd3f8e6126c8db5eb8ae829de018815ddca27a381e185e24661c34997c td=89840008 age=4m50s
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded most recent local snap block" number=45208967 hash=0xfeead671252756c4545f21fa3a5094eac70510728a7df8ed33fa0bb6cfd76497 root=0xf08e7cf30ae1b3c439ed4068e4ac16259213eb563b64868520511832ee804bcd td=89840182 age=22s
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded most recent local finalized block" number=45208965 hash=0xc0321316d4f8a7109afdcff773d0d501841363f14d5bbbbd60bf0c66659e3bc0 root=0x613237e08b1a2294e41621aacfe9b62cc81bd4bec1c599f042292996df5a42fe td=89840178 age=28s
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded last snap-sync pivot marker" number=692057
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded most recent local finalized block" number=45208965 hash=0xc0321316d4f8a7109afdcff773d0d501841363f14d5bbbbd60bf0c66659e3bc0 root=0x613237e08b1a2294e41621aacfe9b62cc81bd4bec1c599f042292996df5a42fe td=89840178 age=28s
t=2024-12-26T10:13:25+0000 lvl=info msg="Loaded last snap-sync pivot marker" number=6920578
t=2024-12-26T10:13:25+0000 lvl=warn msg="Enabling snapshot recovery" chainhead=45208879 diskbase=45208879
t=2024-12-26T10:13:25+0000 lvl=warn msg="Snapshot is not continuous with chain" snaproot=0xf08e7cf30ae1b3c439ed4068e4ac16259213eb563b64868520511832ee804bcd chainroot=0x6922d0fd3f8e6126c8db5eb8ae829de018815ddca27a381e185e24661c34997c
t=2024-12-26T10:13:25+0000 lvl=info msg="Snapshot loaded" diskRoot=0x6922d0fd3f8e6126c8db5eb8ae829de018815ddca27a381e185e24661c34997c root=0x6922d0fd3f8e6126c8db5eb8ae829de018815ddca27a381e185e24661c34997c
t=2024-12-26T10:13:25+0000 lvl=info msg="Initialized transaction indexer" range="entire chain"
t=2024-12-26T10:13:25+0000 lvl=warn msg="Sanitizing invalid txpool reannounce time" provided=0s updated=1m0s
t=2024-12-26T10:13:25+0000 lvl=info msg="Create votePool successfully"
t=2024-12-26T10:13:25+0000 lvl=warn msg="Sanitizing invalid gasprice oracle price cap" provided=<nil> updated=100000000000
t=2024-12-26T10:13:25+0000 lvl=warn msg="Sanitizing invalid gasprice oracle ignore price" provided=<nil> updated=4
t=2024-12-26T10:13:25+0000 lvl=warn msg="Sanitizing invalid gasprice oracle max header history" provided=0 updated=1
t=2024-12-26T10:13:25+0000 lvl=warn msg="Sanitizing invalid gasprice oracle max block history" provided=0 updated=1
t=2024-12-26T10:13:25+0000 lvl=warn msg="Old unclean shutdowns found" count=12
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2021-05-13T03:22:26+0000 age=3y8mo3d
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2021-05-13T03:23:12+0000 age=3y8mo3d
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2021-05-13T03:24:03+0000 age=3y8mo3d
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2021-11-17T07:15:13+0000 age=3y1mo3w
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2023-11-21T08:10:29+0000 age=1y1mo1w
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2024-07-11T07:12:38+0000 age=5mo2w4d
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2024-07-17T17:02:42+0000 age=5mo1w4d
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2024-07-27T06:04:06+0000 age=5mo2d4h
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2024-08-06T10:39:38+0000 age=4mo3w23h
t=2024-12-26T10:13:25+0000 lvl=warn msg="Unclean shutdown detected" booted=2024-08-26T15:20:25+0000 age=4mo1d18h
t=2024-12-26T10:13:25+0000 lvl=info msg="Enabling metrics collection"
t=2024-12-26T10:13:25+0000 lvl=info msg="Starting peer-to-peer node" instance=Geth/v1.4.15-ec318b9c-20240919/linux-amd64/go1.21.13
t=2024-12-26T10:13:26+0000 lvl=info msg="New local node record" seq=1708962647676 id=1b104cdb92cb45e58e9255ddf5abc68fe176c6aeb5f0ac9e6e3c3432fef19271 ip=54.162.32.1 udp=30311 tcp=30311
t=2024-12-26T10:13:26+0000 lvl=info msg="Started P2P networking" self=enode://accbc0a5af0af03e1ec3b5e80544bdceea48011a6928cd82d2c1a9c38b65fd48ec970ba17bd8c0b0ec21a28faec9efe1d1ce55134784b9207146e2f62d8932ba@54.162.32.1:30311
t=2024-12-26T10:13:26+0000 lvl=info msg="IPC endpoint opened" url=/server/data-seed/geth.ipc
t=2024-12-26T10:13:26+0000 lvl=info msg="WebSocket enabled" url=ws://[::]:8545
t=2024-12-26T10:13:26+0000 lvl=info msg="HTTP server started" endpoint=[::]:8545 auth=false prefix="" cors=* vhosts=*
t=2024-12-26T10:13:26+0000 lvl=info msg="Block synchronisation started"
t=2024-12-26T10:13:26+0000 lvl=error msg="Bsc extension registration failed" peer=3e108e39 err="peer connected on bsc without compatible eth support" name=Geth/v1.2.2-33e6f840...
t=2024-12-26T10:13:26+0000 lvl=info msg="Extend chain" add=87 number=45208966 hash=0x13823bafbf84af6315a2b440f92e1a19550b4dd9e3342fd8425de8361c49218a
t=2024-12-26T10:13:26+0000 lvl=error msg="Bsc extension registration failed" peer=f97a78ef err="peer connected on bsc without compatible eth support" name=Geth/v1.2.5-9f574b0f...
t=2024-12-26T10:13:27+0000 lvl=warn msg="Inserted known block" number=45208967 hash=0xfeead671252756c4545f21fa3a5094eac70510728a7df8ed33fa0bb6cfd76497 uncles=0 txs=161 gas=33202115 root=0xf08e7cf30ae1b3c439ed4068e4ac16259213eb563b64868520511832ee804bcd
t=2024-12-26T10:13:27+0000 lvl=error msg="Please file an issue, skip known block execution without receipt" hash=0xfeead671252756c4545f21fa3a5094eac70510728a7df8ed33fa0bb6cfd76497 number=45208967
t=2024-12-26T10:13:27+0000 lvl=error msg="Bsc extension registration failed" peer=d48055d3 err="peer connected on bsc without compatible eth support" name=Geth/v1.2.5-53fef9e8...
t=2024-12-26T10:13:27+0000 lvl=error msg="Bsc extension registration failed" peer=52a8e876 err="peer connected on bsc without compatible eth support" name=Geth/v1.2.10-2b83693...
t=2024-12-26T10:13:28+0000 lvl=info msg="Imported new chain segment" number=45208974 hash=0x741dfad345f0dc187d777a5b26a036a3e1c5a830669d50e612d5f5713eb17fbf miner=0x75B851a27D7101438F45fce31816501193239A83 blocks=7 txs=922 blobs=0 mgas=87.910439 elapsed=953.537ms mgasps=92.19404588319853 snapdiffs="3.19 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B"
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels