Co-authored-by: jwasinger <j-wasinger@hotmail.com>

This pull request removes the node copy operation to reduce memory
allocation. Key Changes as below:

**(a) Use `decodeNodeUnsafe` for decoding nodes retrieved from the trie
node reader**

In the current implementation of the MPT, once a trie node blob is
retrieved, it is passed to `decodeNode` for decoding. However,
`decodeNode` assumes the supplied byte slice might be mutated later, so
it performs a deep copy internally before parsing the node.

Given that the node reader is implemented by the path database and the
hash database, both of which guarantee the immutability of the returned
byte slice. By restricting the node reader interface to explicitly
guarantee that the returned byte slice will not be modified, we can
safely replace `decodeNode` with `decodeNodeUnsafe`. This eliminates the
need for a redundant byte copy during each node resolution.

**(b) Modify the trie in place**

In the current implementation of the MPT, a copy of a trie node is
created before any modifications are made. These modifications include:
- Node resolution: Converting the value from a hash to the actual node.
- Node hashing: Tagging the hash into its cache.
- Node commit: Replacing the children with its hash.
- Structural changes: For example, adding a new child to a fullNode or
replacing a child of a shortNode.

This mechanism ensures that modifications only affect the live tree,
leaving all previously created copies unaffected.

Unfortunately, this property leads to a huge memory allocation
requirement. For example, if we want to modify the fullNode for n times,
the node will be copied for n times.

In this pull request, all the trie modifications are made in place. In
order to make sure all previously created copies are unaffected, the
`Copy` function now will deep-copy all the live nodes rather than the
root node itself.

With this change, while the `Copy` function becomes more expensive, it's
totally acceptable as it's not a frequently used one. For the normal
trie operations (Get, GetNode, Hash, Commit, Insert, Delete), the node
copy is not required anymore.

…cessList (#31336)

closes ethereum/go-ethereum#31335

---------

Co-authored-by: sashabeton <sashabeton2007@gmail.com>

This removes DB schema for LES related db entries. LES has been non-functional
since the merge.

Simple bugfix to include the access-list in the gas-estimation step of
the ABI bindings code.

Add support for state overrides in eth_createAccessList. This will make the method consistent
with other execution methods.

---------

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>

eth/catalyst: force sync of txpool before clearing subpools in Rollback

Fixes #31494

…#31500)

This PR changes log indexer error handling so that if an indexing error
happens then it disables the indexer and reverts to unindexed more
without resetting the database (except in case of a failed database
init).
Resetting the database on the first error would probably be overkill as
a client update might fix this without having to reindex the entire
history. It would also make debugging very hard. On the other hand,
these errors do not resolve themselves automatically so constantly
retrying makes no sense either. With these changes a new attempt to
resume indexing is made every time the client is restarted.
The PR also fixes ethereum/go-ethereum#31491
which originated from the tail indexer trying to resume processing a
failed map renderer.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>

Ignores all hand-built binaries (built with go build, everything built
with make is already ignored)

This PR adds `rawdb.SafeDeleteRange` and uses it for range deletion in
`core/filtermaps`. This includes deleting the old bloombits database,
resetting the log index database and removing index data for unindexed
tail epochs (which previously weren't properly implemented for the
fallback case).
`SafeDeleteRange` either calls `ethdb.DeleteRange` if the node uses the
new path based state scheme or uses an iterator based fallback method
that safely skips trie nodes in the range if the old hash based state
scheme is used. Note that `ethdb.DeleteRange` also has its own iterator
based fallback implementation in `ethdb/leveldb`. If a path based state
scheme is used and the backing db is pebble (as it is on the majority of
new nodes) then `rawdb.SafeDeleteRange` uses the fast native range
delete.
Also note that `rawdb.SafeDeleteRange` has different semantics from
`ethdb.DeleteRange`, it does not automatically return if the operation
takes a long time. Instead it receives a `stopCallback` that can
interrupt the process if necessary. This is because in the safe mode
potentially a lot of entries are iterated without being deleted (this is
definitely the case when deleting the old bloombits database which has a
single byte prefix) and therefore restarting the process every time a
fixed number of entries have been iterated would result in a quadratic
run time in the number of skipped entries.

When running in safe mode, unindexing an epoch takes about a second,
removing bloombits takes around 10s while resetting a full log index
might take a few minutes. If a range delete operation takes a
significant amount of time then log messages are printed. Also, any
range delete operation can be interrupted by shutdown (tail uinindexing
can also be interrupted by head indexing, similarly to how tail indexing
works). If the last unindexed epoch might have "dirty" index data left
then the indexed map range points to the first valid epoch and
`cleanedEpochsBefore` points to the previous, potentially dirty one. At
startup it is always assumed that the epoch before the first fully
indexed one might be dirty. New tail maps are never rendered and also no
further maps are unindexed before the previous unindexing is properly
cleaned up.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>

Implements #31275

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>

It introduces a new variable to store the external port returned by the
addAnyPortMapping function and ensures that the correct external port is
returned even in case of an error.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>

This PR adds metrics related to map rendering and pattern matching to
the `core/filtermaps` package.

…ent` (#31501)

found the failed testcase here
https://ci.appveyor.com/project/ethereum/go-ethereum/builds/51767091/job/rbjke432c05pufja

add a timeout to wait the tx to be mined.

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

This PR refactors the `nodeSet` structure in the path database to use
separate maps for account and storage trie nodes, resulting in
performance improvements. The change maintains the same API while
optimizing the internal data structure.

TruncatePending shows up bright red on our nodes, because it computes
the length of a map multiple times.
I don't know why this is so expensive, but around 20% of our time is
spent on this, which is super weird.

```
//PR: BenchmarkTruncatePending-24    	   17498	     69397 ns/op	   32872 B/op	       3 allocs/op
//Master: BenchmarkTruncatePending-24    	    9960	    123954 ns/op	   32872 B/op	       3 allocs/op
```

```
benchmark                       old ns/op     new ns/op     delta
BenchmarkTruncatePending-24     123954        69397         -44.01%

benchmark                       old allocs     new allocs     delta
BenchmarkTruncatePending-24     3              3              +0.00%

benchmark                       old bytes     new bytes     delta
BenchmarkTruncatePending-24     32872         32872         +0.00%
```
This simple PR is a 44% improvement over the old state


``` 
OUTINE ======================== github.com/ethereum/go-ethereum/core/txpool/legacypool.(*LegacyPool).truncatePending in github.com/ethereum/go-ethereum/core/txpool/legacypool/legacypool.go
     1.96s     18.02s (flat, cum) 19.57% of Total
         .          .   1495:func (pool *LegacyPool) truncatePending() {
         .          .   1496:	pending := uint64(0)
      60ms      2.99s   1497:	for _, list := range pool.pending {
     250ms      5.48s   1498:		pending += uint64(list.Len())
         .          .   1499:	}
         .          .   1500:	if pending <= pool.config.GlobalSlots {
         .          .   1501:		return
         .          .   1502:	}
         .          .   1503:
         .          .   1504:	pendingBeforeCap := pending
         .          .   1505:	// Assemble a spam order to penalize large transactors first
         .      510ms   1506:	spammers := prque.New[int64, common.Address](nil)
     140ms      2.50s   1507:	for addr, list := range pool.pending {
         .          .   1508:		// Only evict transactions from high rollers
      50ms      5.08s   1509:		if uint64(list.Len()) > pool.config.AccountSlots {
         .          .   1510:			spammers.Push(addr, int64(list.Len()))
         .          .   1511:		}
         .          .   1512:	}
         .          .   1513:	// Gradually drop transactions from offenders
         .          .   1514:	offenders := []common.Address{}
```

```go
// Benchmarks the speed of batch transaction insertion in case of multiple accounts.
func BenchmarkTruncatePending(b *testing.B) {
	// Generate a batch of transactions to enqueue into the pool
	pool, _ := setupPool()
	defer pool.Close()
	b.ReportAllocs()
	batches := make(types.Transactions, 4096+1024+1)
	for i := range len(batches) {
		key, _ := crypto.GenerateKey()
		account := crypto.PubkeyToAddress(key.PublicKey)
		pool.currentState.AddBalance(account, uint256.NewInt(1000000), tracing.BalanceChangeUnspecified)
		tx := transaction(uint64(0), 100000, key)
		batches[i] = tx
	}
	for _, tx := range batches {
		pool.addRemotesSync([]*types.Transaction{tx})
	}
	b.ResetTimer()
	// benchmark truncating the pending
	for range b.N {
		pool.truncatePending()
	}
}
```

This PR adds checking for an edgecase which theoretically can happen in
the range-prover. Right now, we check that a key does not overwrite a
previous one by checking that the key is increasing. However, if keys
are of different lengths, it is possible to create a key which is
increasing _and_ overwrites the previous key. Example: `0xaabbcc`
followed by `0xaabbccdd`.

This can not happen in go-ethereum, which always uses fixed-size paths
for accounts and storage slot paths in the trie, but it might happen if
the range prover is used without guaranteed fixed-size keys.

This PR also adds some testcases for the errors that are expected.

This PR applies the config overrides to the new config as well,
otherwise they will not be applied to defined configs, making
shadowforks impossible.

To test:
```
>  ./build/bin/geth --override.prague 123 --dev --datadir /tmp/geth
INFO [04-28|21:20:47.009]  - Prague:                      @123
> ./build/bin/geth --override.prague 321 --dev --datadir /tmp/geth
INFO [04-28|21:23:59.760]  - Prague:                      @321
``

Fixes ethereum/go-ethereum#31737

For PeerDAS, we need to compute cell proofs. Both ckzg and gokzg support
computing these cell proofs.
This PR does the following:

- Update the go-kzg library from "github.com/crate-crypto/go-kzg-4844"
to "github.com/crate-crypto/go-eth-kzg" which will be the new upstream
for go-kzg moving forward
- Update ckzg from v1.0.0 to v2.0.1 and switch to /v2
- Updates the trusted setup to contain the g1 points both in lagrange
and monomial form
- Expose `ComputeCells` to compute the cell proofs

This changes the filtermaps to only pull up the raw receipts, not the
derived receipts which saves a lot of allocations.

During normal execution this will reduce the allocations of the whole
geth node by ~15%.

Since the block hash is not returned for pending blocks, ethclient cannot unmarshal into RPC block. This makes hash optional on rpc block and compute the hash locally for pending blocks to correctly key the tx sender cache.


https://github.com/ethereum/go-ethereum/blob/a82303f4e3cedcebe31540a53dde4f24fc93da80/internal/ethapi/api.go#L500-L504

---------

Co-authored-by: lightclient <lightclient@protonmail.com>

Add tests for GetBlockHeaders that verify client does not disconnect when unlikely block numbers are requested, e.g. max uint64.
---------

Co-authored-by: lightclient <lightclient@protonmail.com>

The functions `rpcRequest` and `batchRpcRequest` call `baseRpcRequest`.
And `resp.Body` will be closed in the function `baseRpcRequest` later by
`t.Cleanup`:

```go
func baseRpcRequest(t *testing.T, url, bodyStr string, extraHeaders ...string) *http.Response {
        // ......
	t.Cleanup(func() { resp.Body.Close() })
	return resp
}
```

This PR improves gas estimation for data-heavy transactions which hit the floor data gas cost.

This PR fixes the out-of-range block number logic of `getBlockLvPointer`
which sometimes caused searches to fail if the head was updated in the
wrong moment. This logic ensures that querying the pointer of a future
block returns the pointer after the last fully indexed block (instead of
failing) and therefore an async range update will not cause the search
to fail. Earier this behaviour only worked when `headIndexed` was true
and `headDelimiter` pointed to the end of the indexed range. Now it also
works for an unfinished index.

This logic is also moved from `FilterMaps.getBlockLvPointer` to
`FilterMapsMatcherBackend.GetBlockLvPointer` because it is only required
by the search anyways. `FilterMaps.getBlockLvPointer` now only returns a
pointer for existing blocks, consistently with how it is used in the
indexer/renderer.

Note that this unhandled case has been present in the code for a long
time but went unnoticed because either one of two previously fixed bugs
did prevent it from being triggered; the incorrectly positive
`tempRange.headIndexed` (fixed in
ethereum/go-ethereum#31680), though caused other
problems, prevented this one from being triggered as with a positive
`headIndexed` no database read was triggered in `getBlockLvPointer`.
Also, the unnecessary `indexLock` in `synced()` (fixed in
ethereum/go-ethereum#31708) usually did prevent
the search seeing the temp range and therefore avoided noticeable
issues.

This PR fixes an initialization bug that in some cases caused the map
renderer to leave the last, partially rendered map as is and resume
rendering from the next map. At initialization we check whether the
existing rendered maps are consistent with the current chain view and
revert them if necessary. Until now this happened through an ugly hacky
solution, a "limited" chain view that was supposed to trigger a rollback
of some maps in the renderer logic if necessary. This whole setup worked
under assumptions that just weren't true any more. As a result it always
tried to revert the last map but also it did not shorten the indexed
range, only set `headIndexed` to false which indicated to the renderer
logic that the last map is fully populated (which it wasn't).
Now an explicit rollback of any unusable (reorged) maps happens at
startup, which also means that no hacky chain view is necessary, as soon
as the new `FilterMaps` is returned, the indexed range and view are
consistent with each other.

In the first commit an extra check is also added to `writeFinishedMaps`
so that if there is ever again a bug that would result in a gapped index
then it will not break the db with writing the incomplete data. Instead
it will return an indexing error which causes the indexer to revert to
unindexed mode and print an error log instantly. Hopefully this will not
ever happen in the future, but in order to test this safeguard check I
manually triggered the bug with only the first commit enabled, which
caused an indexing error as expected. With the second commit added (the
actual fix) the same operation succeeded without any issues.

Note that the database version is also bumped in this PR in order to
enforce a full reindexing as any existing database might be potentially
broken.

Fixes ethereum/go-ethereum#31729

`DefaultBlobSchedule` is actually used downstream to calculate blob fees
(e.g.,
[src](https://github.com/ethereum-optimism/optimism/blob/601a380e47853c2922ea1f8944cda05f0eac16f4/op-service/eth/blob.go#L301)),
this PR makes it explicit that these params are for `Ethereum prod`
instead of `test chains`.

Fixes ethereum/go-ethereum#31732.

This logic was removed in the recent refactoring in the txindexer to
handle history cutoff (#31393). It was first introduced in this PR:
ethereum/go-ethereum#28908.

I have tested it and it works as an alternative to #31745.

This PR packs 3 changes to the flow of fetching txs from the API:

- It caches the indexer tail after each run is over to avoid hitting the
db all the time as was done originally in #28908.

- Changes `backend.GetTransaction`. It doesn't return an error anymore
when tx indexer is in progress. It shifts the responsibility to the
caller to check the progress. The reason is that in most cases we anyway
check the txpool for the tx. If it was indeed a pending tx we can avoid
the indexer progress check.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>

…761)

updates the log entries in `core/filtermaps/indexer.go` to remove double
quotes around keys like "first block" and "last block", changing them to
`firstblock` and `lastblock`. This brings them in line with the general
logging style used across the codebase, where log keys are unquoted
single words.

For example, the log:
`  INFO [...] "first block"=..., "last block"=...`

Is now rendered as:
`  INFO [...] firstblock=..., lastblock=...`

This change improves readability and maintains consistency with logs
such as:
`  INFO [...] number=2 sealhash=... uncles=0 txs=0 ...`

No functional behavior is changed — this is purely a formatting cleanup
for better developer experience.

Issue statement: when user requests eth_simulateV1 to return full
transaction objects, these objects always had an empty `from` field. The
reason is we lose the sender when translation the message into a
types.Transaction which is then later on serialized.

I did think of an alternative but opted to keep with this approach as it
keeps complexity at the edge. The alternative would be to pass down a
signer object to RPCMarshal* methods and define a custom signer which
keeps the senders in its state and doesn't attempt the signature
recovery.

This change adds a limit for RPC method names to prevent potential abuse
where large method names could lead to large response sizes.

The limit is enforced in:
- handleCall for regular RPC method calls
- handleSubscribe for subscription method calls

Added tests in websocket_test.go to verify the length limit
functionality for both regular method calls and subscriptions.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>

Fixes methods debug_standardTraceBlockToFile
and debug_standardTraceBadBlockToFile which were
outputting empty files.

---------

Co-authored-by: maskpp <maskpp266@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>