Skip to content

Migration script fixes#179

Merged
piersy merged 3 commits intocelo7from
piersy/migration-script-fixes
Jul 1, 2024
Merged

Migration script fixes#179
piersy merged 3 commits intocelo7from
piersy/migration-script-fixes

Conversation

@piersy
Copy link
Copy Markdown

@piersy piersy commented Jun 26, 2024

These fixes ensure that the migration script works when there are no
ancients and also when the final celo L1 block is a pre-gingerbread block.

The migration script was assuming that ancients would be present and was
considering the numAncients-1 to be first non ancient block to migrate but when
numAncients is zero that's a problem.

Also removed logic for picking up where db migration left of for the
level db since it was complicating the logic for correctly calculating the
non ancient block to start migrating from.

Finally these changes ensure that the migration block has a non zero gas limit,
even if the parent block had a zero gas limit

@piersy piersy requested review from alecps and palango June 26, 2024 13:35
@piersy piersy marked this pull request as draft June 26, 2024 13:39
@piersy piersy force-pushed the piersy/migration-script-fixes branch from 77fc0a2 to f24dfe4 Compare June 26, 2024 14:39
@piersy piersy marked this pull request as ready for review June 26, 2024 14:45
@palango
Copy link
Copy Markdown

palango commented Jun 28, 2024

Finally these changes ensure that the migration block has a non zero gas limit,
even if the parent block had a zero gas limit

This looks fine to me, the rest I leave for @alecps .

Out of curiosity, the gas limit of zero would be problematic because it would be used in the next block as well or is there another reason?

@piersy
Copy link
Copy Markdown
Author

piersy commented Jun 28, 2024

@palango its because gas limit zero is used to determine whether a block is pre or post gingerbread and so with a gas limit of zero the trasition block wouldn't be decoded properly. It would be missing all the post gingerbread fields and all the extra fields from op-geth. I should probably add a comment to that effect.

@palango
Copy link
Copy Markdown

palango commented Jun 28, 2024

@palango its because gas limit zero is used to determine whether a block is pre or post gingerbread and so with a gas limit of zero the trasition block wouldn't be decoded properly. It would be missing all the post gingerbread fields and all the extra fields from op-geth. I should probably add a comment to that effect.

That makes sense. Yeah, please add a comment and we should also add a page to the spec mentioning all those details.

@palango
Copy link
Copy Markdown

palango commented Jun 28, 2024

Also, the base branch needs to be changed to celo7

Copy link
Copy Markdown

@alecps alecps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@piersy This looks great! Thanks for putting this together. I agree the code is much cleaner just passing in numAncients and getting rid of lastMigratedAncientBlock 👌

My only comment is that it might be helpful to keep the keepNonAncients flag bc we want to try having the rsync command run during the pre-migration (without any transformations applied). To be honest though, it's easy enough to add back so feel free to merge as is. Thanks again!

piersy added 2 commits July 1, 2024 09:14
The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.
@piersy piersy force-pushed the piersy/migration-script-fixes branch from f24dfe4 to 74b8139 Compare July 1, 2024 08:18
@piersy piersy changed the base branch from celo6 to celo7 July 1, 2024 08:40
@piersy
Copy link
Copy Markdown
Author

piersy commented Jul 1, 2024

@palango Added a ticket here for adding to the spec - https://github.com/celo-org/celo-blockchain-planning/issues/356

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jul 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.47%. Comparing base (9907e11) to head (6cedcfa).

Additional details and impacted files
@@             Coverage Diff             @@
##            celo7     #179       +/-   ##
===========================================
+ Coverage   61.32%   75.47%   +14.15%     
===========================================
  Files          20       16        -4     
  Lines        1753     1358      -395     
  Branches       71       12       -59     
===========================================
- Hits         1075     1025       -50     
+ Misses        646      301      -345     
  Partials       32       32               
Flag Coverage Δ
cannon-go-tests 81.03% <ø> (ø)
chain-mon-tests 27.14% <ø> (ø)
sdk-tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 4 files with indirect coverage changes

@piersy piersy merged commit 432380f into celo7 Jul 1, 2024
@piersy piersy deleted the piersy/migration-script-fixes branch July 1, 2024 09:39
karlb pushed a commit that referenced this pull request Sep 10, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Sep 10, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Sep 10, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Sep 13, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Sep 17, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
palango pushed a commit that referenced this pull request Sep 24, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
palango pushed a commit that referenced this pull request Sep 24, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Oct 12, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Oct 14, 2024
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
alecps pushed a commit that referenced this pull request Oct 15, 2024
This works by loading the database of a celo
node. It then removes all existing blocks and
generates a new genesis block including the
existing state tree.

Migrate to urfave/cli/v2

Update op-chain-ops/cmd/op-migrate/main.go

Co-authored-by: Karl Bartel <karl@karl.berlin>

Combine Cel2 migration scripts (#148)

* Initial script to play with celo DB history migration

* Can Read All the headers

Co-authored-by: Alec Schaefer <alecps@users.noreply.github.com>

* Adds new command to migrate ancients db

* Adds comment

* Adds extension methods for transformation

* Implements Transform CeloBody

* Adds impl that runs steps in a concurrent pipeline

* Adds transformHead, verify hashing works

cleanup

* add migration for non-frozen blocks

* copy over entire db and modify in place, works with op-geth at piersy/minimal-data-migration

* remove unecessary copying, cleanup code

* close and reopen DBs

* migrate newdb in place

* saving progress

Co-authored-by: Mariano Cortesi <mcortesi@gmail.com>

* Refactor code to improve database migration process

* better logging

* refactor: inline parMigrateAncientRange

* Remove frozen blocks from nonAncient DB

* check hash matches on nonAncients migration

* clean up branch

Removes unused code, move code for better separation of concerns.

* decode into new types

* fix transformHeader

* make old freezer not readonly so that .meta files are created

* add configurable memory limit

* add comment about memory

* Added celo-dbmigrate Makefile target

* Added dockerfile for celo-dbmigrate and celo-migrate tools

* Workflow for running cel2-migration-tool

* Update cel2-migration-tool image registry

* update op-geth to point to https://github.com/celo-org/op-geth/commits/piersy/for-use-with-migrated-celo-datadir-use-gas-limit-differentiation-rebased-celo6/

* add celo6 logging

* rename scripts to celo-migrate-state and celo-migrate-blocks

* first pass at combining scripts

* saving progress on testing

* fix lint error, use %w to fmt errors

* add updated state migration input files to testdata

* add ability to run block and state migration seperately or together

* add option for migrating only frozen blocks

* remove old scripts

* minor logging improvements in block migrations

* invert clearNonAncients flag logic --> keepNonAncients, make dry-run flag only apply to state migration

* adds README, improves logging

* fix lint err

* Fix Makefile and Dockerfile

* move createNewDbIfNotExists

* rename keep-non-ancients

* update TODO to add more context and state changes

* Remove channel buffers from ancients migration

Co-authored-by: Valentin Rodygin <carterqw@gmail.com>

* bump default batch size to 100000

* add back extended usage string

* add info on state migration to README

* remove --state-dry-run flag

* update default batch size to 50k

* Adding building for op images

* Setting our values for image registry and repository

* update README

* fix logging when newAncients > oldAncients

* fix return value when skipping ancients

* skip transforming block bodies that have already been transformed

* misc. fixes to get re-runs with --keep-non-ancients working

* adds TODO

* addresses cosmetic feedback

* add flag for specifying a buffer

* Show progress on rsync

* Update to latest op-geth

* state-migration: Refactor subtask

* state-migration: Use EIP1559 settings from deploy config

Fixes #135

* state-migration: Enable Fjord hardfork during migration

Fixes #160

* state-migration: Deterministicly set migration block timestamp

Fixes #157

Sets the timestamp to be 5s after the last block.

* state-migration: Set WithdrawalsHash in Cel2 migration block

* fixup! Fix Makefile and Dockerfile

* add note to README about using snapshots for pre-migration

* Set blob gas header fields for transition block

These are now required to be set since cancun was activated.

* Use InitialBaseFee for pre-gingerbread transitionb

* Fix warnings about capitalized error strings

* Output chain config as marshalled JSON

* state-migration: Handle accounts with existing balance

Fixes #158

* remove allocs file, add instructions for how to generate allocs file to README, update TODOs

---------

Co-authored-by: Mariano Cortesi <mariano@clabs.co>
Co-authored-by: Alec Schaefer <alecps@users.noreply.github.com>
Co-authored-by: Mariano Cortesi <mcortesi@gmail.com>
Co-authored-by: Javier Cortejoso <javier.cortejoso@gmail.com>
Co-authored-by: Paul Lange <palango@gmx.de>
Co-authored-by: Valentin Rodygin <carterqw@gmail.com>
Co-authored-by: Piers Powlesland <pierspowlesland@gmail.com>

Set balance of `CeloDistributionSchedule` contract (#162)

* state-migration: Initialize CeloDistributionSchedule

Fixes #155

* state-migration: Don't fail when distribution schedule update errors

* Review comments

state-migration: Set ParentBeaconRoot (#176)

This allows header validation to pass during snap sync

state-migration: Set address of distribution schedule (#177)

state-migration: Read total supply directly from state (#182)

* state-migration: Read totalSupply directly from storage

* Added trigger for updated dependencies

* Removen token bindings

---------

Co-authored-by: Javier Cortejoso <javier.cortejoso@gmail.com>

Fix l2 block older than l1 origin error (#184) (#187)

* Revert to using time.Now() for migration block

Instead of simply adding 5 to the parent block time.

We really do need a deterministic time for the migration block so that
all parties that run the migration arrive at the same migration block
but the problem is that op-geth requires that the L2 migration block
(aka l2 origin) occurs after the l1 origin (I guess the point where you
deploy the bridge contracts to the l1). When we migrate a partially
synced datadir the block before the transition block will be very old,
up to 4 years old! So of course it occurs before the l1 origin. So a fix
just to get things working is to use time.Now(), but probably we should
make this a configurable parameter.

* add flag to specify timestamp

* Update op-chain-ops/cmd/celo-migrate/main.go

---------

Co-authored-by: piersy <pierspowlesland@gmail.com>

Migration script fixes (#179)

* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point

Fix migration script gap in migrated blocks (#189)

* Fix migration script gap in migrated blocks

The range of ancient blocks to remove from the non ancients database was
off by one and resulted in a gap between ancients and non ancients.

Also corrected some log statements that were off by one.

Add pre-migration command to migration script (#192)

* add pre-migration command, rsync and ancients run in parallel, remove onlyAncients flag

* remove block and state migration sub-commands

* make non ancient migration its own step, add flag to measure time

* add more granular timers

* open db without freezer in state migration, remove clearAll

* fix error

* remove update flag from rsync command, add rsync comments

* delete commented out versions of checkForPrevFullMigration

* remove aliases

* remove clearNonAncients flag

* remove measureTime flag, always log time measurements

* remove logging from help text

* remove db reset

* move scan for extra ancients into pre-migration

* update README

* rename extraAncientNumHashes to strayAncientBlocks

state-migration: Fail if account would be overwritten (#202)

* state-migration: Fail if account would be overwritten

* Review changes

* Review changes 2

* Fail in unclear state

* more changes

* Use whitelist to decide if nonce and state are overwritten

Cosmetic changes to the migration script

- Use more lists for added readability
- Capitalize Alfajores and Celo
- Reorder scripting instructions to fit the actual order or operations
- Use GitHub callouts

migration: Add tests (#217)

* migration: Add tests for state migration

* migration: Fix issues shown by tests

* migration: pass allowlist into state migration

Allows for easier testing

* migration: Add test with allowlist

* Correct overwrite counter

* Use in memory DB

migration: Add working allowlist for Alfajores (#220)

* migration: Simplify tests

* migration: Add working allowlist for Alfajores

Adapt migration code to changes in StateDB

StateDB.CreateAccount used to copy existing balance, now it does not any
more.

migration: Set fields correctly for migration block (#212)

migration: Enable Granite (#226)

Write genesis file in state migration (#219)

* squash of #167

* add writeGenesis

* open old freezer in readonly mode, fix locking error

* remove devAlloc

* Revert "open old freezer in readonly mode, fix locking error"

This reverts commit e3fddea.

* fix locking error

* fix lint error, check errors, add comment

* remove comment

* filter extra genesis fields

* fix issue with genesis extra data

* update testdata

---------

Co-authored-by: Javier Cortejoso <javier.cortejoso@gmail.com>

migration: Overwrite create2deployer code (#233)

migration: Allow 'createx' preinstall (#238)

The code already exists on Alfajores and matches the one that would be
deployed, therefore we just allow this address.

add migration-block-number flag (#245)

* add migration-block-number flag

* address feedback

* move migration-block-number flag out of state migration options

Fixes for re-running migration script on same destination db  (#246)

* add reset flag

* add --checksum to rsync options
karlb pushed a commit that referenced this pull request Oct 16, 2024
This works by loading the database of a celo
node. It then removes all existing blocks and
generates a new genesis block including the
existing state tree.

Migrate to urfave/cli/v2

Update op-chain-ops/cmd/op-migrate/main.go

Co-authored-by: Karl Bartel <karl@karl.berlin>

Combine Cel2 migration scripts (#148)

* Initial script to play with celo DB history migration

* Can Read All the headers

Co-authored-by: Alec Schaefer <alecps@users.noreply.github.com>

* Adds new command to migrate ancients db

* Adds comment

* Adds extension methods for transformation

* Implements Transform CeloBody

* Adds impl that runs steps in a concurrent pipeline

* Adds transformHead, verify hashing works

cleanup

* add migration for non-frozen blocks

* copy over entire db and modify in place, works with op-geth at piersy/minimal-data-migration

* remove unecessary copying, cleanup code

* close and reopen DBs

* migrate newdb in place

* saving progress

Co-authored-by: Mariano Cortesi <mcortesi@gmail.com>

* Refactor code to improve database migration process

* better logging

* refactor: inline parMigrateAncientRange

* Remove frozen blocks from nonAncient DB

* check hash matches on nonAncients migration

* clean up branch

Removes unused code, move code for better separation of concerns.

* decode into new types

* fix transformHeader

* make old freezer not readonly so that .meta files are created

* add configurable memory limit

* add comment about memory

* Added celo-dbmigrate Makefile target

* Added dockerfile for celo-dbmigrate and celo-migrate tools

* Workflow for running cel2-migration-tool

* Update cel2-migration-tool image registry

* update op-geth to point to https://github.com/celo-org/op-geth/commits/piersy/for-use-with-migrated-celo-datadir-use-gas-limit-differentiation-rebased-celo6/

* add celo6 logging

* rename scripts to celo-migrate-state and celo-migrate-blocks

* first pass at combining scripts

* saving progress on testing

* fix lint error, use %w to fmt errors

* add updated state migration input files to testdata

* add ability to run block and state migration seperately or together

* add option for migrating only frozen blocks

* remove old scripts

* minor logging improvements in block migrations

* invert clearNonAncients flag logic --> keepNonAncients, make dry-run flag only apply to state migration

* adds README, improves logging

* fix lint err

* Fix Makefile and Dockerfile

* move createNewDbIfNotExists

* rename keep-non-ancients

* update TODO to add more context and state changes

* Remove channel buffers from ancients migration

Co-authored-by: Valentin Rodygin <carterqw@gmail.com>

* bump default batch size to 100000

* add back extended usage string

* add info on state migration to README

* remove --state-dry-run flag

* update default batch size to 50k

* Adding building for op images

* Setting our values for image registry and repository

* update README

* fix logging when newAncients > oldAncients

* fix return value when skipping ancients

* skip transforming block bodies that have already been transformed

* misc. fixes to get re-runs with --keep-non-ancients working

* adds TODO

* addresses cosmetic feedback

* add flag for specifying a buffer

* Show progress on rsync

* Update to latest op-geth

* state-migration: Refactor subtask

* state-migration: Use EIP1559 settings from deploy config

Fixes #135

* state-migration: Enable Fjord hardfork during migration

Fixes #160

* state-migration: Deterministicly set migration block timestamp

Fixes #157

Sets the timestamp to be 5s after the last block.

* state-migration: Set WithdrawalsHash in Cel2 migration block

* fixup! Fix Makefile and Dockerfile

* add note to README about using snapshots for pre-migration

* Set blob gas header fields for transition block

These are now required to be set since cancun was activated.

* Use InitialBaseFee for pre-gingerbread transitionb

* Fix warnings about capitalized error strings

* Output chain config as marshalled JSON

* state-migration: Handle accounts with existing balance

Fixes #158

* remove allocs file, add instructions for how to generate allocs file to README, update TODOs

---------

Co-authored-by: Mariano Cortesi <mariano@clabs.co>
Co-authored-by: Alec Schaefer <alecps@users.noreply.github.com>
Co-authored-by: Mariano Cortesi <mcortesi@gmail.com>
Co-authored-by: Javier Cortejoso <javier.cortejoso@gmail.com>
Co-authored-by: Paul Lange <palango@gmx.de>
Co-authored-by: Valentin Rodygin <carterqw@gmail.com>
Co-authored-by: Piers Powlesland <pierspowlesland@gmail.com>

Set balance of `CeloDistributionSchedule` contract (#162)

* state-migration: Initialize CeloDistributionSchedule

Fixes #155

* state-migration: Don't fail when distribution schedule update errors

* Review comments

state-migration: Set ParentBeaconRoot (#176)

This allows header validation to pass during snap sync

state-migration: Set address of distribution schedule (#177)

state-migration: Read total supply directly from state (#182)

* state-migration: Read totalSupply directly from storage

* Added trigger for updated dependencies

* Removen token bindings

---------

Co-authored-by: Javier Cortejoso <javier.cortejoso@gmail.com>

Fix l2 block older than l1 origin error (#184) (#187)

* Revert to using time.Now() for migration block

Instead of simply adding 5 to the parent block time.

We really do need a deterministic time for the migration block so that
all parties that run the migration arrive at the same migration block
but the problem is that op-geth requires that the L2 migration block
(aka l2 origin) occurs after the l1 origin (I guess the point where you
deploy the bridge contracts to the l1). When we migrate a partially
synced datadir the block before the transition block will be very old,
up to 4 years old! So of course it occurs before the l1 origin. So a fix
just to get things working is to use time.Now(), but probably we should
make this a configurable parameter.

* add flag to specify timestamp

* Update op-chain-ops/cmd/celo-migrate/main.go

---------

Co-authored-by: piersy <pierspowlesland@gmail.com>

Migration script fixes (#179)

* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point

Fix migration script gap in migrated blocks (#189)

* Fix migration script gap in migrated blocks

The range of ancient blocks to remove from the non ancients database was
off by one and resulted in a gap between ancients and non ancients.

Also corrected some log statements that were off by one.

Add pre-migration command to migration script (#192)

* add pre-migration command, rsync and ancients run in parallel, remove onlyAncients flag

* remove block and state migration sub-commands

* make non ancient migration its own step, add flag to measure time

* add more granular timers

* open db without freezer in state migration, remove clearAll

* fix error

* remove update flag from rsync command, add rsync comments

* delete commented out versions of checkForPrevFullMigration

* remove aliases

* remove clearNonAncients flag

* remove measureTime flag, always log time measurements

* remove logging from help text

* remove db reset

* move scan for extra ancients into pre-migration

* update README

* rename extraAncientNumHashes to strayAncientBlocks

state-migration: Fail if account would be overwritten (#202)

* state-migration: Fail if account would be overwritten

* Review changes

* Review changes 2

* Fail in unclear state

* more changes

* Use whitelist to decide if nonce and state are overwritten

Cosmetic changes to the migration script

- Use more lists for added readability
- Capitalize Alfajores and Celo
- Reorder scripting instructions to fit the actual order or operations
- Use GitHub callouts

migration: Add tests (#217)

* migration: Add tests for state migration

* migration: Fix issues shown by tests

* migration: pass allowlist into state migration

Allows for easier testing

* migration: Add test with allowlist

* Correct overwrite counter

* Use in memory DB

migration: Add working allowlist for Alfajores (#220)

* migration: Simplify tests

* migration: Add working allowlist for Alfajores

Adapt migration code to changes in StateDB

StateDB.CreateAccount used to copy existing balance, now it does not any
more.

migration: Set fields correctly for migration block (#212)

migration: Enable Granite (#226)

Write genesis file in state migration (#219)

* squash of #167

* add writeGenesis

* open old freezer in readonly mode, fix locking error

* remove devAlloc

* Revert "open old freezer in readonly mode, fix locking error"

This reverts commit e3fddea.

* fix locking error

* fix lint error, check errors, add comment

* remove comment

* filter extra genesis fields

* fix issue with genesis extra data

* update testdata

---------

Co-authored-by: Javier Cortejoso <javier.cortejoso@gmail.com>

migration: Overwrite create2deployer code (#233)

migration: Allow 'createx' preinstall (#238)

The code already exists on Alfajores and matches the one that would be
deployed, therefore we just allow this address.

add migration-block-number flag (#245)

* add migration-block-number flag

* address feedback

* move migration-block-number flag out of state migration options

Fixes for re-running migration script on same destination db  (#246)

* add reset flag

* add --checksum to rsync options
karlb pushed a commit that referenced this pull request Jan 13, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Jan 15, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Jan 15, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Jan 20, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Jan 27, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Jan 29, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Jan 29, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Jan 31, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Feb 4, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Feb 4, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Feb 12, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Feb 12, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
karlb pushed a commit that referenced this pull request Feb 12, 2025
* Fixed migration for datadirs without ancients

The script was assuming that ancients would have been migrated and was
considering the numAncients-1 to be the next block to migrate but when
numAncients is zero that's a problem.

Also remved logic for  picking up where db migration left of for the
level db since it was complicating the logic and that process takes a
few seconds, which is nothing compared with the minutes taken to migrate
the ancients.

* Ensure that we set gas limit if migrating at pre-gingerbread point
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants