Skip to content

[PSE]: multi-block PSE migration v7#106

Merged
metalarm10 merged 27 commits into
masterfrom
john/multiblock-pse-migration
Mar 18, 2026
Merged

[PSE]: multi-block PSE migration v7#106
metalarm10 merged 27 commits into
masterfrom
john/multiblock-pse-migration

Conversation

@metalarm10

@metalarm10 metalarm10 commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description

This PR closes: https://app.clickup.com/t/868htnh72

Summary

  1. Added LastProcessedDistributionID store to track completed distributions, replacing the implicit ID=0 fallback from PeekNextAllocationSchedule on empty schedule.
  2. Refactored getNextDistributionID to use LastProcessedDistributionID + 1 (so the correct ID is returned even when the schedule is empty).
  3. Added getActiveDistributionID for score queries.
  4. Reject governance schedule proposals where first ID < LastProcessedDistributionID + 1 (prevents reusing already-processed IDs).
  5. Updated v7 migration: re-key DelegationTimeEntries and AccountScoreSnapshot under distributionID=1, rekeys unprocessed old timestamp-keyed schedules with sequential IDs (starting from id=1), init LastProcessedDistributionID=0.
  6. Added genesis proto export/import for LastProcessedDistributionID.
  7. Enabled ID=0 rejection in schedule validation, shift test schedule IDs from "0,1,2" to "1,2,3" to reflect real-world usage where ID=0 is reserved as unset.
  8. Added migration unit tests and chain upgrade integration test for multi-block PSE.

Tests

  1. v7 migration unit tests: verifies old-format entries re-keyed under distributionID=1, time-based keyed old entries moved to sequential IDs, schedule cleared, LastProcessedDistributionID initialized to 0. (Covers migration on empty store as well).
    2. Chain upgrade integration test: verifies params preserved, LastProcessedDistributionID initialized to 0, old time-based schedule migrated to sequential IDs, delegator scores migrated to id based stores & values are preserved.

Reviewers checklist:

  • Try to write more meaningful comments with clear actions to be taken.
  • Nit-picking should be unblocking. Focus on core issues.

Authors checklist

  • Provide a concise and meaningful description
  • Review the code yourself first, before making the PR.
  • Annotate your PR in places that require explanation.
  • Think and try to split the PR to smaller PR if it is big.

This change is Reviewable

@metalarm10 metalarm10 changed the title draft: Multiblock PSE migration v7 [PSE]: multi-block PSE migration v7 Mar 16, 2026
@metalarm10 metalarm10 marked this pull request as ready for review March 16, 2026 15:21
@metalarm10 metalarm10 requested a review from a team as a code owner March 16, 2026 15:21
@metalarm10 metalarm10 requested review from TxCorpi0x, akhlopiachyi, masihyeganeh, miladz68 and ysv and removed request for a team March 16, 2026 15:21

@ysv ysv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ysv reviewed 28 files and all commit messages, and made 6 comments.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on akhlopiachyi, masihyeganeh, metalarm10, miladz68, and TxCorpi0x).


a discussion (no related file):
Thanks for clear description of PR scope 👍


app/upgrade/v7/migrate_pse.go line 19 at r1 (raw file):

// distributionID is the initial distribution ID assigned to all existing entries.
// After this upgrade, a new governance proposal must set the schedule starting at ID 1.
const distributionID uint64 = 1

so If first distribution is executed before we make mainnet upgrade then 2nd one will have ID=1 ?

lets discuss how we approach this


app/upgrade/v7/migrate_pse.go line 34 at r1 (raw file):

	}

	if err := migrateAccountScoreSnapshot(ctx, storeService, distributionID); err != nil {

nit: shouldn't it be plural migrateAccountScoreSnapshot ?


app/upgrade/v7/migrate_pse.go line 46 at r1 (raw file):

// clearAllocationSchedule removes all entries from the AllocationSchedule.
// The schedule will be re-submitted via governance after the upgrade.

lets discuss

Code quote:

// The schedule will be re-submitted via governance after the upgrade.

app/upgrade/v7/migrate_pse_test.go line 159 at r1 (raw file):

	requireT.ErrorIs(err, collections.ErrNotFound)

	// Verify LastProcessedDistributionID = 0.

lets discuss

Code quote:

// Verify LastProcessedDistributionID = 0.

integration-tests/upgrade/pse_test.go line 79 at r1 (raw file):

	requireT.Empty(schedRes.ScheduledDistributions)

	// Validator's score should be preserved (and grown since time has passed).

what is the magnitude of growth here ?
Can we validate it 'inPercentage' or possible range is too big here ?

@metalarm10 metalarm10 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@metalarm10 made 4 comments and resolved 1 discussion.
Reviewable status: 25 of 28 files reviewed, 4 unresolved discussions (waiting on akhlopiachyi, masihyeganeh, miladz68, TxCorpi0x, and ysv).


app/upgrade/v7/migrate_pse.go line 19 at r1 (raw file):

Previously, ysv (Yaroslav Savchuk) wrote…

so If first distribution is executed before we make mainnet upgrade then 2nd one will have ID=1 ?

lets discuss how we approach this

Update: After your review, migration logic is improved: instead of fully pruning the old timestamp-keyed schedule and requiring a new governance proposal, migration now automatically re-keys remaining unprocessed schedule entries with sequential IDs starting from 1.

All existing scores and delegation time entries are intentionally migrated under ID=1, to be processed in the first scheduled multi-block PSE distribution.

Why ID=1 and not 0: The system uses LastProcessedDistributionID to track the last completed distribution. Its initial value is 0 (uint64 default), meaning "nothing processed yet." This value is updated to ongoing.ID after each distribution round completes.
If our first ID were 0, LastProcessedDistributionID would be 0 both before and after the first distribution, so there's no way to distinguish "not set" from "distribution 0 completed," since both return 0.
This ambiguity causes hooks and queries to point to the wrong distribution ID. I see potential issues with starting IDs at 0, but open to discussion if you have better suggestions.


app/upgrade/v7/migrate_pse.go line 46 at r1 (raw file):

Previously, ysv (Yaroslav Savchuk) wrote…

lets discuss

Update: This part is updated in the migration logic. The migration now auto-assigns sequential IDs to existing (unprocessed) schedule entries, so no governance re-submission is needed post-upgrade.


app/upgrade/v7/migrate_pse_test.go line 159 at r1 (raw file):

Previously, ysv (Yaroslav Savchuk) wrote…

lets discuss

Sure.


integration-tests/upgrade/pse_test.go line 79 at r1 (raw file):

Previously, ysv (Yaroslav Savchuk) wrote…

what is the magnitude of growth here ?
Can we validate it 'inPercentage' or possible range is too big here ?

The ~50s test duration (voting + upgrade) adds tokens × 50 to the score. Since the growth depends on elapsed test time, maybe adding a bound here can make the integration test flaky.
Logs from actual test run:
pse_test.go:129: PSE After: params preserved, LastProcessedDistributionID=0, schedule re-keyed (2 entries with IDs), score preserved (1260000000000000 -> 1760000000000000)

@metalarm10 metalarm10 requested a review from ysv March 17, 2026 09:47

@ysv ysv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ysv made 2 comments and resolved 2 discussions.
Reviewable status: 25 of 28 files reviewed, 2 unresolved discussions (waiting on akhlopiachyi, masihyeganeh, metalarm10, miladz68, and TxCorpi0x).


app/upgrade/v7/migrate_pse.go line 19 at r1 (raw file):

Previously, metalarm10 wrote…

Update: After your review, migration logic is improved: instead of fully pruning the old timestamp-keyed schedule and requiring a new governance proposal, migration now automatically re-keys remaining unprocessed schedule entries with sequential IDs starting from 1.

All existing scores and delegation time entries are intentionally migrated under ID=1, to be processed in the first scheduled multi-block PSE distribution.

Why ID=1 and not 0: The system uses LastProcessedDistributionID to track the last completed distribution. Its initial value is 0 (uint64 default), meaning "nothing processed yet." This value is updated to ongoing.ID after each distribution round completes.
If our first ID were 0, LastProcessedDistributionID would be 0 both before and after the first distribution, so there's no way to distinguish "not set" from "distribution 0 completed," since both return 0.
This ambiguity causes hooks and queries to point to the wrong distribution ID. I see potential issues with starting IDs at 0, but open to discussion if you have better suggestions.

06.03-06.04 - PSE 1 -> ID: 1
06.04-06.05 - PSE 2 -> ID: 2

Agreed to query current mainnet config and set it inside upgrade directly. Only add ID field.

NOte: We want to preserve all schedule entries so we don't remove them.


integration-tests/upgrade/pse_test.go line 79 at r1 (raw file):

Previously, metalarm10 wrote…

The ~50s test duration (voting + upgrade) adds tokens × 50 to the score. Since the growth depends on elapsed test time, maybe adding a bound here can make the integration test flaky.
Logs from actual test run:
pse_test.go:129: PSE After: params preserved, LastProcessedDistributionID=0, schedule re-keyed (2 entries with IDs), score preserved (1260000000000000 -> 1760000000000000)

agreed to improve

@TxCorpi0x TxCorpi0x left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I think the poin that i mentioned and also a good integration to cover the update schedule scenarios with multi-block is needed @ysv @metalarm10

@TxCorpi0x partially reviewed 30 files and made 2 comments.
Reviewable status: 28 of 30 files reviewed, 3 unresolved discussions (waiting on akhlopiachyi, masihyeganeh, metalarm10, and miladz68).


x/pse/keeper/hooks.go line 71 at r5 (raw file):

		return 0, err
	}
	return lastProcessed + 1, nil

I am curious about this scenario that may cause a discrepancy in the ID:
If a governance proposal for updating the schedule will be runing getNextDistributionID writes state change to LastProcessedDistributionID + 1 while in distribution.go:337 the schedule starts later, (e.g. lastProcessed=5, but new schedule starts at 10), so the Hok writes the new entries under ID 6, shceduler processed ID 10 first. and distribution.go:40 reads ID 10 (not 6) may produce empty community distribution.

@metalarm10 metalarm10 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unit test added for disallowing schedule ID gaps on gov proposal handler. Detailed integration tests will be added in the next PR. (Thanks for pointing this out).

@metalarm10 made 4 comments.
Reviewable status: 27 of 31 files reviewed, 3 unresolved discussions (waiting on akhlopiachyi, masihyeganeh, miladz68, TxCorpi0x, and ysv).


app/upgrade/v7/migrate_pse.go line 19 at r1 (raw file):

Previously, ysv (Yaroslav Savchuk) wrote…

06.03-06.04 - PSE 1 -> ID: 1
06.04-06.05 - PSE 2 -> ID: 2

Agreed to query current mainnet config and set it inside upgrade directly. Only add ID field.

NOte: We want to preserve all schedule entries so we don't remove them.

Done.


integration-tests/upgrade/pse_test.go line 79 at r1 (raw file):

Previously, ysv (Yaroslav Savchuk) wrote…

agreed to improve

Done.


x/pse/keeper/hooks.go line 71 at r5 (raw file):

Previously, TxCorpi0x wrote…

I am curious about this scenario that may cause a discrepancy in the ID:
If a governance proposal for updating the schedule will be runing getNextDistributionID writes state change to LastProcessedDistributionID + 1 while in distribution.go:337 the schedule starts later, (e.g. lastProcessed=5, but new schedule starts at 10), so the Hok writes the new entries under ID 6, shceduler processed ID 10 first. and distribution.go:40 reads ID 10 (not 6) may produce empty community distribution.

Done. Governance validation now enforces first schedule ID == LastProcessedDistributionID + 1 (no gaps allowed). This ensures hooks and scheduler always write/read the same distribution ID.

@metalarm10 metalarm10 requested review from TxCorpi0x and ysv March 18, 2026 13:12

@ysv ysv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ysv reviewed 12 files and all commit messages, and resolved 2 discussions.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on akhlopiachyi, masihyeganeh, miladz68, and TxCorpi0x).

@TxCorpi0x TxCorpi0x left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TxCorpi0x reviewed 4 files and all commit messages, and resolved 1 discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on akhlopiachyi, masihyeganeh, and miladz68).

@metalarm10 metalarm10 merged commit 7cbb3e8 into master Mar 18, 2026
17 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants