Skip to content

fix: safely allow unbounded mappings in cnight observation pallet#1423

Merged
ozgb merged 21 commits into
mainfrom
oscar-cnight-observation-mapping-storage-split
May 12, 2026
Merged

fix: safely allow unbounded mappings in cnight observation pallet#1423
ozgb merged 21 commits into
mainfrom
oscar-cnight-observation-mapping-storage-split

Conversation

@ozgb

@ozgb ozgb commented Apr 27, 2026

Copy link
Copy Markdown
Contributor

Overview

Replaces the single Mappings storage map with a new Mapping double storage map

We were storing mappings for each Cardano address in an unbounded Vec - every time we add/remove a registration this Vec is loaded into memory - if it gets too big, this could cause allocation failures and liveness issues.

🗹 TODO before merging

  • Ready

📌 Submission Checklist

  • All commits are signed off (git commit -s) for the DCO
  • Changes are backward-compatible (or flagged if breaking)
  • Pull request description explains why the change is needed
  • Self-reviewed the diff
  • I have included a change file, or skipped for this reason:
  • If the changes introduce a new feature, I have bumped the node minor version
  • Update documentation (if relevant)
  • Updated AGENTS.md if build commands, architecture, or workflows changed
  • No new todos introduced

🧪 Testing Evidence

Please describe any additional testing aside from CI:

  • Additional tests are provided (if possible)
  • Existing tests pass
  • Tests added for migration

🔱 Fork Strategy

  • Node Runtime Update
  • Node Client Update
  • Other:
  • N/A

Links

Fixes https://github.com/midnightntwrk/midnight-security/issues/116

ozgb added 2 commits April 27, 2026 09:59
Uses two new maps to replace the previous single `Mappings` storage map - avoids encoding/decoding an unbounded vec in runtime

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
@ozgb ozgb requested a review from a team as a code owner April 27, 2026 09:11
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
@ozgb ozgb added the ai-assisted Created or modified with AI assistance label Apr 27, 2026
Comment thread pallets/cnight-observation/src/migration.rs Outdated
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Comment thread pallets/cnight-observation/src/lib.rs Outdated
Comment thread pallets/cnight-observation/src/lib.rs Outdated
ozgb added 3 commits April 27, 2026 11:07
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
…ture

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
@ozgb

ozgb commented Apr 27, 2026

Copy link
Copy Markdown
Contributor Author

/bot rebuild-metadata

@github-actions

Copy link
Copy Markdown
Contributor

❌ Metadata rebuild failed. Check the workflow logs for details.

Comment thread pallets/cnight-observation/src/migration.rs Outdated
Comment thread pallets/cnight-observation/src/migration.rs Outdated
ozgb added 7 commits April 28, 2026 10:33
Replaces the OnRuntimeUpgrade-style migration with a SteppedMigration so
v0 -> v1 storage drains incrementally over many blocks rather than in
the upgrade block. Per-step weight is bounded by MAX_ENTRIES_PER_ADDR
(observed worst case across networks). Storage version is bumped
inline on completion since pallet_migrations does not bump it for us.

Wires the migration into pallet_migrations::Config::Migrations and
removes it from the executive Migrations tuple. Tests drive
SteppedMigration::step directly against the existing mock.

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
These changes came from switching the pallet to use sidechain_domain::UtxoId

Changed genesis to keep the previous utxo_tx_hash and utxo_index split

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
The mock previously used `type DbWeight = ()`, which yields zero per-step
weight and prevents exercising the SteppedMigration paths that depend on
WeightMeter pressure. Replace with a `parameter_types! pub static` so the
default is non-zero and tests can override per-test via MockDbWeight::set.

Adds two tests now feasible against the v0->v1 migration: that an empty
meter surfaces InsufficientWeight, and that step returns a resume cursor
when the meter exhausts mid-migration.

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
@ozgb

ozgb commented Apr 28, 2026

Copy link
Copy Markdown
Contributor Author

/bot rebuild-metadata

@github-actions

Copy link
Copy Markdown
Contributor

✅ Metadata rebuild complete! Changes have been committed.

@ozgb ozgb closed this Apr 28, 2026
@ozgb ozgb reopened this Apr 28, 2026
ozgb added 2 commits May 1, 2026 23:20
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
@ozgb

ozgb commented May 5, 2026

Copy link
Copy Markdown
Contributor Author

/bot rebuild-metadata

@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

✅ Metadata rebuild complete! Changes have been committed.

@ozgb ozgb closed this May 5, 2026
@ozgb ozgb reopened this May 5, 2026
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
@Klapeyron

Copy link
Copy Markdown
Contributor

/bot rebuild-metadata

@github-actions

Copy link
Copy Markdown
Contributor

✅ Metadata rebuild complete. No changes detected.

@ozgb

ozgb commented May 12, 2026

Copy link
Copy Markdown
Contributor Author

/bot fix-metadata-conflicts accept-incoming

@github-actions

Copy link
Copy Markdown
Contributor

✅ Metadata conflicts resolved successfully!

Strategy used: accept-incoming
Files resolved: metadata/static/midnight_metadata.scale, metadata/static/midnight_metadata_1.0.0.scale

The main branch has been merged into this PR with metadata conflicts resolved.

@ozgb

ozgb commented May 12, 2026

Copy link
Copy Markdown
Contributor Author

/bot rebuild-metadata

…ch changes)

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
@ozgb ozgb force-pushed the oscar-cnight-observation-mapping-storage-split branch from ff0ecba to 17fe44b Compare May 12, 2026 09:07
@ozgb

ozgb commented May 12, 2026

Copy link
Copy Markdown
Contributor Author

/bot rebuild-metadata

@github-actions

Copy link
Copy Markdown
Contributor

✅ Metadata rebuild complete! Changes have been committed.

@ozgb ozgb closed this May 12, 2026
@ozgb ozgb reopened this May 12, 2026
@ozgb ozgb enabled auto-merge May 12, 2026 09:30
@ozgb ozgb added this pull request to the merge queue May 12, 2026
Merged via the queue into main with commit 390ba42 May 12, 2026
54 of 55 checks passed
@ozgb ozgb deleted the oscar-cnight-observation-mapping-storage-split branch May 12, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Created or modified with AI assistance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants