Skip to content

(core, sequencer)!: generate serde traits impls for all protocol protobufs#1260

Merged
SuperFluffy merged 6 commits intomainfrom
superfluffy/serde-for-all-protobuf
Jul 11, 2024
Merged

(core, sequencer)!: generate serde traits impls for all protocol protobufs#1260
SuperFluffy merged 6 commits intomainfrom
superfluffy/serde-for-all-protobuf

Conversation

@SuperFluffy
Copy link
Copy Markdown
Contributor

@SuperFluffy SuperFluffy commented Jul 10, 2024

Summary

This patch ensures extends our generated serde Deserialize and Serialize trait implementations to all protobuf messages defined in astria.protocol.

Background

So far the protocol-specs did not have serde traits generated, but these are necessary to write out bridge-withdrawer actions as JSON.

A side-effect of this patch is that all of proto/protocolapis/astria_vendored is now also commited to the repository (they were previously filtered). Very likely this is not desired but should be addressed in a followup. It is not clear to the author of this patch in how far these types might be necessary.

Changes

  • Generate serde Deserialize and Serialize impls for Rust types generated from astria.protocol and astria_vendored protobufs
  • Use a vendored astria_vendored::tendermint::abci::ValidatorUpdate instead of tendermint_proto: tendermint_proto@v0.34 does not yet have serde impls for its types, which blocks us from having an easy way to use pbjson generated serde impls
  • Introduce a new ValidatorUpdate action to replace tendermint::validator::Update to transform to/from the vendored protobuf type (again necessary to implement serde and foreign type restrictions)
  • Update sequencer to use the new ValidatorUpdate type

Testing

All tests have been upgraded and still pass. Specifically, ingestion of validator updates happens through cometbft, which were already tested end-to-end.

Breaking Changelist

  • Marked as breaking even though none of the breaking change tests are affected: the serialization shape of the validator updates has changed. As they are commited to state using json, this should be breaking.

@github-actions github-actions bot added conductor pertaining to the astria-conductor crate proto pertaining to the Astria Protobuf spec sequencer pertaining to the astria-sequencer crate labels Jul 10, 2024
@SuperFluffy SuperFluffy force-pushed the superfluffy/serde-for-all-protobuf branch from 5c406b2 to 40dc15c Compare July 10, 2024 20:48
this also requires introducing a bespoke ValidatorUpdate
type because tendermint_proto@v0.34 does not derive serde
traits for it ValidatorUpdate type.
@SuperFluffy SuperFluffy force-pushed the superfluffy/serde-for-all-protobuf branch from 40dc15c to 4b26f8b Compare July 10, 2024 20:51
@SuperFluffy SuperFluffy marked this pull request as ready for review July 10, 2024 21:10
@SuperFluffy SuperFluffy requested a review from a team as a code owner July 10, 2024 21:10
@SuperFluffy SuperFluffy requested a review from Fraser999 July 10, 2024 21:10
Copy link
Copy Markdown
Contributor

@Fraser999 Fraser999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - just a couple of non-blocking nits.

@SuperFluffy SuperFluffy enabled auto-merge July 11, 2024 05:42
@SuperFluffy SuperFluffy added this pull request to the merge queue Jul 11, 2024
Merged via the queue into main with commit 5757558 Jul 11, 2024
@SuperFluffy SuperFluffy deleted the superfluffy/serde-for-all-protobuf branch July 11, 2024 05:56
steezeburger added a commit that referenced this pull request Jul 15, 2024
* main:
  feat(cli): add cmd to collect withdrawal events and submit as actions (#1261)
  fix(core, bridge, sequencer)!: dismabiguate return addresses (#1266)
  fix(withdrawer): support withdrawer address that differs from bridge address   (#1262)
  (core, sequencer)!: generate serde traits impls for all protocol protobufs (#1260)
  fix(charts): add resources for sequencer/cometbft (#1254)
  chore(sequencer)!: add metrics (#1248)
  fix(sequencer-utils): fixes issue in `parse_blob` tests (#1243)
  feat(core, proto)!: make bridge unlock memo string (#1244)
  fix(conductor): don't panic during panic (#1252)
  feat(core)!: lowerCamelCase for protobuf json mapping (#1250)
  refactor(bridge-withdrawer)!: refactor startup to a separate subtask and remove balance check from startup (#1190)
  fix: rollup archive node configurations (#1249)
  refactor(core, bridge-withdrawer)!: move bridge-unlock memo to core (#1245)
  fix(sequencer)!: store native asset ibc->trace mapping in init_chain (#1242)
steezeburger added a commit that referenced this pull request Jul 19, 2024
* main: (24 commits)
  chore: update `bytes` and `ics23` crates (#1279)
  fix(sequencer): improve and fix instrumentation (#1255)
  feature(charts): hermes chart fixes, bech32 updates, ibc bridge test (#1130)
  chore(cli): remove unused rollup cli code (#1275)
  chore(test): use a temporary file to not pollute the workspace (#1269)
  chore(sequencer): add mempool benchmarks (#1238)
  fix(bridge-withdrawer)!: fix nonce handling (#1215)
  feat(cli, bridge-withdrawer)!: share code between cli and service (#1270)
  feat(cli): add cmd to collect withdrawal events and submit as actions (#1261)
  fix(core, bridge, sequencer)!: dismabiguate return addresses (#1266)
  fix(withdrawer): support withdrawer address that differs from bridge address   (#1262)
  (core, sequencer)!: generate serde traits impls for all protocol protobufs (#1260)
  fix(charts): add resources for sequencer/cometbft (#1254)
  chore(sequencer)!: add metrics (#1248)
  fix(sequencer-utils): fixes issue in `parse_blob` tests (#1243)
  feat(core, proto)!: make bridge unlock memo string (#1244)
  fix(conductor): don't panic during panic (#1252)
  feat(core)!: lowerCamelCase for protobuf json mapping (#1250)
  refactor(bridge-withdrawer)!: refactor startup to a separate subtask and remove balance check from startup (#1190)
  fix: rollup archive node configurations (#1249)
  ...
bharath-123 pushed a commit that referenced this pull request Jul 25, 2024
…obufs (#1260)

## Summary
This patch ensures extends our generated serde `Deserialize` and
`Serialize` trait implementations to all protobuf messages defined in
`astria.protocol`.

## Background
So far the protocol-specs did not have serde traits generated, but these
are necessary to write out bridge-withdrawer actions as JSON.

A side-effect of this patch is that all of
`proto/protocolapis/astria_vendored` is now also commited to the
repository (they were previously filtered). Very likely this is not
desired but should be addressed in a followup. It is not clear to the
author of this patch in how far these types might be necessary.

## Changes
- Generate serde `Deserialize` and `Serialize` impls for Rust types
generated from `astria.protocol` and `astria_vendored` protobufs
- Use a vendored `astria_vendored::tendermint::abci::ValidatorUpdate`
instead of `tendermint_proto`: `tendermint_proto@v0.34` does not yet
have serde impls for its types, which blocks us from having an easy way
to use `pbjson` generated serde impls
- Introduce a new `ValidatorUpdate` action to replace
`tendermint::validator::Update` to transform to/from the vendored
protobuf type (again necessary to implement serde and foreign type
restrictions)
- Update sequencer to use  the new `ValidatorUpdate` type

## Testing
All tests have been upgraded and still pass. Specifically, ingestion of
validator updates happens through cometbft, which were already tested
end-to-end.

## Breaking Changelist
- Marked as breaking even though none of the breaking change tests are
affected: the serialization shape of the validator updates has changed.
As they are commited to state using json, this should be breaking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conductor pertaining to the astria-conductor crate proto pertaining to the Astria Protobuf spec sequencer pertaining to the astria-sequencer crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants