Skip to content

Plumb events through to TxMeta / LedgerCloseMeta#3516

Merged
latobarita merged 24 commits intostellar:masterfrom
graydon:plumb-events-through
Sep 30, 2022
Merged

Plumb events through to TxMeta / LedgerCloseMeta#3516
latobarita merged 24 commits intostellar:masterfrom
graydon:plumb-events-through

Conversation

@graydon
Copy link
Contributor

@graydon graydon commented Aug 24, 2022

This plumbs ContractEvents through to the preflight endpoint, necessary for #3487

It also plumbs the events through to the LedgerCloseMeta v2 / TransactionMeta v3, as described in CAP-0056.

It also (hopefully correctly) accumulates and hashes the new TransactionResultSetV2 into the ledger header, while still (for now) continuing to publish the old TransactionResultSet values to the database and thus history archives. This will (currently) break offline catchup, which does a download-and-verify job on results files in the archives (which, being defined to contain TransactionResultSet, will still load ok but will not verify against the ledger hash), and will also break archivist and likely anything else that tires to check the identity between results and ledger headers. We should decide how to proceed here before finalizing the design.

@graydon graydon requested a review from sisuresh August 24, 2022 06:25
@graydon graydon changed the title Plumb events through WIP: Plumb events through Aug 26, 2022
@graydon graydon marked this pull request as draft August 26, 2022 06:57
@graydon graydon force-pushed the plumb-events-through branch from 91bcc1a to b817517 Compare September 3, 2022 05:14
@graydon graydon force-pushed the plumb-events-through branch 3 times, most recently from 7dd2847 to 9a65481 Compare September 17, 2022 04:45
@graydon graydon changed the title WIP: Plumb events through Plumb events through to TxMeta / LedgerCloseMeta Sep 18, 2022
@graydon graydon force-pushed the plumb-events-through branch from 9a65481 to b731434 Compare September 18, 2022 05:21
@graydon graydon marked this pull request as ready for review September 18, 2022 05:21
@graydon graydon force-pushed the plumb-events-through branch 6 times, most recently from bf1bbc7 to 056673a Compare September 20, 2022 02:57
Copy link
Contributor

@MonsieurNicolas MonsieurNicolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few issues but otherwise looks good

Copy link
Contributor

@MonsieurNicolas MonsieurNicolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am pushing a possible fix. We should add a test that does something like closeLedger(*app, {tx1, tx2, tx3}, true); and checks (maybe using the "debug meta" functionality) that we produce the meta that we expect?

@MonsieurNicolas
Copy link
Contributor

I pushed a couple commits to that branch:

We should add some tests for this (I suggested in a comment to maybe leverage the "debug stream" in some way: it should be fairly easy to add a few checks against the ledger close meta generated as part of the "meta stream test suite", a ledger containing a fee bump, a regular transaction and maybe a tx that produces events).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you forgot to update normalizeMeta in dumpxdr.cpp

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm actually for hashes to work properly we have to normalize the meta in protocol (so here)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and I'm not sure this can be done more efficiently that what was done in dumpxdr)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@graydon
Copy link
Contributor Author

graydon commented Sep 28, 2022

@MonsieurNicolas I've added a very basic test that does as suggested -- throws some txs at closeLedger and checks the meta is vaguely sensible -- but I'm out of time tonight; will extend it tomorrow, as well as wrap up whatever you mean above in the comment a bout fee bumps, and regenerate the meta test vectors again.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we probably don't want to do this if we're not on v=3 (as this has some unknown perf impact)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't there be a case 3 here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! Yes of course, sorry. Also will add a default assert.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a couple other transactions in there (as the bug we had was that meta was broken with ledgers with more than 1 tx).
For good measure one of these could be a fee bump transaction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just convert to json and do a string comparison of the entire record for "ledger 3"?
That way we'll have in the test the actual expected meta (which will allow us to see when we make any meta changes - as this breaks Horizon).

Also note that vcurrent and vnext have a different format (which we'll see in the test)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, doing so (it adds a lot of test vector files, I eyeballed them a bit but second opinions on whether they're correct welcome!)

@graydon graydon force-pushed the plumb-events-through branch from 74df22d to 1aa67c7 Compare September 30, 2022 02:07
@graydon graydon force-pushed the plumb-events-through branch from 1aa67c7 to d0c42b7 Compare September 30, 2022 03:20
@graydon graydon force-pushed the plumb-events-through branch from ffb848c to 301bfaa Compare September 30, 2022 04:37
@graydon
Copy link
Contributor Author

graydon commented Sep 30, 2022

@MonsieurNicolas at last I believe this is rebased, passing all tests, and (I hope!) addressing all your comments. Should be ready to land.

@MonsieurNicolas
Copy link
Contributor

r+ d4b28ee

@latobarita latobarita merged commit cbd1ea4 into stellar:master Sep 30, 2022
@graydon graydon deleted the plumb-events-through branch October 31, 2025 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants