The following was originally posted on Slack. Waiting for comments.

I ran a benchmark, which sends 500 minting requests to Devnet (internally, this produces an invoke tx to STRK ERC20). I ran this 10 times for the latest Devnet commit on main (uses blockifier 0.15.0-rc.2) and 10 times for Devnet with cairo_native enabled (uses blockifier 0.15.0-rc.3). Both Devnets were run as Docker containers. Here's what I got (the measured values are in seconds):

Latest Devnet:

Time sample:   DescribeResult(nobs=10, minmax=(64.17280006408691, 66.57202363014221), mean=65.35412051677704, variance=0.6618370423397418, skewness=-0.3165210792540387, kurtosis=-1.123343669071298)

Devnet with cairo_native:

Time sample:   DescribeResult(nobs=10, minmax=(61.838404178619385, 64.88976907730103), mean=63.579264760017395, variance=1.0566302223600166, skewness=-0.3388753169287398, kurtosis=-1.099763810594258)

I ran an independent t-test, with alternative hypothesis set to "greater" (testing the hypothesis that latest Devnet uses more time):

TtestResult(statistic=4.281465331376874, pvalue=0.00022459226077571598, df=18.0)

Try out cairo-native #826

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions