Skip to content

Added benchmarks#56

Merged
bwplotka merged 2 commits intobwplotka:mainfrom
saswatamcode:benchmark
Jul 12, 2021
Merged

Added benchmarks#56
bwplotka merged 2 commits intobwplotka:mainfrom
saswatamcode:benchmark

Conversation

@saswatamcode
Copy link
Copy Markdown
Collaborator

This PR adds benchmarks for mdformatter and mdgen.

Run for mdformatter:

go test -run=^$ -bench=. -benchtime 20s -memprofile mem.out -cpuprofile cpu.out -benchmem github.com/bwplotka/mdox/pkg/mdformatter

Run for mdgen:

go test -run=^$ -bench=. -benchtime 20s -memprofile mem.out -cpuprofile cpu.out -benchmem github.com/bwplotka/mdox/pkg/mdformatter/mdgen 

@saswatamcode saswatamcode requested a review from bwplotka July 3, 2021 14:24
@saswatamcode saswatamcode self-assigned this Jul 3, 2021
@saswatamcode
Copy link
Copy Markdown
Collaborator Author

For me it shows:

go test -run=^$ -bench=. -benchtime 20s -memprofile mem.out -cpuprofile cpu.out -benchmem github.com/bwplotka/mdox/pkg/mdformatter
goos: darwin
goarch: amd64
pkg: github.com/bwplotka/mdox/pkg/mdformatter
cpu: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
Benchmark_Mdformatter/testdata/not_formatted.md-4               1000000000               0.0001016 ns/op               0 B/op          0 allocs/op
PASS
ok      github.com/bwplotka/mdox/pkg/mdformatter        0.578s
go test -run=^$ -bench=. -benchtime 20s -memprofile mem.out -cpuprofile cpu.out -benchmem github.com/bwplotka/mdox/pkg/mdformatter/mdgen          
goos: darwin
goarch: amd64
pkg: github.com/bwplotka/mdox/pkg/mdformatter/mdgen
cpu: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
Benchmark_Mdgen_Sleep2/benchdata/sleep2.md-4            1000000000               0.0008666 ns/op               0 B/op          0 allocs/op
Benchmark_Mdgen_Sleep5/benchdata/sleep5.md-4            1000000000               0.0001499 ns/op               0 B/op          0 allocs/op
Benchmark_Mdgen_GoHelp/benchdata/gohelp.md-4            1000000000               0.0001487 ns/op               0 B/op          0 allocs/op
PASS
ok      github.com/bwplotka/mdox/pkg/mdformatter/mdgen  22.254s

Copy link
Copy Markdown
Owner

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I wish all my programs has so fast execution XD

for n := 0; n < b.N; n++ {
f := New(context.Background())

b.Run(filename, func(b *testing.B) {
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's important to put b.N loop INSIDE the nested b.Run as this is the test case that will be benchmarked. Since you don't have we have misleading results I think.

Copy link
Copy Markdown
Collaborator Author

@saswatamcode saswatamcode Jul 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Updated results:

go test -run=^$ -bench=. -benchtime 20s -memprofile mem.out -cpuprofile cpu.out -benchmem github.com/bwplotka/mdox/pkg/mdformatter/mdgen          
goos: darwin
goarch: amd64
pkg: github.com/bwplotka/mdox/pkg/mdformatter/mdgen
cpu: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
Benchmark_Mdgen_Sleep2/benchdata/sleep2.md-4              361028             65180 ns/op           42746 B/op        375 allocs/op
Benchmark_Mdgen_Sleep5/benchdata/sleep5.md-4              403152             62930 ns/op           42748 B/op        375 allocs/op
Benchmark_Mdgen_GoHelp/benchdata/gohelp.md-4              432640             57559 ns/op           42744 B/op        375 allocs/op
PASS
ok      github.com/bwplotka/mdox/pkg/mdformatter/mdgen  100.219s
go test -run=^$ -bench=. -benchtime 20s -memprofile mem.out -cpuprofile cpu.out -benchmem github.com/bwplotka/mdox/pkg/mdformatter
goos: darwin
goarch: amd64
pkg: github.com/bwplotka/mdox/pkg/mdformatter
cpu: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
Benchmark_Mdformatter/testdata/not_formatted.md-4                 396523             61845 ns/op           42747 B/op        375 allocs/op
PASS
ok      github.com/bwplotka/mdox/pkg/mdformatter        44.476s
go test -run=^$ -bench=. -benchtime 20s -memprofile mem.out -cpuprofile cpu.out -benchmem github.com/bwplotka/mdox/pkg/mdformatter/linktransformer
goos: darwin
goarch: amd64
pkg: github.com/bwplotka/mdox/pkg/mdformatter/linktransformer
cpu: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
Benchmark_Linktransformer/Validator-4             250507            108464 ns/op           43955 B/op        388 allocs/op
PASS
ok      github.com/bwplotka/mdox/pkg/mdformatter/linktransformer        29.197s

f := New(context.Background())

b.Run(filename, func(b *testing.B) {
buf := bytes.Buffer{}
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you don't use this buf, there is a strong risk that compiler will optimize away the WHOLE thing (because result is unused).

I think it's unlikely in your case, because I don't think compiler can tell if Format has side effects or not, but it's advise to create a global testBuf variable and use that (allocate new buffer all the time)

Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
Copy link
Copy Markdown
Owner

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks better, did you have more meaningful results now?

@saswatamcode
Copy link
Copy Markdown
Collaborator Author

Yes, as mentioned here. 🙂

@saswatamcode saswatamcode requested a review from bwplotka July 10, 2021 11:38
Copy link
Copy Markdown
Owner

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing, let's merge but IMO we should generate bigger examples so we can see some patterns (: We can do in next PRs!

Great job LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants