Differential fuzzing of pulldown-cmark and commonmark.js#663
Differential fuzzing of pulldown-cmark and commonmark.js#663Martin1887 merged 2 commits intopulldown-cmark:masterfrom
Conversation
|
Hey there! This is the fuzzer I've been using to find
I don't think it makes sense to merge this as-is since it keeps finding more corner cases. Instead of me opening 10+ issues, I hope the fuzzer can be a useful tool for the maintainers of this crate. I run it with this command: |
|
The dingus deployed version is the latest release (0.30), and so it should be the version used here. In addition, commonmark.js code should be inside a new directoy inside Thanks for your contribution! |
Good point, let me change the JavaScript to a released version to have a stable reference. I'll also move it to the |
|
Any news here? I also think that creating issues reported by the fuzzer is not bad, though a new tag |
No news, I didn't do the cleanups yet. Let me take a look and do that now. |
|
Hi @Martin1887, I moved things around and cleaned up the PR a little. I found that I could use https://lib.rs/rquickjs to run the JavaScript and only pay the setup cost once per fuzzer run. This brings a ~5x speedup on my machine compared to before (from about 30 exec/s to around 150 exec/s). When I run the fuzzer, it normally finds a parsing difference after a few thousand attempts.
I'm glad you like the issues, I'll keep reporting them then 😄 A tag sounds good so that one can look for common patterns. |
|
Great! I will review more carefully the changes and merge the pull request. The first finding I think can be fixed is the big number of new dependencies: they should be put inside dev-dependencies to avoid compiling them in the final binary and also avoiding the increase of vulnerabilities surface. Thanks! |
This adds a new fuzzer which will run random Markdown texts through
both pulldown-cmark and the commonmark.js reference implementation.
The commonmark.js output is turned into Markdown events and the two
event streams are compared.
The bundled `commonmark.min.js` file is from
https://unpkg.com/commonmark@0.30.0/dist/commonmark.min.js
This is the same file you get when you `npm install commonmark`.
Thanks for reviewing it!
The dependencies are only for the fuzzers, so in a sense, they're already dev-dependencies 😄 The |
|
I have tried it in x86_64 and it works, so I have created the following issue and the pull request can be merged. Thanks. |



This adds a new fuzzer which will run random Markdown texts through both pulldown-cmark and commonmark.js. The output from commonmark.js is turned into
Events and the two are compared.The bundled
commonmark.jsandcommonmark.min.jsfiles were generated by runningat revision 20b52e5 of https://github.com/commonmark/commonmark.js/. The output differs slightly from what we see on https://spec.commonmark.org/dingus/ so I’m not sure what version is deployed there.