Skip to content

Make dos-fuzzer part of the workspace#945

Merged
Martin1887 merged 1 commit intopulldown-cmark:masterfrom
kdarkhan:fuzz-workspaces
Sep 3, 2024
Merged

Make dos-fuzzer part of the workspace#945
Martin1887 merged 1 commit intopulldown-cmark:masterfrom
kdarkhan:fuzz-workspaces

Conversation

@kdarkhan
Copy link
Contributor

This is possible because of the change introduced in cargo-fuzz
in rust-fuzz/cargo-fuzz#357.

Using the same workspace enables reusing same deps and sharing cache between members. It also has better editor support.

@kdarkhan
Copy link
Contributor Author

kdarkhan commented Aug 30, 2024

I attempted to do the same for fuzz dir but faced compilation errors. I believe they are caused by outdated patched dependency on moz-js. If the project maintainers approve of the approach, I can replace the existing JS based parser with reference implementation in C in a separate PR. The C library could be used with bindgen generated FFI calls. Let me know what you think. The library seems to be easy to install in Debian/Ubuntu/Archlinux based distros.

Copy link
Collaborator

@ollpu ollpu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks alright if all the scripts still work. Do note that dos-fuzzer doesn't use cargo-fuzz, it's a custom thing entirely.

@ollpu
Copy link
Collaborator

ollpu commented Aug 31, 2024

warning: profiles for the non root package will be ignored, specify profiles at the workspace root:

The root already has lto = true. You can probably just remove it from dos-fuzzer/Cargo.toml.

@kdarkhan kdarkhan force-pushed the fuzz-workspaces branch 2 times, most recently from 8600e20 to fe397b9 Compare August 31, 2024 09:23
@kdarkhan
Copy link
Contributor Author

kdarkhan commented Aug 31, 2024

warning: profiles for the non root package will be ignored, specify profiles at the workspace root:

The root already has lto = true. You can probably just remove it from dos-fuzzer/Cargo.toml.

Removed the config and also adjusted the relative path in dos-fuzzer/run script.

Do note that dos-fuzzer doesn't use cargo-fuzz, it's a custom thing entirely.

I see, did not realize that. The proposed changes should still be good though. If I get an approval for my proposal in this comment, I will work on making fuzz dir part of the workspace.

Using the same workspace enables reusing same deps
and sharing cache between members. It also has better editor
support.
@Martin1887
Copy link
Collaborator

OK, so I think that this can be merged as it is and another pull request could be opened for the other fuzzer, right?

@kdarkhan
Copy link
Contributor Author

kdarkhan commented Sep 2, 2024

OK, so I think that this can be merged as it is and another pull request could be opened for the other fuzzer, right?

Yes, I can work on the other part separately.

Just to be sure, can somebody confirm if I can replace the commonmark JS based implementation with the one implemented in C? It will require anybody who wants to run fuzzers installing the compiled library on machines.

@Martin1887 Martin1887 merged commit 0f97de5 into pulldown-cmark:master Sep 3, 2024
@ollpu
Copy link
Collaborator

ollpu commented Sep 3, 2024

Just to be sure, can somebody confirm if I can replace the commonmark JS based implementation with the one implemented in C? It will require anybody who wants to run fuzzers installing the compiled library on machines.

I would rather try to find a solution for moz-js or change how the JS is run, but I am not very familiar with the setup.

maxdeviant referenced this pull request in zed-industries/zed Sep 5, 2024
This PR upgrades `pulldown_cmark` to v0.12.

There were a few breaking changes that needed to be accounted for:

- The `BlockQuote` variant now has a `kind` attached. Right now we're
ignoring it.
- `pulldown_cmark` now emits tags for definition lists. This codepath
has been left unimplemented, for now.

### Release Notes

<details>
<summary>raphlinus/pulldown-cmark (pulldown-cmark)</summary>

###
[`v0.12.1`](https://redirect.github.com/pulldown-cmark/pulldown-cmark/releases/tag/v0.12.1):
0.12.1

[Compare
Source](https://redirect.github.com/raphlinus/pulldown-cmark/compare/v0.12.0...v0.12.1)

##### Security

- Fix O(n\*\*2) comment parser by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/941](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/941)

##### New features

- impl From<CowStr> for String by
[@&#8203;oconnor663](https://redirect.github.com/oconnor663) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/943](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/943)

##### Developers

- Make dos-fuzzer part of the workspace by
[@&#8203;kdarkhan](https://redirect.github.com/kdarkhan) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/945](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/945)

##### New Contributors

- [@&#8203;oconnor663](https://redirect.github.com/oconnor663) made
their first contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/943](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/943)
- [@&#8203;kdarkhan](https://redirect.github.com/kdarkhan) made their
first contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/945](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/945)

**Full Changelog**:
pulldown-cmark/pulldown-cmark@v0.12.0...v0.12.1

###
[`v0.12.0`](https://redirect.github.com/pulldown-cmark/pulldown-cmark/releases/tag/v0.12.0):
0.12.0

[Compare
Source](https://redirect.github.com/raphlinus/pulldown-cmark/compare/v0.11.3...v0.12.0)

Thanks to all contributors! This release mainly adds the long awaited
commonmark-hs description lists (under a flag) and enables the
blockquote kind in `TagEnd` reverted in 0.11.2.

#### Breaking changes

- feat: re-add kind for BlockQuote in TagEnd by
[@&#8203;Martin1887](https://redirect.github.com/Martin1887) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/940](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/940)
- Refactor TextMergeStream by
[@&#8203;ollpu](https://redirect.github.com/ollpu) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/931](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/931)

#### New features

- Implement commonmark-hs compatible definition lists by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/915](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/915)

#### Other changes

- Rename superlinear time fuzzer to `dos-fuzzer` by
[@&#8203;ollpu](https://redirect.github.com/ollpu) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/938](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/938)

**Full Changelog**:
pulldown-cmark/pulldown-cmark@v0.11.2...v0.12.0

###
[`v0.11.3`](https://redirect.github.com/pulldown-cmark/pulldown-cmark/releases/tag/v0.11.3):
0.11.3

[Compare
Source](https://redirect.github.com/raphlinus/pulldown-cmark/compare/v0.11.2...v0.11.3)

#### Security

- Fix O(n\*\*2) comment parser by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/944](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/944)

**Full Changelog**:
pulldown-cmark/pulldown-cmark@v0.11.2...v0.11.3

###
[`v0.11.2`](https://redirect.github.com/pulldown-cmark/pulldown-cmark/releases/tag/v0.11.2)

[Compare
Source](https://redirect.github.com/raphlinus/pulldown-cmark/compare/v0.11.1...v0.11.2)

Revert BlockQuote kind to avoid breaking change.

###
[`v0.11.1`](https://redirect.github.com/pulldown-cmark/pulldown-cmark/releases/tag/v0.11.1)

[Compare
Source](https://redirect.github.com/raphlinus/pulldown-cmark/compare/v0.11.0...v0.11.1)

Thanks to all people involved in this release! The main change of this
release is the reduction of the MSRV to 1.71.1, but it also includes a
lot of bug fixes and a new mdBook for user-friendly documentation.

#### Breaking changes

- Add BlockQuoteKind to BlockQuote TagEnd by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/926](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/926)
(reverted in v0.11.2)

#### What's Changed

- fix: CowStr deserialization when escaping by
[@&#8203;aatifsyed](https://redirect.github.com/aatifsyed) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/895](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/895)
- fix(test): fix generating spec tests doesn't work on Windows due to
line-endings by [@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/903](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/903)
- feat: add `-G` CLI option to enable GFM support by
[@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/905](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/905)
- feat: set `DefaultBrokenLinkCallback` as the default broken link
callback of `OffsetIter` by
[@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/901](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/901)
- chore(doc): use `cargo add` to instruct how to install this crate as
dependency by [@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/904](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/904)
- fix typo by [@&#8203;jmbhughes](https://redirect.github.com/jmbhughes)
in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/909](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/909)
- Fix parsing blocks inside alert body by
[@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/908](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/908)
- fuzz: fix building fuzzer and improve fuzzing coverage by enabling
more parse options by [@&#8203;rhysd](https://redirect.github.com/rhysd)
in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/910](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/910)
- fix: fix warnings reported from nightly rustc by
[@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/911](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/911)
- fix: fix infinite loop when metadata delimiter is indented by
[@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/913](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/913)
- Raise the link cutoff from 5 to 32 by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/917](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/917)
- Reduce MSRV to 1.71.1 by separating benchmarks into a new crate with
CI enhancements by [@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/916](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/916)
- Add guide book and deploy script for it by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/883](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/883)
- ci: fix deploying the document to GitHub Pages and make the deploy job
faster by [@&#8203;rhysd](https://redirect.github.com/rhysd) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/920](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/920)
- Fix lone task list item bug by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/924](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/924)
- Fix offset range around footnotes that look like images by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/925](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/925)
- Update old footnote format to interrupt paragraph by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/928](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/928)
- Fix confusing bug with back-to-back footnotes by
[@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/930](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/930)
- Add reproduction cases for
[#&#8203;927](https://redirect.github.com/raphlinus/pulldown-cmark/issues/927)
by [@&#8203;zoni](https://redirect.github.com/zoni) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/929](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/929)
- Add regression test for
[#&#8203;655](https://redirect.github.com/raphlinus/pulldown-cmark/issues/655)
by [@&#8203;ollpu](https://redirect.github.com/ollpu) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/932](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/932)
- Renovate the superlinear time fuzzer by
[@&#8203;ollpu](https://redirect.github.com/ollpu) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/935](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/935)

#### New Contributors

- [@&#8203;aatifsyed](https://redirect.github.com/aatifsyed) made their
first contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/895](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/895)
- [@&#8203;jmbhughes](https://redirect.github.com/jmbhughes) made their
first contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/909](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/909)
- [@&#8203;zoni](https://redirect.github.com/zoni) made their first
contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/929](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/929)

**Full Changelog**:
pulldown-cmark/pulldown-cmark@v0.11.0...v0.11.1

###
[`v0.11.0`](https://redirect.github.com/pulldown-cmark/pulldown-cmark/releases/tag/v0.11.0):
0.11.0

[Compare
Source](https://redirect.github.com/raphlinus/pulldown-cmark/compare/v0.10.3...v0.11.0)

##### Finally, the so long awaited math mode is here! Enable the option
to use it.

This release also includes other improvements and bugfixes, please see
the changelog below for more details. Thanks to all contributors that
has made possible this release!

#### Breaking changes

- Change `write_to_html` to allow `fmt::Write` by
[@&#8203;stepantubanov](https://redirect.github.com/stepantubanov) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/870](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/870)

#### New features

-   Math mode

#### Bugfixes

- \[0.11] Don't exit `scan_attribute` with the ix pointing at block
quote by [@&#8203;notriddle](https://redirect.github.com/notriddle) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/873](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/873)
- (Re)introduce simd feature to pulldown-cmark-escape by
[@&#8203;ollpu](https://redirect.github.com/ollpu) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/880](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/880)
- fix: remove unnecessary end_newline set by
[@&#8203;tomcur](https://redirect.github.com/tomcur) in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/885](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/885)

#### New Contributors

- [@&#8203;duskmoon314](https://redirect.github.com/duskmoon314) made
their first contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/874](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/874)
- [@&#8203;stepantubanov](https://redirect.github.com/stepantubanov)
made their first contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/870](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/870)
- [@&#8203;tomcur](https://redirect.github.com/tomcur) made their first
contribution in
[https://github.com/pulldown-cmark/pulldown-cmark/pull/885](https://redirect.github.com/pulldown-cmark/pulldown-cmark/pull/885)

**Full Changelog**:
pulldown-cmark/pulldown-cmark@v0.10.3...v0.11.0

</details>

Release Notes:

- N/A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants