Skip to content

perf: deprecate mime_guess in favor of a lighter guessing function.#1698

Merged
IWANABETHATGUY merged 3 commits intorolldown:mainfrom
7086cmd:perf/use_mime_more
Jul 26, 2024
Merged

perf: deprecate mime_guess in favor of a lighter guessing function.#1698
IWANABETHATGUY merged 3 commits intorolldown:mainfrom
7086cmd:perf/use_mime_more

Conversation

@7086cmd
Copy link
Copy Markdown
Contributor

@7086cmd 7086cmd commented Jul 23, 2024

I wrapped the mime, and infer crate to mime_more (benchmark result also in README). It also provides lighter mime guessing functions, with the benchmark from using mime_guess in about 289 ns to 37 ns and less rare extensions.

Also, the behavior of the Data URL is aligned with esbuild (except the percent decoding).

@netlify
Copy link
Copy Markdown

netlify Bot commented Jul 23, 2024

Deploy Preview for rolldown-rs canceled.

Name Link
🔨 Latest commit 0456d1a
🔍 Latest deploy log https://app.netlify.com/sites/rolldown-rs/deploys/66a2f1f47c76070008201755

@7086cmd 7086cmd changed the title perf: use mime_more crate. perf: use mime_more crate to handle dataurl. Jul 23, 2024
@7086cmd 7086cmd changed the title perf: use mime_more crate to handle dataurl. perf: use mime_more crate to handle dataurl. Jul 23, 2024
@hyf0
Copy link
Copy Markdown
Member

hyf0 commented Jul 23, 2024

I will suggest to inline this package if code is proven to be better. However, in this case, using outside crate is not friendly for future maintaining, if we want to change something.

@hyf0 hyf0 added on-hold and removed needs-triage labels Jul 23, 2024
@7086cmd
Copy link
Copy Markdown
Contributor Author

7086cmd commented Jul 23, 2024

Ok. How about using the crate's extension_light which can reduce not only the program size but also the bundling time, and ignore other features?

Other implementations are the same as the original one in this repo.

@7086cmd 7086cmd changed the title perf: use mime_more crate to handle dataurl. perf: use mime_more crate to guess mime. Jul 23, 2024
@7086cmd 7086cmd marked this pull request as ready for review July 24, 2024 05:20
@hyf0
Copy link
Copy Markdown
Member

hyf0 commented Jul 24, 2024

Ok. How about using the crate's extension_light which can reduce not only the program size but also the bundling time, and ignore other features?

Other implementations are the same as the original one in this repo.

If it's better I will suggest just copy those code to rolldown's repo. Using outside crate is not friendly for future maintaining, if we want to change something.

@7086cmd
Copy link
Copy Markdown
Contributor Author

7086cmd commented Jul 24, 2024

OK, I see.

@7086cmd 7086cmd force-pushed the perf/use_mime_more branch 2 times, most recently from ad0b8ca to 27f72ef Compare July 24, 2024 06:15
@7086cmd 7086cmd changed the title perf: use mime_more crate to guess mime. perf: deprecate mime_guess in favor of a lighter guessing function. Jul 24, 2024
Comment thread crates/rolldown_utils/src/light_guess.rs
@IWANABETHATGUY
Copy link
Copy Markdown
Member

All snapshot you could found under https://github.com/evanw/esbuild/tree/main/internal/bundler_tests/snapshots, just convert our snake_case into PascalCase and prefix with Test

@7086cmd
Copy link
Copy Markdown
Contributor Author

7086cmd commented Jul 24, 2024

Okay, thanks. I will fix them when I'm free.

Comment thread crates/rolldown_utils/src/light_guess.rs
Comment thread crates/rolldown_utils/src/light_guess.rs
Comment thread crates/rolldown_utils/src/mime.rs Outdated
@7086cmd 7086cmd marked this pull request as draft July 24, 2024 13:37
@7086cmd
Copy link
Copy Markdown
Contributor Author

7086cmd commented Jul 25, 2024

It should be a problem in our test case, and sadly I don't have a way to handle it (indistinguishable file is not supported in GitHub, see it in #1668). In esbuild, the \x should be the transferred expression, for example, when writing

hexString := "\x48\x65\x6C\x6C\x6F"

The hexString is just hello in output.

@7086cmd 7086cmd marked this pull request as ready for review July 25, 2024 04:32
@IWANABETHATGUY
Copy link
Copy Markdown
Member

It should be a problem in our test case, and sadly I don't have a way to handle it (indistinguishable file is not supported in GitHub, see it in #1668). In esbuild, the \x should be the transferred expression, for example, when writing

hexString := "\x48\x65\x6C\x6C\x6F"

The hexString is just hello in output.

Looks like our output is already wrong before https://github.com/rolldown/rolldown/pull/1668/files#diff-d0d7250843734347c76a1838a8e0b2d2f19752da56098bfea90a4e9fc7ac67c0L18,
esbuild

---------- /out.js ----------
// test.custom
var require_test = __commonJS({
  "test.custom"(exports, module) {
    module.exports = "data:application/octet-stream;base64,YQBigGP/ZA==";
  }
});

// entry.js
console.log(require_test());

@7086cmd
Copy link
Copy Markdown
Contributor Author

7086cmd commented Jul 25, 2024

It should be a problem in our test case, and sadly I don't have a way to handle it (indistinguishable file is not supported in GitHub, see it in #1668). In esbuild, the \x should be the transferred expression, for example, when writing

hexString := "\x48\x65\x6C\x6C\x6F"

The hexString is just hello in output.

Looks like our output is already wrong before https://github.com/rolldown/rolldown/pull/1668/files#diff-d0d7250843734347c76a1838a8e0b2d2f19752da56098bfea90a4e9fc7ac67c0L18, esbuild

---------- /out.js ----------
// test.custom
var require_test = __commonJS({
  "test.custom"(exports, module) {
    module.exports = "data:application/octet-stream;base64,YQBigGP/ZA==";
  }
});

// entry.js
console.log(require_test());

Even if before #1668, the behavior with the test may still be "incorrect" because we may regard every \ as a char instead of the synthesis char with the x00.

@7086cmd 7086cmd force-pushed the perf/use_mime_more branch from 71bc2e2 to d2f48bf Compare July 25, 2024 14:32
@IWANABETHATGUY IWANABETHATGUY added this pull request to the merge queue Jul 26, 2024
Merged via the queue into rolldown:main with commit cbcb511 Jul 26, 2024
@7086cmd 7086cmd deleted the perf/use_mime_more branch July 26, 2024 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants