I gave it a try on my mac but used nu --config /path/to/config.nu --env-config /path/to/env/nu --plugin-config /path/to/plugin.nu -c exit to make sure it loaded my real config. Here are the results.

Benchmark 1 - existing nushell
  Time (mean ± σ):      67.1 ms ±   1.0 ms    [User: 57.1 ms, System: 7.9 ms]
  Range (min … max):    65.3 ms …  75.4 ms    500 runs

Benchmark 2 - this pr
  Time (mean ± σ):      51.8 ms ±   0.7 ms    [User: 44.4 ms, System: 5.2 ms]
  Range (min … max):    50.3 ms …  55.8 ms    500 runs

Benchmark 2 is 1.30 ± 0.03 times faster than Benchmark 1

I gave it a try on my mac but used nu --config /path/to/config.nu --env-config /path/to/env/nu --plugin-config /path/to/plugin.nu -c exit to make sure it loaded my real config. Here are the results.

Benchmark 1 - existing nushell
  Time (mean ± σ):      67.1 ms ±   1.0 ms    [User: 57.1 ms, System: 7.9 ms]
  Range (min … max):    65.3 ms …  75.4 ms    500 runs

Benchmark 2 - this pr
  Time (mean ± σ):      51.8 ms ±   0.7 ms    [User: 44.4 ms, System: 5.2 ms]
  Range (min … max):    50.3 ms …  55.8 ms    500 runs

Benchmark 2 is 1.30 ± 0.03 times faster than Benchmark 1

30% faster, even better 🔥🔥🔥

Same test (with config files) on Windows 11 laptop

Benchmark 1 - existing nushell
  Time (mean ± σ):     231.9 ms ±   9.5 ms    [User: 155.7 ms, System: 36.4 ms]
  Range (min … max):   216.6 ms … 350.2 ms    500 runs

Benchmark 2 - this PR
  Time (mean ± σ):     138.4 ms ±   5.5 ms    [User: 78.9 ms, System: 19.3 ms]
  Range (min … max):   127.6 ms … 174.3 ms    500 runs

Benchmark 2 is 1.68 ± 0.10 times faster than Benchmark 1

It's a little shocking how much faster it is.

@FilipAndersson245 The current thinking is that we want to wait and try this after the next release. JT has seen some projects switch allocators and have odd issues. I'm wondering if there's any way to create tests for this allocator in order to avoid such problems?

Another question is if this allocator is supported on all our platforms that we release to.

And here's another point that was brought up, "I am not sure about is if setting the rust level global allocator messes with tooling that instruments allocations."

@FilipAndersson245 The current thinking is that we want to wait and try this after the next release. JT has seen some projects switch allocators and have odd issues. I'm wondering if there's any way to create tests for this allocator in order to avoid such problems?

Another question is if this allocator is supported on all our platforms that we release to.

And here's another point that was brought up, "I am not sure about is if setting the rust level global allocator messes with tooling that instruments allocations."

seems reasonable,
The supported platforms from their github does not state specific but it seem to be quite good Windows, macOS, Linux, BSD, etc.

After switching, it still works with tool such as valgrind, maybe this could be merged, and then yanked before release if it shows to produce problems 🤔.

I agree tests would be nice, I'm unsure about how the global alloc is set, as its done in the main.rs file, if it is applied to tests, and benchmarks, or just the bin.

its surprising that you got such a large reduction in startup time @fdncred. It seem alot of overhead in Nushell is from allocations.

Thanks for your other comments. I'll pass that along.

its surprising that you got such a large reduction in startup time @fdncred. It seem alot of overhead in Nushell is from allocations.

I'm assuming this is because that i'm loading a few completion files, a few modules, a few scripts where most people may not load all that stuff. But that is just a guess.

multiple platforms

We have a nightly CI that runs now. Maybe we'll be able to see if it fails tomorrow?

We have a nightly CI that runs now. Maybe we'll be able to see if it fails tomorrow?

that would be nice,

I'm assuming this is because that i'm loading a few completion files, a few modules, a few scripts where most people may not load all that stuff. But that is just a guess.

Great that it performs better in more real usecases 🔥

BTW: The nightly CI may work when it's get merged to main branch

BTW: The nightly CI may work when it's get merged to main branch

I thought it was already merged?

BTW: The nightly CI may work when it's get merged to main branch

I thought it was already merged?

The nightly CI was merged, I mean this PR will be checked by the CI after it's been merged

So we merge this and if problems are found we yank it before next release? @fdncred

@FilipAndersson245 I've been looking into this a bit more. Here's what I'd like to do for now. Following how qsv uses different allocators, I think it would nice to enable this functionality behind some features.

So, that would mean something like this.

mimalloc = { version = "0.1", default-features = false, optional = true }
jemallocator = { version = "0.5", optional = true }

and in the code

#[cfg(feature = "mimalloc")]
#[global_allocator]
static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;

#[cfg(feature = "jemallocator")]
#[global_allocator]
static GLOBAL: jemallocator::Jemalloc = jemallocator::Jemalloc;

The difference in qsv vs nushell is that I think it would be good to initially default to the rust "standard" allocator, but have mimalloc and jemallocator in features so that we could compile with any of the 3 allocators for testing.

Searching the qsv repo also revealed this in qsv-x86_64-pc-windows-gnu.txt

The mimalloc memory allocator is not enabled on this prebuilt because of
cross-compilation limitations using GitHub's Action Runners.

Compile qsv natively on this platform if you want to use mimalloc.

One other thing. If you can make these changes, please ensure that the feature shows up when we run the version command. I think it'll be helpful seeing standard, mimalloc, or jemallocator` somehow in version. It could be with the other features or could have a "allocator" row all to itself, up to you.

did a comparason with jemalloc included, it beformed worse then baseline
hyperfine --warmup 50 --runs 1000 -N -L program nu,nu-mim,nu-jel "./{program} -c 'exit'" --export-markdown res.md

should we include it for completeness sake?

I would say no. Having two features that collide is also a cargo crime as cargo assumes that all features are additive only.

Should we enable the feature in some nightly build and see if something blows up?

Should we enable the feature in some nightly build and see if something blows up?

Ya, that would be good. We just added nushell/nightlies.

Nightly release of today: https://github.com/nushell/nightly/releases/tag/nightly-b907bf3

Nightly release of today: https://github.com/nushell/nightly/releases/tag/nightly-b907bf3

awesome, where can we tweak the feature parameters, or is that already included in those builds?

awesome, where can we tweak the feature parameters, or is that already included in those builds?

I'm afraid we can't tweak the feature parameters currently, you can get some information from: https://github.com/nushell/nushell/blob/main/.github/workflows/release-pkg.nu