Skip to content

[openwrt-23.05] rust: Improve build performance#22376

Merged
1715173329 merged 12 commits intoopenwrt:openwrt-23.05from
jefferyto:rust-build-performance-openwrt-23.05
Oct 15, 2023
Merged

[openwrt-23.05] rust: Improve build performance#22376
1715173329 merged 12 commits intoopenwrt:openwrt-23.05from
jefferyto:rust-build-performance-openwrt-23.05

Conversation

@jefferyto
Copy link
Member

Maintainer: @lu-zero, me (for python-setuptools-rust)
Compile tested: none (cherry picked from #22339)
Run tested: none

Description:
Please see #22339 for details about these changes.

There is one change while cherry picking these commits; it appears arp-whisper and procs have not been backported to 23.05 so I omitted the changes for those packages (in 0dfc1b5). If there is interest to have those backported I can wait for that to happen, then re-cherry-pick the commit.

The build system already requires Python to be installed.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit a00dae9)
Features to be enabled for host may not be the same as those for target.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit d24095b)
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 35768bf)
* Compress dist archives with gzip instead of xz; gzip is faster to
  compress and decompress

* Use a for loop instead of calling find to extract archives

* Use libdeflate's gzip to decompress instead of gzip

* Limit search for install scripts to top level of extracted archives

This also runs the install scripts with bash instead of sh, in
accordance with the shebang lines inside the scripts.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 855623d)
This allows rustc/cargo/etc to be called without having to set PATH, as
$(STAGING_DIR)/host/bin is already in PATH.

This also fixes CARGO_HOME not being set during Host/Configure and
Host/Compile.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 49aaf19)
This also:

* Modify the "release" profile in place of adding the "stripped" profile

  Only the profile for target is modified; there are no file size
  constraints for host.

* For host, build with the "release" profile

* For target, build with either the "dev" or "release" profile based on
  CONFIG_DEBUG

There is no environment variable to specify the "strip" option, but
enabling this option is not necessary as the build system will already
strip binaries based on CONFIG_NO_STRIP / CONFIG_USE_STRIP /
CONFIG_USE_SSTRIP.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 5c5123f)
As CARGO_HOME mainly functions as a download and source cache[1], moving
it into $(DL_DIR) allows it to persist and be reused between different
buildroots/sdks (when DL_DIR is set to a custom/external location).

[1]: https://doc.rust-lang.org/cargo/guide/cargo-home.html

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 853c9c9)
This consolidates all environment variables for cargo into:

* CARGO_HOST_CONFIG_VARS / CARGO_PKG_CONFIG_VARS

  These contain all cargo-specific environment variables, i.e. without
  "common" variables like CC.

* CARGO_HOST_VARS / CARGO_PKG_VARS (renamed from CARGO_VARS)

  These contain all environment variables to be passed to cargo.

This also:

* Set the CARGO_BUILD_TARGET environment variable instead of using the
  --target command-line option

* Update Python include files to use CARGO_HOST_CONFIG_VARS /
  CARGO_PKG_CONFIG_VARS

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 9db7284)
This allows cargo to use make's jobserver when building packages, by
marking the cargo command as recursive (with the + prefix[1]) and
setting MAKEFLAGS.

This also:

* Give cargo/x.py the build directory instead of having to change the
  current directory (and opening subshells)

* Set PKG_BUILD_PARALLEL/HOST_BUILD_PARALLEL for Rust packages to enable
  the use of make's jobserver

[1]: https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 0dfc1b5)
[omit changes to arp-whisper and procs]
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Using sccache makes recompilation of rustc and Rust packages faster.

This also makes the rust package visible in menuconfig, in order for the
sccache options to be accessible.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit b4ec017)
* codegen-units, lto, opt-level - Set to values to optimize binary
  size[1].

* overflow-checks - Enabled because in release mode, integer overflows
  are defined as two's complement wrap[2]. It is highly unlikely that
  any program is intentionally relying on this behaviour; it would be
  better to panic instead of continue execution in this case.

* debug, debug-assertions, panic, rpath - Set to their default (release)
  values, to override any settings made by packages, e.g. ripgrep sets
  debug = 1[3].

[1]: https://github.com/johnthagen/min-sized-rust
[2]: https://huonw.github.io/blog/2016/04/myths-and-legends-about-integer-overflow-in-rust/
[3]: https://github.com/BurntSushi/ripgrep/blob/13.0.0/Cargo.toml#L79-L80

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 8bf2725)
This adds a patch (submitted upstream in
PyO3/setuptools-rust#364), to read the profile
to pass to cargo from an environment variable.

This also updates the Python include files to set the environment
variable based on values from rust-values.mk.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 29ca979)
@jefferyto jefferyto added the OpenWrt 23.05 (end of support) Issue/PR on branch 23.05 label Oct 12, 2023
@1715173329 1715173329 merged commit cea77ca into openwrt:openwrt-23.05 Oct 15, 2023
@jefferyto jefferyto deleted the rust-build-performance-openwrt-23.05 branch October 15, 2023 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OpenWrt 23.05 (end of support) Issue/PR on branch 23.05

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants