Skip to content

Faster build & install#153

Closed
agrande wants to merge 2 commits intorust-mobile:masterfrom
agrande:fast-build
Closed

Faster build & install#153
agrande wants to merge 2 commits intorust-mobile:masterfrom
agrande:fast-build

Conversation

@agrande
Copy link
Contributor

@agrande agrande commented Jul 17, 2021

Here are some tweaks that increase the build and install speed a lot for me.
Disabling compression on the apk entries in particular reduced the build time from ~12 seconds to just 2 seconds on my project, which helps a lot with development iteration speed.

This PR is not meant to be merged as it is, but to open a conversation about how this sort of build optimisations could be included in cargo apk, whether as one or multiple command line arguments, or Cargo.toml option(s).

file_name.to_str().unwrap()
));
aapt.arg("add")
.arg("-0")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-0 "" disables compression for any file extension.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be rather added to the aapt package command

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msiglreith I think it should be on both, as the argument only has effect on the files added through that particular command. We don't add any (e.g. heavy .so) files through aapt package, only through aapt add.

@msiglreith
Copy link
Contributor

Thanks! As general feature sth like RUSTFLAGS environment variables for aapt and adb would be interesting, which would cover this. But this combination of flags seems useful enough to provide a shortcut in terms of a command line option.

@MarijnS95
Copy link
Member

If there are no downsides to --fastdeploy that's a welcome flag to add by default. Disabling compression on the other hand sounds like something that's useful in debug scenarios only, and should probably be left enabled when building apk releases.

adb.arg("install").arg("-r").arg(&self.path);
adb.arg("install")
.arg("-r")
.arg("--fastdeploy")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be conditional (ref https://stackoverflow.com/a/50009451)?

@blaind
Copy link

blaind commented Jul 26, 2021

This is an interesting approach, and definitely helpful. In my tests I can get apt installs (excluding rust compile time) down from 12 seconds to 3-4 seconds by using this patch.

However, I noticed one caveat - the "fastdeploy" detects random changes of entries inside APK at different compile times. Sometimes its faster (3-4 secs), sometimes slower (~7-10 secs).

Figured out it might be because of offsets / order that files are added into the APK by aapt. You can see the file listings with aapt list -v ./target/release/apk/examples/[your_example].apk

Did one iteration for sorting the library files by modification time (some debug code and other tests included as well) here:
master...blaind:performance

This seems to help, but if a file is added to asset folder (added by cargo-apt by aapt -A [assets-folder], that causes a full(er) reupload.

I wonder how the APK's actually work. Another approach could be to try if cargo-apk should edit the existing APK (instead of a new?)

This all applies to development mode. Release should not use fastdeploy I guess.

@blaind
Copy link

blaind commented Jul 26, 2021

Another (very very crude) way to speed up the build & install in some cases: stripping out the debug symbols of bundled libraries.

This can be set in Cargo.toml too, but I see cases (esp in dev) where that could be behind a feature flag in cargo apt too.

In my test project I get APT binary size reduction from 103MB down to 39MB with this.

Note: the example below may strip .so's outside target dir too (in runtime_libs folder if defined) - do not run!

Example

diff --git a/ndk-build/src/apk.rs b/ndk-build/src/apk.rs
index 1e7eea0..324e5c1 100644
--- a/ndk-build/src/apk.rs
+++ b/ndk-build/src/apk.rs
@@ -78,6 +78,12 @@ impl<'a> UnalignedApk<'a> {
         if !path.exists() {
             return Err(NdkError::PathNotFound(path.into()));
         }
+
+        std::process::Command::new("aarch64-linux-gnu-strip")
+            .arg(path)
+            .output()
+            .unwrap();
+
         let abi = target.android_abi();
         let file_name = path.file_name().unwrap();
         let out = self.0.build_dir.join("lib").join(abi);

@MarijnS95
Copy link
Member

MarijnS95 commented Jun 3, 2022

@agrande Would you mind if I respin the -0 "" change, wrapped in debug conditionals? I'm starting to work more with cargo-apk and larger libraries, and compression of even relatively small blobs takes insanely long compared to the time it takes to copy it uncompressed and uploading it over a USB cable to the device.

@blaind Is there anything actionable on what you found above, that we can work into a separate PR that enables --fastdeploy? I found these related arguments in help:

     --fastdeploy: use fast deploy
     --no-fastdeploy: prevent use of fast deploy
     --force-agent: force update of deployment agent when using fast deploy
     --date-check-agent: update deployment agent when local version is newer and using fast deploy
     --version-check-agent: update deployment agent when local version has different version code and using fast deploy
     --local-agent: locate agent files from local source build (instead of SDK location)

I have no idea how to read this (without looking up better docs) nor what "agent files" are, but in some way it feels like this allows us to push "source build" files to the device without even packaging them up through aapt first? For a native lib that could mean only adb pushing what has changed which must be super quick.

Primarily because I've been using Mozilla's Rust gradle plugin lately, and remembered how quick Android debug deployment actually is.

@agrande
Copy link
Contributor Author

agrande commented Jun 4, 2022

@MarijnS95 Do whatever you want with this PR, I'm not currently using cargo-apk, and probably won't be in the foreseeable future.

@MarijnS95
Copy link
Member

MarijnS95 commented Jun 5, 2022

Thanks @agrande! I've gone ahead and submitted the -0 change in #283. Unfortunately --fastdeploy isn't working on all my dev devices, some are bailing out with incremental-install being disallowed/disabled, and the fallback to a streamed install getting stuck 🤷. Will queue that up for looking into some time later.

@blaind
Copy link

blaind commented Jun 6, 2022

For me, it's been a while since using cargo-apk. But the gains were substantial, would probably be good to investigate ways to use it in the future (through a feature flag for example). Maybe creating a new issue for future reference, if someone willing to work on it?

@MarijnS95
Copy link
Member

With the compression disablement squared away I'll do some profiling on my project to see how big the gains are for --fastdeploy, and see if we can make an educated decision whether it should be enabled or disabled by default (given that it was flaky for me, the user should have the option to opt-out or opt-in).

@MarijnS95
Copy link
Member

Thanks! Compression disablement was applied in #283, and IIRC empirical testing found that fastdeploy, streaming install and friends seem to be enabled/picked by default.

The cargo-apk portion of this repo moved to https://github.com/rust-mobile/cargo-apk so I'll close it here under ndk; one can always respin this PR there 🤞

@MarijnS95 MarijnS95 closed this Dec 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants