Closed
Conversation
5 tasks
18a52d9 to
ae6c557
Compare
79f3fea to
8cb71d4
Compare
Earlier 54G -> 25G Now 53G -> 16G (10G excluded for docker image)
Does this work? Yes Is this cursed? Not as cursed as using bash for build system
This should allow for more flexible configuration of docker containers by allowing devs to pass on their own flags to docker Also ~/clean.sh will now not remove /home/builder/.termux-build to account for cases where /home/builder/.termux-build is a volume mounted to the docker image, where it is not possible to remove the directory
We now are always using docker for builds, so can't do this
Do not clutter what we do in "Gather build summary" step
Also make sure we use this optimization in package_updates.yml
~/.termux-build on host Should be helpful for local builds for using IDEs and host tools for development
In commits, %ci:free-space will force freeing space in commits In workflow dispatch, a new checkbox should be available
Instead of only supporting one of the flags, we now support passing multiple flags at the same time for more convenience. The command line argument parser will exit as soon as it detects an argument/flag it doesn't handle to preserve maximum compatibility with existing commands
rid of long length variable name
Logic error found by GitHub Copilot
**BREAKING CHANGES** This now requires AppArmor to be installed and running with docker for limiting the capabilities that `CAP_SYS_ADMIN` provides to containers. Host kernel must support fuse. The host /dev/fuse device is passed onto the container. **DETAILED DESCRIPTION** ./scripts/run-docker.sh first starts with relaxed profile, and then after changing the uid and gid of the builder user and group, drops to restricted profile. Each container get's it's own profile so that if ./scripts/run-docker.sh is run parallel with multiple containers, there is no race condition for the when we are changing the builder uid/gid, where the other container will run with higher privileges than needed. For ensuring least privileges, only mount and umount2 syscalls have been permitted in seccomp profile. Additionally rules for allowing clone, clone3 and similar syscalls when certain contain conditions are met and only when CAP_SYS_ADMIN is not set have been removed as we aren't allowing these syscalls when CAP_SYS_ADMIN is set. The AppArmor profile is based on Docker's default AppArmor profile. The profile was extracted using nerdctl (which is an alternate CLI interface to Docker CLI). The profile can be extracted using `nerdctl apparmor inspect`. There are two AppArmor profiles we have setup, one restricted and relaxed. Currently there is little difference between relaxed and restricted profile. The only difference is that relaxed profile allows any kind of mount syscall, while the restricted profile only allows mount syscalls only for fuse.fuse-overlayfs Regarding security of passing /dev/fuse to containers, the Linux kernel documentation specifies that it should be fine to pass this to namespaces. The CAP_SYSTEM_ADMIN is needed only for the mount syscall to work. Due to some historic reasons this needs this dangerous capability. Although the syscall is needed we are only allowing mount and umount syscalls to happen inside the container, so seccomp profile and apparmor profile should be doing the damage control. Linux kernel documentation for fuse-passthrough: https://docs.kernel.org/6.16/filesystems/fuse-passthrough.html Upstream Linux kernel commit for fuse documentation: torvalds/linux@18ee43c Even without apparmor, things should be fine as we aren't fiddling around much with apparmor for security reasons, just currently limiting where the fuse filesystem can be used. In future, AppArmor profiles can also be used for further hardening of docker image
@licy183 and me for Seccomp profile Me for apparmor profile Feel free to add yourself if you believe that you can deal with this nicely. Mostly for others looking for maintenance of the apparmor profile. Just grab the docker's default config using nerdctl apparmor inspect, diff with the current config and figure it out For the seccomp profile, just diff with the exact commit of moby/moby's seccomp profile and store the updated JSON
AppArmor isn't configured by default on distributions other than Ubuntu, so don't mandate it. AppArmor proper configuration and setup is a huge pain especially if you aren't familiar with it and containers in general. Even a lot of the maintainers aren't already familiar with it and using it already so let's just keep it optional and do not use it if not detected on host.
only respect TERMUX_DOCKER_USE_SUDO for running docker commands Even if docker is setup in rootless mode, apparmor needs to be run with sudo. This was the cause of CI failure and me trying to bang my head on why things weren't working on CI. Thanks, a "sudo" is all it takes. I wish it worked in real life as well on people
We assume in a lot of places that all files in termux-packages are trusted. So make sure that the container isn't able to modify anything in this environment. Also do not allow any changes to ~/lib/ where we are storing Android NDK and SDK.
This is a slightly modified version of robotpy/crossenv@3ef761b We only need the simple shebang, so only use that always.
Also update setuptools to 80.9.0 while we are at full rebuilds with python 3.13
fixes crossenv broken
Relevant upstream commit python/cpython@1f8b24e Also apply this same fix in host build of python as cross-python uses it's own cutils instead of what we have built for target in our cross builds.
…roid api version `python -m build` builds android wheels as android_$TERMUX_ARCH.whl but pip during installation expects it to be android_$TERMUX_API_LEVEL_$TERMUX_ARCH.whl so we need make sure that it is installable without renaming the wheel
[[TODO: add more descriptive commit message]]
ac_cv_func_getlogin_r=no is needed to ensure that it doesn't try to use getlogin_r on Android during on-device builds. Although I have enabled on-device builds, I don't see any point in keeping this as the same version of python needs to be available for building the same version of python. Pretty much defeats the purpose unless you are rebuilding with some changes for the same minor version
Needed to fix hardcoded python version in find_package for Boost.Python. Needed to fix builds for ledger
Also use ninja for building pyarrow for faster builds than unix makefiles. Also remove logic for unsetting PYTHONPATH, as it is needed for Cython to be used.
The libxml2-python subpackage needs to be rebuilt
%ci:free-disk
c83e84c to
29b3f46
Compare
Member
Author
|
Merged in #27739 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.