Skip to content

Use static TLS implementation#1165

Closed
jensmeindertsma wants to merge 27 commits intovolta-cli:mainfrom
jensmeindertsma:use-static-tls-impl
Closed

Use static TLS implementation#1165
jensmeindertsma wants to merge 27 commits intovolta-cli:mainfrom
jensmeindertsma:use-static-tls-impl

Conversation

@jensmeindertsma
Copy link
Copy Markdown

@jensmeindertsma jensmeindertsma commented Feb 23, 2022

This pull request implements volta-cli/rfcs#47.

Tasks

  • Enable rustls feature of attohttpc
  • Update install script now that we don't specify OpenSSL version in download URL anymore
  • Test x86_64-unknown-linux-gnu binary on CentOS 6
  • Test MUSL-based binaries

@jensmeindertsma
Copy link
Copy Markdown
Author

This pull request is ready for review, just not ready to be merged yet until I completed the last few tasks.

Requesting for review

Reviews from anyone else would be welcome as well 😊

Have a good day!

@jensmeindertsma
Copy link
Copy Markdown
Author

As requested by @charlespierce it may be better to switch back to Docker-based builds as that way we can support older Linux distro's. We should base the Docker image off of CentOS 6, this will take some adjustment to the install-build-tools.sh script.

@jensmeindertsma
Copy link
Copy Markdown
Author

@charlespierce what do you think about the change in upload URL's? There's no longer a version of OpenSSL bundled, so we don't need that in the upload URL anymore. I've removed the OpenSSL version from the upload URL, but does this break installing older Volta versions?

Comment on lines +31 to +57
use_clang=1
install_packages \
qemu-user \
gcc-aarch64-linux-gnu \
libc6-dev-arm64-cross
;;
--target=aarch64-unknown-linux-musl)
use_clang=1
install_packages \
qemu-user
;;
--target=x86_64-unknown-linux-musl)
use_clang=1
;;
--target=*)
;;
esac

if [ -n "$use_clang" ]; then
# https://github.com/rustls/rustls/pull/1009 upgraded Rust's LLVM version to
# 14
llvm_version=14
sudo apt-key add ./ci/llvm-snapshot.gpg.key
sudo add-apt-repository "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-$llvm_version main"
sudo apt-get update
install_packages clang-$llvm_version llvm-$llvm_version
fi
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@charlespierce my biggest concern with building in a Docker image based on CentOS 6 is whether these lines here will still work. Surely I can update this to use yum, but a major issue is that LLVM-14 and Clang-14 may not be available for CentOS 6. I can try https://www.vultr.com/docs/how-to-install-llvm-and-clang-on-centos-6 but there may be problems with building from source on such an old Linux distro.

@jensmeindertsma
Copy link
Copy Markdown
Author

Reduced the scope of this PR to purely the use of the static TLS implementation, we can figure out ARM and MUSL later in a different PR.

@charlespierce
Copy link
Copy Markdown
Contributor

FYI @jensmeindertsma I just updated our existing Dockerfile to use a mirror of the CentOS vault provided by CERN, rather than the default vault.centos.org one, which appears to be more stable.

@jensmeindertsma
Copy link
Copy Markdown
Author

Thanks!

I've spend quite some hours now struggling with this. It looks like we need to compile GCC from source but after having tried that I've ran into problems. Not to mention, to support ARM and MUSL we'll need to build LLVM and CLang from source as well. I'm going to let this PR rest for a little bit, can you add a help-wanted label?

@charlespierce
Copy link
Copy Markdown
Contributor

I wonder if it's possible to use the Vault versions of the "Software Collections" repo to get a more up-to-date version of GCC to enable ring to complete? For reference, I found this issue: briansmith/ring#1417

And related, I found some descriptions of how to update GCC from a different repo:
https://stackoverflow.com/questions/55345373/how-to-install-gcc-g-8-on-centos
https://vault.centos.org/6.10/sclo/

That may enable us to avoid having to build from source (which should significantly speed things up). Also, for ARM / MUSL, since it's a new platform that we didn't support before, I think it's reasonable to base it on a more recent Linux distro version, to simplify that process. Then when we get to releasing Volta 2.0, we can include pulling the build toolchain up to parity with that version, to unify the build steps across the board.

@charlespierce
Copy link
Copy Markdown
Contributor

See also this comment for how to get ring to build on CentOS 6: briansmith/ring#673 (comment)

@jensmeindertsma
Copy link
Copy Markdown
Author

@charlespierce that's a start, but since Rust now requires clang14, and devtools7 doesn't include clang14, we still need to build that from source. Which is where I'm getting stuck.

@jensmeindertsma
Copy link
Copy Markdown
Author

If you'd like to learn more about what's required, check the setup and build files used in ring's CI

@jensmeindertsma
Copy link
Copy Markdown
Author

Which would be https://github.com/briansmith/ring/blob/main/mk/install-build-tools.sh and the cargo.sh file in the same directory

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants