Skip to content

Commit e19b2e0

Browse files
adferrandbmw
adferrand
and
bmw
committed
Migrate certbot-auto users on CentOS 6 to Python 3.6 (#7268)
Fixes #7007 Python 3.4 is [EOL](https://www.python.org/dev/peps/pep-0429/), and only Python 3.x version available for CentOS 6 through EPEL is this version, and so is used by `certbot-auto`, the only official way to install Certbot on this platform. This unpleasant situation becomes a little more uncomfortable, considering that the newest `pip` version (19.2) [just dropped Python 3.4 support](pypa/pip#6685) and will refuse to start on this Python version. We can expect a lot of dependencies to follow this path now. One direct result of this situation is that a fix to support correctly the ARM platforms requires to upgrade `pip` to 19.2 for `certbot-auto`. So this is not possible right now. Then, let's upgrade Certbot instances on CentOS 6 to a supported version of Python 3. This PR proposes a new bootstrap approach for CentOS 6 platform, `BootstrapRpmPython3Legacy`, that will install Python 3.6 from [SCL](https://www.softwarecollections.org) (the latest one available for now on CentOS 6). In term of Python 3 specific bootstrap methods, I take the occasion here to completely separate the bootstrap of CentOS 6 as a legacy system, from the RPM-based newest systems (like Fedora 29+) that are simply dropping support for Python 2.x. This is in prevision of future migration for all systems on Python 3.x, that is a different problematic than supporting old systems. * Add logic * Rebuilt letsencrypt-auto * Fix logic * Focus on specific packages * Maintain PATH for further invocations of letsencrypt-auto after bootstrap. * Various corrections * Fix farm test for RHEL6 * Working centos6 letsencrypt-auto self tests * Fix test_sdist for CentOS 6 * Corrections * Work in progress * Working configuration * Fix typo * Remove EPEL. Add a test. * Update letsencrypt-auto-source/letsencrypt-auto.template Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Improvements after review * Improvements * Add a comment * Add a test * Update a test * Corrections * Update function return * Work in progress * Correct behavior on oracle linux 6. * Corrections * Rebuild script * Add letsencrypt-auto tests for oraclelinux6 * Update tox.ini Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto.template Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto.template Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Remove specific code for scientific linux * Change some variables names * Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Various corrections * Fix tests * Add a comment * Update message * Fix test message * Update letsencrypt-auto-source/letsencrypt-auto.template Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update scripts * More focused assertion * Add back a test * Update script * Update letsencrypt-auto-source/letsencrypt-auto.template Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update letsencrypt-auto-source/letsencrypt-auto.template Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Check quiet mode * Add changelog * Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
1 parent 2dbe47f commit e19b2e0

File tree

13 files changed

+540
-168
lines changed

13 files changed

+540
-168
lines changed

‎.travis.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,9 @@ matrix:
234234
- sudo: required
235235
env: TOXENV=le_auto_centos6
236236
services: docker
237+
- sudo: required
238+
env: TOXENV=le_auto_oraclelinux6
239+
services: docker
237240
<<: *extended-test-suite
238241
- sudo: required
239242
env: TOXENV=docker_dev

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Certbot adheres to [Semantic Versioning](https://semver.org/).
1515
staging server instead of the live server when `--dry-run` is used.
1616
* Updated certbot-dns-google to depend on newer versions of
1717
google-api-python-client and oauth2client.
18+
* Migrated CentOS 6 certbot-auto users from Python 3.4 to Python 3.6.
1819

1920
### Fixed
2021

letsencrypt-auto-source/Dockerfile.centos6 renamed to letsencrypt-auto-source/Dockerfile.redhat6

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,13 @@
11
# For running tests, build a docker image with a passwordless sudo and a trust
22
# store we can manipulate.
33

4-
FROM centos:6
4+
ARG REDHAT_DIST_FLAVOR
5+
FROM ${REDHAT_DIST_FLAVOR}:6
56

6-
RUN yum install -y epel-release
7+
ARG REDHAT_DIST_FLAVOR
8+
9+
RUN curl -O https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm \
10+
&& rpm -ivh epel-release-latest-6.noarch.rpm
711

812
# Install pip and sudo:
913
RUN yum install -y python-pip sudo
@@ -27,11 +31,18 @@ RUN mkdir -p /home/lea/certbot
2731
COPY ./tests/certs/ca/my-root-ca.crt.pem /usr/local/share/ca-certificates/
2832
RUN update-ca-trust
2933

30-
# Copy code:
34+
# Copy current letsencrypt-auto:
3135
COPY . /home/lea/certbot/letsencrypt-auto-source
3236

37+
# Fetch previous letsencrypt-auto that was installing python 3.4
38+
RUN curl https://raw.githubusercontent.com/certbot/certbot/v0.38.0/letsencrypt-auto-source/letsencrypt-auto \
39+
-o /home/lea/certbot/letsencrypt-auto-source/letsencrypt-auto_py_34 \
40+
&& chmod +x /home/lea/certbot/letsencrypt-auto-source/letsencrypt-auto_py_34
41+
42+
RUN cp /home/lea/certbot/letsencrypt-auto-source/tests/${REDHAT_DIST_FLAVOR}6_tests.sh /home/lea/certbot/letsencrypt-auto-source/tests/redhat6_tests.sh \
43+
&& chmod +x /home/lea/certbot/letsencrypt-auto-source/tests/redhat6_tests.sh
44+
3345
USER lea
3446
WORKDIR /home/lea
3547

36-
RUN sudo chmod +x certbot/letsencrypt-auto-source/tests/centos6_tests.sh
37-
CMD sudo certbot/letsencrypt-auto-source/tests/centos6_tests.sh
48+
CMD ["sudo", "certbot/letsencrypt-auto-source/tests/redhat6_tests.sh"]

‎letsencrypt-auto-source/letsencrypt-auto‎

Lines changed: 153 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -256,20 +256,28 @@ DeprecationBootstrap() {
256256
fi
257257
}
258258

259-
MIN_PYTHON_VERSION="2.7"
260-
MIN_PYVER=$(echo "$MIN_PYTHON_VERSION" | sed 's/\.//')
259+
MIN_PYTHON_2_VERSION="2.7"
260+
MIN_PYVER2=$(echo "$MIN_PYTHON_2_VERSION" | sed 's/\.//')
261+
MIN_PYTHON_3_VERSION="3.5"
262+
MIN_PYVER3=$(echo "$MIN_PYTHON_3_VERSION" | sed 's/\.//')
261263
# Sets LE_PYTHON to Python version string and PYVER to the first two
262-
# digits of the python version
264+
# digits of the python version.
265+
# MIN_PYVER and MIN_PYTHON_VERSION are also set by this function, and their
266+
# values depend on if we try to use Python 3 or Python 2.
263267
DeterminePythonVersion() {
264268
# Arguments: "NOCRASH" if we shouldn't crash if we don't find a good python
265269
#
266270
# If no Python is found, PYVER is set to 0.
267271
if [ "$USE_PYTHON_3" = 1 ]; then
272+
MIN_PYVER=$MIN_PYVER3
273+
MIN_PYTHON_VERSION=$MIN_PYTHON_3_VERSION
268274
for LE_PYTHON in "$LE_PYTHON" python3; do
269275
# Break (while keeping the LE_PYTHON value) if found.
270276
$EXISTS "$LE_PYTHON" > /dev/null && break
271277
done
272278
else
279+
MIN_PYVER=$MIN_PYVER2
280+
MIN_PYTHON_VERSION=$MIN_PYTHON_2_VERSION
273281
for LE_PYTHON in "$LE_PYTHON" python2.7 python27 python2 python; do
274282
# Break (while keeping the LE_PYTHON value) if found.
275283
$EXISTS "$LE_PYTHON" > /dev/null && break
@@ -285,7 +293,7 @@ DeterminePythonVersion() {
285293
fi
286294
fi
287295

288-
PYVER=`"$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//'`
296+
PYVER=$("$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//')
289297
if [ "$PYVER" -lt "$MIN_PYVER" ]; then
290298
if [ "$1" != "NOCRASH" ]; then
291299
error "You have an ancient version of Python entombed in your operating system..."
@@ -368,7 +376,9 @@ BootstrapDebCommon() {
368376

369377
# Sets TOOL to the name of the package manager
370378
# Sets appropriate values for YES_FLAG and QUIET_FLAG based on $ASSUME_YES and $QUIET_FLAG.
371-
# Enables EPEL if applicable and possible.
379+
# Note: this function is called both while selecting the bootstrap scripts and
380+
# during the actual bootstrap. Some things like prompting to user can be done in the latter
381+
# case, but not in the former one.
372382
InitializeRPMCommonBase() {
373383
if type dnf 2>/dev/null
374384
then
@@ -388,26 +398,6 @@ InitializeRPMCommonBase() {
388398
if [ "$QUIET" = 1 ]; then
389399
QUIET_FLAG='--quiet'
390400
fi
391-
392-
if ! $TOOL list *virtualenv >/dev/null 2>&1; then
393-
echo "To use Certbot, packages from the EPEL repository need to be installed."
394-
if ! $TOOL list epel-release >/dev/null 2>&1; then
395-
error "Enable the EPEL repository and try running Certbot again."
396-
exit 1
397-
fi
398-
if [ "$ASSUME_YES" = 1 ]; then
399-
/bin/echo -n "Enabling the EPEL repository in 3 seconds..."
400-
sleep 1s
401-
/bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..."
402-
sleep 1s
403-
/bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 second..."
404-
sleep 1s
405-
fi
406-
if ! $TOOL install $YES_FLAG $QUIET_FLAG epel-release; then
407-
error "Could not enable EPEL. Aborting bootstrap!"
408-
exit 1
409-
fi
410-
fi
411401
}
412402

413403
BootstrapRpmCommonBase() {
@@ -488,13 +478,88 @@ BootstrapRpmCommon() {
488478
BootstrapRpmCommonBase "$python_pkgs"
489479
}
490480

481+
# If new packages are installed by BootstrapRpmPython3 below, this version
482+
# number must be increased.
483+
BOOTSTRAP_RPM_PYTHON3_LEGACY_VERSION=1
484+
485+
# Checks if rh-python36 can be installed.
486+
Python36SclIsAvailable() {
487+
InitializeRPMCommonBase >/dev/null 2>&1;
488+
489+
if "${TOOL}" list rh-python36 >/dev/null 2>&1; then
490+
return 0
491+
fi
492+
if "${TOOL}" list centos-release-scl >/dev/null 2>&1; then
493+
return 0
494+
fi
495+
return 1
496+
}
497+
498+
# Try to enable rh-python36 from SCL if it is necessary and possible.
499+
EnablePython36SCL() {
500+
if "$EXISTS" python3.6 > /dev/null 2> /dev/null; then
501+
return 0
502+
fi
503+
if ! scl --list 2>/dev/null | grep -q rh-python36; then
504+
return 0
505+
fi
506+
set +e
507+
. scl_source enable rh-python36
508+
set -e
509+
}
510+
511+
# This bootstrap concerns old RedHat-based distributions that do not ship by default
512+
# with Python 2.7, but only Python 2.6. We bootstrap them by enabling SCL and installing
513+
# Python 3.6. Some of these distributions are: CentOS/RHEL/OL/SL 6.
514+
BootstrapRpmPython3Legacy() {
515+
# Tested with:
516+
# - CentOS 6
517+
518+
InitializeRPMCommonBase
519+
520+
if ! "${TOOL}" list rh-python36 >/dev/null 2>&1; then
521+
echo "To use Certbot on this operating system, packages from the SCL repository need to be installed."
522+
if ! "${TOOL}" list centos-release-scl >/dev/null 2>&1; then
523+
error "Enable the SCL repository and try running Certbot again."
524+
exit 1
525+
fi
526+
if [ "${ASSUME_YES}" = 1 ]; then
527+
/bin/echo -n "Enabling the SCL repository in 3 seconds... (Press Ctrl-C to cancel)"
528+
sleep 1s
529+
/bin/echo -ne "\e[0K\rEnabling the SCL repository in 2 seconds... (Press Ctrl-C to cancel)"
530+
sleep 1s
531+
/bin/echo -e "\e[0K\rEnabling the SCL repository in 1 second... (Press Ctrl-C to cancel)"
532+
sleep 1s
533+
fi
534+
if ! "${TOOL}" install "${YES_FLAG}" "${QUIET_FLAG}" centos-release-scl; then
535+
error "Could not enable SCL. Aborting bootstrap!"
536+
exit 1
537+
fi
538+
fi
539+
540+
# CentOS 6 must use rh-python36 from SCL
541+
if "${TOOL}" list rh-python36 >/dev/null 2>&1; then
542+
python_pkgs="rh-python36-python
543+
rh-python36-python-virtualenv
544+
rh-python36-python-devel
545+
"
546+
else
547+
error "No supported Python package available to install. Aborting bootstrap!"
548+
exit 1
549+
fi
550+
551+
BootstrapRpmCommonBase "${python_pkgs}"
552+
553+
# Enable SCL rh-python36 after bootstrapping.
554+
EnablePython36SCL
555+
}
556+
491557
# If new packages are installed by BootstrapRpmPython3 below, this version
492558
# number must be increased.
493559
BOOTSTRAP_RPM_PYTHON3_VERSION=1
494560

495561
BootstrapRpmPython3() {
496562
# Tested with:
497-
# - CentOS 6
498563
# - Fedora 29
499564

500565
InitializeRPMCommonBase
@@ -505,12 +570,6 @@ BootstrapRpmPython3() {
505570
python3-virtualenv
506571
python3-devel
507572
"
508-
# EPEL uses python34
509-
elif $TOOL list python34 >/dev/null 2>&1; then
510-
python_pkgs="python34
511-
python34-devel
512-
python34-tools
513-
"
514573
else
515574
error "No supported Python package available to install. Aborting bootstrap!"
516575
exit 1
@@ -769,31 +828,50 @@ elif [ -f /etc/redhat-release ]; then
769828
RPM_DIST_VERSION=0
770829
fi
771830

772-
# Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
773-
# RHEL 8 also uses python3 by default.
774-
if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 -o "$PYVER" -eq 26 ]; then
775-
RPM_USE_PYTHON_3=1
776-
elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
777-
RPM_USE_PYTHON_3=1
778-
elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
779-
RPM_USE_PYTHON_3=1
780-
else
781-
RPM_USE_PYTHON_3=0
782-
fi
831+
# Handle legacy RPM distributions
832+
if [ "$PYVER" -eq 26 ]; then
833+
# Check if an automated bootstrap can be achieved on this system.
834+
if ! Python36SclIsAvailable; then
835+
INTERACTIVE_BOOTSTRAP=1
836+
fi
783837

784-
if [ "$RPM_USE_PYTHON_3" = 1 ]; then
785838
Bootstrap() {
786-
BootstrapMessage "RedHat-based OSes that will use Python3"
787-
BootstrapRpmPython3
839+
BootstrapMessage "Legacy RedHat-based OSes that will use Python3"
840+
BootstrapRpmPython3Legacy
788841
}
789842
USE_PYTHON_3=1
790-
BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
843+
BOOTSTRAP_VERSION="BootstrapRpmPython3Legacy $BOOTSTRAP_RPM_PYTHON3_LEGACY_VERSION"
844+
845+
# Try now to enable SCL rh-python36 for systems already bootstrapped
846+
# NB: EnablePython36SCL has been defined along with BootstrapRpmPython3Legacy in certbot-auto
847+
EnablePython36SCL
791848
else
792-
Bootstrap() {
793-
BootstrapMessage "RedHat-based OSes"
794-
BootstrapRpmCommon
795-
}
796-
BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
849+
# Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
850+
# RHEL 8 also uses python3 by default.
851+
if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 ]; then
852+
RPM_USE_PYTHON_3=1
853+
elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
854+
RPM_USE_PYTHON_3=1
855+
elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
856+
RPM_USE_PYTHON_3=1
857+
else
858+
RPM_USE_PYTHON_3=0
859+
fi
860+
861+
if [ "$RPM_USE_PYTHON_3" = 1 ]; then
862+
Bootstrap() {
863+
BootstrapMessage "RedHat-based OSes that will use Python3"
864+
BootstrapRpmPython3
865+
}
866+
USE_PYTHON_3=1
867+
BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
868+
else
869+
Bootstrap() {
870+
BootstrapMessage "RedHat-based OSes"
871+
BootstrapRpmCommon
872+
}
873+
BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
874+
fi
797875
fi
798876

799877
LE_PYTHON="$prev_le_python"
@@ -1078,8 +1156,15 @@ if [ "$1" = "--le-auto-phase2" ]; then
10781156
# If the selected Bootstrap function isn't a noop and it differs from the
10791157
# previously used version
10801158
if [ -n "$BOOTSTRAP_VERSION" -a "$BOOTSTRAP_VERSION" != "$PREV_BOOTSTRAP_VERSION" ]; then
1081-
# if non-interactive mode or stdin and stdout are connected to a terminal
1082-
if [ \( "$NONINTERACTIVE" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
1159+
# Check if we can rebootstrap without manual user intervention: this requires that
1160+
# certbot-auto is in non-interactive mode AND selected bootstrap does not claim to
1161+
# require a manual user intervention.
1162+
if [ "$NONINTERACTIVE" = 1 -a "$INTERACTIVE_BOOTSTRAP" != 1 ]; then
1163+
CAN_REBOOTSTRAP=1
1164+
fi
1165+
# Check if rebootstrap can be done non-interactively and current shell is non-interactive
1166+
# (true if stdin and stdout are not attached to a terminal).
1167+
if [ \( "$CAN_REBOOTSTRAP" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
10831168
if [ -d "$VENV_PATH" ]; then
10841169
rm -rf "$VENV_PATH"
10851170
fi
@@ -1090,12 +1175,21 @@ if [ "$1" = "--le-auto-phase2" ]; then
10901175
ln -s "$VENV_PATH" "$OLD_VENV_PATH"
10911176
fi
10921177
RerunWithArgs "$@"
1178+
# Otherwise bootstrap needs to be done manually by the user.
10931179
else
1094-
error "Skipping upgrade because new OS dependencies may need to be installed."
1095-
error
1096-
error "To upgrade to a newer version, please run this script again manually so you can"
1097-
error "approve changes or with --non-interactive on the command line to automatically"
1098-
error "install any required packages."
1180+
# If it is because bootstrapping is interactive, --non-interactive will be of no use.
1181+
if [ "$INTERACTIVE_BOOTSTRAP" = 1 ]; then
1182+
error "Skipping upgrade because new OS dependencies may need to be installed."
1183+
error "This requires manual user intervention: please run this script again manually."
1184+
# If this is because of the environment (eg. non interactive shell without
1185+
# --non-interactive flag set), help the user in that direction.
1186+
else
1187+
error "Skipping upgrade because new OS dependencies may need to be installed."
1188+
error
1189+
error "To upgrade to a newer version, please run this script again manually so you can"
1190+
error "approve changes or with --non-interactive on the command line to automatically"
1191+
error "install any required packages."
1192+
fi
10991193
# Set INSTALLED_VERSION to be the same so we don't update the venv
11001194
INSTALLED_VERSION="$LE_AUTO_VERSION"
11011195
# Continue to use OLD_VENV_PATH if the new venv doesn't exist

0 commit comments

Comments
 (0)