Fix slave on reinstall python wheel#24727
Conversation
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
9e7cdb5 to
17692aa
Compare
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
This PR fixes an issue where Python wheels are not reinstalled during the build process if they're already installed in the slave container. The fix adds the --force-reinstall flag to the pip install command to ensure wheels are always reinstalled for the $(PYTHON_WHEELS_PATH)/%-install target.
Key changes:
- Added
--force-reinstallflag to pip install command in the standard build path
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
for sonic-py-common (not available in pypi)
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if mkdir $(PYTHON_WHEELS_PATH)/pip_lock &> /dev/null; then | ||
| ifneq ($(CROSS_BUILD_ENVIRON),y) | ||
| { sudo -E SKIP_BUILD_HOOK=Y pip$($*_PYTHON_VERSION) install $(PYTHON_WHEELS_PATH)/$* $(LOG) && rm -d $(PYTHON_WHEELS_PATH)/pip_lock && break; } || { rm -d $(PYTHON_WHEELS_PATH)/pip_lock && exit 1 ; } | ||
| { sudo -E SKIP_BUILD_HOOK=Y pip$($*_PYTHON_VERSION) install --force-reinstall --no-deps $(PYTHON_WHEELS_PATH)/$* $(LOG) && rm -d $(PYTHON_WHEELS_PATH)/pip_lock && break; } || { rm -d $(PYTHON_WHEELS_PATH)/pip_lock && exit 1 ; } |
There was a problem hiding this comment.
With --no-deps, if this package is being installed for the first time, but dependencies aren't installed, wouldn't this result in a broken package?
There was a problem hiding this comment.
I understand your concern, and just confirm it is true. Below is the test result. Actually this is a good thing if we
consider this is a build system. Recursively automatically install all the dependencies from external source (pypi in this case) is dangerous. We required all the pypi dependences explicitly installed during build slave image, if anything is missing, the new behavior will be a broken package in the dependency, and old behavior will just recursively automatically install it from pypi (even a local build one like sonic_config_engine, which lead to an exploit opportunity).
user@585593c72ce8:/sonic$ pip3 list -l | grep Jinja2
Jinja2 3.1.2
user@585593c72ce8:/sonic$ sudo pip3 uninstall Jinja2
Found existing installation: Jinja2 3.1.2
Uninstalling Jinja2-3.1.2:
Would remove:
/usr/lib/python3/dist-packages/Jinja2-3.1.2.egg-info
/usr/lib/python3/dist-packages/jinja2
Proceed (Y/n)? y
Successfully uninstalled Jinja2-3.1.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
user@585593c72ce8:/sonic$ sudo pip3 install --force-reinstall --no-deps target/python-wheels/bookworm/sonic_config_engine-1.0-py3-none-any.whl
sed: -e expression #1, char 11: extra characters after command
Processing ./target/python-wheels/bookworm/sonic_config_engine-1.0-py3-none-any.whl
Installing collected packages: sonic-config-engine
Attempting uninstall: sonic-config-engine
Found existing installation: sonic-config-engine 1.0
Uninstalling sonic-config-engine-1.0:
Successfully uninstalled sonic-config-engine-1.0
Successfully installed sonic-config-engine-1.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
[notice] A new release of pip is available: 24.2 -> 25.3
[notice] To update, run: python3 -m pip install --upgrade pip
user@585593c72ce8:/sonic$ pip3 list -l | grep Jinja2
user@585593c72ce8:/sonic
There was a problem hiding this comment.
Would be nice if pip were to consider this as a failure and immediately fail the build, but agreed, this is better.
There was a problem hiding this comment.
In above example, the build process is silent about a broken package is installed.
However, during build the debian host image or docker images, we are still using normal pip3 (Recursively automatically install all the dependencies from external source), so there will be no extra broken package introduced by this PR. so this PR does not harm prod features.
| # Link python script and data expected location to the cross python virtual env istallation locations | ||
| { PATH=$(VIRTENV_BIN_CROSS_PYTHON$($*_PYTHON_VERSION)):${PATH} sudo -E $(VIRTENV_BIN_CROSS_PYTHON$($*_PYTHON_VERSION))/pip$($*_PYTHON_VERSION) install $(PYTHON_WHEELS_PATH)/$* $(LOG) && $(if $(findstring $(SONIC_CONFIG_ENGINE_PY3),$*),(sudo ln -s $(VIRTENV_BIN_CROSS_PYTHON$($*_PYTHON_VERSION))/sonic-cfggen /usr/local/bin/sonic-cfggen 2>/dev/null || true), true ) && $(if $(findstring $(SONIC_YANG_MODELS_PY3),$*),(sudo ln -s $(VIRTENV_BASE_CROSS_PYTHON3)/yang-models /usr/local/yang-models 2>/dev/null || true), true ) && rm -d $(PYTHON_WHEELS_PATH)/pip_lock && break; } || { rm -d $(PYTHON_WHEELS_PATH)/pip_lock && exit 1 ; } | ||
| # Link python script and data expected location to the cross python virtual env installation locations | ||
| { PATH=$(VIRTENV_BIN_CROSS_PYTHON$($*_PYTHON_VERSION)):${PATH} sudo -E $(VIRTENV_BIN_CROSS_PYTHON$($*_PYTHON_VERSION))/pip$($*_PYTHON_VERSION) install --force-reinstall --no-deps $(PYTHON_WHEELS_PATH)/$* $(LOG) && $(if $(findstring $(SONIC_CONFIG_ENGINE_PY3),$*),(sudo ln -s $(VIRTENV_BIN_CROSS_PYTHON$($*_PYTHON_VERSION))/sonic-cfggen /usr/local/bin/sonic-cfggen 2>/dev/null || true), true ) && $(if $(findstring $(SONIC_YANG_MODELS_PY3),$*),(sudo ln -s $(VIRTENV_BASE_CROSS_PYTHON3)/yang-models /usr/local/yang-models 2>/dev/null || true), true ) && rm -d $(PYTHON_WHEELS_PATH)/pip_lock && break; } || { rm -d $(PYTHON_WHEELS_PATH)/pip_lock && exit 1 ; } |
There was a problem hiding this comment.
any failure log to check if install fails or succeeds?
There was a problem hiding this comment.
like sonic_config_engine-1.0-py3-none-any.whl-install.log ? yes, it is already generated during build process.
This reverts commit 4dd3b8a.
#### Why I did it If one python wheel is already installed inside slave container, it will not install again. Below is a sample log: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/user/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl sonic-yang-models is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 22:53:07 UTC 2025 Elapsed time: 0h 0m 1s ``` However, we expect to reinstall the python wheel for target `$(PYTHON_WHEELS_PATH)/%-install` ##### Work item tracking - Microsoft ADO **(number only)**: #### How I did it Update slave.mk to enasure force install the python wheel. #### How to verify it After this change, local build will successfully force install the python wheel. See new logs: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/qiluo/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl Installing collected packages: sonic-yang-models Attempting uninstall: sonic-yang-models Found existing installation: sonic-yang-models 1.0 Uninstalling sonic-yang-models-1.0: Successfully uninstalled sonic-yang-models-1.0 Successfully installed sonic-yang-models-1.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 23:59:31 UTC 2025 ```
…-net#24760) This reverts commit 4dd3b8a.
#### Why I did it If one python wheel is already installed inside slave container, it will not install again. Below is a sample log: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/user/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl sonic-yang-models is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 22:53:07 UTC 2025 Elapsed time: 0h 0m 1s ``` However, we expect to reinstall the python wheel for target `$(PYTHON_WHEELS_PATH)/%-install` ##### Work item tracking - Microsoft ADO **(number only)**: #### How I did it Update slave.mk to enasure force install the python wheel. #### How to verify it After this change, local build will successfully force install the python wheel. See new logs: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/qiluo/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl Installing collected packages: sonic-yang-models Attempting uninstall: sonic-yang-models Found existing installation: sonic-yang-models 1.0 Uninstalling sonic-yang-models-1.0: Successfully uninstalled sonic-yang-models-1.0 Successfully installed sonic-yang-models-1.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 23:59:31 UTC 2025 ``` Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
…-net#24760) This reverts commit 4dd3b8a. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
#### Why I did it If one python wheel is already installed inside slave container, it will not install again. Below is a sample log: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/user/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl sonic-yang-models is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 22:53:07 UTC 2025 Elapsed time: 0h 0m 1s ``` However, we expect to reinstall the python wheel for target `$(PYTHON_WHEELS_PATH)/%-install` ##### Work item tracking - Microsoft ADO **(number only)**: #### How I did it Update slave.mk to enasure force install the python wheel. #### How to verify it After this change, local build will successfully force install the python wheel. See new logs: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/qiluo/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl Installing collected packages: sonic-yang-models Attempting uninstall: sonic-yang-models Found existing installation: sonic-yang-models 1.0 Uninstalling sonic-yang-models-1.0: Successfully uninstalled sonic-yang-models-1.0 Successfully installed sonic-yang-models-1.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 23:59:31 UTC 2025 ``` Signed-off-by: xiaweijiang <xiaweijiang@microsoft.com>
…-net#24760) This reverts commit 4dd3b8a. Signed-off-by: xiaweijiang <xiaweijiang@microsoft.com>
#### Why I did it If one python wheel is already installed inside slave container, it will not install again. Below is a sample log: ``` sed: -e expression #1, char 11: extra characters after command WARNING: The directory '/var/user/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl sonic-yang-models is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 22:53:07 UTC 2025 Elapsed time: 0h 0m 1s ``` However, we expect to reinstall the python wheel for target `$(PYTHON_WHEELS_PATH)/%-install` ##### Work item tracking - Microsoft ADO **(number only)**: #### How I did it Update slave.mk to enasure force install the python wheel. #### How to verify it After this change, local build will successfully force install the python wheel. See new logs: ``` sed: -e expression #1, char 11: extra characters after command WARNING: The directory '/var/qiluo/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl Installing collected packages: sonic-yang-models Attempting uninstall: sonic-yang-models Found existing installation: sonic-yang-models 1.0 Uninstalling sonic-yang-models-1.0: Successfully uninstalled sonic-yang-models-1.0 Successfully installed sonic-yang-models-1.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 23:59:31 UTC 2025 ```
…-net#24760) This reverts commit 4dd3b8a.
#### Why I did it If one python wheel is already installed inside slave container, it will not install again. Below is a sample log: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/user/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl sonic-yang-models is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 22:53:07 UTC 2025 Elapsed time: 0h 0m 1s ``` However, we expect to reinstall the python wheel for target `$(PYTHON_WHEELS_PATH)/%-install` ##### Work item tracking - Microsoft ADO **(number only)**: #### How I did it Update slave.mk to enasure force install the python wheel. #### How to verify it After this change, local build will successfully force install the python wheel. See new logs: ``` sed: -e expression sonic-net#1, char 11: extra characters after command WARNING: The directory '/var/qiluo/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. Processing ./target/python-wheels/bookworm/sonic_yang_models-1.0-py3-none-any.whl Installing collected packages: sonic-yang-models Attempting uninstall: sonic-yang-models Found existing installation: sonic-yang-models 1.0 Uninstalling sonic-yang-models-1.0: Successfully uninstalled sonic-yang-models-1.0 Successfully installed sonic-yang-models-1.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 25.3 [notice] To update, run: python3 -m pip install --upgrade pip Build end time: Wed Dec 3 23:59:31 UTC 2025 ``` Signed-off-by: Feng Pan <fenpan@microsoft.com>
…-net#24760) This reverts commit 4dd3b8a. Signed-off-by: Feng Pan <fenpan@microsoft.com>
Why I did it
If one python wheel is already installed inside slave container, it will not install again. Below is a sample log:
However, we expect to reinstall the python wheel for target
$(PYTHON_WHEELS_PATH)/%-installWork item tracking
How I did it
Update slave.mk to enasure force install the python wheel.
How to verify it
After this change, local build will successfully force install the python wheel. See new logs:
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)