Skip to content

Reduce binary size for Linux and Mac package#3337

Merged
askhade merged 26 commits intoonnx:masterfrom
jcwchen:reduce-linux
Mar 25, 2021
Merged

Reduce binary size for Linux and Mac package#3337
askhade merged 26 commits intoonnx:masterfrom
jcwchen:reduce-linux

Conversation

@jcwchen
Copy link
Copy Markdown
Member

@jcwchen jcwchen commented Mar 16, 2021

Description

  • Instead of building C++ protobuf with default configuration, release pipelines should build it with release mode. It can remarkably reduce binary size for Linux package and slightly reduce for Mac package.

Linux package -- 18.9MB ->11.5MB
Mac package -- 12MB -> 11.3MB

Meanwhile; the total running time will be much faster as well: (Linux 59mins -> 8 mins; Mac 26mins -> 9mins)

  • Deprecate Python 3.5 to prevent failure caused by Python 3.5 incompatibility

Motivation
ONNX 1.8.1 PyPI package becomes much larger than 1.8.0, especially Linux package. The main reason is ONNX upgraded its protobuf-compiler version from 2.6 to 3.11. We need to figure out a reasonable way to shrink protobuf dependency even though it needs to be upgraded.

jcwchen added 2 commits March 16, 2021 16:04
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen jcwchen requested a review from a team as a code owner March 16, 2021 23:07
jcwchen added 12 commits March 16, 2021 18:14
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
pull_request:
branches: [rel-*]
branches: [rel-*, master] # TODO: remove it before merge

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mac CI fails because of this pypa/setuptools-scm#542. Will try to reduce binary size as well.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ONNX decided to deprecate python 3.5 in the following 1.9. Will remove it in this PR to prevent failure caused by Python 3.5 incompatibility

jcwchen added 3 commits March 18, 2021 07:24
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen jcwchen changed the title [WIP] Reduce binary size for Linux package Reduce binary size for Linux and Mac package Mar 18, 2021
jcwchen added 2 commits March 18, 2021 10:34
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@askhade askhade added this to the 1.9 milestone Mar 18, 2021
jcwchen added 3 commits March 18, 2021 13:32
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen jcwchen changed the title Reduce binary size for Linux and Mac package Reduce binary size for Linux and Mac package; Deprecate Python 3.5 Mar 19, 2021
@jcwchen jcwchen linked an issue Mar 22, 2021 that may be closed by this pull request
Comment thread .github/workflows/release_mac.yml Outdated
- name: Install Python dependencies
run: |
python -m pip install -q --upgrade pip
# Latest setuptools-scm-6.0.1 (dependency of setuptools) does not support Python 3.5 anymore
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still relevant if we want to remove support for Python 3.5?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Removed. Thanks

@askhade
Copy link
Copy Markdown
Contributor

askhade commented Mar 22, 2021

can we separate this in 2 PRs... to keep things clean.... Create a new 1 for python deprecation and keep this for binary size reduction. Thanks!

@jcwchen jcwchen changed the title Reduce binary size for Linux and Mac package; Deprecate Python 3.5 Reduce binary size for Linux and Mac package Mar 22, 2021
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen
Copy link
Copy Markdown
Member Author

jcwchen commented Mar 22, 2021

can we separate this in 2 PRs... to keep things clean.... Create a new 1 for python deprecation and keep this for binary size reduction. Thanks!

Sure thing. A new PR (#3353) decouple from this PR

@askhade askhade merged commit b51f033 into onnx:master Mar 25, 2021
@jcwchen jcwchen deleted the reduce-linux branch September 6, 2022 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants