[ONNX] De-duplicate initializers (#68202) by BowenBao · Pull Request #69547 · pytorch/pytorch

BowenBao · 2021-12-07T18:25:31Z

Stack from ghstack:

ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same data_ptr, strides and sizes.

Co-authored-by: BowenBao bowbao@microsoft.com

Differential Revision: D32994271

ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Co-authored-by: BowenBao <bowbao@microsoft.com> [ghstack-poisoned]

facebook-github-bot · 2021-12-07T18:25:47Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/69547
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit f7ceb62 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

msaroufim · 2021-12-09T18:55:58Z

@msaroufim has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

malfet · 2021-12-10T00:00:18Z

torch/csrc/jit/passes/onnx/helper.h

 static const int OPSET_VERSION_11 = 11;
 static const int OPSET_VERSION_12 = 12;
 static const int OPSET_VERSION_13 = 13;
+static const int OPSET_VERSION_14 = 14;
+static const int OPSET_VERSION_15 = 15;


BE question: shouldn't it be enum?

Suggested change

static const int OPSET_VERSION_11 = 11;

static const int OPSET_VERSION_12 = 12;

static const int OPSET_VERSION_13 = 13;

static const int OPSET_VERSION_14 = 14;

static const int OPSET_VERSION_15 = 15;

enum {

OPSET_VERSION_11 = 11,

OPSET_VERSION_12,

OPSET_VERSION_13,

OPSET_VERSION_14,

OPSET_VERSION_15,

};

malfet · 2021-12-10T00:01:14Z

torch/onnx/__init__.py

 producer_name = "pytorch"
 producer_version = _C._onnx.PRODUCER_VERSION
-constant_folding_opset_versions = [9, 10, 11, 12, 13, 14]
+constant_folding_opset_versions = [9, 10, 11, 12, 13, 14, 15]


Could this list ever be contiguous? If not, lets ask python to generate it

Suggested change

constant_folding_opset_versions = [9, 10, 11, 12, 13, 14, 15]

constant_folding_opset_versions = list(range(9, 16))

ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Co-authored-by: BowenBao <bowbaomicrosoft.com> ghstack-source-id: d17dfa4 Pull Request resolved: pytorch#69547

ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Co-authored-by: BowenBao <bowbaomicrosoft.com> Differential Revision: [D32994271](https://our.internmc.facebook.com/intern/diff/D32994271) [ghstack-poisoned]

pytorch-probot · 2022-01-07T00:09:33Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/28c3526caf1bf1dcccfe5e8da4accd5cd0e6b5c2/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default

Workflows	Labels (bold enabled)	Status
Triggered Workflows
linux-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/trunk`	✅ triggered
linux-docs	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/docs`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-vulkan-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
docker-builds	`ciflow/all`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
ios-12-5-1-x86-64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
linux-binary-conda	`ciflow/binaries`, `ciflow/binaries/conda`	🚫 skipped
linux-binary-libtorch-cxx11-abi	`ciflow/binaries`, `ciflow/binaries/libtorch`	🚫 skipped
linux-binary-libtorch-pre-cxx11	`ciflow/binaries`, `ciflow/binaries/libtorch`	🚫 skipped
linux-binary-manywheel	`ciflow/binaries`, `ciflow/binaries/wheel`	🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`, `ciflow/trunk`	🚫 skipped
linux-docs-push	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	🚫 skipped
periodic-linux-xenial-cuda11.1-py3.7-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	🚫 skipped
periodic-win-vs2019-cuda11.1-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
periodic-win-vs2019-cuda11.5-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	🚫 skipped

You can add a comment to the PR and tag @pytorchbot with the following commands:

# ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun

# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slow

For more information, please take a look at the CI Flow Wiki.

msaroufim · 2022-01-07T21:10:42Z

@msaroufim has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Co-authored-by: BowenBao <bowbaomicrosoft.com> Differential Revision: [D32994271](https://our.internmc.facebook.com/intern/diff/D32994271) [ghstack-poisoned]

msaroufim · 2022-01-13T18:42:55Z

@msaroufim has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

msaroufim · 2022-01-13T18:48:47Z

@msaroufim has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Co-authored-by: BowenBao <bowbaomicrosoft.com> ghstack-source-id: d17dfa4 Pull Request resolved: pytorch#69547

ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Co-authored-by: BowenBao <bowbaomicrosoft.com> Differential Revision: [D32994271](https://our.internmc.facebook.com/intern/diff/D32994271) [ghstack-poisoned]

malfet · 2022-02-10T19:38:50Z

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Pull Request resolved: #69547 ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Test Plan: Imported from OSS Reviewed By: msaroufim Differential Revision: D32994271 Pulled By: malfet fbshipit-source-id: 10ac66638b6255890875272472aa9ed07a5b1d9a Co-authored-by: BowenBao <bowbao@microsoft.com>

Summary: Pull Request resolved: pytorch/pytorch#69547 ScriptModule export introduces duplicated ONNX initializers for shared weights, unnecessarily increases ONNX model size. This PR de-duplicates ONNX initializers for model exported in eval mode, by checking if the underlying tensors share the same `data_ptr`, `strides` and `sizes`. Test Plan: Imported from OSS Reviewed By: msaroufim Differential Revision: D32994271 Pulled By: malfet fbshipit-source-id: 10ac66638b6255890875272472aa9ed07a5b1d9a Co-authored-by: BowenBao <bowbao@microsoft.com> (cherry picked from commit d7cbde940c5c259a3feff5af870b01dd21fbf3e0)

BowenBao requested a review from shubhambhokare1 as a code owner December 7, 2021 18:25

BowenBao mentioned this pull request Dec 7, 2021

Add missing docstrings for ONNX converter API. Fixes #67393 (#67640) #68489

Closed

facebook-github-bot added the cla signed label Dec 7, 2021

BowenBao mentioned this pull request Dec 7, 2021

Allow caffe2-specific graph transformations for OperatorExportTypes.ONNX_ATEN_FALLBACK when BUILD_CAFFE2 is ON (#67460) #68490

Closed

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Dec 7, 2021

pytorchbot added the open source label Dec 7, 2021

malfet reviewed Dec 10, 2021

View reviewed changes

pytorch-probot bot added the ciflow/default label Jan 7, 2022

BowenBao mentioned this pull request Feb 10, 2022

[ONNX] Link to the wiki (#68505) #72663

Closed

pytorchmergebot closed this in cc79274 Feb 11, 2022

facebook-github-bot deleted the gh/BowenBao/173/head branch February 15, 2022 15:17

pommedeterresautee mentioned this pull request Jul 1, 2022

Pytorch 1.12 breaks GPT-2 (and probably T5) ELS-RD/transformer-deploy#107

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ONNX] De-duplicate initializers (#68202)#69547

[ONNX] De-duplicate initializers (#68202)#69547
BowenBao wants to merge 4 commits intogh/BowenBao/173/basefrom
gh/BowenBao/173/head

BowenBao commented Dec 7, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Dec 7, 2021 •

edited

Loading

Uh oh!

msaroufim commented Dec 9, 2021

Uh oh!

malfet Dec 10, 2021

Uh oh!

malfet Dec 10, 2021

Uh oh!

pytorch-probot bot commented Jan 7, 2022 •

edited

Loading

⚛️ CI Flow

Uh oh!

msaroufim commented Jan 7, 2022

Uh oh!

msaroufim commented Jan 13, 2022

Uh oh!

msaroufim commented Jan 13, 2022

Uh oh!

malfet commented Feb 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	constant_folding_opset_versions = [9, 10, 11, 12, 13, 14, 15]
	constant_folding_opset_versions = list(range(9, 16))

Conversation

BowenBao commented Dec 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Dec 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

msaroufim commented Dec 9, 2021

Uh oh!

malfet Dec 10, 2021

Choose a reason for hiding this comment

Uh oh!

malfet Dec 10, 2021

Choose a reason for hiding this comment

Uh oh!

pytorch-probot bot commented Jan 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚛️ CI Flow

Uh oh!

msaroufim commented Jan 7, 2022

Uh oh!

msaroufim commented Jan 13, 2022

Uh oh!

msaroufim commented Jan 13, 2022

Uh oh!

malfet commented Feb 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

BowenBao commented Dec 7, 2021 •

edited

Loading

facebook-github-bot commented Dec 7, 2021 •

edited

Loading

pytorch-probot bot commented Jan 7, 2022 •

edited

Loading