Fuse module enhancements#26457
Closed
raghuramank100 wants to merge 5 commits intogh/raghuramank100/25/basefrom
Closed
Fuse module enhancements#26457raghuramank100 wants to merge 5 commits intogh/raghuramank100/25/basefrom
raghuramank100 wants to merge 5 commits intogh/raghuramank100/25/basefrom
Conversation
Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/) [ghstack-poisoned]
raghuramank100
pushed a commit
that referenced
this pull request
Sep 19, 2019
Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/) ghstack-source-id: 90402269 Pull Request resolved: #26457
dzhulgakov
reviewed
Sep 19, 2019
Collaborator
dzhulgakov
left a comment
There was a problem hiding this comment.
Looking good, some more suggestions
| """ | ||
| return torch.nn._intrinsic.LinearReLU(linear, relu) | ||
|
|
||
| def fuse_conv_bn(conv, bn): |
Collaborator
There was a problem hiding this comment.
this function (and probably fuse_conv_bn_eval) misses setting the training mode. Something like result.train(original.training)
Contributor
Author
There was a problem hiding this comment.
There is no need to do that here, we just create a sequential here with the original modules, so all attributes are preserved
torch/quantization/fuse_modules.py
Outdated
| _set_module(model, modules_to_fuse[0], new_mod) | ||
|
|
||
| for item in modules_to_fuse[1:]: | ||
| _set_module(model, item, torch.nn.Identity()) |
Collaborator
There was a problem hiding this comment.
for cleanness - set the training mode here too
torch/quantization/fuse_modules.py
Outdated
| for item in modules_to_fuse[1:]: | ||
| _set_module(model, item, torch.nn.Identity()) | ||
|
|
||
| def fuse_modules(model, modules_to_fuse): |
Collaborator
There was a problem hiding this comment.
I wonder whether we should do special handling for Sequential. The good thing about sequantials is that we know what forward looks like. So we could just iterate through them and try to find a match.
I'm also fine doing it later in a separate PR as it doesn't appear to be a critical feature.
Contributor
Author
There was a problem hiding this comment.
This is useful only when we have a sequential module at the leaf of the module tree having fusable modules, so lets do this in a later PR
jerryzh168
reviewed
Sep 19, 2019
Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/) [ghstack-poisoned]
raghuramank100
pushed a commit
that referenced
this pull request
Sep 30, 2019
Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion ghstack-source-id: 91046110 Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/)
Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/) [ghstack-poisoned]
dzhulgakov
approved these changes
Sep 30, 2019
Collaborator
dzhulgakov
left a comment
There was a problem hiding this comment.
Minor comments - otherwise good to go
Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/) [ghstack-poisoned]
Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/) [ghstack-poisoned]
raghuramank100
pushed a commit
that referenced
this pull request
Oct 1, 2019
Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Differential Revision: [D17466382](https://our.internmc.facebook.com/intern/diff/D17466382/)
Contributor
|
This pull request has been merged in dddae3f. |
jamesr66a
pushed a commit
that referenced
this pull request
Oct 3, 2019
Summary: Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Test Plan: buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details Differential Revision: D17466382 fbshipit-source-id: 0a548f8f4c366f3ecc59db693bac725ccd62328e
jamesr66a
pushed a commit
that referenced
this pull request
Oct 3, 2019
Summary: Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Test Plan: buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details Differential Revision: D17466382 fbshipit-source-id: 0a548f8f4c366f3ecc59db693bac725ccd62328e
jamesr66a
pushed a commit
that referenced
this pull request
Oct 3, 2019
Summary: Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Test Plan: buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details Differential Revision: D17466382 fbshipit-source-id: 0a548f8f4c366f3ecc59db693bac725ccd62328e
jamesr66a
pushed a commit
that referenced
this pull request
Oct 4, 2019
Summary: Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Test Plan: buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details Differential Revision: D17466382 fbshipit-source-id: 0a548f8f4c366f3ecc59db693bac725ccd62328e
jamesr66a
pushed a commit
that referenced
this pull request
Oct 4, 2019
Summary: Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Test Plan: buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details Differential Revision: D17466382 fbshipit-source-id: 0a548f8f4c366f3ecc59db693bac725ccd62328e
soumith
pushed a commit
that referenced
this pull request
Oct 7, 2019
Summary: Pull Request resolved: #26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Test Plan: buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details Differential Revision: D17466382 fbshipit-source-id: 0a548f8f4c366f3ecc59db693bac725ccd62328e
pdlive215
pushed a commit
to pdlive215/pytorch
that referenced
this pull request
Nov 27, 2019
Summary: Pull Request resolved: pytorch#26457 Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict. Also add support for Conv-Relu and linear-relu fusion Also support inplace and out of place fusion of models. ghstack-source-id: 91076386 Test Plan: buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details Differential Revision: D17466382 fbshipit-source-id: 0a548f8f4c366f3ecc59db693bac725ccd62328e
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Enhancement to fuse module to support sequentials, fuse list can now be just like the state dict.
Also add support for Conv-Relu and linear-relu fusion
Differential Revision: D17466382