[pytorch][mobile] remove backward functions from jit-op-registry for mobile build by ljk53 · Pull Request #26657 · pytorch/pytorch

ljk53 · 2019-09-23T20:01:21Z

Stack from ghstack:

[pytorch][perf] add mobile friendly at:parallel_for backend #26702 [pytorch][perf] add mobile friendly at:parallel_for backend
[pytorch] fix ParallelNative.h/cpp build #26746 [pytorch] fix ParallelNative.h/cpp build
[pytorch][mobile] remove backward functions from jit-op-registry for mobile build #26657 [pytorch][mobile] remove backward functions from jit-op-registry for mobile build
[pytorch] rename caffe2::mobile_threadpool to caffe2::mobile_pthreadpool #26700 [pytorch] rename caffe2::mobile_threadpool to caffe2::mobile_pthreadpool

Summary:
Add codegen option to remove backward ops from jit-op-registry as they are not
likely to be used for inference only mobile build.

Measured ARM-v7 AAR build size change: 5,804,182 -> 5,331,219.

Test Plan:

build and integrate with demo app;

Differential Revision: D17533887

…mobile build Summary: Add codegen option to remove backward ops from jit-op-registry as they are not likely to be used for inference only mobile build. Measured ARM-v7 AAR build size change: 5,804,182 -> 5,331,219. Test Plan: - build and integrate with demo app; [ghstack-poisoned]

…mobile build Summary: Add codegen option to remove backward ops from jit-op-registry as they are not likely to be used for inference only mobile build. Measured ARM-v7 AAR build size change: 5,804,182 -> 5,331,219. Test Plan: - build and integrate with demo app; ghstack-source-id: e0530c5 Pull Request resolved: #26657

ljk53 · 2019-09-23T20:03:03Z

tools/jit/gen_jit_dispatch.py



+def is_backward_op(decl):
+    return decl['name'].endswith('_backward') or decl['name'].endswith('_backward_out')


This looks a bit hacky but there is an example above searching for "_out" and another example searching for "backward": https://github.com/pytorch/pytorch/blob/master/tools/autograd/gen_variable_type.py#L574

can we use the same set of filters as in python bindings? (and add a comment about it)

can we use the same set of filters as in python bindings? (and add a comment about it)

Do you think I should use the entire SKIP_PYTHON_BINDINGS / even directly references it?

It has a lot other stuff like '.*_forward', 'index', .. - I haven't verified but do you think these can all be safely deleted from JIT binding?

ljk53 · 2019-09-23T20:05:04Z

tools/setup_helpers/generate_code.py

+            declarations_path or DECLARATIONS_PATH,
+            jit_gen_dir,
+            'tools/jit/templates',
+            disable_autograd=disable_autograd)


I think technically backward ops are not coupled with autograd? Maybe I should rename all "disable_autograd" to "inference_only"? Just don't want to create too many different concepts while the main purpose is just to shrink prebuilt mobile lib size...

I think the same setting makes sense - and the backward ops ARE coupled with autograd. They are not exposed to python, so the only way to use them is through autograd

ljk53 · 2019-09-23T20:05:44Z

tools/jit/gen_jit_dispatch.py

                                             lvalues=lvalues)
        return constructor

+    def filter_decls(jit_decls, disable_autograd):


I'm going to add another option to take a while list of ops to create custom mobile build so making this a separate function.

…gistry for mobile build" Summary: Add codegen option to remove backward ops from jit-op-registry as they are not likely to be used for inference only mobile build. Measured ARM-v7 AAR build size change: 5,804,182 -> 5,331,219. Test Plan: - build and integrate with demo app; Differential Revision: [D17533887](https://our.internmc.facebook.com/intern/diff/D17533887) [ghstack-poisoned]

ljk53 · 2019-09-24T16:48:49Z

Add @zdevito and @suo to help review.
@gchanan made a good point during offline discussion: we don't bind these backward functions in python, probably should simply skip all of them by default?

pytorch/tools/autograd/gen_python_functions.py

Line 21 in 128a65e

'.*_backward', '.*_backward_(out|input|weight|bias)', '.*_forward',

…gistry for mobile build" Summary: Add codegen option to remove backward ops from jit-op-registry as they are not likely to be used for inference only mobile build. Measured ARM-v7 AAR build size change: 5,804,182 -> 5,331,219. Test Plan: - build and integrate with demo app; Differential Revision: [D17533887](https://our.internmc.facebook.com/intern/diff/D17533887) [ghstack-poisoned]

dzhulgakov

@gchanan made a good point during offline discussion: we don't bind these backward functions in python, probably should simply skip all of them by default?

I think they should be still bound to JIT because of symbolic gradients

dzhulgakov · 2019-09-25T04:07:38Z

tools/jit/gen_jit_dispatch.py



+def is_backward_op(decl):
+    return decl['name'].endswith('_backward') or decl['name'].endswith('_backward_out')


can we use the same set of filters as in python bindings? (and add a comment about it)

dzhulgakov · 2019-09-25T04:09:02Z

tools/setup_helpers/generate_code.py

+            declarations_path or DECLARATIONS_PATH,
+            jit_gen_dir,
+            'tools/jit/templates',
+            disable_autograd=disable_autograd)


I think the same setting makes sense - and the backward ops ARE coupled with autograd. They are not exposed to python, so the only way to use them is through autograd

ljk53 · 2019-09-25T05:58:53Z

Seems I hit some weird bug with ghstack here - #26700 used to be after this PR, I landed it first as it doesn't really depend on this PR and swapped their order. It somehow marked this PR as merged.
I can recreate this PR but hope it doesn't cause any other problem...

cc: @ezyang

ezyang · 2019-09-25T14:15:36Z

Yeah just make a new PR. Sorry about the trouble. If you can send me some rages I can look into it.

…mobile build Summary: Add codegen option to remove backward ops from jit-op-registry as they are not likely to be used for inference only mobile build. Measured ARM-v7 AAR build size change: 5,804,182 -> 5,331,219. Test Plan: - build and integrate with demo app; ghstack-source-id: 03a3854 Pull Request resolved: pytorch/pytorch#26657

pytorchbot added oncall: jit Add this issue/PR to JIT oncall triage queue module: build Build system issues labels Sep 23, 2019

ljk53 commented Sep 23, 2019

View reviewed changes

ljk53 requested review from dzhulgakov, ezyang and gchanan September 23, 2019 20:05

This was referenced Sep 24, 2019

[pytorch] rename caffe2::mobile_threadpool to caffe2::mobile_pthreadpool #26700

Closed

[pytorch][perf] add mobile friendly at:parallel_for backend #26702

Closed

ljk53 added this to the 1.3 milestone Sep 24, 2019

ljk53 requested review from suo and zdevito September 24, 2019 16:46

ljk53 mentioned this pull request Sep 24, 2019

[pytorch] fix ParallelNative.h/cpp build #26746

Closed

dzhulgakov reviewed Sep 25, 2019

View reviewed changes

ljk53 merged commit 3dc5a3d into gh/ljk53/54/base Sep 25, 2019

ljk53 removed this from the 1.3 milestone Sep 25, 2019

ljk53 mentioned this pull request Sep 25, 2019

[pytorch][mobile] remove backward functions from jit-op-registry for mobile build #26783

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[pytorch][mobile] remove backward functions from jit-op-registry for mobile build#26657

[pytorch][mobile] remove backward functions from jit-op-registry for mobile build#26657
ljk53 merged 3 commits intogh/ljk53/54/basefrom
gh/ljk53/54/head

ljk53 commented Sep 23, 2019 •

edited

Loading

Uh oh!

ljk53 Sep 23, 2019

Uh oh!

dzhulgakov Sep 25, 2019

Uh oh!

ljk53 Sep 25, 2019

Uh oh!

ljk53 Sep 23, 2019

Uh oh!

dzhulgakov Sep 25, 2019

Uh oh!

ljk53 Sep 23, 2019

Uh oh!

ljk53 commented Sep 24, 2019

Uh oh!

dzhulgakov left a comment

Uh oh!

dzhulgakov Sep 25, 2019

Uh oh!

dzhulgakov Sep 25, 2019

Uh oh!

ljk53 commented Sep 25, 2019

Uh oh!

ezyang commented Sep 25, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants



		def is_backward_op(decl):
		return decl['name'].endswith('_backward') or decl['name'].endswith('_backward_out')

Conversation

ljk53 commented Sep 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ljk53 commented Sep 24, 2019

Uh oh!

dzhulgakov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ljk53 commented Sep 25, 2019

Uh oh!

ezyang commented Sep 25, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ljk53 commented Sep 23, 2019 •

edited

Loading