Skip to content

[unified build] Additional changes to make GPU builds work#10507

Closed
orionr wants to merge 3 commits intopytorch:masterfrom
orionr:dll
Closed

[unified build] Additional changes to make GPU builds work#10507
orionr wants to merge 3 commits intopytorch:masterfrom
orionr:dll

Conversation

@orionr
Copy link
Contributor

@orionr orionr commented Aug 14, 2018

A continuation of #10504 for GPU, torch, etc. builds.

I was testing with

# edit caffe2/CMakeLists.txt to have
# target_compile_options(caffe2 PRIVATE "-fvisibility=hidden")
# enabled
# and then
FULL_CAFFE2=1 python setup.py build_deps | tee ~/log.txt
cat ~/log.txt | egrep 'undefined refer' | sort | less

I'll rebase on master when @Yangqing's changes in 10504 land, but putting up for some testing.

cc @mingzhe09088 @anderspapitto @ezyang

@orionr orionr changed the title [unified build] Changes to make GPU, torch, etc. builds work [WIP][unified build] Changes to make GPU, torch, etc. builds work Aug 14, 2018
@orionr
Copy link
Contributor Author

orionr commented Aug 14, 2018

@pytorchbot retest this please

@Yangqing
Copy link
Contributor

Yangqing commented Aug 16, 2018

#10504 just landed :)

@orionr orionr changed the title [WIP][unified build] Changes to make GPU, torch, etc. builds work [unified build] Additional changes to make GPU builds work Aug 16, 2018
facebook-github-bot
facebook-github-bot reviewed Aug 16, 2018
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

orionr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@orionr
Copy link
Contributor Author

orionr commented Aug 16, 2018

I'm going to land these changes even though we still have the following to debug before turning on hidden visibility

build/lib/libcaffe2_gpu.so: undefined reference to `ncclAllGather'
build/lib/libcaffe2_gpu.so: undefined reference to `ncclAllReduce'
build/lib/libcaffe2_gpu.so: undefined reference to `ncclBcast'
build/lib/libcaffe2_gpu.so: undefined reference to `ncclCommInitAll'
build/lib/libcaffe2_gpu.so: undefined reference to `ncclGetErrorString'
build/lib/libcaffe2_gpu.so: undefined reference to `ncclReduce'
build/lib/libcaffe2_gpu.so: undefined reference to `ncclReduceScatter'

build/lib/libcaffe2.so: undefined reference to `google::FlagRegisterer::FlagRegisterer<std::string>(char const*, char const*, char const*, std::string*, std::string*)'
build/lib/libcaffe2.so: undefined reference to `google::SetUsageMessage(std::string const&)'

lib/libgflags.so.2.2.1: undefined reference to `__cxa_throw_bad_array_new_length@CXXABI_1.3.8'
lib/libgflags.so.2.2.1: undefined reference to `operator delete(void*, unsigned long)@CXXABI_1.3.9'
lib/libgflags.so.2.2.1: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'

zdevito pushed a commit to zdevito/ATen that referenced this pull request Aug 16, 2018
@orionr @facebook-github-bot
Additional changes to make GPU builds work (#10507)
691079c 
Summary:
A continuation of pytorch/pytorch#10504 for GPU, torch, etc. builds.

I was testing with

```
FULL_CAFFE2=1 python setup.py build_deps | tee ~/log.txt
cat ~/log.txt | egrep 'undefined refer' | sort | less
```

I'll rebase on master when Yangqing's changes in 10504 land, but putting up for some testing.

cc mingzhe09088 anderspapitto ezyang
Pull Request resolved: pytorch/pytorch#10507

Reviewed By: Yangqing

Differential Revision: D9359606

Pulled By: orionr

fbshipit-source-id: c2a3683b3ea5839689f5d2661da0bc9055a54cd2
@ezyang ezyang added the merged label Jun 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@apaszke apaszke Awaiting requested review from apaszke

@colesbury colesbury Awaiting requested review from colesbury

@ezyang ezyang Awaiting requested review from ezyang

@gchanan gchanan Awaiting requested review from gchanan

@soumith soumith Awaiting requested review from soumith

@zdevito zdevito Awaiting requested review from zdevito

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@orionr @Yangqing @facebook-github-bot @ezyang