Skip to content

Unable to build with Spack/from source, ffmpeg libraries not found #2591

Closed
#2612
Closed
Unable to build with Spack/from source, ffmpeg libraries not found#2591
#2612
Labels
needs discussiontopic: build
@adamjstewart

Description

@adamjstewart
adamjstewart
opened on Aug 16, 2020

🐛 Bug

I'm trying to install torchvision from source using the Spack package manager, but the build can't find my ffmpeg libraries, which are in a non-standard location. There are a couple of reasons for this:

  1. Mismatch of compilers. At some points of the build, it is using the compiler I defined in CXX, which is a compiler wrapper designed to automatically inject the appropriate linker args. At other points of the build, it is using the system compiler, possibly picked up from Python's distutils, which is not a compiler wrapper.
  2. TORCHVISION_LIBRARY seems to be ignored. The README says to set TORCHVISION_LIBRARY with the path to all dependency libraries, but this isn't working either.

To Reproduce

$ spack install -v py-torchvision
...
[15/15] /Users/Adam/spack/lib/spack/env/clang/clang++ -MMD -MF /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/video_reader/VideoReader.o.d -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder -I/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/video_reader -I/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/ffmpeg-4.2.2-hoilb6sa2j6ggatrb2xvmuazqrtgcoa7/include -I/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc -I/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/py-torch-1.6.0-7zhzk2mza5nl3hvalsoql43dq56p7vs6/lib/python3.7/site-packages/torch/include -I/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/py-torch-1.6.0-7zhzk2mza5nl3hvalsoql43dq56p7vs6/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/py-torch-1.6.0-7zhzk2mza5nl3hvalsoql43dq56p7vs6/lib/python3.7/site-packages/torch/include/TH -I/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/py-torch-1.6.0-7zhzk2mza5nl3hvalsoql43dq56p7vs6/lib/python3.7/site-packages/torch/include/THC -I/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/python-3.7.8-skt55xepunmc5rj2diqoljahpau3zwrm/include/python3.7m -c -c /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/video_reader/VideoReader.cpp -o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/video_reader/VideoReader.o -std=c++14 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=video_reader -D_GLIBCXX_USE_CXX11_ABI=0
...
/usr/bin/clang++ -bundle -undefined dynamic_lookup /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/video_reader/VideoReader.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/seekable_buffer.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/cc_stream.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/util.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/subtitle_sampler.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/audio_stream.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/subtitle_stream.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/audio_sampler.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/stream.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/time_keeper.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/memory_buffer.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/decoder.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/sync_decoder.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/video_stream.o /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/build/temp.macosx-10.15.6-x86_64-3.7/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-py-torchvision-0.7.0-732klihrqrcpdkrv4mispep23skf2cbr/spack-src/torchvision/csrc/cpu/decoder/video_sampler.o -L/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/py-torch-1.6.0-7zhzk2mza5nl3hvalsoql43dq56p7vs6/lib/python3.7/site-packages/torch/lib -L/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/python-3.7.8-skt55xepunmc5rj2diqoljahpau3zwrm/lib -lavcodec -lavformat -lavutil -lswresample -lswscale -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.macosx-10.15.6-x86_64-3.7/torchvision/video_reader.so -std=c++14
ld: library not found for -lavcodec
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/bin/clang++' failed with exit status 1

As you can see, at some points of the build, it is using the compiler wrapper /Users/Adam/spack/lib/spack/env/clang/clang++, while at other parts it's using the actual compiler /usr/bin/clang++. This is true whether or not I use the ninja backend for building.

Note that I have a few local patches to Spack's torchvision recipe to add TORCHVISION_LIBRARY and TORCHVISION_INCLUDE, see below.

Expected behavior

If CXX is set to a particular compiler, I would expect the build to use that compiler for all compilation. Secondly, if TORCHVISION_LIBRARY is mentioned in the README as a way to locate libraries, I would expect that to work.

Environment

  • PyTorch / torchvision Version: 1.6.0 / 0.7.0 (also reproduced with 1.5.1 / 0.6.1)
  • OS: macOS 10.15.6
  • Compiler: Apple Clang 11.0.3
  • How you installed PyTorch / torchvision: spack (source)
  • Build command you used (if compiling from source): python setup.py build && python setup.py install
  • Python version: 3.7.8
  • CUDA/cuDNN version: N/A
  • GPU models and configuration: N/A
  • Any other relevant information: build environment contains the following env vars:
TORCHVISION_INCLUDE=/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/python-3.7.8-skt55xepunmc5rj2diqoljahpau3zwrm/include/python3.7m:/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/py-torch-1.6.0-7zhzk2mza5nl3hvalsoql43dq56p7vs6/lib/python3.7/site-packages/torch/include:/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/ffmpeg-4.2.2-hoilb6sa2j6ggatrb2xvmuazqrtgcoa7/include
TORCHVISION_LIBRARY=/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/python-3.7.8-skt55xepunmc5rj2diqoljahpau3zwrm/lib:/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/py-torch-1.6.0-7zhzk2mza5nl3hvalsoql43dq56p7vs6/lib/python3.7/site-packages/torch/lib:/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-11.0.3/ffmpeg-4.2.2-hoilb6sa2j6ggatrb2xvmuazqrtgcoa7/lib

Additional context

I realize that you likely don't get a lot of bug reports from people installing torchvision with Spack. I'm a Spack developer, so I'm happy to handle all of the Spack-specific questions. Also, if any torch/torchvision developers would like to help me maintain our Spack recipes, let me know!

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs discussiontopic: build

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions