THC.h : no such file or directory

[Gouvernathor]

Traceback (most recent call last):
  File "/home/ppavia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1740, in _run_ninja_build
    subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "train.py", line 9, in <module>
    from model import SSNModel
  File "/data3/ppavia/ssn-pytorch/model.py", line 4, in <module>
    from lib.ssn.ssn import ssn_iter, sparse_ssn_iter
  File "/data3/ppavia/ssn-pytorch/lib/ssn/ssn.py", line 4, in <module>
    from .pair_wise_distance import PairwiseDistFunction
  File "/data3/ppavia/ssn-pytorch/lib/ssn/pair_wise_distance.py", line 8, in <module>
    pair_wise_distance_cuda = load_inline(
  File "/home/ppavia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1293, in load_inline
    return _jit_compile(
  File "/home/ppavia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1357, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/home/ppavia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1469, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/home/ppavia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1756, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'pair_wise_distance': [1/3] /usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=pair_wise_distance -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/TH -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /usr/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 --compiler-options '-fPIC' -std=c++14 -c /home/ppavia/.cache/torch_extensions/py38_cu102/pair_wise_distance/cuda.cu -o cuda.cuda.o 
FAILED: cuda.cuda.o 
/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=pair_wise_distance -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/TH -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /usr/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 --compiler-options '-fPIC' -std=c++14 -c /home/ppavia/.cache/torch_extensions/py38_cu102/pair_wise_distance/cuda.cu -o cuda.cuda.o 
/home/ppavia/.cache/torch_extensions/py38_cu102/pair_wise_distance/cuda.cu:17:10: fatal error: THC/THC.h: Aucun fichier ou dossier de ce type
   17 | #include <THC/THC.h>
      |          ^~~~~~~~~~~
compilation terminated.
[2/3] c++ -MMD -MF main.o.d -DTORCH_EXTENSION_NAME=pair_wise_distance -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/TH -isystem /home/ppavia/.local/lib/python3.8/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /usr/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /home/ppavia/.cache/torch_extensions/py38_cu102/pair_wise_distance/main.cpp -o main.o 
ninja: build stopped: subcommand failed.

[654493176]

This might help: pytorch/pytorch#73520

[Gouvernathor]

But I don't have "horovod"

[654493176]

但我没有“horovod”

open-mmlab/mmdetection3d#1332 (comment)
horovod/horovod@7afa032
So you need the old version of pytorch or to remove TH/THC include.

[Gouvernathor]

Thanks, it worked.