[BUG] Cannot install transformers4rec through pip

[mvidela31]

Bug description

Hi everyone, I tried to install the transformers4rec library through pip but it fails when trying to install the rapids-dask-dependency requirement.

Steps/Code to reproduce bug

pip install transformers4rec[nvtabular]

Collecting rapids-dask-dependency==24.4.* (from dask-cuda>=22.12.0->merlin-core>=23.4.0->merlin-models>=23.4.0->transformers4rec==23.04->transformers4rec[nvtabular]==23.04)
  Using cached rapids_dask_dependency-24.4.0.tar.gz (2.0 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [79 lines of output]
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/wheel.py", line 226, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/wheel.py", line 160, in download_manual
          index_response = urlopen_with_retry(project_url)
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/wheel.py", line 83, in urlopen_with_retry
          return urlopen(url, **kwargs)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 216, in urlopen
          return opener.open(url, data, timeout)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 525, in open
          response = meth(req, response)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 634, in http_response
          response = self.parent.error(
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 563, in error
          return self._call_chain(*args)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 496, in _call_chain
          result = func(*args)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 643, in http_error_default
          raise HTTPError(req.full_url, code, msg, hdrs, fp)
      Traceback (most recent call last):
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/wheel.py", line 226, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/wheel.py", line 160, in download_manual
          index_response = urlopen_with_retry(project_url)
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/wheel.py", line 83, in urlopen_with_retry
          return urlopen(url, **kwargs)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 216, in urlopen
          return opener.open(url, data, timeout)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 525, in open
          response = meth(req, response)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 634, in http_response
          response = self.parent.error(
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 563, in error
          return self._call_chain(*args)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 496, in _call_chain
          result = func(*args)
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/urllib/request.py", line 643, in http_error_default
          raise HTTPError(req.full_url, code, msg, hdrs, fp)
      urllib.error.HTTPError: HTTP Error 404: Not Found
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/buildapi.py", line 29, in build_wheel
          return download_wheel(pathlib.Path(wheel_directory), config_settings)
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/wheel.py", line 228, in download_wheel
          report_install_failure(distribution, version, config, exception_context)
        File "/tmp/pip-build-env-pmmppziy/overlay/lib/python3.10/site-packages/wheel_stub/error.py", line 67, in report_install_failure
          raise InstallFailedError(
      wheel_stub.error.InstallFailedError:
      *******************************************************************************
      
      The installation of rapids-dask-dependency for version 24.4.0 failed.
      
      This is a special placeholder package which downloads a real wheel package
      from https://pypi.nvidia.com/. If https://pypi.nvidia.com/ is not reachable, we
      cannot download the real wheel file to install.
      
      You might try installing this package via
      ```
      $ pip install --extra-index-url https://pypi.nvidia.com/ rapids-dask-dependency
      ```
      
      Here is some debug information about your platform to include in any bug
      report:
      
      Python Version: CPython 3.10.14
      Operating System: Linux 5.10.228-219.884.amzn2.x86_64
      CPU Architecture: x86_64
      Driver Version: 550.127
      CUDA Version: 12.4
      
      *******************************************************************************
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Expected behavior

Successful installation of the transformers4rec library.

Environment details

• Transformers4Rec version:
• Platform: Linux 5.10.228-219.884.amzn2.x86_64
• Python version: Python 3.10.14
• Huggingface Transformers version:
• PyTorch version (GPU?): torch==2.2.2
• Tensorflow version (GPU?):

Additional context

I'm currently trying to use this library on a AWS SageMaker ml.g4dn.xlarge notebook instance.

Tasks

Beta Give feedback

No tasks being tracked yet.

[rnyak]

@mvidela31 hello. Merlin has Pytorch and Tensorflow docker images. Would that can be an option for you to use Merlin?

[mvidela31]

Hi @rnyak, I was able to use run the Merlin library using the provided docker images, thanks for your suggestion. Anyway, it would be better for my specific use case if you could fix the pip installation of the library. Best regards.