simplify wheel CI scripts, other small packaging changes

[jameslamb]

Fixes #6023

Proposes some miscellaneous packaging cleanup:

• updates rapids-dependency-file-generator to its latest version (1.17.0) in pre-commit config
• removes unused PACKAGE_CUDA_SUFFIX in build_wheel.sh
• removes unnecessary pip install cmake in test_wheel.sh
  • there are aarch64 treelite wheels now: https://pypi.org/project/treelite/#files
• CMake option cleanup:
  • adds BUILD_PRIMS_BENCH OFF, removes BUILD_BENCH OFF in get_raft.cmake (matching changes to RAFT from 23.04: Add end-to-end CUDA ann-benchmarks to raft raft#1304)
  • removes unnecessary BUILD_CUML_PRIMS_BENCH (nothing uses this)
  • removes unnecessary CUML_USE_FAISS_STATIC (nothing uses this)
• dependencies.yaml changes:
  • breaks some dependencies out into depends_on_* groups to reduce duplication, and for consistency with other RAPIDS projects (docs explaining this)
  • alphabetizes dependency lists

[jameslamb]

/ok to test

[jameslamb]

/ok to test

[bdice]

We could move/remove libcumlprims and libraft-headers into their own depends_on_X lists. Or maybe we can remove libraft-headers here? We did that in cugraph, iirc. This is not a blocker, I defer to your judgment.

[jameslamb]

I don't think we want to remove libraft-headers here. That was fine in rapidsai/cugraph#4805 (comment) because there, it was coming through transitively via libraft.

cuml's environments and conda recipes don't depend on libraft, so I think it's still needed here.

I just pushed b9a0bb0 moving libcumlprims and libraft-headers into depends_on lists. They were only used once so it doesn't remove any duplication today, but it's helpful for applying this depends_on convention consistently across RAPIDS, and keeps the diff of a future change using those packages in more places smaller.

I'll merge this once CI passes.

[jameslamb]

CI here is failing with this dask-related issue:

E ImportError: cannot import name 'meta_nonempty' from 'dask.dataframe.core' (/opt/conda/envs/test/lib/python3.12/site-packages/dask/dataframe/core.py)

(build link)

That should have been fixed by rapidsai/rapids-dask-dependency#77.

But for some reason CI here is not picking up the newest rapids-dask-dependency ... it's getting 25.02.00a8 when the latest (the one with that new pinning) is 25.02.00a9. (ref: https://anaconda.org/rapidsai-nightly/rapids-dask-dependency/files).

Trying one more re-run of CI. If that fails, I'll investigate why conda isn't choosing that newer rapids-dask-dependency.

[bdice]

@jameslamb We have had to purge old alpha builds of rapids-dask-dependency several times in the past when the dask pinnings have changed. In those situations, the solver was preferring an older rapids-dask-dependency build with a different dask pinning for unclear reasons. We may need to do that again.

[jameslamb]

Ok yeah sounds good @bdice , I'll go ask for that.

By the way, I think CI here will now also be blocked until #6187 is merged. Seeing this in wheel builds:

/__w/cuml/cuml/python/cuml/build/cp310-cp310-linux_aarch64/_deps/cuvs-src/cpp/include/cuvs/cluster/kmeans.hpp(88): error: identifier "RAFT_LEVEL_INFO" is undefined

(build link)

Which I think is a result of rapidsai/raft#2530 being merged about an hour ago.

[vyasr]

Ah yes unfortunately that is correct. I figured today was a fairly low-impact day to start merging those since most people are off. I'm currently waiting for the latest raft builds to deploy, then I'll merge the corresponding cuvs PR, then the cuml one.

[jameslamb]

Yep all good! Thanks for keeping that rapids-logger stuff moving, I definitely think the bumps along the way will be worth it for the end state it gets us to.

[jameslamb]

/merge