Utilize GPU when training

[FutureGoose]

I have a lot of training data, so I'm trying to get BirdNET to utilize the computer's GPU. I have been unsuccessful so far and am pulling my hair out. Any help would be greatly appreciated. Has anyone been successful in running the model on a GPU? How did you make it work? Did you adapt BirdNET for TensorFlow version 2.10, or did you use WSL?

My understanding is that these are the options for TensorFlow to make it run on the GPU:

1. Python 3.9 with TensorFlow 2.10
2. Python 3.10 with the DirectML plugin (TensorFlow 2.10)
3. Linux (WSL in this case)

Since the current requirement for BirdNET is TensorFlow 2.15, we are left with only option 3.

I've been following the installation instructions here:

and here

I.e. Cuda 11.8 and cuDNN 8.6.

These are the errors I'm getting when checking if the GPU is available:

(venv) gustaf@DESKTOP-EMR42IA:~/code/biosonic$ python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

2024-05-30 21:40:47.425057: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

2024-05-30 21:40:48.004912: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

2024-05-30 21:40:48.594078: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:984] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.

2024-05-30 21:40:48.613551: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2251] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[]

OS platform and distribution: WSL Ubuntu 22.04.3 (Windows 10)

Python version: 3.10

CUDA/cuDNN version: 11.8/8.6

GPU model and memory: 4095MB NVIDIA GeForce RTX 2070 SUPER