GPU accelerated GameCI docker containers

work in progess, these docs are only partially complete.

This repo provides 3 examples of custom Dockerfiles for use with GameCI which provide GPU-acceleration on NVIDIA GPU's via the nvidia-container-toolkit.

Image Types

1. Standard Images

   Standard images uses the entrypoint.sh script to query the nvidia-container-toolkit for the version of the current driver running on the host machine, then downloads and installs the user-space components into the conatianer. This provides compatibility across disparate hardware, but comes at the cost of a slower start-up time.

2. Baked Images

   Baked images have had a specific version of the Nvidia driver pre-downloaded and extracted during the build process. This means the runtime components of the driver are available to the container immediately upon boot. This results is a much faster startup at the cost of tying the image to a specific set of hardware and requiring the user to maintain the custom image.

3. Self-Authenticating Images These images contain a selenium-based automation script that will create a new license-file and activate it at startup. This is useful for using the editor images as a remote-desktop session.

Demo

Motivation

Addresses the following issues:

• game-ci/unity-test-runner#102
• game-ci/unity-test-runner#113
• game-ci/documentation#323

Enables the following features inside a CI/CD pipeline:

• Light Baking
• Screen Recording
• Performance Testing & Benchmarking
• Play Testing
• Basic Remote Development Environments

Changes

The new images are built by sourcing from starndard GameCI editor images and adding the following:

• User account w/ password for non-root operation
• X11-based virtual display
• Basic desktop environment (Xfce4 or KDE)
• Firefox-esr browser (non-snap)
• Remote access to the grphical display via VNC (port 5900) or browser (port 8080)
• Nvidia user-space driver components

Host Preparation

Tested on Ubuntu22.04 and Debain12 hosts.

Requires Nvidia drivers + Container Runtime, or GPU Operator to be installed on the host. See https://github.com/small-hack/smol-metal for instructions.

Auth notes

• Generate the .alf file

# Dont export usn or pw with quotes

unity-editor -quit \
    -batchmode \
    -nographics \
    -logFile /dev/stdout \
    -createManualActivationFile \
    -username "$USERNAME" \
    -password "$PASSWORD"

• Run selenium

cd unity-self-auth && \
./license.py ../Unity_v*.alf

1. Log into the docker-host

2. Generate a License

• from https://github.com/buildstar-online/unity-webgl-nginx

  # Create a temporary directoy to work in
  mkdir -p /tmp/scratch
  cd /tmp/scratch
  
  # Export important variables
  export EDITOR_VERSION="2022.1.23f1"
  export EDITOR_IMAGE="deserializeme/gcicudaeditor:latest"
  export SLENIUM_IMAGE="deserializeme/gcicudaselenium:latest"
  export USERNAME="YOUR_EMAIL_HERE"
  export PASSWORD="YOUR_PASSWORD_HERE"
  
  # Change ownership of local dir so the container user can save data to mounted volumes
  chown 1000:1000 .
  touch Unity_v${EDITOR_VERSION}.alf
  
  # Generate the ALF file using the Editor 
  docker run --rm -it -v $(pwd)/Unity_v${EDITOR_VERSION}.alf:/Unity_v${EDITOR_VERSION}.alf \
      --user root \
     $EDITOR_IMAGE \
     unity-editor -quit \
     -batchmode \
     -nographics \
     -logFile /dev/stdout \
     -createManualActivationFile \
     -username "$USERNAME" \
     -password "$PASSWORD"

3. Authorize the License or Answer 2FA prompt

• Next try to authorize the license. This may fail due to a 2Factor prompt from Unity. If that happens, you will need to run the [grafical login process].

  docker run --rm -it --user 1000:1000 \
      --mount type=bind,source="$(pwd)",target=/home/player1/unity-self-auth/Downloads \
      -e USERNAME="$USERNAME" \
      -e PASSWORD="$PASSWORD" \
      -e HEADLESS="True" \
      $SLENIUM_IMAGE \
      ./license.py "./Downloads/Unity_v${EDITOR_VERSION}.alf"

4. Run the Unity Editor container

• Mount your project's repo directory as a volume. I'm using https://github.com/buildstar-online/unity-webgl-nginx as an example below.

  docker run --gpus all --rm -it \
    -p 5900:5900 \
    -p 2222:22 \
    --mount type=bind,source=$(pwd),target=/home/player1/Downloads \
    --mount type=bind,source=$(pwd)/unity-webgl-nginx,target=/home/player1/unity-webgl-nginx \
    --user root \
    $EDITOR_IMAGE

5. Install GPU user-space driver, create xorg.conf, and start X sevices

wget https://raw.githubusercontent.com/buildstar-online/gameci-docker-extras/main/gpu-enable.sh
export DISPLAY=:0
bash gpu-enable.sh

6. Connect to container using VNC

7. Sign-in and start Unity

# Sign-in
sudo /opt/unity/Editor/Unity -quit \
    -batchmode \
    -logFile /dev/stdout \
    -manualLicenseFile /home/player1/Downloads/*.ulf

# Start Editor
sudo /opt/unity/Editor/Unity

8. Enjoy