Iommu update for Rocky Linux 9 #21
Conversation
roles/iommu/tasks/main.yml
Outdated
| @@ -3,11 +3,11 @@ | |||
| ansible.builtin.include_role: | |||
| name: stackhpc.linux.grubcmdline | |||
| vars: | |||
| kernel_cmdline: # noqa: var-naming[no-role-prefix] | |||
| - intel_iommu=on | |||
| kernel_cmdline: "{{ ['intel_iommu=on'] + (['vfio-pci.ids=' + vfio_pci_ids] if vfio_pci_ids is defined else []) }}" # noqa: var-naming[no-role-prefix] | |||
There was a problem hiding this comment.
Is the vfio-pci.ids parameter actually related to the IOMMU? You can use the grubcmdline role directly to add kernel args.
There was a problem hiding this comment.
Also, vfio_pci_ids is not included in the README or defaults, and does not have the role name as a prefix.
There was a problem hiding this comment.
The VFIO driver is an IOMMU/device agnostic framework for exposing direct device access to userspace, in a secure, IOMMU protected environment. In other words, this allows safe 2, non-privileged, userspace drivers.
https://docs.kernel.org/driver-api/vfio.html
I put it in here based on this. Do you think I shouldn't include it?
I've also mentioned it in the docs now
There was a problem hiding this comment.
For GPU passthrough there are a few other changes we can make, such as blacklisting the nouveau driver. Should we create a gpu-passthrough role to add this configuration?
roles/iommu/README.md
Outdated
| @@ -4,6 +4,10 @@ | |||
|
|
|||
| - `iommu_pt`: Default true, Configure passthrough mode, which doesn't require DMA translation. | |||
|
|
|||
| ## Host Vars | |||
|
|
|||
| - `vfio_pci_ids`: Can optionally be set with the pci id of the device to pass-through. | |||
There was a problem hiding this comment.
Would it be worth using the iommu prefix to stop variable collisions? Also I guess it can go in the "Variable" section of the README with the other variable.
| kernel_cmdline_remove: # noqa: var-naming[no-role-prefix] | ||
| - ^intel_iommu= | ||
| when: "'Intel' in ansible_facts.processor.0" | ||
| - ^vfio-pci\.ids= | ||
| when: ansible_facts.processor | select('search', 'Intel') | list | length > 0 |
There was a problem hiding this comment.
I believe vfio-pci is relevant for non-Intel processors too.
There was a problem hiding this comment.
Also I am not sure how the previous when clause even worked. Even on CentOS Stream 8, I see ansible_facts.processor looks like this:
ansible_facts.processor:
- '0'
- GenuineIntel
- Intel(R) Xeon(R) Gold 6146 CPU @ 3.20GHz
- '1'
- GenuineIntel
- Intel(R) Xeon(R) Gold 6146 CPU @ 3.20GHz
So ansible_facts.processor.0 is always a string set to 0.
There was a problem hiding this comment.
I tested with different Ansible versions for Yoga and for 2023.1.
There was a problem hiding this comment.
cpu_facts come from /proc/cpuinfo - I've seen it returning 0 in the first line and GenuineIntel in the first line, maybe there was some bug in the past that caused lack of '0' in the first line
technowhizz
force-pushed
the
iommu-rl9
branch
5 times, most recently
from
5d6e3d9 to
92edcff
Compare
technowhizz
force-pushed
the
iommu-rl9
branch
2 times, most recently
from
874531e to
38ad1c1
Compare
| ansible.builtin.shell: |- | ||
| #!/bin/bash | ||
| set -eux | ||
| dracut -v -f /boot/initramfs-$(uname -r).img $(uname -r) |
There was a problem hiding this comment.
Need to be careful not to break ubuntu
There was a problem hiding this comment.
Changes the conditional to search for 'Intel' in the ansible_facts.processor variable as the first item in the list is not always consistent.
When enabling iommu on RL9 hypervisors the check
when: "'Intel' in ansible_facts.processor.0"kept failing as it returned 0. Not sure how the check was working before but nowansible_facts.processor.1contains the line that hasIntelin it as ansible combines the lists into one list. We now use a seach for Intel in the list to pass the check:when: ansible_facts.processor | select('search', 'Intel') | list | length > 0Additionally, added support for specifying vfio-pci Id's in the cmdline.