This is a CNI (container network interface) plugin that passes one physical SR-IOV nic into the namespace of a pod that requests it.
We use this CNI plugin in our CI to test SR-IOV functionality in KubeVirt.
We use multus-cni to attach the VFs and
PFs to the pod as a secondary network. We then use a DaemonSet
to deploy the
plugin on all the relevant K8s nodes.
.
├── deploy # The manifests used to deploy the plugin and its pre-requisites.
│ └── multus-cni.yaml
├── Dockerfile # The Dockerfile that we use to pack the plugin in.
├── install-plugin # The script that actually installs the plugin.
├── plugin # The plugin itself.
│ └── sriov-passthrough-cni
└── README.md
In order to use the CNI and allocate one SR-IOV PF:
-
Add one of the following annotation to the prow job yaml (both are doing the same):
[1]
k8s.v1.cni.cncf.io/networks: multus-cni-ns/sriov-passthrough-cni
[2]
k8s.v1.cni.cncf.io/networks: '[{"interface":"net1","name":"sriov-passthrough-cni","namespace":"multus-cni-ns"}]'
-
Check if
pull-kubevirt-e2e-kind-1.17-sriov
job at [3] has requests/limits ofprow/sriov
, if it does, make sure it request only 1prow/sriov
. see [4] for example. This will let k8s know that only one PF is allocated (each PF is represented by oneprow/sriov
resource). K8s uses these resources in order to know how many jobs can run simultaneously. Each SR-IOV node of the CI cluster hasprow/sriov
capacity according to the amount of it's available physical PFs.
[4]
requests:
prow/sriov: "1"
limits:
prow/sriov: "1"
- In case 2 PFs are required, please do the following changes:
-
Use one of the following annotations instead those in the previous [1] section.
[1]
k8s.v1.cni.cncf.io/networks: 'multus-cni-ns/sriov-passthrough-cni,multus-cni-ns/sriov-passthrough-cni'
[2]
k8s.v1.cni.cncf.io/networks: '[{"interface":"net1","name":"sriov-passthrough-cni","namespace":"multus-cni-ns"}, {"interface":"net2","name":"sriov-passthrough-cni","namespace":"multus-cni-ns"}]'
-
Follow [4] of the previous section, but instead of 1
prow/sriov
, use 2, in order to let k8s know that 2 PFs are allocated per job, each PF is represented by aprow/sriov
resource.
requests:
prow/sriov: "2"
limits:
prow/sriov: "2"