-
Notifications
You must be signed in to change notification settings - Fork 33
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Restore readme after accidental deletion.
- Loading branch information
Showing
1 changed file
with
191 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,191 @@ | ||
# Linux kernel driver for Compute Engine Virtual Ethernet | ||
|
||
This repository contains the source for building an out-of-tree Linux kernel | ||
module for the Compute Engine Virtual Ethernet device. | ||
|
||
# Supported Hardware | ||
|
||
The driver here binds to a single PCI device id used by the virtual Ethernet | ||
device found in some Compute Engine VMs. | ||
|
||
Field | Value | Comments | ||
------------- | -------- | -------- | ||
Vendor ID | `0x1AE0` | Google | ||
Device ID | `0x0042` | | ||
Sub-vendor ID | `0x1AE0` | Google | ||
Sub-device ID | `0x0058` | | ||
Revision ID | `0x0` | | ||
Device Class | `0x200` | Ethernet | ||
|
||
# Supported Kernels | ||
|
||
This driver s supported on any of the [distros listed as supporting gVNIC](https://cloud.google.com/compute/docs/images/os-details#networking). | ||
Those distros have native drivers for gVNIC, but this driver can be used to | ||
replace the native driver to get the latest enhancements. Note that native | ||
drivers are likely to report version 1.0.0, this should be ignored. The | ||
upstream community has deprecated the use of driver versions it has not | ||
been updated since the initial upstream version. | ||
|
||
This driver is also supported on [clean Linux LTS kernels that are not EOL](https://www.kernel.org/category/releases.html). | ||
Linux kernels starting 5.4 have the driver built in, but this driver can be | ||
installed on any LTS kernel to get the latest enhancements. | ||
|
||
Debian 9 and 10 are supported since they use clean Linux LTS kernels | ||
(4.9 and 4.19 respectively). | ||
|
||
Versions that are not marked as a release candidate (rc) correspond to upstream | ||
versions of the driver. It is our intention that release candidates | ||
will be upstreamed in the near future, but when and in what form this happens | ||
depends on the Linux community and the upstream review process. We can't | ||
guarantee that a release candidate will land upstream as-is or if it | ||
will be accepted upstream at all. | ||
|
||
# Installation | ||
|
||
## GitHub | ||
|
||
If you downloaded a release tarball from GitHub: This source code should | ||
already be multi-kernel compatible. You can skip down to Compiling the Driver. | ||
|
||
### Building the multi-kernel compatible driver source | ||
|
||
If you downloaded the source from GitHub: To install this driver on anything | ||
other than the current upstream kernel, you will need to download Coccinelle, | ||
and untar it. It can be found here: https://coccinelle.gitlabpages.inria.fr/website/download.html | ||
|
||
NOTE: Most distros will include a version of Coccinelle in their package manager | ||
but we require version 1.0.6 or newer, available from the official website. | ||
|
||
To build the multi-kernel compatible driver source: | ||
|
||
```bash | ||
export SPATCH='/path/to/coccinelle/spatch' | ||
./build_src.sh --target=oot | ||
``` | ||
|
||
TIP: The spatch path may be omitted if it has been installed on the search path. | ||
|
||
### Compiling the driver | ||
|
||
Building and installing this driver requires that you have the headers installed | ||
for your current kernel version. On CentOS, you might have to additionally | ||
install the `kernel-devel` package. | ||
|
||
Ensure the headers are installed, then build, | ||
install, and load `gve.ko`: | ||
|
||
```bash | ||
make -C /lib/modules/`uname -r`/build M=$(pwd)/build modules modules_install | ||
depmod | ||
modprobe gve | ||
``` | ||
|
||
Check via `ethtool -i devname` that the new driver is installed. If not, you can | ||
install it manually. | ||
|
||
> [!WARNING] | ||
> Run this as a single line, as running `rmmod` alone will remove the existing | ||
> driver and disconnect you if connected over SSH. | ||
```bash | ||
sudo rmmod gve; sudo insmod ./build/gve.ko | ||
``` | ||
|
||
# Configuration | ||
|
||
## Ethtool | ||
|
||
gve supports common ethtool operations including controlling offloads and | ||
multi-queue operation. | ||
|
||
Viewing/Changing the number of queues per traffic class is done via ethtool. | ||
|
||
```bash | ||
ethtool --show-channels devname | ||
``` | ||
|
||
Returns the number of transmit and receive queues per traffic class, along with | ||
those maximums. | ||
|
||
```bash | ||
ethtool --set-channels devname [rx N] [tx N] [combined N] | ||
``` | ||
|
||
combined: attempts to set both rx and tx queues to N rx: attempts to set rx | ||
queues to N tx: attempts to set tx queues to N | ||
|
||
## XDP | ||
|
||
To attach an XDP program to the driver, the number of RX and TX queues must be | ||
no more than half their maximum values. The maximum values are based on the | ||
number of CPUs available. | ||
|
||
### Manual Configuration | ||
|
||
To manually configure gVNIC, you'll need to complete the following steps: | ||
|
||
* Install a driver capable of driving the new device installed in the guest OS. | ||
The Linux driver source is available on GitHub and is upstreamed into the Linux kernel. | ||
* Create a boot disk tagged with the 'GVNIC' guest OS feature. | ||
* Ensure that the nic-type is set to GVNIC for all interfaces intended to use gVNIC. To switch back to VirtioNet (the default option), using the same image, recreate the VM with change the nic-type set to VIRTIO_NET. | ||
|
||
|
||
The crucial step for both is ensuring you have a capable driver installed. An instance created with a gVNIC interface and no driver will have no internal or external network connectivity, including SSH and RDP. The interactive serial console may be useful for debugging if this situation arises. | ||
|
||
#### Instructions | ||
To build and install the kernel driver from source you must have a supported kernel with headers installed. Our driver supports building against mainline and major distribution kernels versions 3.10 and later. Follow your distro's instructions for how to do that. Source code and driver packages can be obtained from GitHub under [GoogleCloudPlatform/compute-virtual-ethernet-linux](https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux). | ||
|
||
1. Install or update kernel and headers to version 3.10 or later. To complete this step, review the documentation for your operating system. Source code and driver packages can be obtained from GitHub, see [GoogleCloudPlatform/compute-virtual-ethernet-linux](https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux). | ||
|
||
For example, if you are using Debian, you can install the kernel driver using the deb package either by running: | ||
```shell | ||
dpkg -i gve-dkms_1.1.0_all.deb | ||
``` | ||
|
||
or | ||
|
||
```shell | ||
apt update | ||
apt install ./gve-dkms_1.1.0_all.deb | ||
``` | ||
Where gve-dkms_1.1.0_all.deb is the driver deb package that you have been supplied with. This will install the driver into the correct place in your filesystem. | ||
|
||
2. To load the driver run: | ||
```shell | ||
modprobe gve | ||
``` | ||
This loads the driver until you remove it with rmmod or you reboot. | ||
|
||
3. To have the driver automatically load on boot if you are using systemd run: | ||
```shell | ||
echo gve > /etc/modules-load.d/gve.conf | ||
``` | ||
|
||
Optional: If you are using debian run the following instead of the above command: | ||
```shell | ||
echo gve >> /etc/modules | ||
``` | ||
|
||
4. Stop the VM. | ||
|
||
5. Create an image from the disk that is attached to the VM that you just stopped. | ||
```shell | ||
gcloud compute images create IMAGE_NAME\ | ||
--source-disk DISK_NAME \ | ||
--guest-os-features=GVNIC | ||
``` | ||
|
||
6. Replace the following: | ||
|
||
IMAGE_NAME: The name of the image that you want to create. This image has the Compute Engine virtual network driver installed. | ||
DISK_NAME: The name of the boot disk on the VM instance that you just stopped. | ||
|
||
7. Use the image, which now has the gVNIC driver installed, to create and start a VM with a gVNIC network interface. | ||
```shell | ||
gcloud compute instances create INSTANCE_NAME\ | ||
--source-image IMAGE_NAME \ | ||
--zone <preferred compute zone> \ | ||
--network-interface=nic-type=GVNIC | ||
``` | ||
|
||
Note: Before conducting a manual installation of the gVNIC driver, you will want to consult with your distro's out-of-tree (oot) support policy and [tainted](https://www.kernel.org/doc/html/latest/admin-guide/tainted-kernels.html) kernel support policy. |