Skip to content

Commit

Permalink
Merge branch 'gtrevi/update-release-process' of https://github.com/gt…
Browse files Browse the repository at this point in the history
…revi/ebpf-for-windows into gtrevi/update-release-process
  • Loading branch information
gtrevi committed Mar 21, 2024
2 parents b7fa9be + b3407e6 commit 5723728
Show file tree
Hide file tree
Showing 32 changed files with 356 additions and 1,296 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/cicd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ jobs:
uses: ./.github/workflows/reusable-build.yml
with:
ref: ${{ github.ref }}
repository: ${{ github.repository }}
build_artifact: Build-x64
generate_release_package: true
build_msi: true
Expand All @@ -63,6 +64,7 @@ jobs:
uses: ./.github/workflows/reusable-build.yml
with:
ref: ${{ github.ref }}
repository: ${{ github.repository }}
build_artifact: Build-x64-native-only
build_msi: true
build_nuget: true
Expand Down Expand Up @@ -266,6 +268,7 @@ jobs:
uses: ./.github/workflows/reusable-build.yml
with:
ref: ${{ github.ref }}
repository: ${{ github.repository }}
build_artifact: Build-x64-Analyze
# Analysis on external projects is conditional, as on small CI/CD VMs the compiler can run OOM
build_options: /p:Analysis='True' /p:AnalysisOnExternal='False'
Expand All @@ -277,6 +280,7 @@ jobs:
uses: ./.github/workflows/reusable-build.yml
with:
ref: ${{ github.ref }}
repository: ${{ github.repository }}
build_artifact: Build-x64-Sanitize
build_options: /p:AddressSanitizer='True'

Expand Down Expand Up @@ -476,6 +480,7 @@ jobs:
uses: ./.github/workflows/reusable-build.yml
with:
ref: ${{ github.ref }}
repository: ${{ github.repository }}
build_artifact: Build-x64-CodeQl
build_codeql: true

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/dependency-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ jobs:
egress-policy: audit

- name: 'Checkout Repository'
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2
- name: 'Dependency Review'
uses: actions/dependency-review-action@9129d7d40b8c12c1ed0f60400d00c92d437adcce # v4.1.3
4 changes: 2 additions & 2 deletions .github/workflows/nuget_update.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
with:
egress-policy: audit

- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
with:
# Only check out main repo, not submodules.
ref: ${{ github.event.workflow_run.head_branch }}
Expand All @@ -61,7 +61,7 @@ jobs:
nuget update -noninteractive -verbosity detailed ${{env.SOLUTION_FILE_PATH}}
- name: Create Pull Request
uses: peter-evans/create-pull-request@a4f52f8033a6168103c2538976c07b467e8163bc
uses: peter-evans/create-pull-request@70a41aba780001da0a30141984ae2a0c95d8704e
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: Update NuGet packages
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ossar-scan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
paths_ignore: '["**.md", "**/docs/**"]'

# Checking out the branch is needed to correctly log security alerts.
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
if: steps.skip_check.outputs.should_skip != 'true'
with:
# Only check out main repo, not submodules.
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/reusable-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ on:
ref:
required: true
type: string
# repository to be used (needed for self-hosted runner setups)
repository:
required: true
type: string
# Name associated with the output of this build.
build_artifact:
required: true
Expand Down Expand Up @@ -84,10 +88,10 @@ jobs:
powershell.exe "echo 'msvc_tools_version=%VCToolsVersion%' | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append"
powershell.exe "echo 'ASAN_WIN_CONTINUE_ON_INTERCEPTION_FAILURE=true' | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append"
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
if: steps.skip_check.outputs.should_skip != 'true'
with:
repository: microsoft/ebpf-for-windows
repository: ${{inputs.repository}}
submodules: 'recursive'
ref: ${{inputs.ref}}

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/reusable-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,15 +96,15 @@ jobs:
paths_ignore: '["**.md", "**/docs/**"]'

# Checking out the branch is needed to gather correct code coverage data.
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
# Only check out source code if code coverage is being gathered.
if: (inputs.code_coverage == true) && (steps.skip_check.outputs.should_skip != 'true')
with:
submodules: 'recursive'
ref: ${{ github.event.workflow_run.head_branch }}

# Perform shallow checkout for self-hosted runner.
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
if: (inputs.environment == 'ebpf_cicd_tests_ws2019' || inputs.environment == 'ebpf_cicd_tests_ws2022' || inputs.environment == 'ebpf_cicd_perf_ws2022') && (steps.skip_check.outputs.should_skip != 'true')
with:
ref: ${{ github.event.workflow_run.head_branch }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/scorecards-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
- name: "Checkout code"
if: github.ref_name == 'main'
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
with:
persist-credentials: false

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/update-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
run: |
sudo apt install doxygen
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633

- name: Clone docs
run: |
Expand Down
2 changes: 1 addition & 1 deletion docs/DevelopmentGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ If you see unexpected formatting changes in the code, verify that you are runnin

The following license header **must** be included at the top of every code file:

```
```c
// Copyright (c) Microsoft Corporation
// SPDX-License-Identifier: MIT
```
Expand Down
60 changes: 34 additions & 26 deletions docs/InstallEbpf.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,49 +16,40 @@ Do the following from within the VM:
* **Runtime Components** (mandatory): this feature adds the eBPF runtime and core components, which are also required by the other components. If you select only this
feature, only [native code generation](NativeCodeGeneration.md) is enabled.
* **JIT** (optional): this sub-feature adds support for JIT-compiled eBPF programs and (in a Debug build only) interpreted eBPF programs.
* **Development** (optional): this feature adds headers and libraries used for development. If you only want to use eBPF for development
rather than running programs, you can [use the NuGet package](GettingStarted.md#using-ebpf-in-development)
instead of the MSI.
* **Testing** (optional): this feature adds tests for the eBPF runtime for use by eBPF runtime developers.

An **command line install/uninstall** is also supported, through the direct use of `C:\Windows\system32\msiexec.exe` from an *administrative Command Prompt*:

- The installation folder can be customized by assigning the desired path to the `INSTALLFOLDER` parameter (path with spaces must be put between double quotes), i.e.:
* The installation folder can be customized by assigning the desired path to the `INSTALLFOLDER` parameter (path with spaces must be put between double quotes), i.e.:

```bash
INSTALLFOLDER="C:\Program Files\ebpf-for-windows"
```

- The following feature-components are available for customization, and must be assigned as comma-separated values to the `ADDLOCAL` parameter:
* The following feature-components are available for customization, and must be assigned as comma-separated values to the `ADDLOCAL` parameter:

- `eBPF_Runtime_Components` (**mandatory**): runtime components (installed in `[Installation folder]\*`, `[Installation folder]\drivers`).
- `eBPF_Runtime_Components_JIT` (optional): JIT compiler service (installed in `[Installation folder]\JIT`).
- `eBPF_Development` (optional): header files, scripts and libraries for development (installed in `[Installation folder]\include`,`[Installation folder]\lib`,`[Installation folder]\scripts`).
- `eBPF_Testing` (optional): header files for development (installed in `[Installation folder]\testing`).
* `eBPF_Runtime_Components` (**mandatory**): runtime components (installed in `[Installation folder]\*`, `[Installation folder]\drivers`).
* `eBPF_Runtime_Components_JIT` (optional): JIT compiler service (installed in `[Installation folder]\JIT`).

e.g., (full featured):

```bash
ADDLOCAL=eBPF_Runtime_Components,eBPF_Runtime_Components_JIT,eBPF_Development,eBPF_Testing
ADDLOCAL=eBPF_Runtime_Components,eBPF_Runtime_Components_JIT
```


Below are some examples of CLI installations/uninstallation, using "`C:\Program Files\ebpf-for-windows`" as the installation folder:

* Installation:
> **Note**: add the "`/qn`" switch for **unattended install**.

```bash
# Debug MSI - fully-featured installation
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components,eBPF_Runtime_Components_JIT,eBPF_Development,eBPF_Testing
# Debug MSI - fully-featured installation, including the JIT compiler (available on pre-release versions only)
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components,eBPF_Runtime_Components_JIT
# Debug MSI - fully-featured installation, no JIT compiler
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components,eBPF_Development,eBPF_Testing
# Debug MSI - minimal installation (only runtime components)
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components
# Release MSI - fully-featured installation, including the JIT compiler (available on pre-release versions only)
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components,eBPF_Runtime_Components_JIT,eBPF_Development,eBPF_Testing
# Release MSI - fully-featured installation (no JIT compiler on future post-release versions)
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components,eBPF_Development,eBPF_Testing
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components,eBPF_Runtime_Components_JIT
# Release MSI - minimal installation (only runtime components)
C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi INSTALLFOLDER="C:\Program Files\ebpf-for-windows" ADDLOCAL=eBPF_Runtime_Components
Expand All @@ -79,27 +70,33 @@ C:\Windows\system32\msiexec.exe /i eBPF-for-Windows.x.x.x.msi <other options> /l
```

### Method 2 (Install files you built yourself)

This method uses a machine that
has already built the binaries for `x64/Debug` or `x64/Release`.

1. Deploy the binaries to `C:\Temp` in your VM, as follows (from within a "*Developer PowerShell for VS 2022*"):

- If you **built the binaries from inside the VM**, then from your `ebpf-for-windows` directory in the VM, run:
* If you **built the binaries from inside the VM**, then from your `ebpf-for-windows` directory in the VM, run:

```ps
.\x64\debug\deploy-ebpf -l
```
- Otherwise, if you **built the binaries on the host machine**, then from your `ebpf-for-windows`

* Otherwise, if you **built the binaries on the host machine**, then from your `ebpf-for-windows`
directory on the host machine, start an admin Powershell on the host machine and run:

```ps
.\x64\debug\deploy-ebpf --vm="<test-vm-name>"
```

or, to also copy files needed to run various tests, run:

```ps
.\x64\debug\deploy-ebpf --vm="<test-vm-name>" -t
```

or, to copy files to a specific directory, including file shares, run:

```ps
.\x64\debug\deploy-ebpf -l="c:\some\path"
```
Expand All @@ -112,24 +109,31 @@ has already built the binaries for `x64/Debug` or `x64/Release`.
powershell -ExecutionPolicy Bypass .\scripts\setup-ebpf.ps1
```

### Method 3 (Install files you built yourself, with a VM checkpoint)

This method uses a machine that
has already built the binaries for `x64/Debug` or `x64/Release`.

Copy the build output in `\x64\[Debug|Release]` to the host of the test VM and run the following in a Powershell
command prompt:

1. Create a snapshot of the test VM named **baseline**, by running:

```ps
Checkpoint-VM -Name <test-vm-name> -CheckpointName baseline
```

1. Store the VM administrator credential, by running the following commands:

```ps
Install-Module CredentialManager -force
```

```ps
New-StoredCredential -Target TEST_VM -Username <VM Administrator> -Password <VM Administrator account password> -Persist LocalMachine
```

> Note that "`TEST_VM`" is literal and is used in step 5 below; it need not be the name of any actual test VM.
1. Enter the `\x64\[Debug|Release]` directory (`cd`) where the build artifacts are stored.
1. Modify `.\vm_list.json` to specify the name of the test VM under `VMList`, eg:
Expand All @@ -146,11 +150,14 @@ command prompt:
]
}
```

1. Run the following commands to setup to use the credentials saved with `TEST_VM` in step 2,
for logging into each of the VMs named in `vm_list.json`:

```ps
Set-ExecutionPolicy unrestricted -Force
```

```ps
.\setup_ebpf_cicd_tests.ps1
```
Expand All @@ -162,26 +169,27 @@ to install eBPF on all Windows nodes in a Kubernetes cluster.

1. Download the `.msi` file from the [latest release on GitHub](https://github.com/microsoft/ebpf-for-windows/releases) and copy it over to [images](../images) directory.


2. Build ebpf-for-windows image.

* To **build the image on the Windows Host**, make sure docker is installed. [Install docker on Windows Server](https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?tabs=Windows-Server/).
Start an admin Powershell on the Windows Host and run the following command and provide parameters for `repository`, `tag` and `OSVersion`:

```ps
.\images\build-images.ps1
````
```

* To **build the image on a Linux machine** (e.g. Ubuntu), make sure docker is installed (see [install docker on Ubuntu](https://docs.docker.com/engine/install/ubuntu/)), and do the following:

- Run the following command and provide parameters for `repository`, `tag` and `OSVersion`:
* Run the following command and provide parameters for `repository`, `tag` and `OSVersion`:

```bash
$HOME/ebpf-for-windows-image/build-images.sh
````
```

3. Push the `ebpf-for-windows` image to your repository.

4. Update `manifests/Kubernetes/ebpf-for-windows-daemonset.yaml` with the container image pointing to your image path. Run the following command:

```cmd
kubectl apply -f manifests/Kubernetes/ebpf-for-windows-daemonset.yaml
```
8 changes: 4 additions & 4 deletions docs/NativeCodeGeneration.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ required, and to import the helper functions.

Every generated C file contains a single global entry point of type metadata_table_t:

```
```c
typedef struct _metadata_table
{
void (*programs)(program_entry_t** programs, size_t* count);
Expand All @@ -111,7 +111,7 @@ is not valid in a C variable name. This variable is the only globally visible va
## Exported programs
Each program in the generated C file is exported via a program_entry_t:
```
```c
typedef struct _program_entry
{
uint64_t (*function)(void*);
Expand All @@ -138,7 +138,7 @@ The skeleton framework then uses NMR to publish this information to the eBPF exe
The generated C code exposes a table containing the address of each helper function, name, ID, and additional meta-data
of the helper function. The C code generator emits a table for the helper functions referenced by the program:

```
```c
typedef struct _helper_function_entry
{
uint64_t (*address)(uint64_t, uint64_t, uint64_t, uint64_t, uint64_t);
Expand All @@ -161,7 +161,7 @@ in the generated code are called indirectly via the address field.

Each map referenced by any of the eBPF programs is added as a map_entry_t:

```
```c
typedef struct _map_entry
{
void* address;
Expand Down
Loading

0 comments on commit 5723728

Please sign in to comment.