Skip to content

Commit

Permalink
Get multiport-eswitch mode from metalnet rundir
Browse files Browse the repository at this point in the history
  • Loading branch information
PlagueCZ committed Oct 16, 2024
1 parent 025d117 commit c5ebb2c
Show file tree
Hide file tree
Showing 3 changed files with 33 additions and 2 deletions.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Build the manager binary
FROM --platform=$BUILDPLATFORM golang:1.23 as builder
FROM --platform=$BUILDPLATFORM golang:1.23 AS builder

ARG GOARCH=''

Expand Down
22 changes: 22 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,25 @@

## CRD usage
* [Usage](./usage/crd_usage.md)

## Multiport-eswitch mode
When running dpservice with Mellanox in multiport-eswitch mode, it is important to tell metalnet about it:
```
metalnet --multiport-eswitch
```
or (overrides the above)
```
echo -n "eswitch" > /var/lib/metalnet/mode
```
This changes the way metalnet generates identifiers for virtual function representors that are sent over to dpservice.

If pf1-proxy is also in use, it is important to mark it as used in the metalnet VF database:
```
mkdir -p /var/lib/metalnet/netfns/claims
echo -n "$pf1_proxy_vf_pci" > /var/lib/metalnet/netfns/claims/00000001-0000-4000-0000-000000000000
```
Where the `$pf1_proxy_vf_pci` is the PCI address of the VF representor for pf1-proxy. This should be the only VF using `mlx5_core` driver instead of the `vfio-pci` driver. One of many ways to retrieve such address is as follows:
```
pf1_proxy_vf_name=$(/opt/local/bin/dpdk-devbind.py -s | grep "mlx5Gen Virtual Function" | grep "drv=mlx5_core" | awk -F'if=' '{print $2}' | awk '{print $1}')
pf1_proxy_vf_pci=$(/opt/local/bin/dpdk-devbind.py -s | grep "mlx5Gen Virtual Function" | grep $pf1_proxy_vf_name | awk '{print $1}')
```
11 changes: 10 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ func main() {
flag.BoolVar(&enableIPv6Support, "enable-ipv6", false, "Enable IPv6 support")
flag.IntVar(&publicVNI, "public-vni", 100, "Virtual network identifier used for public routing announcements.")
flag.IPVar(&routerAddress, "router-address", net.IP{}, "The address of the next router.")
flag.BoolVar(&multiportEswitchMode, "multiport-eswitch", false, "Enable multiport eswitch support")
flag.BoolVar(&multiportEswitchMode, "multiport-eswitch", false, "Enable multiport eswitch support (can be overridden in metalnet-dir)")
flag.BoolVar(&enableLeaderElection, "leader-elect", false,
"Enable leader election for controller manager. "+
"Enabling this will ensure there is only one active controller manager.")
Expand All @@ -125,6 +125,15 @@ func main() {
log.SetLevel(log.DebugLevel)
}

// detect multiport-eswitch mode automatically (overrides command-line)
// Usage example: 'echo "eswitch" > /var/lib/metalnet/mode'
modeFilePath := filepath.Join(metalnetDir, "mode")
content, err := os.ReadFile(modeFilePath)
if err == nil {
multiportEswitchMode = strings.TrimSpace(string(content)) == "eswitch"
}
setupLog.Info(fmt.Sprintf("Multiport Eswitch mode set to: %v", multiportEswitchMode))

defaultRouterAddr.PublicVNI = uint32(publicVNI)
defaultRouterAddr.SetBySubsciption = false

Expand Down

0 comments on commit c5ebb2c

Please sign in to comment.