WireGuard interface support

[b-m-f]

Draft 1

WireGuard interfaces in Podman

Have a dedicated WireGuard interface for a container.

Prerequisites

• WireGuard configuration file
  • the path to the config should be passed through from Podman
    • Error check on file permissions
• Kernel > 5.6 or correct kernel modules for WireGuard installed
  • Error check availability of the module

How?

• Interfaces will be created on the host
  • Naming: wg-$HASH
  • $HASH should be 12 chars long
• After creation the interface will be moved into the containers namespace
  • more info at https://www.wireguard.com/netns/
• Once inside the namespace the routing can be set up
  • to stay compatible with existing networks no default route will be set
  • AllowedIPs from the configs [Peers] array are aggregated and saved in a Set.
  • A route is set for each CIDR of the Set

Tool choices

• Use available libraries for routing setup

ipnet
netlink-packet-route
netlink-packet-core
netlink-sys

RFC

• Should we simply ignore PostUp and PostDown
  • If we choose not to then they should be run inside the containers network namespace using the permissions that podman has available.
• In the first iteration it might make sense to ignore DNS settings of the [Interface] section as well

History

# Initial Idea
This is an idea for integrating WireGuard interfaces into containers. 
The use case I have at the moment is App monitoring and hosting multiple services with different intranet hostnames on the same VM/Bare Metal machine without a reverse proxy. But I am sure that this could be a feature many other people can appreciate for private network setups.

# Process

The process would probably look like follows:

- Setup bridge network if desired
- Create WireGuard interface inside the container namspace
- Set up routing
  - if no bridge -> all traffic goes over WireGuard via default route
  - if bridge -> only specified  Ranges go over WireGuard interface. Default route still over bridge network
- set up port-rebinding via ip/nftables if desired

# Things to consider

- WireGuard endpoints with a hostname need the Bridge network alive if DNS should be resolved (unless the interface is created on the host and then moved inside the container namespace)

# Implementation

Have a `.yaml` file that takes the common WireGuard configuration parameters:

\```yaml
# ? -> optional
peers: 
  - name:
    endpoint:
    public_key:
    persistent_keep_alive?:
    allowed_ips:
interface:
  private_key:
  MTU?:
  port:
  force_default_route?:
\```

I can implement this feature if you (the maintainers) think this is desirable functionality.
At the moment I achieve this functionality with https://github.com/b-m-f/wg-pod, but having it directly integrated in Podman would surely be the most elegant way to integrate containers into WireGuard networks.

The biggest problem is probably the decision on whether or not the WireGuard interface should be the default route. 
This could be done via a configuration param (simple) or by checking whether a default route to a bridge network exists(smart). 

[Luap99]

I have thought about this recently. I personally would like to add wireguard support in the not so far future.

My thoughts:

• Create the wireguard interface and the host and them move it into the namespace, take a look at https://www.wireguard.com/netns/ If I understand it correctly with that we can only use the wiregurad interface in the container and do not have to worry about other interfaces in the netns to enable connectivity for wireguard.
  This will make setup extremely simple.

• I don't like to introduce a new yaml config format? Wireguard already has a specific config file format which is used by wg-quick so we should use the same.

• Also I would like to not mess with the firewall here to enable port forwarding. Since you should be connected via VPN you should just directly connect to that continuer ip.

• As far as default route is considered every network gets a default route at the moment, I think that should be the same for wireguard. There is an internal option which is used to disable the default route.

• It should work the same regardless of other drivers used, the wg driver itself should not check anything about other used networks, so --network <bridge> ---network <wg> and only --network <wg> should result in the same task for the wireguard implementation.

• Lastly, should we implement this directly via netlink API or shell out to the wg tools? I am not sure I would prefer to use it

In any case we should not do anything directly related to cryptography, i.e. key generation.

[b-m-f]

Create the wireguard interface and the host and them move it into the namespace, take a look at https://www.wireguard.com/netns/ If I understand it correctly with that we can only use the wiregurad interface in the container and do not have to worry about other interfaces in the netns to enable connectivity for wireguard.
This will make setup extremely simple

Yup, this also helps when a hostname is used for the Endpoint as the initial request to initiate the connection resolves it to an IP via the hosts DNS resolver settings.

I don't like to introduce a new yaml config format? Wireguard already has a specific config file format which is used by wg-quick so we should use the same.

Where do you think this .conf file should live? Just pass any path to podman and then have it passed through to this project where it could be read?

Also I would like to not mess with the firewall here to enable port forwarding. Since you should be connected via VPN you should just directly connect to that continuer ip.

How could we deal with sth like -p 8080:80 that is passed through a container that is connected to a VPN exclusively though?

As far as default route is considered every network gets a default route at the moment, I think that should be the same for wireguard. There is an internal option which is used to disable the default route.

It should work the same regardless of other drivers used, the wg driver itself should not check anything about other used networks, so --network ---network and only --network should result in the same task for the wireguard implementation.

So, would you say that --network <bridge> is always the default route and if all traffic should flow through the WG network it would require setting AllowedIPs = 0.0.0.0/0 in the WireGuard config?

Lastly, should we implement this directly via netlink API or shell out to the wg tools? I am not sure I would prefer to use it

I think it might be better to do this via the netlink API. As long as we document that the feature needs the WireGuard module in the Kernel.

In any case we should not do anything directly related to cryptography, i.e. key generation.
Totally agree with you :)

I have thought about this recently. I personally would like to add wireguard support in the not so far future.

Currently I have a lot of free time as I am stuck in german university burocracy and also still looking for a job.
If you have time we could jump in a call and talk about whether or not we can split the work on this somehow?

[Luap99]

You can work on it, any help is welcome. I already have enough things to do anyways.
But we should first agree on the design.

I would also like the opinion of the other members @flouthoc @mheon @baude or anybody else who might be interested it this.

Where do you think this .conf file should live? Just pass any path to podman and then have it passed through to this project where it could be read?

Also I would like to not mess with the firewall here to enable port forwarding. Since you should be connected via VPN you should just directly connect to that continuer ip.

Yes I think something like --network <name>:config_path=/some/path for podman then pass it down the stack. This would obviously mean that we need changes in podman and containers/common (where most of the network code is).

How could we deal with sth like -p 8080:80 that is passed through a container that is connected to a VPN exclusively though?

Just ignore it like, this is already done by the macvlan driver. I don't see how we can support it when the peer endpoint is on other system. We cannot route anything directly to the wireguard container interface in this case since the host netns is not part of that subnet.

So, would you say that --network is always the default route and if all traffic should flow through the WG network it would require setting AllowedIPs = 0.0.0.0/0 in the WireGuard config?

Mhh, I think you are right. Maybe it is best to just setup the route for the AllowedIPs. In this case the user has full control.

[b-m-f]

You can work on it, any help is welcome. I already have enough things to do anyways.
But we should first agree on the design.
I would also like the opinion of the other members @flouthoc @mheon @baude or anybody else who might be interested it this.

Sure thing. Lets wait for some feedback and then I can compile what we gathered so far.

Just ignore it like, this is already done by the macvlan driver. I don't see how we can support it when the peer endpoint is on other system. We cannot route anything directly to the wireguard container interface in this case since the host netns is not part of that subnet.

What do you think about the following when -p 8080:80 is given:

netns exec CONTAINER_NAMESPACE nft add rule prerouting iifname WIREGUARD_INTERFACE protocol dport 8080 redirect to 80

This could be executed inside the containers network namespace after the interface was moved inside of it and remote peers would be able to use the port that was rebound.
For remote containers that rely on certain default ports this could be pretty handy.

[baude]

either start a design doc (@Luap99 knows where that is) or start a discussion that mimics it ?

[b-m-f]

I have updated the initial discussion. Not sure what the best way is to get everyone notified on changes here. Hope it works

[mheon]

I have no real comments except RE: port forwarding - I feel like we should explicitly disable it with VPN'd networks to ensure all traffic goes through the VPN?

[b-m-f]

either start a design doc (@Luap99 knows where that is) or start a discussion that mimics it ?

Sounds like a good idea.
Is this a Redhat internal thing or can I get access too @Luap99 ?

[baude]

this is now a discussion.

[b-m-f]

Cool, thanks for transferring this.
After the weekend I can add more details and update the initial suggestions with info gathered.

[Luap99]

Should we simply ignore PostUp and PostDown
If we choose not to then they should be run inside the containers network namespace using the permissions that podman has available.

Yes let ignore them, if there is a use case for this we can always add it later.

[Luap99]

Kernel > 5.16 or correct kernel modules for WireGuard installed

Did you mean 5.6? AFAIK wireguard was added in 5.6 to the kernel

[b-m-f]

True. Updated

[Luap99]

Interfaces will be created on the host
Naming: wg-$CONTAINER_NAME-$HASH

This will not work, interface names can only be 15 chars long. The container name can be much longer. I think wg-$HASH is enough and $HASH should be 12 chars long.

[b-m-f]

Updated!

[Luap99]

Can you add a description of the general user workflow, e.g. podman network create --driver wireguard wg-net, podman run --network wg-net:config...

Do you plan to also make the required changes in podman and cotnainers/common? What should podman network inspect display, I assume that all fields are empty except the driver name and date?

[Luap99]

We have stable types, changing them to add driver specific options is not desirable. The only way would be to put them in the map[string]string options for network inspect or DriverOpts for contianer inspect , which will make it very hard to parse for users.
Right now network inspect only shows the raw network config file as it is used in network create. I don't think it is information is required for users and if they want it we can always documented the nsenter -n -t $containerpid wg show command.

My main ask here was is what to do podman network create --subnet ...? Since everything is in the wg conf file, there is no need to supply anything (and thus should error).

[b-m-f]

My main ask here was is what to do podman network create --subnet ...? Since everything is in the wg conf file, there is no need to supply anything (and thus should error).

We could whitelist allowed params and error on the rest?

But this got me thinking some more:

Network necessary?

I am somewhat unsure about the following thing:
podman run --network wg-net:config...

• Why are we even creating the network in the first place?
• Is there a piece I am missing?

I can not find the added value of that abstraction if podman network inspect wg-net returns:

[
     {
          "name": "wg-net",
          "id": "1234",
          "driver": "wireguard",
          "created": "2022-10-19T10:01:34.345610709+02:00",
     }

Alternative

Do you think that podman run --wireguard path_to_config ... could be an option?
Since the actual network could be spread around n machines and the container will be part of this WireGuard setups might break the general

• create network on host
• hook container into network
  workflow.

But given the (most-likely) distributed nature of the VPN I think this way actually reflects the nature of the network best.

Documenting the nsenter command + adding info into DriverOpts could then be sufficient.

[Luap99]

Why are we even creating the network in the first place?
Is there a piece I am missing?

This is how the code works. You need a network in order to add it with --network ..., there exceptions such as none, host or slirp4netns. However all of them do not call either CNI or netavark so it is fine. We call them network modes.

For anything that uses netavark we need to pass the information from podman to netavark, this requires a network per json config format, see

netavark/src/network/types.rs

Line 56 in d9e76ab

pub struct NetworkOptions {

We could hide this away in podman but there is a problem with backwards compatibility. Since we support arbitrary network names --network wireguard could already be used, even if this is unlikely we must support this at least until the next major podman version.
I agree with you, it would be nice if the user does not need to create a "dummy" network since all info will be in the per container conf anyway. What might be possible is to do the same as the default podman bridge network, if the config exists on disk use that otherwise creater a in memory network: https://github.com/containers/common/blob/5c3399234f97a6fa7105e3d9e0a4c152706c7f09/libnetwork/netavark/network.go#L227-L233

Check out the code here: https://github.com/containers/common/tree/main/libnetwork

[b-m-f]

Thanks for all the info @Luap99 . I will be out for the weekend and think I have enough info to start on code next week and get a first draft PR out soon.

I also have some tech tasks for a job I am currently considering, so I might be a bit slow on progress. But I am determined on finishing this one :)

[b-m-f]

Ok. Started in podman and containers common.

Next step would be in netavark to get

working.

So far I have not checked out the testing suite but will dig into it once the functionality is there

[moschroe]

Just found this after successfully experimenting with a manual container setup modeled after https://github.com/jcarrano/wg-podman. I can't upvote this enough, would be just perfect for setting up services only reachable through a VPN. This is my current task and so far, I'll probably have to work with routing, port-forwarding and maybe even some NAT. I'm not sure I can sell some fiddly bash script as solid infrastructure :)

Unfortunately, I'm not in a situation where I can commit to regular work, but I am an experienced rustacean and if necessary, I'd be willing to try and contribute.

Port-forwarding would not be a requirement for me at all at this point.

I know it's another project, but the absolute cherry on top would be if one could (in the end) re-use existing docker compose files with podman-compose and just amend the network config to have a wireguard connection. But I assume that, as soon as podman can pass on the configuration, that should be possible as well?

[b-m-f]

You could use quadlet with wg-pod.
This is what I am running for now, but as you can see it bugs me as well :)

No idea regarding the compose question.

[Luap99]

compose could be more difficult, you just would need to somehow set the wg config file.
There is a generic map[string]string DriverOpts in the docker API but I do not know if that is used by docker-compose at all. And podman-compose would definitely need patches too.

[moschroe]

I have in the meantime built a rust program that takes a container name and wg config and uses the crate netns-rs and the wg and ip executables with command line parameters to set up the wg interface. Similar to wg-pod but for docker, as we currently have services running with compose.

It works very nicely, especially since it removes the host from all networking/firewalling issues and the company wireguard gateway can handle it centrally. The downside of this is that the command must be run every time after the container is started/restarted :(

[b-m-f]

Hi @Luap99 @mheon ,

Do you have a good resource for the netlink API messages?
I will have to add the namespace switch as a new function.
The iproute2 source code is what I would use to do this, but maybe you know something better?

[Luap99]

I also used ip commands but I just run them with strace, this will decode the netlink messages into something more human friendly.
Other than that rtnetlink(7) contains some info but it is missing most of the options.

netavark/src/network/netlink.rs

Line 151 in 9cb258f

pub fn set_link_name(&mut self, id: u32, name: String) -> NetavarkResult<()> {

could be a starting point for you. You want to use Nla::NetNsFd() instead of the name.

[b-m-f]

Thanks. This helped out and I got the namespace switch done.

Now its onto setting the WireGuard device properties.

[b-m-f]

Hi @mheon @Luap99,

I think I have covered almost everything we discussed (excecpt interface name. see #472 (comment)).

There are 3 PRs at

• WireGuard Support podman#16291
• WireGuard support common#1209
• Adds wireguard network support #472

and the netavark one is quite huge.

If I can make the review easier somehow please let me know! Over the next few days I have some interviews (Redhat is also on the list :)) and will prepare for them, so I might take a bit to respond.