Multiple networks?

[bj0]

I sometimes have a machine on multiple networks at once. In wireguard, this is easy as I can just create a new config with a different interface name, and they can both run without conflict.

Is this use case supported on netbird? I can't find any information about it in the docs or issues.

[mlsmaycon]

Hi @bj0, currently, we don't offer straight support for multiple Wireguard connections as you would need to set the daemon listening address, a custom config path, and the new interface name needs to be updated in the configuration file.

Let me know if you need a guide for it to try this out.

we will evaluate the options to make this possible in an easier manner and perhaps work on it in the next few days.

[bj0]

A guide would be cool, but I can figure out those options pretty easy. My main question is how would you "manage" or "auth" this second network? Would you need a second oauth account (im currently just using a google account)?

[mlsmaycon]

Hello @bj0 with the release v0.9.0 you are able to run the following steps to achieve that. Below are the steps:

For a 2 connection example on a single Linux with SystemD, the steps are:

stop and uninstall the daemon

sudo netbird service stop
sudo netbird service uninstall

Create custom login files and login:

Assuming we have two accounts, ACCOUNT-A and ACCOUNT-B:

netbird login --config ./config-wt1.json --log-file console --setup-key ACCOUNT-A-AAA...
netbird login --config ./config-wt2.json --log-file console --setup-key ACCOUNT-B-BBB...

Now, we need to edit each configuration file and change the WgIface, WgPort and update the IFaceBlackList:

# FROM
    "WgIface": "wt0",
    "WgPort": 51820,
    "IFaceBlackList": [
        "wt0",
        ...
        
# TO 
# interface wt1:       
        "WgIface": "wt1",
        "WgPort": 50001,
    "IFaceBlackList": [
        "wt",
        ...
# interface wt2:       
        "WgIface": "wt2",
        "WgPort": 50002,
    "IFaceBlackList": [
        "wt",
        ...        

move the files to the default config location /etc/netbird

sudo mv  ./config-wt1.json /etc/netbird/config-wt1.json
sudo mv  ./config-wt2.json /etc/netbird/config-wt2.json

install the wt1 service pointing to the config location:

sudo netbird service install --config /etc/netbird/config-wt1.json

let's edit the systemd file /etc/systemd/system/netbird.service and update the ExecStart and rename it

# FROM
ExecStart=/usr/bin/netbird "service" "run" "--config" "/etc/netbird/config-wt1.json" "--log-level" "info"

# TO
ExecStart=/usr/bin/netbird "service" "run" "--config" "/etc/netbird/config-wt1.json" "--log-level" "info" "--daemon-addr" "unix:///var/run/netbird-wt1.sock" "--log-file" "/var/log/netbird/client-wt1.log"

# rename
sudo mv /etc/systemd/system/netbird.service /etc/systemd/system/netbird-wt1.service 

make a copy of the service file for the interface wt2, then update its ExecStart

sudo cp /etc/systemd/system/netbird-wt1.service /etc/systemd/system/netbird-wt2.service 

# FROM
ExecStart=/usr/bin/netbird "service" "run" "--config" "/etc/netbird/config-wt1.json" "--log-level" "info" "--daemon-addr" "unix:///var/run/netbird-wt1.sock" "--log-file" "/var/log/netbird/client-wt1.log"
# TO
ExecStart=/usr/bin/netbird "service" "run" "--config" "/etc/netbird/config-wt2.json" "--log-level" "info" "--daemon-addr" "unix:///var/run/netbird-wt2.sock" "--log-file" "/var/log/netbird/client-wt2.log"

reload systemd

sudo systemctl daemon-reload

start each interface service

sudo systemctl start netbird-wt1
sudo systemctl start netbird-wt2

Note For direct connection to work, you will need peers to be running the latest version

[fti7]

@mlsmaycon thx for the Tutorial
Please consider to implement this kind of multi- tenancy/instances in all of your components natively.
This might work for an Advanced Linux User, but its difficult for e.g. Windows GUI Client User

There will be enough use-cases for this in the future (Both, Client and UI/Management Side)
and it might be easier to design/implement this directly in the beginning 😀

Thx!

[bj0]

Thanks @mlsmaycon ! That's super helpful, I'll give it a try when I get a second oidp/network going.

I was thinking about this while reading through the access control documentation. Another potential solution might be, instead of a separate network, a "shared" group where you could add peers from a different network (obviously on the same mediation server). That way you wouldn't need to run double everything, but it would rely on the mediation server much more.

[mlsmaycon]

@mlsmaycon thx for the Tutorial Please consider to implement this kind of multi- tenancy/instances in all of your components natively. This might work for an Advanced Linux User, but its difficult for e.g. Windows GUI Client User

There will be enough use-cases for this in the future (Both, Client and UI/Management Side) and it might be easier to design/implement this directly in the beginning 😀

Thx!

Thanks, @fti7 and @bj0 we are definitely thinking about improving that experience. We built the ground for it with smaller CGNAT and random networks.

Moving forward, maybe we can use a bit of input from you folks, how would you see a multi-tenancy setup? Would be more based on company/personal domain, accounts, or networks?

[fti7]

Sure, here is my POV:
Multi Tenancy for me means full Isolation.

The scenario what mostly like gonna happen is that in multiple Parties are using Netbird with either your shared Management Server or setting up an own one.
They are completely isolated and dont talk to each other. The only thing, what you already mentioned is the Collision of the CGNAT Subnet.
But i guess that shouldnt be an big issue if they are getting smaller and the reserved Subnet per "Management Server Tenant" gets randomized.
If this still happens, it should be possible to change this reserved Subnet as an Admin on one of the Servers

Scenario 1

Im a Freelancer and use Netbird for my private Network using app.netbird.io and working for 2 Companies which each have their own selfhosted Management Server.

In this Scenario only the Clients need some Modifications.
How it should look like (Example Windows GUI Client):

The Clients have a list where i can add/remove multiple "Profiles"
A Profile is basically some Config which i get from an Admin via a QR Code/File/Copy&Paste and contains

• Tenant ID (I would use some UUID; Can help for further Scenarios where you uniquely need to identify tenants )
• Tenant/Profile Name (Possible to overwrite by the User)
• Management URL
• Admin URL
• Optionally: Setup Key
• CGNAT Subnet, to directly see if there are Collisions... but that could also be checked after calling the Management Server; It should only allow one Enabled/Active Connection per Subnet... multiple Entries with the same Subnet in general should be allowed (Example: Your company has a Main VPN Mgmt Server and some Disaster Recovery Backup one in a different Datacenter; Both have the same CGNAT Subnet)

Each Profile List entry will have some ON/OFF Toggle for globally turning off the Connection (e.g. im Working for Company #2 only 1 Week per Month, and dont want to have this Connection open all the time)

For the Linux Client, you can implement some "Add/Delete/Enable/Disable Profile" command through the CLI

Scenario 2

Multi Tenancy for the Management Server
Scenario: I want to have multiple Isolated "Networks" for whatever reason. (e.g. one for my private Smarthome Network and one only for my Friends, sharing a Minecraft Server)

• Just add an Dropdown Field on the upper left Navigation bar where the User can choose which Tenant is currently active in the UI
• One User can belong to multiple Tenants with different Roles (Admin/Regular Peer)
• You need also a "Superuser" Role which is able to manage the tenants per Management Server

I guess this two are the most Important Scenarios which is needed to build up complex Architectures

Later you could for example add an fancy Gateway function to connect two Tenants (Kind of Site2Site VPN) -> Example: You have two Companies which each use their own Netbird and want to share some Services Transparently (Without adding the opposite Profiles to all employees)....
But...., thats stuff for some advanced Use Cases in the Future..... 😀

[mlsmaycon]

@fti7 thank you so much for your suggestions and my apologies for not giving feedback earlier, we are aligned on the use cases, for the profile switching I think we can improve a bit more and make things more simpler, just by connecting and the app will handle the rest, and in case you are connecting to multiple self-hosted we can add a Add profile with custom manager.

We are looking at a major account refactor that will allow for better multi-tenancy and isolation. The same goes for the network range management.

Time wise, these changes might come in Q4 or early Q1/2023.

[alexlyee]

@mlsmaycon This is great! It seems like this is a solution for combining meshnets of my friends' networks and my own? I came accross this because I opened a case for something very similar over on innernet here. If you wouldn't mind reading my use-case over there, would you mind clarifying if and how this would work for it?

[helmut72]

We are looking at a major account refactor that will allow for better multi-tenancy and isolation. The same goes for the network range management.

Time wise, these changes might come in Q4 or early Q1/2023.

Are these features released or still in development? If in development, is there a new roadmap?

[lfarkas]

i think that one machine can be part of multiple network is a very basic requirements. even openvpn knows this feature from the very beginning. of course this should support multiple interface and multiple config file (or one config file support for multiple interface). imho multiple config file would be more robust. what's more with multiple config file we can use systemd's template services the same way as openvpn do it systemd.unit.html. in this case netbird@work, netbird@home service can be used. of course this requires different network interface and different CGNAT.

is there any progress with it?

[nazarewk]

I'm running multiple systemd services: 1 for each network I connect to, don't remember the exact reason, but I had trouble getting it to run using instantiated systemd.unit (I guess it boiled down to running Wireguard listener on different port for each instance)

• allow specifying --wg-iface and --wg-port #659 - to set ports on first start without editing later on

[bc24fl]

I'm evaluating Netbird and super impressed so far! Our use case requires key employees to have access to multiple client isolated networks from a single device. Any updates on this?

[Fantu]

one other thing that I see should be modified for support multiple netbird instance is windows firewall rule, actually the name of rule is fixed to "Netbird", I suppose is enough a simple change the rule name to "Netbird-"+$WgIface (for example "Netbird-wt0")

[fti7]

Any update on this?

[jyolo]

Any update on this?

[pete1019]

Multi Tenancy would be so nice. What are the current news on this? Thanks a lot.

[the-project-group]

@mlsmaycon any update on this?
How would this work on macOS regarding the service?

[sh00t3r]

I converted the guide from @mlsmaycon to use it on windows with the help of nssm. only DNS is not working that well. you can find it on my wiki: link

[florian-obradovic]

@mlsmaycon I would love to see both options.
I use Netbird for my HomeNetwork and plan to use it as a self hosted Tailscale-competitor option at my corp.
Additionally at the corp, I'd love to see multi tenancy, here an example use case:
Network 1 is for road warriors / remote access for all clients with their personal username (SSO)
Network 2 is for admin only stuff like remotely accessing these client machines and they're automatically onboarded via access key and always runs as a service. This would make managing these machines much easier. Example: rdp access, vnc from login screen via netbird-network2 only, etc.

Are there any plans for payed self-hosted enterprise plans incl. logging / reporting, support, etc?
We want to self host but are "blind" and don't have any access and traffic reports.
We want to support the project.
We want to get customer support if needed.

[florian-obradovic]

@mlsmaycon I would love to see both options.
I use Netbird for my HomeNetwork and plan to use it as a self hosted Tailscale-competitor option at my corp.
Additionally at the corp, I'd love to see multi tenancy, here an example use case:
Network 1 is for road warriors / remote access for all clients with their personal username (SSO)
Network 2 is for admin only stuff like remotely accessing these client machines and they're automatically onboarded via access key and always runs as a service. This would make managing these machines much easier. Example: rdp access, vnc from login screen via netbird-network2 only, etc.

Are there any plans for payed self-hosted enterprise plans incl. logging / reporting, support, etc?
We want to self host but are "blind" and don't have any access and traffic reports.
We want to support the project.
We want to get customer support if needed.

[florian-obradovic]

Trying to figure out the steps on macOS:
WgIface must use inteface prefix utun, Example: utun90 for home and utun91 for work (check if free with ifconfig).
Otherwise you get this:
2024-09-06T16:01:47+02:00 ERRO client/internal/connect.go:263: error while starting Netbird Connection Engine: create wg interface: Interface name must be utun[0-9]

sudo launchctl load -w /Library/LaunchDaemons/netbird-home.plist
sudo launchctl load -w /Library/LaunchDaemons/netbird-work.plist

To check status

netbird status --config /etc/netbird/config-home.json --log-level info --daemon-addr unix:///var/run/netbird-home.sock
netbird status --config /etc/netbird/config-work.json --log-level info --daemon-addr unix:///var/run/netbird-tpg.sock

Currently I still struggle a bit...
They are connected successfully but even can't ping their own address if both tunnels are connected...
I checked for subnet overlap > No (100.116.0.0/16 & 100.102.0.0/16)

Could this due to the link local addresses being the same?

[GhaziTriki]

Hello,

I have successfuly working 2 netbird on 2 networks for Windows guest. The DNS remains a problem. I am using a DNS proxy, Acrylic, however it is possible to start a single netbird instance with 127.0.0.1 as a custom DNS. The second netbird instance fails then. What is blocking pultiple netbird instances to use the same DNS server?

[tgutzler]

@mlsmaycon, you seemed quite motivated to push this along 2 years ago but I cannot see an option for switching profiles in the latest client for windows. Has this been shelved?

[Echutaa]

@mlsmaycon, you seemed quite motivated to push this along 2 years ago but I cannot see an option for switching profiles in the latest client for windows. Has this been shelved?

Its in the roadmap and there is at least one draft. The roadmap called for it to come in Q3 '24 so status is unknown but seems like its at least still in the plan and being worked on at some level.

[EdouardVanbelle]

Hello I am correctly using 2 netbird instances on my Mac:

/Library/LaunchDaemons/netbird-work.plist with a configuration file mapped to /var/run/netbird-work.sock
/Library/LaunchDaemons/netbird-home.plist with a configuration file mapped to /var/run/netbird-home.sock

I saw that DNS configuration is set via the latest daemon started.
which is logic, seeing the source code you are dealing with scutil on Darwin systems and using uniq keys State:/Network/Service/NetBird-Match/DNS & State:/Network/Service/NetBird-Search/DNS

How do you plan to deal with multiple tenancy ?
I think it can be possible to use namespaced keys to permit multitennancy
for example:
State:/Network/Service/NetBird-Work-Match/DNS vs State:/Network/Service/NetBird-Home-Match/DNS
but I guess it will be different on Windows & Linux (I saw your trick with eBPF)

While waiting a solution on a Darwin system, I did this very simple override:

# cat /etc/resolver/<NETBIRD DOMAIN FOR WORK>
nameserver 100.120.255.254

# cat /etc/resolver/<NETBIRD DOMAIN FOR HOME>
nameserver 100.101.255.254