GNS3 VM on VMware Fusion tech preview for M1

[grossmj]

I built a GNS3 VM for ARM which can run on Apple silicon M1. You can download the disks (VMDK) here

Please note this is EXPERIMENTAL, do not expect it to work like on Intel/AMD. VMware Fusion for M1 itself is just a preview. See below for limitations, workarounds and other issues.

Installation on VMware Fusion

Create a custom virtual machine

It is not possible to import an OVA at the moment, see #3257 (comment)

Choose the operating system

The GNS3 VM is based on Ubuntu 20.04 LTS for ARM64 architecture (aarch64).

Choose a virtual disk

Select the first GNS3 VM disk.

Customize the name

Add the second hard drive

Go to the VM settings and add an additional device

Select the second disk

Configure network adapters

Go back to the VM settings and add another device

Make sure it is configured for NAT

Go to the first adapter settings and make sure it is configured to be private

Start the VM

Launch GNS3 and use the VM as a remote server

Limitations and workarounds

Docker

Trying to run a container out of the box will result in an exec format error because the busybox binary that is distributed part of the GNS3 server package is compiled for the x86_64 architecture, not ARM.

The workaround is to SSH to the GNS3 VM and type this command:

sudo cp /usr/bin/busybox /usr/local/lib/python3.8/dist-packages/gns3server/compute/docker/resources/bin/busybox

Note that this will not be needed with GNS3 version >= 3.0, the server will automatically use the host busybox: GNS3/gns3-server#1908

However, any image or container you want to run must be built for the arm64v8 architecture, most images like Ubuntu, Alpine etc. support this. You can quickly check on Docker Hub what architectures are supported, for instance https://hub.docker.com/_/ubuntu

Compare this with a container we distribute, https://hub.docker.com/r/gns3/network_automation/tags, which only supports amd64. We aim to have our containers built for multiple architectures at a later stage.

This is what happens if you want to run Ubuntu image made for amd64 architecture:

gns3@gns3vm:~$ docker run --rm -t amd64/ubuntu uname -m
Unable to find image 'amd64/ubuntu:latest' locally
latest: Pulling from amd64/ubuntu
7b1a6ab2e44d: Pull complete 
Digest: sha256:7cc0576c7c0ec2384de5cbf245f41567e922aab1b075f3e8ad565f508032df17
Status: Downloaded newer image for amd64/ubuntu:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
standard_init_linux.go:228: exec user process caused: exec format error

It is currently only possible to run images / containers made for other architectures on a x86_64 host, see https://www.stereolabs.com/docs/docker/building-arm-container-on-x86/ and https://github.com/multiarch/qemu-user-static#supported-host-architectures.

Conclusion it is only possible to Docker containers made for arm64/v8

Qemu

Qemu is the big issue because KVM hardware acceleration is not supported.

My understanding is that to use KVM, the firmware must start Linux in hypervisor (EL2 / HYP) mode and the kernel should print something like this:

CPU: All CPU(s) started at EL2
...
kvm [1]: Hyp mode initialized successfully

As you can see GNS3 VM doesn't start with Hypervisor mode or EL2:

Also, I read the following nested virtualization including the ARMv8.4 improvements are present on Apple M1. However, Apple's virtualization solution doesn't currently support those.

Also, I was successful to run the GNS3 VM using a patched QEMU leveraging Hypervisor.framework (hvf acceleration): see https://gist.github.com/akihikodaki/87df4149e7ca87f18dc56807ec5a1bc5 or https://gist.github.com/citruz/9896cd6fb63288ac95f81716756cb9aa

However, as expected, nested virtualization isn't possible even after trying to enable it on the Qemu command line (virtualization=on).

So I don't expect we will see nested virtualization until Apple or VMware does something.

I was able to manually create a template for IOSv-L2 and I had to force to not using KVM by adding -machine accel=tcg otherwise the server would complain that KVM is not available.

Conclusion: supported but likely to be really slow

Dynamips

Dynamips seems to work okay. I only tested with a c3725 IOS image.

However, Dynamips has been compiled without JIT (Just In Time) support which may slow things down or not be as optimized.

Conclusion: supported but may be slower or unstable

IOU

It is not possible to run IOU because IOU images are compiled for i386 or x86_64. I remember of some Solaris Sparc images back in the days but nothing about ARM.

gns3@gns3vm:/opt/gns3/images/IOU$ file i86bi-linux-l3-adventerprisek9-ms.155-2.T.bin
i86bi-linux-l3-adventerprisek9-ms.155-2.T.bin: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.9, stripped

gns3@gns3vm:/opt/gns3/images/IOU$ ./i86bi-linux-l3-adventerprisek9-ms.155-2.T.bin 
bash: ./i86bi-linux-l3-adventerprisek9-ms.155-2.T.bin: cannot execute binary file: Exec format error

One workaround is to use Qemu User Emulation: https://wiki.debian.org/QemuUserEmulation

I was successful to manually running an IOU image:

gns3@gns3vm:/opt/gns3/images/IOU$ qemu-i386-static i86bi-linux-l3-adventerprisek9-ms.155-2.T.bin 1
...

Cisco IOS Software, Linux Software (I86BI_LINUX-ADVENTERPRISEK9-M), Version 15.5(2)T, DEVELOPMENT TEST SOFTWARE
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Thu 26-Mar-15 07:36 by prod_rel_team

...

Linux Unix (Intel-x86) processor with 87588K bytes of memory.Installed image archive

Processor board ID 2048001
8 Ethernet interfaces
8 Serial interfaces
64K bytes of NVRAM.

and after installing qemu-user-binfmt I was able to execute IOU transparently.

gns3@gns3vm:/opt/gns3/images/IOU$ ./i86bi-linux-l3-adventerprisek9-ms.155-2.T.bin 1
...

Cisco IOS Software, Linux Software (I86BI_LINUX-ADVENTERPRISEK9-M), Version 15.5(2)T, DEVELOPMENT TEST SOFTWARE
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Thu 26-Mar-15 07:36 by prod_rel_team
*Nov 19 07:59:55.937: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is OFF
*Nov 19 07:59:55.937: %CRYPTO-6-GDOI_ON_OFF: GDOI is OFF
Router>

I will write a howto set Qemu User Emulation up in the GNS3 VM.

Conclusion: supported but may be slow since it is emulation

VPCS

No issue here since the vpcs Debian package was compiled for ARM.

gns3@gns3vm:~$ file /usr/bin/vpcs 
/usr/bin/vpcs: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=2403ecb5c20e3f8264d0bafa15c022dd6e7b6759, for GNU/Linux 3.7.0, stripped

gns3@gns3vm:~$ vpcs 

Welcome to Virtual PC Simulator, version 0.8.2
Dedicated to Daling.
Build time: Aug 23 2021 11:15:00
Copyright (c) 2007-2015, Paul Meng ([email protected])
All rights reserved.

VPCS is free software, distributed under the terms of the "BSD" licence.
Source code and license can be found at vpcs.sf.net.
For more information, please visit wiki.freecode.com.cn.

Press '?' to get help.

VPCS[1]>

Conclusion: supported, no issues detected

[jnatschev]

Wow! Nice. Cheers, John Natschev

…

On 19 Nov 2021, at 13:47, Jeremy Grossmann ***@***.***> wrote: I built a GNS3 VM for ARM which can run on Apple silicon M1. You can download the disks (VMDK) here <https://www.dropbox.com/s/up5d8qxopmafu9u/GNS3VM.ARM64.2.2.27.zip?dl=0> Please note this is EXPERIMENTAL, do not expect it to work like on Intel/AMD. VMware Fusion for M1 itself is just a preview. See #3257 <#3257> for limitations, workarounds and other issues. Installation on VMware Fusion Create a custom virtual machine It is not possible to import an OVA at the moment, see #3257 (comment) <#3257 (comment)> <https://user-images.githubusercontent.com/4752614/142552930-f49b20e0-568d-4394-a0c3-dd35044c7dcb.png> Choose the operating system The GNS3 VM is based on Ubuntu 20.04 LTS for ARM64 architecture (aarch64). <https://user-images.githubusercontent.com/4752614/142553094-a0a998e0-0d83-4c2b-b493-51165d27dffc.png> Choose a virtual disk Select the first GNS3 VM disk. <https://user-images.githubusercontent.com/4752614/142553414-64e354b4-82f2-4221-bef1-2752f1b584ca.png> <https://user-images.githubusercontent.com/4752614/142553834-2e25276a-8ad2-4602-9e27-53c2a94ec2cd.png> <https://user-images.githubusercontent.com/4752614/142554107-ef523274-8d98-4b80-b455-a0d1d37dbed6.png> Customize the name <https://user-images.githubusercontent.com/4752614/142554214-6d526098-c7f4-431e-a1b7-c9caa7312d37.png> Add the second hard drive Go to the VM settings and add an additional device <https://user-images.githubusercontent.com/4752614/142554318-34e9a9fe-8eda-4125-b333-47819dfcfadb.png> <https://user-images.githubusercontent.com/4752614/142554443-32283ff3-4b25-4602-9492-d544f4169729.png> Select the second disk <https://user-images.githubusercontent.com/4752614/142554473-7d31dbcd-3dcd-4256-86fc-1a1c266ce4ff.png> <https://user-images.githubusercontent.com/4752614/142554737-33801b2e-8e0d-45dc-bbe8-5dd6c347cefc.png> Configure network adapters Go back to the VM settings and add another device <https://user-images.githubusercontent.com/4752614/142554885-29adff32-b145-415e-bb3d-a2b8385f1dcb.png> Make sure it is configured for NAT <https://user-images.githubusercontent.com/4752614/142554986-d29ae08a-70b6-4ae3-aa2e-af3c24985326.png> Go to the first adapter settings and make sure it is configured to be private <https://user-images.githubusercontent.com/4752614/142555072-405bf404-36bd-4aa3-afad-e68176ad24f3.png> Start the VM <https://user-images.githubusercontent.com/4752614/142555198-de3402f9-26ee-41b2-96ad-7f6672615672.png> <https://user-images.githubusercontent.com/4752614/142556113-1de1871b-5f1c-499c-998b-ee10c4b0d0c9.png> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3261>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACC57OOWAYEJ3BWK5A6OAZLUMW3FDANCNFSM5ILDXFNA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[grossmj]

Haha no worries, hang on I am going to post about limitations / workarounds ;)

[grossmj]

I have updated the post

[grossmj]

Here are the instructions to run IOU

SSH to the VM, install Qemu User Emulation

sudo apt update
sudo apt install qemu binfmt-support qemu-user qemu-user-binfmt

Edit /etc/apt/sources.list and replace by this content:

deb [arch=i386] http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb [arch=i386] http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
deb [arch=i386] http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
deb [arch=i386] http://security.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse

deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ focal main restricted universe multiverse
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ focal-updates main restricted universe multiverse
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ focal-backports main restricted universe multiverse
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ focal-security main restricted universe multiverse
deb [arch=arm64] https://download.docker.com/linux/ubuntu focal stable

Finally, install gns3-iou which will in turn install the necessary IOU dependencies (SSL lib)

sudo dpkg --add-architecture i386
sudo apt update
sudo apt install libc6:i386 gns3-iou

[Ondjultomte]

Is this running ios locally ? even though perhaps slow ?

I need to run some sort of IOS locally on my m1 mac, just for fast testing when scripting. For full test I'll connect to my real lab setup.

[grossmj]

Regarding Qemu and the lack of KVM support in the VM, I think the problem is even bigger because even if we have it, this would only work for ARM (qemu-system-aarch64) not for qemu-system-x86_64, the major issue is that most virtual appliances from the big companies are all x86_64 based.

[ghost]

An ARM based computer like the M1 Macs will never be able to run x86 code in hardware, just because its CPU doesn't understand x86 code. So x86 code can only be executed by software emulation, what will be slow. As long as the major network vendors don't offer VM images for ARM based computer, buying a M1 Mac is not an option for me. Any decent intel computer will outperform a M1 Mac, when running x86 VMs.

[grossmj]

Maybe network vendors will switch to ARM or more realistically have both x86 and ARM appliances but this will most likely take years. So, indeed I wouldn't recommended buying a M1 Mac if you need to run x86 VMs.

[adilnas62711411]

Hello,

I downloaded the experimental vm and set it up in Vmware fusion tech preview per your instructions and have observed that the vm is not booting properly.

[grossmj]

Hum, I don't see anything wrong. Have you selected "Ubuntu 64-bit Arm" for the operating system?

[sstah188]

Hey @grossmj! I'm having the same issue. I selected "Ubuntu 64-bit Arm" for the operating system.

[sstah188]

@grossmj @adilnas62711411 I figured out the problem. @adilnas62711411, you have to use the disk images @grossmj provides at the very beginning of the article:

So I deleted my virtual machine, started over, and downloaded the disk @grossmj provided and now it boots up properly when I start the VM after going through the steps again! Going to go ahead and proceed with the setup, thanks @grossmj!

[sstah188]

The problem was that we apparently downloaded the wrong GNS3 VM for this instance. For this experiment this download will not work.

[Jverbist]

Do you have to use an ubuntu arm image as well? or how does it boot?

[acefer]

Hi!

I've just followed the guide and tried docker containers with amd64 images. At first, i couldn't make it work properly, but after installing qemu-user-static i was able to run amd64 images on containers.

sudo apt-get install -y qemu-user-static

I've tried with the following images:
This one for routers:
https://hub.docker.com/r/ajnouri/quagga_alpine/
And this one for computers:
https://hub.docker.com/r/gns3/endhost

Idk if i misunderstand something or it just works as intended.

[grossmj]

Hum, I thought it is not possible to run amd64 images. I will try again, thanks 👍

[network-shark]

Wow thank you ! As soon as I get my hands on my m1 , I will help with testing !

[network-shark]

I got my Mac and started to compile dynamips from git . Good news it compiled, but without jit . Nevertheless , it seems the idle pc calculation is broken. I got a lot of values and none of them are working. Tested with c7200-advipservicesk9-mz.152-4.S5.bin . @grossmj Does idle pc calculation works for you ?

[Lifeng-sg]

Hi,

I am very new to GNS3
I have one Macbook pro ( 2021) running Monterey 12.2, would like to run GNS3 on my Mackbook.
Currently, I have the following apps on it.

1. Parallels Desktop 17 for Mac
2. Ubuntu 20.04.2 ARM64 (installed automatically via Parallels desktop 17)
   The Ubuntu is running seem ok.
   I am a network guy I would like to know what is next?
3. what is the necessary configuration of Ubuntu
4. is there any procedure for the installation of GNS3?

thanks
Lifeng

[bronsoun]

@grossmj sincerely I'm sorry but I can't add new device templates due to hash sum mismatch :( what i'd do?

[bejoybkn]

Hello @grossmj

Thanks a ton for this post and i finally managed to install GNS3 VM on my M1 (almost lost hope as vmware/VIrtual box support is still officially not there)

Is there any a way i can install Cisco NXOSv/Arista VEOS appliances on this VM as they both are x86?

I tried selecting the different qemu options but its stuck at waiting for the server to respond and does not proceed.

Again massive applauds for releasing this post.

[network-shark]

I am using a x86 server to run the vm. On my m1 I start gns3 and connect to the server it runs perfectly. Due to the lack of x86 emulation on m1 I don’t think there will be a solution to run gn3 directly on your mac. Maybe never , until all vendors are shipping vm’s for arm64 and this will take years

[gfhfhoo]

Nice solution! @grossmj
But I still have some problem here:
I want to try to run IOU device on arm platform you mentioned, then, I have installed related libraries of qemu. My question is how can I start my iou device images? Just directly run them by command and will be shown in GNS3 or another way? I don't understand, thank you again. Hope for your reply.

[cheungtitus]

was able to manually create a template for IOSv-L2 and I had to force to not using KVM by adding -machine accel=tcg otherwise the server would complain that KVM is not available.

@grossmj Tried this with IOSv using the default QEMU binary which is /usr/bin/qemu-system-x86_64 (v4.2.1) but when the device starts it is stuck in the "Booting from Hard Disk..." stage.

Now if I select to use /usr/bin/qemu-system-aarch64 (v4.2.1) it'd crash with the following in the console.

QEMU process has stopped, return code: 1
Start QEMU with /usr/bin/qemu-system-aarch64 -name IOSv-1 -m 256M -smp cpus=1,sockets=1 -boot order=c -drive file=/opt/gns3/projects/da555b4a-3cef-4778-b7d0-d107722664c9/project-files/qemu/2b3f3a1a-4ad3-44ed-ad82-23357298d9e0/hda_disk.qcow2,if=ide,index=0,media=disk,id=drive0 -uuid 2b3f3a1a-4ad3-44ed-ad82-23357298d9e0 -vnc 0.0.0.0:0 -monitor tcp:127.0.0.1:33681,server,nowait -net none -device e1000,mac=0c:3f:3a:1a:00:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:20001,localaddr=127.0.0.1:20000 -machine accel=tcg

Execution log: qemu-system-aarch64: -machine accel=tcg: No machine specified, and there is no default Use -machine help to list supported machines

And with the following pop up.

Can you comment if there are any additional steps you did? I'm running this on gns3-vm 2.2.32.

[parveshtaneja2005]

getting below error.

GNS3 management console.
Running GNS3 version 2.2.34 on Darwin (64-bit) with Python 3.8.5 Qt 5.15.2 and PyQt 5.15.6.
Copyright (c) 2006-2022 GNS3 Technologies.
Use Help -> GNS3 Doctor to detect common issues.

=> Cannot load the GNS3 VM settings in the preferences dialog: not connected to the controller
Cannot load the IOU license in the preferences dialog: not connected to the controller
Client version 2.2.34 is not the same as server (controller) version 2.2.28
Error while getting templates list: Client version 2.2.34 is not the same as server (controller) version 2.2.28
Error while getting compute list: Client version 2.2.34 is not the same as server (controller) version 2.2.28
Error while getting appliances list: Client version 2.2.34 is not the same as server (controller) version 2.2.28
Error while getting the GNS3 VM settings : Client version 2.2.34 is not the same as server (controller) version 2.2.28
Error while getting the IOU license information: Client version 2.2.34 is not the same as server (controller) version 2.2.28

[sangiiii]

Facing the same issue. Anyone got a workaround for this ?

[parveshtaneja2005]

GNS3 VM shows 2.2.28. unable to upgrade it.

gns3@gns3vm:$ sudo apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
libfwupdplugin1
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
gns3@gns3vm:$

[EagleYing]

You should run sudo apt update first.

[termaulmaul]

Transport (VMDB) error -14: Pipe connection has been broken

[lhtex]

Very nice! Thanks!!! Works like a charm!
Now I am waiting for how I can update to 2.2.29 !

[pmanolak]

Don't know if I am doing something wrong, but I was never able to start the gns2 vm on fusion in my MacBook M1.
Always failed with the error "This virtual machine cannot be powered on because it requires the X86 machine architecture, which is incompatible with this Arm machine architecture host."
See KB-84273.
I don't think that this is an issue with the Fusion itself but rather than with the GNS3 VM which is compatible only for x86 platforms.
I am using version VMware Fusion 13.0.2 (for Intel-based and Apple silicon Macs).

Any ideas?

[hanyeolk]

@grossmj First, Thank you for your all the work. I have followed your steps and was able to install GNS3 VM on my MacBook pro M1 chip. Now, I tried to set up some testing PJ , adding Paloalto VM on GNS3. I found one youtube tutorial about adding PA VM in GNS3 MacBook m1, https://youtu.be/OFu2WXENWdI?si=982vvU1hJD05B912, but when i about to install PA VM, the option "Install the appliance on the GNS3 VM (recommended) is grayed out. Then once i hit next then I get the error message ("The selected server doesn't support KVM.A Linux server of the GNS3 VM running in VMware is required." Please see attached screenshot. Appreciate if anyone can help!!