BGP with NHRP over GRE+IPSEC Problem

[ahmad-rz]

Hi, sorry if my question is simple or something else. I want to connect FRR as spoke to Cisco as a hub. So, I chose the BGP routing protocol and selected DMVPN style in FRR (using the nhrpd) because there are multiple branches. I also configured strongSwan for IPsec and phases 1 and 2 established without any problems.

Without using an IPsec profile under the GRE tunnel, BGP will be established with Cisco and NHRP is working too, everything is fine as expected. However, BGP gets down and change state to Connect when activating the IPSEC Profile under the tunnel interface(gre tunnel). The output of "sudo journalctl -u frr -f" is:

2024-08-26 13:03:05.795 [ERR!] bgpd: [TXY0T-CYY6F][EC 100663299] Can't get remote address and port: Transport endpoint is not connected
2024-08-26 13:03:05.795 [ERR!] bgpd: [P90Z5-FP0GJ][EC 33554461] 2.1.99.1: nexthop_set failed, local: 100.106.9.114:46591 remote: (null)p update_if: ens224 resetting connection - intf (Unknown)

Aug 26 12:56:49 frr-virtual bfdd[1097]: [G308J-GJ9KQ][EC 100663301] interface Tunnel328 address 2.1.99.2/32 with peer flag set, but no peer address!
Aug 26 13:00:54 frr-virtual bgpd[1047]: [TXY0T-CYY6F][EC 100663299] Can't get remote address and port: Transport endpoint is not connected
Aug 26 13:00:54 frr-virtual bgpd[1047]: [P90Z5-FP0GJ][EC 33554461] 2.1.99.1: nexthop_set failed, local: 100.106.9.114:42765 remote: (null)p update_if: ens224 resetting connection - intf (Unknown)
Aug 26 13:00:58 frr-virtual bgpd[1047]: [G308J-GJ9KQ][EC 100663301] interface Tunnel328 address 2.1.99.2/32 with peer flag set, but no peer address!

I suspect the /32 prefix used in the GRE tunnel is causing NHRP formation issues, and as a result, the next IP address (2.1.99.1) is unreachable for the BGP daemon, despite running NHRP as described below:

frr-virtual# show ip nhrp Iface Type Protocol NBMA Claimed NBMA Flags Identity Tunnel328 static 2.1.99.1 32.8.8.8 - Tunnel328 local 2.1.99.2 100.106.9.114 100.106.9.114 -
Also, no ping to 2.1.99.1(cisco).

"The attachment contains the configuration of Cisco and FRR as required."

cisco.txt
frr.txt

FRR Version:
10-1

Kernel version:
6.8.0-31-generic

Linux Version:
Ubuntu 24.04

[ahmad-rz]

someone help plz ...

[pguibert6WIND]

someone help plz ...

it looks like the remote peer is unreachable, because IPSec did not mount.
when using dmvpn with IKe, you must have IPsec first, then NHRP, then BGP.

where is the cisco crypto config?

[ahmad-rz]

someone help plz ...

it looks like the remote peer is unreachable, because IPSec did not mount. when using dmvpn with IKe, you must have IPsec first, then NHRP, then BGP.

where is the cisco crypto config?

IPSec is established and working with GRE tunnel and BGP is up without NHRP. However, when NHRP configuration is added, BGP goes down.

here is the the configuration of cisco & strongswan ipsec:

netdpt@frr-virtual:~$ cat /etc/ipsec.conf
# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
    charondebug="all"
    uniqueids=no

conn ipsec
    authby=secret
    auto=start
    type=tunnel
    keyexchange=ikev1
    dpdaction=clear
    dpddelay=30s
    dpdtimeout=120s
    left=100.106.9.114
    right=32.8.8.8
    ike=aes256-sha1-modp1024
    esp=aes256-sha256-modp1024
    keyingtries=%forever

netdpt@frr-virtual:~$ sudo cat /etc/ipsec.secrets
# This file holds shared secrets or RSA private keys for authentication.

# RSA private key for this host, authenticating it to any other host
# which knows the public part.

100.106.9.114 32.8.8.8 : PSK "iS@kmpFRR"

crypto isakmp policy 2
 encr aes 256
 authentication pre-share
 group 2

crypto isakmp key iS@kmpFRR address 100.106.9.114
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 10
crypto ipsec security-association replay window-size 1024

crypto ipsec transform-set iS@kmpFRR esp-aes 256 esp-sha-hmac
 mode tunnel

crypto ipsec profile ipsec-frr
 set transform-set iS@kmpFRR

[ahmad-rz]

also maybe this helps, after clearing and stopping ipsec service and reload everything, output of tail -f /var/log/syslog:

2024-08-31T23:21:09.918848+03:30 frr-virtual charon: 08[IKE] deleting IKE_SA ipsec[9] between 100.106.9.114[100.106.9.114]...32.8.8.8[32.8.8.8]
2024-08-31T23:22:28.915198+03:30 frr-virtual charon: 08[CFG] vici initiate CHILD_SA 'ipsec'
2024-08-31T23:22:28.915337+03:30 frr-virtual charon: 08[IKE] initiating Main Mode IKE_SA ipsec[10] to 32.8.8.8
2024-08-31T23:22:28.986739+03:30 frr-virtual charon: 15[IKE] IKE_SA ipsec[10] established between 100.106.9.114[100.106.9.114]...32.8.8.8[32.8.8.8]
2024-08-31T23:22:30.216385+03:30 frr-virtual nhrpd[1058]: [G68G8-7T7EG] cache (peer check failed: online?0 requested?1 ipsec?1)
2024-08-31T23:22:30.218247+03:30 frr-virtual nhrpd[1058]: [G68G8-7T7EG] cache (peer check failed: online?0 requested?1 ipsec?1)

[ahmad-rz]

anyone there ?