Ingress is allways open for any IP and I can't do anything about it in iptables

[CrazyPilot]

Hi! I installed RKE2 cluster, and placed there a website with nginx-ingress. Now I want to ban several IPs from accessing this website. Actually I plan to maintain set of IPs to be entirely blocked in iptables. For now, I'm truing to block 1.2.3.4 on my nodes and it does not work.

So I run iptables -I INPUT -s 1.2.3.4 -j DROP and get this result:

# iptables -L INPUT -n --line-numbers
Chain INPUT (policy DROP)
num  target     prot opt source               destination
1    cali-INPUT  0    --  0.0.0.0/0            0.0.0.0/0            /* cali:Cz_u1IQiXIMmKD4c */
2    DROP       0    --  1.2.3.4              0.0.0.0/0
3    KUBE-PROXY-FIREWALL  0    --  0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes load balancer firewall */
4    KUBE-NODEPORTS  0    --  0.0.0.0/0            0.0.0.0/0            /* kubernetes health check service ports */
5    KUBE-EXTERNAL-SERVICES  0    --  0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes externally-visible service portals */
6    CROWDSEC_CHAIN  0    --  0.0.0.0/0            0.0.0.0/0
7    KUBE-FIREWALL  0    --  0.0.0.0/0            0.0.0.0/0
8    ufw-before-logging-input  0    --  0.0.0.0/0            0.0.0.0/0
9    ufw-before-input  0    --  0.0.0.0/0            0.0.0.0/0
10   ufw-after-input  0    --  0.0.0.0/0            0.0.0.0/0
11   ufw-after-logging-input  0    --  0.0.0.0/0            0.0.0.0/0
12   ufw-reject-input  0    --  0.0.0.0/0            0.0.0.0/0
13   ufw-track-input  0    --  0.0.0.0/0            0.0.0.0/0

I placed my drop rule on a first place, but rke2 moved chain cali-INPUT to the 1st place again.
After that when I try to access this node from ip 1.2.3.4, all ports are inaccessible except 80 and 443. RKE2 just ignores this rule and exposes ingress to everyone. Seems like it's because of cali-INPUT chain, and I can't put any rule in front of it.

RKE2 version: v1.31.2+rke2r1

[brandond]

Use network policy. Don't try to manipulate the Kubernetes-managed iptables rules directly.

[CrazyPilot]

If i have a list of 500-1000 ip addresses I want to ban, and this list is changing in time, how can I put set it with network policy?

[brandond]

I'd probably move that into an ingress middleware, or do it on a system external to Kubernetes and the hosts themselves (ie a firewall or WAF in front of your site). The Kubernetes components (Calico in this case) expect to be able to exclusively manage the iptables rules.

[CrazyPilot]

I have several bare metal servers at Hetzner, there's no any systems external to Kubernetes which I can control. Only one firewall in front of my site is iptables of the node. So I just applied some iptables rules to make Kubernetes ports available only from my servers' IPs and it works. Everything except my attempts to hide https from spammers.

I also tried to use Crowdsec inside the cluster, it analyses ingress logs well, but I also had to patch ingress itself to add there a crowdsec nginx bouncer (and there I gave up, didn't found any working examples how to get bouncer into ingress)

Maybe I just should remove ingress from the cluster and build my own Nginx image with Crowdsec and everything else I need? Does this approach look fine? I have just couple of domains and only 3 working nodes, so it can be handled in more traditional way if theres no other easy options.

Sorry for noob questions, just trying to migrate to RKE2 from Proxmox. It will be my first RKE2 cluster)

[CrazyPilot]

Finally I installed crowdsec into cluster using this manual: https://docs.crowdsec.net/u/getting_started/installation/kubernetes/
Only one thing in this manual had to be changed -- to add bouncer to ingress I used this command:

helm -n kube-system upgrade -f crowdsec-ingress-bouncer.yaml rke2-ingress-nginx rke2-ingress-nginx --repo https://rke2-charts.rancher.io