fail2ban whitelist dynamic ip update hostname's associated ip

Question

this is my first question here so I apologize if don't comply with the best practices.

Recently I've been a victim of a POST HTTP slow DDoS attack using different IPs on similar and different ranges on my server.

I've managed to mitigate it using fail2ban by creating a personalized filter for my specific needs:

[Definition]

failregex = ^<HOST> .*POST .*xmlrpc\.php.*
^<HOST> .*HEAD .*m.y.i.p.:80.*
ignoreregex =

These two were the most recursive attempts at my server: 'POST xmlrpc.php' request and a 'HEAD http://m.y.i.p/{phpmyadmin|phpwhatevervariation|etc...}'.

I successfully managed to block them using my local jail as this

[nginx-xmlrpc]
enabled = true
filter = nginx-xmlrpc
action = route
logpath = /var/log/nginx/access.log
maxretry = 3
findtime = 10800
bantime = 86400

The problem is that I was keeping myself locked out every so often during development. So I decided to whitelist myself. Unfortunately, my ISP provides dynamic ips, so I had to associate a hostname to zonomi and use DDNS to update my subdomain with my new assigned IP addresses every so often. I then added my hostname to my ignoreip entry on the local jail as this:

# MISCELLANEOUS OPTIONS
#

# "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not
# ban a host which matches an address in this list. Several addresses can be
# defined using space separator.
ignoreip = 127.0.0.1/8 my.hostname.sub.domain

Today I was working on the webserver and I got blocked, so I checked my hostname and it hasn't updated my IP. I manually did it and after the DNS spread over and the hostname's ip changed, I tried to access my website/server with no success.

It appears to me that either: (1) once the ban was set I would have to restart fail2ban to flush the block on my IP (which I dislike due to the fact that all the other IP's blocked are forgotten, the real threats) or (2) somehow fail2ban wasn't able to update my hostname's associated IP.

My question is: If it's (1), is there a way to lift the block automatically without restarting fail2ban or, in case it's (2), is there a way to update my hostname's ip automatically?

Does fail2ban uses IPTABLES? Should I cron a chain flush with my hostname's IP on iptables every minute?

Kindly, A.

Answer 1

hi i was a victim of the same attack i never posted on my stackoverflow account so here is my 1st post to help you with how i resolve this issue.

and i felt better about it as i did research on the xmlrpc.php and determined it is a deprecated feature in WordPress im not ever going to use and however you cannot remove the file as its a core object and WordPress goes haywire when removed.

in the end this resolved my issue by not trying to catch traffic to xmlrcp.php but by removing the venerability and then measuring and banning those trying to use it.

---

Step1 deny access to xmlrpc.php

edit your nginx config for the enabled site

nano /etc/nginx/sites-enabled/YOURCONFIG.conf

add the following lines inside your config for this site

this will reject all access to this file, it will reject the traffic and log it as http error code 403 where it is then picked up in fail2ban in later.

location = /xmlrpc.php {
    deny all;
}

---

Step2 configure fail2ban to watch and ban 403 events due to excessive xmlrpc.php requests 404 added also because we need it

actions to create definitions:

touch /etc/fail2ban/filter.d/nginx-403.conf
touch /etc/fail2ban/filter.d/nginx-404.conf
nano /etc/fail2ban/filter.d/nginx-403.conf
nano /etc/fail2ban/filter.d/nginx-404.conf

definitions contain:

/etc/fail2ban/filter.d/nginx-403.conf

##nginx-403.conf
[Definition]
failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 403
ignoreregex =

/etc/fail2ban/filter.d/nginx-404.conf

##nginx-404.conf
[Definition]
failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 404
ignoreregex =

edit jail.conf to add in the new definitions:

nano /etc/fail2ban/jail.conf

add following lines for triggers:

/etc/fail2ban/jail.conf

[nginx-403]
enabled = true
port    = http,https
logpath = /var/log/nginx/access.log
maxretry = 5
findtime = 300

[nginx-404]
enabled = true
port    = http,https
logpath = /var/log/nginx/access.log
maxretry = 10
findtime = 300

---

Step3 reload configs so new changes take effect

after configs edited reload services for changes to take effect

systemctl reload nginx.service fail2ban.service

ensure services started correctly by checking the status

systemctl status nginx.service fail2ban.service

---

Step4 validate impact

use what ever method you like to see the nginx access logs i use:

tail /var/log/nginx/access.log -f
OR
tail /var/log/nginx/access.log -f|grep xmlrpc

you should see the traffic get logged as a new error code:

• from the original http code 200 in bold

• 35.xxx.xx.xxx - - [11/May/2023:21:52:08 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"

• to the new http code 403 in bold

• 35.xxx.xx.xxx - - [11/May/2023:21:52:09 -0400] "POST /xmlrpc.php HTTP/1.1" 403 548 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"

use what ever method you like to review the sys logs i use:

journalctl -ef|grep fail2ban

watch fail2ban pickup the new http code 403 definition and trigger the ban:

see the bot get banned

May 11 22:22:11 myhostnamehere.Com fail2ban.actions[17191]: NOTICE [nginx-403] Ban 35.xxx.xxx.xx

[eof] done.

my web server CPU and php process feels better. but now im off to address the 300 redirect spam now happy banning!

Answer 2

Fail2Ban uses iptables. As per fail2ban's documentation, it allows whitelisting based on hostname or ip addresses: http://www.fail2ban.org/wiki/index.php/Whitelist

You should use a Dynamic DNS service, set a small TTL for your hostname (like 600 which amounts for 10 minutes). You can go even with 300 (it's not complaint with the standard but it will the job). Then check and see. If your DDNS hostname was created with a default TTL which in most of the cases for A records is between 3600 and 14400 (1 hour - 4 hours) then that might be the reason.