Block User-Agent on AWS Elastic Beanstalk

Question

I am running a Django application on AWS Elastic Beanstalk. I keep having alerts (for days now) because the following user agent constantly tries to access some pages: "Mozilla/5.0 Jorgee"

[30/Jul/2017:13:55:55 +0000] "HEAD /pma/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:55 +0000] "HEAD /db/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:56 +0000] "HEAD /admin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:56 +0000] "HEAD /mysql/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:56 +0000] "HEAD /database/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:57 +0000] "HEAD /db/phpmyadmin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:57 +0000] "HEAD /db/phpMyAdmin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:57 +0000] "HEAD /sqlmanager/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:58 +0000] "HEAD /mysqlmanager/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:58 +0000] "HEAD /php-myadmin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:58 +0000] "HEAD /phpmy-admin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:59 +0000] "HEAD /mysqladmin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:59 +0000] "HEAD /mysql-admin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"
[30/Jul/2017:13:55:59 +0000] "HEAD /admin/phpmyadmin/ HTTP/1.1" 403 - "-" "Mozilla/5.0 Jorgee"

It used to return 404, but I managed to block it to 403 thanks to the following line in settings.py:

DISALLOWED_USER_AGENTS = (re.compile(r'Mozilla\/5.0 Jorgee'), )

Is there a way to simply block it from even getting to the Django level? Or a way to stop getting it written to the logs? It generates Health Check alerts :-/

Answer 1

You could use AWS WAF rules to block anything you want from there. Nevertheless I assume that you've created environment with Classic Load Balancer. Migration to the Application Load Balancer is not a pain indeed. And what it does is way more than Classic Load Balancer, with some chances for upgrade in the future. If you would consider moving out to the ALB (we did this long time ago, so every new environment has it), the there's easy way to block these things out, or just move them aside so they don't interrupt.

For us, the easiest approach was to define AWS WAF rule to have string condition matching the Host header against some defined domains we publish our sites with. If there's any request, that is not using this domain, it will get 403 and won't generate things like requests to the ELB are failing with 5xx on Elastic Beanstalk. If you take a closer look at logs on ELB level, these bots tend to use direct IP adresses, not DNS names, so it generates errors when accessing website with defined server_names in Nginx, for instance. So thanks to this, we have clearer logs and there's no more health warning on EB environment, which was a standard before.

I described this case in details here https://www.mkubaczyk.com/2017/10/10/use-aws-waf-block-bots-jorgee-500-status-elastic-beanstalk if you need some guidance or screenshots.

Answer 2

Try to use ModSecurity , Where you can control your traffic to apache server

https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual

https://github.com/SpiderLabs/owasp-modsecurity-crs/tree/v3.0/master/rules

add user agent to be blocked to this file in your server

https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.0/master/rules/crawlers-user-agents.data

Answer 3

You could create an AWS Web Application Firewall with a rule to reject traffic using that user agent string. Then attach the WAF to the Elastic Load Balancer in your Elastic Beanstalk environment.

Alternatively, you could create a rule in the reverse-proxy running on your Elastic Beanstalk EC2 instances to block that traffic before it gets to Django. I'm not sure if Django apps on EB default to using Apache or Nginx for the reverse proxy. You'd have to figure out which one you are using and then look up how to configure that to block traffic based on a user-agent string.

It's not clear to me how this traffic is causing health check alerts in your application. If it is spamming your app with so much traffic that your server becomes overloaded and unresponsive, then I would recommend using a WAF to block it so that your server(s) will never have to see the traffic at all.