Permanently Blocking IP Addresses with AWS WAF Rate-Based Rule Limit
Preventing DDoS Attacks and Permanently Blocking Attacker IP Addresses Using AWS WAF Rate-Based Rule Limit
What is WAF?
AWS WAF is a web-application firewall that lets you control access to your content based on the criteria that you specify. In simpler words, it protects your web-applications from threats and provides you ability to control the access of traffic that is visiting your Site.
What is this blog about?
AWS WAF provides multiple ways to manage access to the traffic coming to your web-application. One of the most important things we can do with this is to stop DDoS (Distributed Denial-of-Service). Rate-based rule that counts incoming requests and rate limit requests when they are coming too fast a rate. We can tweak the rules according to our requirement and prevent DDoS attacks on our server. But there are some limitations to it and in this blog, we are going to see what the limitations are and how we can surpass these limitations.
WAF and other resource Pricing
WAF is not a free service and there is no free tier for that. But if you are learning then for testing it is really very cheap. During my testing period it cost me nearly $0.18. and if you want to set up permanently then refer to the official documents from AWS.
Few things need to note down first
AWS WAF can only be used with CloudFront distributions and Regional Resources.
Regional Resources:
- Amazon API Gateway REST API
- Application Load Balancer
- AWS AppSync GraphQL API
- Amazon Cognito user pool
- AWS App Runner service
- AWS Verified Access instance
PS: Here I am implementing this whole thing for application load balancer.
Creating Web ACL and Rules
In this section we will create Web ACL with rule that blocks IP which exceed defined rate limit.
- Go to AWS WAF → Web ACLs → Select region in which your load balancer or regional resource → Create Web ACL.
2. Select resource type: (for me) Regional resources, Region: Asia Pacific (Mumbai), Name: xxxx
3. Add application load balancer: Add Resources → Application Load Balancer → Select alb.
4. Add Rate-base Rules: Add rules → Add my own rules and rule groups.
5. Rule Set-Up that block IPs based on specified rate limit:
Rule type: Rule builder
Type: Rate-based rule
Rate Limit: 100 (You can specify according to your traffic)
Evaluation window: 1 minute (That means if someone make more than 100 requests in 1 minute then their IP will be blocked)
Request aggregation: IP address in header (Why this option? Let’s discuss it later in this blog)
Header field name: X-Forwarded-For
6. Create Rule → Set rule Priority → Enable sampled request.
Here we have created Web ACL with a rule that blocks IP addresses if it exceeds the rate limit of 100 requests per minute. Are we done??? Nope.
Problem with this rate-base rule
Don’t know why but for some kind of reason this rate-base rule blocks IP addresses very well but for a few minutes only. After a few minutes it releases blocked IPs and the attacker again able to attack our server.
Solution?
Get help from Lambda Functions. There is another service from AWS WAF called IP sets. IP sets are a set of IP addresses stored in the IP set. We can utilize these IP sets to create new Web ACL rule that allows or blocks IPs from these IP sets.
Here’s what we are going to do:
- Create an IP Set that contains IP addresses of attackers.
- Create rule in Web ACL that blocks IP addresses from attackers IP set.
- Create Lambda Function that reads the list of Blocked IPs by Rate-Base Rule, and Add those IPs to the attacker's IP set.
- Create one Eventbridge Rule that triggers lambda functions every minute. So, we can keep attackers IP set list updated with newly blocked IPs.
Too much hectic tasks? Don’t worry CloudFormation has your back. I have created CloudFormation template which will create IP set, Lambda Function, IAM roles and permissions and Eventbridge Rule.
Creating Lambda, IP set, IAM role and Eventbridge rule.
Using CloudFormation template we will create IP set, Lambda Function, IAM roles and permissions and Eventbridge Rule.
- You can find CloudFormation template here.
- Go to CloudFormation → Stacks → Create Stack → With New Resources (Standard).
- Prepare template: Choose and existing template. Template source: Upload a template file. (here I have saved template to the S3 so using the S3 URL but you can directly upload .YAML file.
4. Give name to the stack and fill the following Parameters.
- RateBasedRuleName: Name of our Rate-base- rule created in Web ACL.
- Scope: REGIONAL ( I am using ALB so it is regional resource.)
- WebACLId: It can be found at Web ALC dashboard (AWS WAF → Web ACLs)
- WebACLName: Name of Web ACL.
5. Keep everything else default and at the end Select the checkbox of “I acknowledge that AWS CloudFormation might create IAM resources” and Submit.
Now IP set, Lambda function, IAM roles and Eventbridge Rule will be created by CloudFormation. We have to create Rule that block IPs from created IP set.
Create IP set blocking rule
We are going to create a new Web ACL rule that blocks IP addresses from the IP set created by CloudFormation.
- Go to AWS WAF → Web ACLs → your Web ACL → Rules → Add my own rules.
2. Block reuqest from the IPs of attackers IP set.
- Rule type: IP set
- Name: xxxxx
- IP set: Select IP set create by CloudFormation.
- IP address to use as the originating address: IP address in header
- Header field name: X-Forwarded-For
- Position inside header: Any IP addresses
- Action: Block
3. Set priority of rule: select this rule and move this rule to the top.
We are done with our implementation. Now, it's time for testing.
Testing
I am using ApacheBench for sending 200 requests in a minute to my server. FYI, I have created one EC2 instance running on Ubuntu and going to use it for testing, you can install this on your local machine as well.
- Install ApacheBench on Ubuntu.
sudo apt-get update
sudo apt-get install apache2-utils- Creating shell script that send 200 requests in minute to the domain (in other word to the server)
#!/bin/bash
for i in {1..20}
do
ab -n 10 -c 1 -H "X-Forwarded-For: 1.2.3.4" http://your-server-endpoint/
sleep 3
done- To run script run following commands
chmod +x test.sh
./test.shNow check the IP set list (AWS WAF → IP sets → …) there you can find this 1.2.3.4/32 IP which means everything is working properly, (Maybe list won’t get update immediately as lambda triggers at every 1 minute.
Why are we using ‘X-Forwarded-For’?
For a moment assume that you are DDoS attacker. What are the things that need to be taken care of when you are DDoSing? Your IP address. As an attacker your IP address is a very important thing that you don’t want to reveal to the victim or anyone else. So, to hide their IP attacker use HTTP proxies (Zombies) to attack webserver. HTTP proxies will send requests to the webserver on behalf of attackers. HTTP proxies are hard to find as they may be situated at any random geo-location and their IPs keep changing. The diagram below is a very simple representation of DDoS attacks.
In this case, blocking Source IP Addresses is not helpful at all, as source IP addresses are constantly changing and attackers are still able to attack your server easily from another HTTP Proxies.
Is it impossible to stop DDoS attacks?
Nope, as per the research no clear pattern could be found as IPs were keep changing and from entirely different locations across the planet, but one thing was clearly visible after inspecting HTTP requests, they all had header that indicates that they are being forwarded from HTTP proxies: ‘X-Forwarded-For’.
Any requests that are coming from HTTP proxies or load balancer have this common header ‘X-Forwarded-For’, which contains the originating client IP. The meaning of this header is that the request is being sent on behalf of the originating IP. This header is only present if the request is coming from HTTP proxies or load balancer. This header is non-standard, there is no guarantee you’ll get it, and the way it is handled can differ on the proxy implementation. You also have no guarantee that it will contain a proper IP.
Advantages of using ‘X-Forwarded-For’:
- If you use Source IP address to block request then there is a high chance that your regular user might get blocked if they exceed limit, while the attacker will still be able to increase load of your server by using proxies.
- ‘X-Forwarded-For’ header is only present in requests that are coming from HTTP proxies or load balancer, making it easier to differentiate between regular users and users with bad intentions.
References
