Skip to main content

Me and varnish win against a DDOS attack.

This past month one of my servers experienced her first DDOS - a distributed denial of service attack. A denial of service attack (or DOS) just means an attempt to shut down an internet-based service by overwhelming it with requests. A simple DOS attack is usually relatively easy to deal with using the standard linux firewall called iptables.  The way iptables works is by filtering the traffic based on the incoming request source (i.e., the IP of the attacking machine). The attacking machine's IP can be added into your custom ip tables 'blacklist' to block all traffic from it, and it's quite scalable so the only thing that can be overwhelmed is your actual internet connection, which is hard to do.

The reason a distributed DOS is harder is because the attack is distributed from multiple machines. I first noticed an increase in my traffic about a day after it had started - it wasn't slowing down my machine, but it did show up as a spike in traffic. I quickly saw that a big chunk of traffic was all of the same form - a POST to a domain that wasn't actually in use except as a redirect. There were several requests per second, and each attacking machine would do the same request about 8 times. So it was coming from a lot of different machines, making it not feasible to manually keep adding in these ip's into my blacklist.

It certainly could have been a lot worse. Because it was attacking a domain that was being redirected, it was using up an apache process, but no php, so it was getting handled very easily without making a noticeable dent in regular services. But it was worrisome, in case the traffic picked up. It was also a curious attack - why make an attack on an old domain that wasn't even in use? My best guess is that it was either a mistake, or a way of keeping someone's botnet busy. I have heard that there are a number of these networks of "zombie" machines, presumably a kind of mercenary force for hire, and maybe if there are no contracts, they get sent out on scurrilous missions to keep them busy.

In any case, I also thought a bit about why Varnish wasn't being useful here. Varnish is my reverse-proxy protective bubble for my servers (yes, kind of like how a layer of varnish protects your furniture). The requests weren't getting cached by Varnish because in general, it's not possible to responsibly cache POST requests (which is presumably why a DDOS would favour this kind of traffic). To see why, just imagine a login request , which is a POST - each request will have a unique user/pass and the results of the request will need to get handled by the underlying CMS (Drupal in my case).

But, in this case, I wasn't getting any valid POST requests to that domain anyway, so that made it relatively easy to add the following stanza to my varnish configuration:

 if (req.http.host ~ "example.com" && req.request == "POST") {
   return (lookup);
 }

And indeed, now all the traffic is bouncing off my varnish and I'm not worrying. In case it was a domain that was actively in use, I could have added an extra path condition (since no one should be POST'ing to the front page of most of my domains anyway), but it would have started getting trickier. Which is why you won't find Varnish too helpful for DDOS POST attacks in general. As usual, the details matter, and in this case, since I was being attacked by a collection of mindless machines, the good guys won.

Popular posts from this blog

The Tyee: Bricolage and Drupal Integration

The Tyee is a site I've been involved with since 2006 when I wrote the first, 4.7 version of a Drupal module to integrate Drupal content into a static site that was being generated from bricolage. About a year ago, I met with Dawn Buie and Phillip Smith and we mapped out a number of ways to improve the Drupal integration on the site, including upgrading the Drupal to version 5 from 4.7. Various parts of that grand plan have been slowly incorporated into the site, but as of next week, there'll be a big leap forward that coincides with a new design [implemented in Bricolage by David Wheeler who wrote and maintains Bricolage] as well as a new Drupal release of the Bricolage integration module . Plans Application integration is tricky, and my first time round had quite a few issues. Here's a list of the improvements in the latest version: File space separation. Before, Drupal was installed in the apache document root, which is where bricolage was publishing it's co...

A Strange Passion for Security

I'm not a computer security expert, but it's been part of my work for many years, in different forms.  A very long time ago, a friend hired me to write up a primer for internet security, and ever since then it's been a theme that's sat in the background and pops up every now and then . But lately, it's started to feel like more than a theme, and but indeed a passion. You may consider computer and internet security to be a dry subject, or maybe you imagine feelings of smugness or righteousness, but "passion" is the right word for what I'm feeling. Here's google's definition: Passion: 1. a strong and barely controllable emotion. 2. the suffering and death of Jesus. Okay, let's just go with number 1. for now. If you followed my link above to other posts about security, you'll notice one from eight years ago where I mused on the possibility of the discovery of a flaw in how https works. Weirdly enough, a flaw in https was discovered shortly...

IATS and CiviCRM

Update, Nov 2009: I've just discovered and fixed a bug I introduced in the 2.2 branch for the IATS plugin. The bug was introduced when i updated the API files from IATS and failed to notice that the legacy method for C$ one-time donations was no longer supported. If you're using a version greater than or equal to 2.2.7, and are using IATS for C$, non-recurring donations, then you're affected . To fix it edit the file : CRM/Core/Payment/IATS.php, and remove the line that looks like this: $canDollar = ($params['currencyID'] == 'CAD'); //define currency type The full fix removes a conditional branch based on that value a little further on, but by removing this line, it'll never actually use that branch. Drop me a line if you have any questions. Update, May 2009: This post is still getting quite a bit of traffic, which is great. Here are a few important things to note: The IATS plugin code is in CiviCRM, you don't need to add any code. Y...