Skip to main content

Varnish, Drual and ical (.ics): expiry issue fixed

My normal configuration of a public site on my servers involves using varnish for the page cache and setting expire page to 1 day. This mostly works quite well (the varnish module in Drupal takes care of clearing the varnish cache when you're creating/editing content).

We recently launched a new Drupal version of the Calgary French & International School (okay, I was just along for the tail end to help with the launch, Karin and Rob get the credit for all the work), which includes an ical feed for parents (generated from views of course).

That's an excellent thing - parents can subscribe to the feed and have all the upcoming events on their mobile device (or google calendar, or both). But we discovered that although it works great on the Mac desktop, it wasn't working well for iOS (i.e. the iPhone). It would poll frequently enough, but only actually update once a day.

It turned out that these two devices are interpreting the http header 'cache-control' differently - on the iphone, it appeared to interpret it to say don't both looking for fresh data more than once a day. The header is not very well defined unfortunately, but it is used by Drupal/Varnish to control the maximum expiry date, so we didn't want to crank it too low (or risk a badly performing site, since most access is anonymous).

The solution was actually simple: a little help in the varnish vcl file, in my vcl_deliver function, below. The piece I added was the second if, and it's just modifying the cache-control header on output if it's delivering a file with extension 'ics'.

sub vcl_deliver {
  if (resp.http.magicmarker) {
    unset resp.http.magicmarker;
    set resp.http.age = "0";
  }
  if (req.url ~ "\.ics$") {
    set resp.http.cache-control = "public, max-age=60";
  }
}



Popular posts from this blog

Confused by online payment processing? You're not alone.

In the old days during "polite" conversation, it was considered rude to talk about sex, politics, religion and money. You might think we're done with taboos, we're not (and I'll leave Steven Pinker to make the general argument about that, as he does so well in The Better Angels of Our Nature).

The taboo I'm wrestling with is about money - not how much you make, but about online payment processing, how it works, and what it costs. In this case, I think the taboo exists mainly because of the stakes at hand (i.e. lots of money) and the fact that most of those who are involved don't get much out of explaining how it really works - i.e. the more nuanced communications are overwhelmed by sales-driven messaging, and the nuanced stuff is either proprietary secrets or likely to get slapped down by the sales department.

In other words, if you want to really understand about online payment processing because you want to decide between one system and another, you'…

Me and varnish win against a DDOS attack.

This past month one of my servers experienced her first DDOS - a distributed denial of service attack. A denial of service attack (or DOS) just means an attempt to shut down an internet-based service by overwhelming it with requests. A simple DOS attack is usually relatively easy to deal with using the standard linux firewall called iptables.  The way iptables works is by filtering the traffic based on the incoming request source (i.e., the IP of the attacking machine). The attacking machine's IP can be added into your custom ip tables 'blacklist' to block all traffic from it, and it's quite scalable so the only thing that can be overwhelmed is your actual internet connection, which is hard to do.

The reason a distributed DOS is harder is because the attack is distributed from multiple machines. I first noticed an increase in my traffic about a day after it had started - it wasn't slowing down my machine, but it did show up as a spike in traffic. I quickly saw that…

drupal, engagement, mailing lists, email

I lived, worked and studied in Costa Rica from 1984 to 1989. Ostensibly, I was there to study Mathematics at the University, and indeed I graduated with an MSc. in Mathematics supervised by Ricardo Estrada (check that page, he even advertises me as one of his past students). And yes, I do have a nine page thesis that I wrote and defended in Spanish somewhere in my files, on a proof and extension of one of Ramanujan's theories. But mathematics is a pretty lonely endeavour, and what drew me back to Central America (after the first visit, which was more of an accident), was the life and politics. The time I lived there was extremely interesting (for me as an outsider, though also painful and tragic for it's inhabitants) because of the various wars that were largely fuelled by US regional hegemonic interests (of the usual corporate suspects and individuals) and neglect (of the politicians and public) - the Contra war in Nicaragua, the full-scale guerrilla wars in El Salvador and …