Skip to main content

Docker: Putting things together and pulling them apart

My favourite fictional scenes involve groups of people eating around a table, behaving badly. I think of Margaret Atwood's Edible Woman, or the movie "August: Osage County". Or the tea party in Alice in Wonderland.

What works well in narrative is often the opposite of what works for computers - the worst computer mess-ups often involve a collection of badly behaving pieces that manage to make a mess much larger than any of the badly behaving pieces could do on their own.

As a timely example, a client of mine that ill-advisedly used a generic host for their WordPress site recently had the site go down a few times. The host first told them they were a victim of a Denial of Service attack, but now thinks it was due to an incompatibility between their server and some backup software that was generating an unexpectedly high load. At the same time, the contact at the service provider was trying to fly down to Brazil for a family emergency and had gotten stuck in an airport due to weather.

This is of course one of the hardest problems with complexity - the unexpected interaction between different pieces, each of which can break under the wrong circumstances.

Which brings me to Docker, and Dockerfiles, and my second installment about diving into the world of Linux containers. My previous post detailed how I came to embrace Docker and containers as the solution (in theory) to my goal of providing managed Drupal/CiviCRM hosting for non-profits.

The Dockerfile is a text file that tells the Docker engine how to build a container "image". An "image" is a file on disk that is used to launch the container, which is then the live copy of the image. It's live in the sense that code is running in the container, and it's a virtualised operating system running inside a "host" operating system.

After running the usual hello world type scenarios, my first serious Dockerfile was an attempt to replicate my current old server. So I started with a CentOS base image, and then replicated some of the steps I'd normally go through to build a server - i.e. using yum to install various packages, adding some additional repositories, and some configuration customization. The simplest way to do that is just to use the RUN command. My mental model of what's happening here is: it creates a container from the base image, runs all the RUN commands, and then writes the result back down as an image.

If I used my new image in place of my current server, it would be a nice improvement - I could implement each site in a separate container from this image and gain the efficiency benefits of the shared container image.

But the reason I started this blog post with my narrative examples is because Docker can and should provide more than this. The "Docker Way" is microservices, which means that instead of having one container that does everything an individual server used to do, each container is only responsible for one microservice, and these are assembled together into the application. There are lots of good arguments for this on the Internet, but what helped me embrace this idea was to think of a docker container as a more sophisticated analogue to a Unix "process". To go sideways for a second: recall that in programming, there was an upheaval in coding when people started to embrace object orientation. That change allowed us to use a more sophisticated mental model for bits of code that might previously have been implemented as a procedure or function - now we could bundle together multiple functions and properties into a single object that exposed only specific pieces to the rest of the application. In a similar way, a Docker container is now a more sophisticated version of what used to be a program that you'd install onto your server. Only now you can restrict the ways that it might unexpectedly interact with the rest of the things that make up your application.

So instead of a big Docker image that replicates a complex mini-server to power the website, the Docker Way is to create multiple Docker images that do the things the individual programs used to do. A minimal version of that is in fact two containers: a container for apache, and a container for mysql. A simple development environment for Drupal can happily use just that. Of course, in a production environment, it gets more complicated.

In fact, as you go down the rabbit hole of micro-services, another holy grail is disposability - i.e. you're not depending on any one container instance for your application to continue working. If you can do that, then you can start thinking of your containers as a layer of abstraction that doesn't need to think about the host it's on. If that seems over the top for your small website, then you're right, but remember that people are using containers to solve much different problems. To give you an idea about this: consider how when you log into gmail, your service is not tied to any specific machine (google has about a zillion machines, which are constantly rotating, undergoing maintenance, breaking down, etc.).

But even for our smaller scale, it's a nice benefit that allows us to do other cool stuff which we can talk about later. The obvious question is - if your container is disposed of, what happens to the stuff that you were working on, ah, like your website contents? That turns out to be a challenge. One tool that simplifies the problem a bit is a thing called a "volume" container - a special container that holds all the stuff you don't want to loose, and that doesn't get disposed of when you dispose of the container that uses it. That allows you to only worry about the persistence of the volume, but creates the new problem of managing the volume - not only keeping track of it, but also being able to share it with multiple containers if you're doing that sort of thing.

The other big issue that arises with containers and microservices is the "connecting them together" - not just the ability for them to talk to each other, but to handle the disposing of some of them, etc. This is pleasantly called "orchestration", and is really where the hard work starts.

Until next time, TTFN.


Popular posts from this blog

Confused by online payment processing? You're not alone.

In the old days during "polite" conversation, it was considered rude to talk about sex, politics, religion and money. You might think we're done with taboos, we're not (and I'll leave Steven Pinker to make the general argument about that, as he does so well in The Better Angels of Our Nature).

The taboo I'm wrestling with is about money - not how much you make, but about online payment processing, how it works, and what it costs. In this case, I think the taboo exists mainly because of the stakes at hand (i.e. lots of money) and the fact that most of those who are involved don't get much out of explaining how it really works - i.e. the more nuanced communications are overwhelmed by sales-driven messaging, and the nuanced stuff is either proprietary secrets or likely to get slapped down by the sales department.

In other words, if you want to really understand about online payment processing because you want to decide between one system and another, you'…

Me and varnish win against a DDOS attack.

This past month one of my servers experienced her first DDOS - a distributed denial of service attack. A denial of service attack (or DOS) just means an attempt to shut down an internet-based service by overwhelming it with requests. A simple DOS attack is usually relatively easy to deal with using the standard linux firewall called iptables.  The way iptables works is by filtering the traffic based on the incoming request source (i.e., the IP of the attacking machine). The attacking machine's IP can be added into your custom ip tables 'blacklist' to block all traffic from it, and it's quite scalable so the only thing that can be overwhelmed is your actual internet connection, which is hard to do.

The reason a distributed DOS is harder is because the attack is distributed from multiple machines. I first noticed an increase in my traffic about a day after it had started - it wasn't slowing down my machine, but it did show up as a spike in traffic. I quickly saw that…

IATS and CiviCRM

Update, Nov 2009: I've just discovered and fixed a bug I introduced in the 2.2 branch for the IATS plugin. The bug was introduced when i updated the API files from IATS and failed to notice that the legacy method for C$ one-time donations was no longer supported.
If you're using a version greater than or equal to 2.2.7, and are using IATS for C$, non-recurring donations, then you're affected.
To fix it edit the file : CRM/Core/Payment/IATS.php, and remove the line that looks like this:

$canDollar = ($params['currencyID'] == 'CAD'); //define currency type The full fix removes a conditional branch based on that value a little further on, but by removing this line, it'll never actually use that branch. Drop me a line if you have any questions.
Update, May 2009: This post is still getting quite a bit of traffic, which is great. Here are a few important things to note:
The IATS plugin code is in CiviCRM, you don't need to add any code.You do still …