Skip to main content


Managed Services: A Creative Tension

What Are Managed Services? " Managed services " is an offering of many businesses that provide Internet services. In the past couple of years, it's a term I've used to describe what I offer in my Blackfly Solutions Drupal and CiviCRM hosting business . You may not know whether you would want such a thing, since it's a very badly named thing.  This post will try and give a reason for why managed services is a thing at all, how it can be a good thing, and why it may be what you want. Here's the short version: managed services exist to fill the gap between what machine automation can reasonably provide and what people actually want. In a subsequent post, I'll explain how containers can be a useful tool for delivering managed services. What Do You Mean by Services? A "service" in the "managed services" context is the consumer-oriented one, i.e. something a consumer wants that they get from a service provider. For example: "hosting for
Recent posts

Successful mass mailing

Happy families are all alike; every unhappy family is unhappy in its own way. - Leo Tolstoy Like happy families, successful mass mailings are easily taken for granted as normal, but they are more an exception than a rule. Most mass mailings fail in one or more ways. And the ways that a mass mailing can fail are probably more diverse and interesting than you think. So when you ask "why am I not receiving my mass mail" or "so-and-so isn't getting my mass mail", it's rarely a simple answer. Even if it used to work fine. To put some perspective on this - if you're getting only 10% of your mass mail opened, it's not bad. Industry-wide, a 7% click through seems to be about average. In this post, I'm going to follow a piece of mass mail and show you all the different ways it can fail to be successful. I use CiviMail for my mass mailings, and but I think most of it will be tool agnostic.  To keep it simple, I won't try and go through all the ways you

An apache OOM (out of memory) emergency in a container

On Sunday last, a (Linux) server in my infrastructure that was running a fairly conservative number of docker containers in production was brought to its knees. The monitoring data (from prometheus) showed that cpu was all gobbled up (from an average of less than 2% to a steady 75%-ish) and remained gobbled up until the server was rebooted. Notably, the disk usage and throughput went down during the event, and memory usage did not change notably, nor was it notable high. On review of the messages log, one of the last entries before the event was documentation of an apache OOM (out of memory) event. On this server, apache is only running inside containers, which are generally limited to 500Mb (by docker). So presumably, a docker container running apache ran out of memory and tried to recover some memory and that was what triggered the event. Reviewing the log of requests before the emergency, it's not clear which container or url or urls might have been generating so much memory u

Hello Ruby

Rebuilding the Garage, v. 2

13 years ago, I spent a few weeks rebuilding my garage, as documented on this blog . At the time, I chose the less expensive clear plastic roofing tiles, because I wasn't really convinced the project was going to work and I imagined I'd have to tear it all down and replace it with a real garage in a few years. But sure enough, some projects last a lot longer than you expect, and it's still there, more due to benign neglect than any plan. Here are a few photos to give you an idea of how it's weathered the time. "Post-apocalyptic" is a good description. Of course, 13 years is a long time, and in the interim, my 5 year old is now 18 years old, and bigger, stronger and in some ways smarter than I am. He's off to university in the fall and had a gap in his summer employment, so I've employed him to fix it up. I'm looking forward to the outcome as well as the process.

Orchestrating Drupal + CiviCRM containers into a working site: describing the challenge

In my previous posts, I've provided my rationale for making use of Docker and the microservices model for a boutique-sized Drupal + CiviCRM hosting service. I've also described how to build and maintain images that could be used for the web server (micro) service part of such a service. The other essential microservice for a Drupal + CiviCRM website is a database, and fortunately, that's reasonably standard. Here's a project that minimally tweaks the canonical Mariadb container by adding some small configuration bits: That leaves us now with the problem of "orchestration", i.e. how would you launch a collection of such containers that would serve a bunch of Drupal + CiviCRM sites. More interestingly, can we serve them in the real world, over time, in a way that is sustainable? i.e. handle code updates, OS updates, backups, monitoring, etc? Not to mention the various crons that need to run, and how about things

Building and maintaining Drupal + CiviCRM application containers

In my previous two posts, I provided some background into why I decided on using containers for a boutique Drupal + CiviCRM hosting platform, and why Docker and its micro-services approach is a good choice for building and maintaining containers. Although I promised to talk about orchestration, that was getting ahead of the story - first I'm going to look at the challenge of keeping your application containers up-to-date with OS and application-level updates. There's a fair amount of work in that, but the tooling is mature and there is lots of good documentation. A great place to start is to visit the official Drupal docker hub page . From there, you can pull a working Drupal code container, and it gets re-built frequently with all the OS and Drupal-code updates, so you just refresh your containers whenever you want (i.e. whenever a security release comes out, or more often to stay up-to-date). A nice thing about that project is that it demonstrates a technique for mainta