Skip to main content

Confused by online payment processing? You're not alone.

In the old days during "polite" conversation, it was considered rude to talk about sex, politics, religion and money. You might think we're done with taboos, we're not (and I'll leave Steven Pinker to make the general argument about that, as he does so well in The Better Angels of Our Nature).

The taboo I'm wrestling with is about money - not how much you make, but about online payment processing, how it works, and what it costs. In this case, I think the taboo exists mainly because of the stakes at hand (i.e. lots of money) and the fact that most of those who are involved don't get much out of explaining how it really works - i.e. the more nuanced communications are overwhelmed by sales-driven messaging, and the nuanced stuff is either proprietary secrets or likely to get slapped down by the sales department.

In other words, if you want to really understand about online payment processing because you want to decide between one system and another, you're in for a rough ride.

Several years ago I wrote a couple of payment processors for CiviCRM, and more recently I've been working on a new version of one of them. At the same time, two clients have recently been trying to understand their existing payment processor services in order to integrate those processes into their CiviCRM installations. So this is my "Payment Processor primer for CiviCRM administrators" blog post.

What You Need To Know

Here's a simplified but useful model of what's happening. A typical online payment has three phases, and each phase may be the responsibility of a different (or the same) service provider. I'm talking about a typical real-time transaction via credit card - other flavours will introduce new complications.

Phase 1: The Form

The web form is the public interface where the visitor inputs things like a name and credit card number. Sometimes, it's a two part form. Depending on the transaction, you'll want this form customized so that your visitor doesn't get confused and leave. The "depending" bit is really about your visitor's relationship to you. If they already know and love you, it probably matters less. If they're new and not yet convinced they want to give you money, it's big.

CiviCRM can support the form, but also supports payment processors that insist on doing the form themselves (e.g. paypal standard). The big advantage to CiviCRM doing the form is customization and not-alarming-or-confusing-the-visitor (e.g. the paypal form allows credit cards, but many people get to that form and bail because they think they need to sign up for a paypal account). The big disadvantage is that you need to worry about your server and something called PCI compliance, which is another topic.

Phase 2: The Transaction Processing or The Payment Gateway

This phase starts after the visitor clicks the submit or confirm button and may happen entirely in the background, or may involve the visitor in supplementary forms and clicking. This phase is the responsibility of a "Payment Gateway", a service that you have to buy unless you're a large corporation that builds their own. This payment gateway service has business contracts and software relationships with the phase 3 section. The key services they provide are to abstract away the individual details of the different card company interfaces and to take responsibility for financial compliance stuff (e.g. they need to keep those credit card numbers very, very safe ...).

CiviCRM does not try and do this, but provides interfaces to many payment gateways and in theory allows you to write an interface to any payment gateway that publishes some kind of "API" or programmer interface. It can be confusing because many payment gateways also try to be in the business of providing Phase 1 services (e.g. 'hosted forms') and it may not be obvious if there is such a thing as an API, and sometimes they call it something else.

An excellent, more technical description is here on wikipedia:

Phase 3: Transaction Completion

This is the murkiest phase, where the payment gateway, the institution behind the card (issuing bank or card association) and the "merchant account" all exchange information and some kind of electronic trail gets laid that eventually results in money being transferred from the card holder to the "merchant account". The "merchant account" is a special kind of bank account that is enabled for credit card payments. What makes it special is that the credit card companies have a noose around it's neck - i.e. they take a chunk of money before it gets to the account, and reserve the right to take the money back if there's a problem. The "merchant account" might be directly owned by you the site owner, or it might be owned by someone else who then passes the money to you.

It's not unreasonable to confuse this phase with phase 2 since they happen together, and since phase 2 without phase 3 is kind of pointless, but it's important to separate it out in terms of costs and responsibilities. Phase 2 is really a technical and business relationship service that is handling the details of the transaction (kind of like an electronic teller, or maybe the hired gun in a drug deal). Phase 3 is where the money ends up and is accounted for (the backroom settling of accounts ...).

It's also important to separate them out because you can have phase 3 stuff going on without phase 2. For example: a 'recurring transaction' where a donor says they'll give you money every month. Once the initial deal is sealed, the subsequent transactions don't need to go through phase 1 or 2 (but might anyway).

What You Actually Get and How Much it Costs

So the challenge of comparing various payment processor "solutions" is to figure out the apples and oranges. With CiviCRM you have to buy services from at least one company in order to get online payments from credit cards, but any company you find may offer a mix of services covering these three phases. Paypal standard will only give you a soup-to-nuts end-to-end solution. A merchant account will only get you the last phase and you still need a payment gateway service. If you don't have an ssl certificate for your site, you will need phase 1 services, but if you do have ssl, you probably don't want phase 1 services. Most payment gateway services will offer to bundle in a merchant account and/or a hosted payment form service. And each of these offerings will be better than the competition for reasons x, y and z. And each one will use a different vocabulary to describe what they are giving you.

So, here's what you should expect and look at:

1. Phase 1 services. You probably shouldn't pay anything for this unless they are providing really good customization - it should be a one-time and change fee for the customization of the form. Getting CiviCRM to host this form is usually a better idea unless you're doing cheap hosting and/or can't get an ssl certificate.

2. Phase 2 services. These will typically cost a monthly or yearly fixed fee, sometimes a set up fee, and probably always a per-transaction fee. There's no particular reason that there should be a % fee, since the costs of providing these service are basically per transaction + a setup and account maintenance, unless the company is trying to do some kind of gambling, which is stupid.

3. Phase 3 services. The merchant account part of the service is really about paying off the card mafioso, plus an extra handling charge to the bank. Each major card has it's own rates that they charge the bank, but MasterCard and Visa are similar and Amex costs about an extra 1%. If you're a small-medium organization, then you'll probably pay a pretty standard amount, but if you're a large organization, you can usually negotiate a better rate, which is just going to reduce the extra amount that the bank charges you. It'll never go below the industrial rate, which is complicated (i.e. it fluctuates and depends on lots of things), but I'd hazard lives at around 1.5% (why not? for a start, consider all those points reward systems out there and put that together with a certainty that card companies aren't losing money ...).

One thing that this model helps you do is to compare the bundled services, which will typically be the monthly or yearly + per transaction costs of the payment gateway plus the % costs of the merchant account. You can sometimes see how they gamble on the % costs and give you a single 'blended rate', and sometimes gamble on the size by shifting costs back and forth from per transaction to % rates.

Popular posts from this blog

Orchestrating Drupal + CiviCRM containers into a working site: describing the challenge

In my previous posts, I've provided my rationale for making use of Docker and the microservices model for a boutique-sized Drupal + CiviCRM hosting service. I've also described how to build and maintain images that could be used for the web server (micro) service part of such a service.

The other essential microservice for a Drupal + CiviCRM website is a database, and fortunately, that's reasonably standard. Here's a project that minimally tweaks the canonical Mariadb container by adding some small configuration bits:

That leaves us now with the problem of "orchestration", i.e. how would you launch a collection of such containers that would serve a bunch of Drupal + CiviCRM sites. More interestingly, can we serve them in the real world, over time, in a way that is sustainable? i.e. handle code updates, OS updates, backups, monitoring, etc? Not to mention the various crons that need to run, and how about things like…

Building and maintaining Drupal + CiviCRM application containers

In my previous two posts, I provided some background into why I decided on using containers for a boutique Drupal + CiviCRM hosting platform, and why Docker and its micro-services approach is a good choice for building and maintaining containers.

Although I promised to talk about orchestration, that was getting ahead of the story - first I'm going to look at the challenge of keeping your application containers up-to-date with OS and application-level updates. There's a fair amount of work in that, but the tooling is mature and there is lots of good documentation.

A great place to start is to visit the official Drupal docker hub page. From there, you can pull a working Drupal code container, and it gets re-built frequently with all the OS and Drupal-code updates, so you just refresh your containers whenever you want (i.e. whenever a security release comes out, or more often to stay up-to-date).

A nice thing about that project is that it demonstrates a technique for maintaining …

The Tyee: Bricolage and Drupal Integration

The Tyee is a site I've been involved with since 2006 when I wrote the first, 4.7 version of a Drupal module to integrate Drupal content into a static site that was being generated from bricolage. About a year ago, I met with Dawn Buie and Phillip Smith and we mapped out a number of ways to improve the Drupal integration on the site, including upgrading the Drupal to version 5 from 4.7. Various parts of that grand plan have been slowly incorporated into the site, but as of next week, there'll be a big leap forward that coincides with a new design [implemented in Bricolage by David Wheeler who wrote and maintains Bricolage] as well as a new Drupal release of the Bricolage integration module.PlansApplication integration is tricky, and my first time round had quite a few issues. Here's a list of the improvements in the latest version:File space separation. Before, Drupal was installed in the apache document root, which is where bricolage was publishing it's content. This …