Using varnish in front of a webhosting canada or other shared hosting site? Pay attention to your probe!
I've had a varnish installation protecting http://www.fairvote.ca -- a wordpress site hosted at webhosting canada (whc.ca) -- for about a year. Last Friday in the evening, it started spitting out 503 errors, for no obvious reason. I spent a few hours yesterday in vain trying to get the hosts to tell me what, if anything might have changed at their end.
What I eventually figured out is that they changed how their servers respond to http requests without a domain - previously it was a 200, and it became something else (a redirect, I believe). That's not unreasonable (though relatively pointless to change at this stage, I would imagine), except that, by default, varnish probes its backends with requests that don't have a domain. What that meant is that varnish started thinking that the backend was broken, so stopped talking to it (even though it was perfectly capable of doing so).
The fix was just to add a customized probe that included a valid domain to the request.
What I eventually figured out is that they changed how their servers respond to http requests without a domain - previously it was a 200, and it became something else (a redirect, I believe). That's not unreasonable (though relatively pointless to change at this stage, I would imagine), except that, by default, varnish probes its backends with requests that don't have a domain. What that meant is that varnish started thinking that the backend was broken, so stopped talking to it (even though it was perfectly capable of doing so).
The fix was just to add a customized probe that included a valid domain to the request.