Doubtless you've heard this little nugget of inanity from more than one indignant apologist, and it's usually the sort of thing they resort to when everything else they've thrown your way has been flattened. The glib response is usually something along the lines of, "Yeah, like baldness is a hair color!" Then this is followed by tortuous explanations where you find yourself trying to describe the difference between belief that there are no gods and disbelief in gods, to a mind not exactly skilled in grasping nuance.
But there's an easier way to deal with this one, a way even Christians might understand, and it's illustrated by a post today from PZ.
Atheism is not a religion for the same reason theism is not a religion. The terms refer solely to the disbelief or belief in gods. But religion implies a ritualized, or at least organized practice. Indeed, a person can be theistic and yet not the least bit religious. Theism is not a religion, but Christianity, Islam, et al, are.
Similarly, atheism is not a religion, but...there are atheistic religions. And they are just as irrational and lacking in evidence as theistic ones. Buddhism, on the whole, seems generally benign, though its embrace of such fantasies as reincarnation (which is something you're encouraged to avoid) puts it squarely in the realm of delusion and woo. But then there are the Raelians, a gang of raving nitwits who reject God...only to replace him with aliens. It may be generous even to call Raelianism (if that's the term) atheistic, since they just put God in a UFO and only reject the traditional notion of a supernatural god. But to some, and to themselves, they are considered atheistic on those grounds alone.
We at The Atheist Experience have all encountered self-proclaimed atheists who go on to voice their eager support for other irrational ideas, like 9/11 Trutherism or "alternative" medicine.
So no, atheism itself is not a religion. But there are atheist religions, and there are individual atheists just as lost to reason and confused as many theists. It isn't enough to reject gods simply because you don't like Pat Robertson or the Pope or the Tea Party or what have you. Skepticism and critical thinking must inform everything you do. A person can get to atheism by means other than critical thinking, but it's possible to adopt even ideas that are right for the wrong reasons. Put critical thinking first, and atheism should not only flow naturally from that, but it will have a much more sound intellectual footing, and you'll be better inoculated against other slippery falsehoods that sneak through the back door of your confirmation bias too.