Let me start off by saying that I don't think there is an atheist out there who knows with absolute certainty that there isn't a God. We can't know anything with total certainty. Therefore, things are weighed via probability. I personally would put the probability of God's existence at about a googolth (a decimal place followed by 99 zeroes and a 1), where 1=total certainty of existence and 0=total certainty of non-existence.
I don't believe in God for a number of reasons.
First, the same reasons you would use to say it is unreasonable to believe in things like invisible dragons in your garage and Santa and a teapot in orbit between Earth and Mars also tell you that 1) theism is also unreasonable, and 2) theism is more unreasonable than those things.
Then, there's Occam's Razor. If supposedly "God created the Universe and God always existed", why not go with the simpler position of "the Universe always existed". The Big Bang, by the way, marks the beginning of time, so "always existed" and "has existed for a finite time" are not contradictory.
Thirdly, if God is used as an explanation of natural phenomena then you inevitably run into the problem of explaining the complexity of something with the greater complexity of something else (God). If complexity needs an explanation then you are worse off by answering complexity with greater complexity. Some have tried to answer this by saying that, in some theologies, God is seen as supremely simple. However, just because one is labelled or called something doesn't make it true a priori. Just because Nazis called themselves socialists doesn't mean they weren't really fascists. God's supposed simplicity can be questioned by noting that a God who can't make a universe and physical laws is simpler then a God who can't.