Well, for example some of the beliefs, such as the creation of the world, is based on the scientific theory of the "big bang" rather than a creator. Also, it's foundation is that of very logical and non-superstitious mannerisms. The idea of luck, for example is disregarded; here's a quote from Dalai Llama himself:
"I know a person who makes a living selling lucky charms. He claims that his charms can give good luck, prosperity and he guarantees that you will be able to pick three numbers. But if what he says is true then why isn't he himself a multi-millionaire? If his lucky charms really work, then why doesn't he win the lottery week after week? The only luck he has is that there are people silly enough to buy his magic charms."
And, as for the God idea, from Ven S. Dhammika:
"Primitive humans found selves in a dangerous and hostile world, the fear of wild animals, of not being able to find enough food, of injury or disease, and of natural phenomena like thunder, lightning and volcanoes were constantly with them. Finding no security, they created the idea of gods in order to give them comfort in good times, courage in times of danger and consolation when things went wrong. To this day, you will notice that people become more religious at times of crises, you will hear them say that the belief in a god or gods gives them the strength they need to deal with life. You will hear them explain that they believe in a particular god because they prayed in time of need and their prayer was answered. All this seems to support the Buddha's teaching that the god-idea is a response to fear and frustration. The Buddha taught us to try to understand our fears, to lessen our desires and to calmly and courageously accept the things we cannot change. He replaced fear, not with irrational belief but with rational understanding.
The second reason the Buddha did not believe in a god is because there does not seem to be any evidence to support this idea. There are numerous religions, all claiming that they alone have god's words preserved in their holy book, that they alone understand god's nature, that their god exists and that the gods of other religions do not. Some claim that god is masculine, some that she is feminine and others that it is neuter. They are all satisfied that there is ample evidence to prove the existence of their god but they laugh in disbelief at the evidence other religions use to prove the existence of another god. It is not surprising that with so many different religions spending so many centuries trying to prove the existence of their gods that still no real, concrete, substantial or irrefutable evidence has been found. Buddhists suspend judgement until such evidence is forthcoming.
The third reason the Buddha did not believe in a god is that the belief is not necessary. Some claim that the belief in a god is necessary in order to explain the origin of the universe. But this is not so. Science has very convincingly explained how the universe came into being without having to introduce the god-idea. Some claim that belief in god is necessary to have a happy, meaningful life. Again we can see that this is not so. There are millions of atheists and free-thinkers, not to mention many Buddhists, who live useful, happy and meaningful lives without belief in a god. Some claim that belief in god's power is necessary because humans, being weak, do not have the strength to help themselves. Once again, the evidence indicates the opposite. One often hears of people who have overcome great disabilities and handicaps, enormous odds and difficulties through their own inner resources, through their own efforts and without belief in a god. Some claim that god is necessary in order to give man salvation. But this argument only holds good if you accept the theological concept of salvation and Buddhists do not accept such a concept. Based on his own experience, the Buddha saw that each human being had the capacity to purify the mind, develop infinite love and compassion and perfect understanding. He shifted attention from the heavens to the heart and encouraged us to find solutions to our problems through self-understanding."
And how the universe was made:
"All religions have myths and stories which attempt to answer this question. In ancient times, when humankind simply did not know, such myths were adequate, but in the 20th century, in the age of physics, astronomy and geology, such myths have been superseded by scientific fact. Science has explained the origin of the universe without recourse to the god-idea."
Buddhism is very scientifically and/or factually based for the majority of it. Of course, being a philosophy there is going to be dubious areas, like your question about reincarnation and karma.