The serious mathematical study of the idea of probability was started in the 17th Century, by Pascal and Fermat, and a little later by Bernouilli, who produced one of its first treatises, the "Ars Conjectandi", that also stands at the beginning of the mathematical study of statistics. There were some forerunners, like Cardano and De Witt, who were respectively concerned with games of chance and the profitability of insurance, but it remains a somewhat curious fact that although the concept of chance is very old, since the ancient Greeks already knew it and wondered about it, the mathematics of chance is relatively new.

After Pascal, Fermat and Bernouilli, quite a lot was done in probability by mathematicians - De Moivre, Laplace, Gauss, Poisson, Boole, Pearson, to name some - but the whole subject only got a sound axiomatic foundation by A. Kolmogorov in the 1930ies, who proposed the following axioms for it.

Let s and t be arbitrary formulas, and let "p(.)" be a function that is read "the probability of":

K1. 0 <= p(s) <= 1.
K2. If s is logically true, then p(s)=1.
K3. If s logically implies t, then p(s) <= p(t).
K4. If s and t are logically inconsistent, then p(s or t)=p(s)+p(t).

These axioms allow the derivation of all theorems that were commonly accepted by mathematicians for finite probabilities.

Since then, the mathematics of probability is a branch of what is known as measure theory, which is concerned with sums and integrals, and which has a fine abstract introduction by P. Halmos: "Measure Theory".

Also, Kolmogorov's axiom 4 is supplemented or strengthened in such measure theoretical treatments by an axiom that allows the formation of infinite sums, which is required for many applications and theorems.

Even so, the philosophical foundations of probability are far less certain than the by now widely received mathematics of it, and there are quite a few distinct interpretations of what probability and/or chance really are, which differ from the assumption that in the end they are wholly subjective or a mere measure of human ignorance to the assumption that there really is chance and indeterminism in nature, and probability theory represents it, to various positions inbetween.

What increased these problems in the 20th Century were the arisal of indeterminism in physics (quantum mechanics) and the arisal of social statistics and the testing of hypotheses, both of which have a fairly firm mathematical foundation in measure theory, but a far less firm foundation when one inquires into their possible factual truth or validity.

There are very many texts about probability and its interpretations. If we suppose for the moment that the mathematics of it is widely accepted and well expounded by measure-theoretical mathematical texts, or textbooks that are derived from this treatment, there still remain many problems with what it all might mean. Seven able and interesting expositions are:

W. Stegmüller: Probleme und Resultaten der analytischen und Wissenschaftsphilosophie.
E. Adams: A Primer of Probability Logic.
C. Howson & P. Urbach: Scientific Reasoning - The Bayesian Approach.
R. Weatherford: Philosophical Foundations of Probability Theory.
A.A. Sheshnikov: Problems in Probability Theory, Mathematical. Statistics and Theory of Random Functions.
H. Freudenthal: Waarschijnlijkheidstheorie.
T. Fine: Theories of probability.