The Algebra of Probable Inference — Richard T. Cox

This is a lovely, lovely book and I can’t believe it has taken me this long to find and read it (November 2005: I was lead to this book via Jaynes, who was the author that also recommended Stove). Cox, a physicist, builds the foundations of logical probability using Boolean algebra and just two axioms, which are so concise and intuitive that I repeat them here:

1. “The probability of an inference on given evidence determines the probability of its contradictory on the same evidence.”

2. “The probability on given evidence that both of two inferences are true is determined by their separate probabilities, one on the given evidence, the other on this evidence with the additional assumption that the first inference is true.”

Cox then begins to build. He shows that logic can be and is represented by probability, the type of function probability is, the relation of uncertainty and entropy, and what expectation is. He ends with deriving Lapace’s rule of succession, and argues when this rule is valid and when it is invalid. And he does it all in 96 pages!. This is one of the rare books that I also recommend you read each footnote. If you have any interest in probability or statistics, you have a moral obligation to read this book.

The Rationality of Induction — David Stove

Is deductive logic empirical? (No) Is inductive logic also empirical? (No) Is induction justified and, if so, is it just an extension of logic? (Yes)

Stove takes Hume (and his nuttier current-day followers such as Popper) to task and shows that, yes, induction is rational. He also shows that the common belief that ordinary is formal is a myth. Knowledge of the validness of certain arguements must come from intution, as Carnap argued, and Stove proves. He shows that certain forms of logical arguments do not always give valid conclusions, and that all arguments must be judged individually. In his words, “Cases Rule”.

This is another in a series of books that I think are largely unknown by most statisticians and probabilists, especially those who tend toward so-called pure mathematics. But this book, like Jaynes and Cox, argue the case for logical, as opposed to subjective, probability forcefully and conclusively. They deserve to be more widely read because, I believe, they have a great deal to say on the foundations of our field.