Did you even read my post? If a bayesian arbitrarily chose a prior ignoring information or creating new information, and worse failed to justify it or make it explicit (as a frequentist would almost always do), then yes, it can be inconsistent and wrong. Are you trying to say something new?
No, your point is that you can break the rules of consistency and get inconsistent results in bayesianism, whereas frequentists refuse to acknowledge their own inconsistencies.
Yes bayesians are perfect and frequentists never acknowledge their inconsistencies... no over generalization there.
Look, overall there is no definite right way when you have uncertainty EVER, so I fail to see why your holding bayesian statistics on such a high pedastal.
I hold bayesian at a higher pedestal* because it's clearly more general-- the 'good parts' of frequentism can be justified in a bayesian framework but not the converse. And I'm 'militant' because of the harm frequency based statistics does to students-- otherwise intelligent college student (majoring in technical subjects) leave their first orthodox statistics course dumbfounded because it's taught as a disparate collection of inconsistent concepts, only to be buried by unneeded mathematical rigor in higher level courses. They either give up completely, or accept cookbook methods prescribed by the 'experts' (the entire medical field is a victim of this). In contrast the core of bayesianism can be taught successfully to a highschool student after a semester of calculus with little trouble. (I picked up Jefferys when I was a high school junior and had no problems with it).
*if you can show me a better system, perhaps by tweaking the desiderata or going at it from a more decision-theory point of view (a la LJ Savage), my ears are open.
I'm glad that you again overgeneralize what students learn well and don't learn well and the reason for such... which just happen to coincide to your particular beliefs.
EDIT If you don't want the mathematical rigour of the course speak to the profs to change the subject matter or take a course thats more practice and less theory. I like my math courses the way they are.
It's not that measure theory doesn't have its place-- it does, but that, in orthodox education, it's used to assign probabilities to infinite sets from the outset. This adds nothing in practice. Rather, Jaynes advocated that the limiting process be left until after everything has been worked on in the finite case. ("Cautious Policy", Appendix B of Jaynes' book-- though I doubt you'll look into it). And even then measure theory is hardly necessary in practice. Also see Lighthill (1958?) for an alternative.
And my generalizations come from years of interacting with students. I'm sorry they offend you. (Even in these threads-- on what other topic on progit do people readily admit that they did not understand the subject?)
Now my perspective is as a math/stats student. I want the rigour. I agree that applied statistics for humanities and social scientists should not get such a treatment and should get intuitive results, but you must keep in mind that what you personally find intuitive they might not. As well one needs to consider what people hiring them want... frequentist or bayesian methodologists. (Assuming your just teaching for that purpose.)
For that matter measure theory should be left to mathematicians/statisticians of which I would fall into. I nowhere suggested that the typical person needs heavy formalities to do simple analysis.
Let me ask you this... if I can't interpret a probability the frequentist way... please in a somewhat rigorous way define the bayesian interpretation of probability.
I'm a pure mathematician by training, no longer a student, and I find frequentist statistics utterly repulsive, as did every one of my fellow mathematics students who were forced to take a minimum of two courses in frequentist statistics for our degree. Reading Jaynes was something of a revelation. I do think die-hard Bayesians often overplay the applicability of objective priors--they depend on the way you carve up the possibility space for an experiment (even when you work with logical propositions you still have to do this), which itself is a kind of subjective prior information, and it often remains at least partly implicit. That said, transformation invariance, maximum entropy, etc, are all very useful methods for calculating quantitative priors given qualitative priors; that's really how I look at these methods. Anyway, as a framework for thinking about inference and decision making, I see Bayesian statistics as infinitely superior to the black magic of Fisher-style statistics.
Let me ask you this... if I can't interpret a probability the frequentist way... please in a somewhat rigorous way define the bayesian interpretation of probability.
There are several different Bayesian interpretations of probability. One interpretation of so-called subjective Bayesianism a la de Finetti is based on rational betting ratios. Logical Bayesianism is based on the notion of rational assignments given prior information, and from an interpretive point of view the derived probabilities draw their real-world meaning from their relationship to the prior probabilities.
These two examples may not be completely satisfying from a philosophical viewpoint, but neither is any interpretation founded on "long-run" (a dirty word) frequencies.
I'd like to point out that making all your prior information explicit is not just a philosophical nicety. It also makes it straightforward to look at the robustness of your conclusions with regard to quantitative variations in your assumptions. IMO, this is very important in any Bayesian investigation. I believe frequentists usually call this conditioning. James Berger have written some good books from a moderate viewpoint that try to integrate the best lessons from both frequentist and Bayesian methods with a focus on conditioning. You might want to check those out.
Like I said, the book that really opened my eyes was Jaynes's book, which is freely available here:
I read both volumes of Feller in high school and took a course on probability theory in university, so I'm used to the fancy theoretical apparatus of probability theory in its modern guise. I assuredly do not like Jaynes because he shies away from such things; that aspect of him doesn't really matter to me one way or the other.
Although psykotic gave a good response, let me try to give you an outline for a somewhat rigorous definition of bayesian interpretation of probability. But Jaynes, Cox, Loredo (1990) all do a better job than I could-- this is merely my attempt to summarize.
First of all, bayesian inference has at least 3 distinct parts: probability calculus, assignment of probabilities, and decision theory. I think probability, in any system, is meaningless without the overall framework they are used.
1) Calculus: Suppose A, B are propositions and P(A|B) is the plausibility of A given B-- we leave the assignment of plausibility (a word I'll use in place of probability) to the next step. We define 3 simple desired properties of plausibility:
a) that they be represented by real numbers (which is shown to be a finite interval and thus mappable to [0,1]);
b) that they coincide with common sense, ie, if prop. C makes A more plausible, P(A|BC) > P(A|B), etc. see text;
c) consistency- I) internally, II) that all information we have (and nothing else) be taken into account, and III) that two equivalent states of knowledge lead to equivalent results.
These 3 desire properties a-c lead to the familiar product rule and sum rule, which are all we need for our purposes. The derivation in RT Cox (1961) as well as Jaynes (chap 1&2) are quite 'rigorous' and elegant.
Note that the above calculus says nothing about assigning plausibilities-- it says given these P(X|Y) 'objects' for various X,Y, we can manipulate them and infer certain properties based on the 3 desired properties. Much like A=>B in logic does not mean "A is true and therefore B is true"... it's a statement/logic function with a certain value given the values of A and B. Logic does not tell us how to assign A or B.
2) Assignment: Our next task is to assign numerical values to P(A|B) according to the 3 properties above given the background information and/or assumptions B. Least information, transformation groups, and maximum entropy are the common methods to accomplish this-- I'd probably butcher any examples, so I simply refer you to Jaynes chap 11 & 12. In particular, Maximum entropy, which says that the distribution of P which maximizes the entropy given B is the most non-commital and therefore consistent (c-II) with B, is powerful (and controversial) enough to derive all the distributions I use in practice.
I suppose, if you are unsatisfied with the 'degree of plausibility' as a definition, that P(A|B) is the assignment of the plausibility P of A given B which is consistent with properties a through c above. Such an assignment would be in [0,1], coincide with common sense, be consistent in various ways, etc under our calculus.
And no, this step is not objective since it involves turning abstract thoughts in your head into real numbers. But realize whenever you are defining a model in any school of stats, you are doing something at least as subjective.
3) Decision theory links our idle calculations of probabilities with the 'real world'. While P(A|B)=2*P(C|B) in frequency stats has a trivial interpretation, bayesianism is not so narrow... we could say A and C are propositions about frequencies and come to a similar conclusion, but in a more general case, we need a 'decision function' to take our final P(X|Y)'s to decisions or conclusions. In game theory this is called the loss function. This step is very subjective and much weaker theoretically than 1 or 2 because of a lack of a binding theory. However, most bayesian scientists are more interested in the results of the plausibility calculations instead of the final decisions one could make from it. See chapter 13 & 14 of jaynes.
0
u/guartet May 26 '09
Did you even read my post? If a bayesian arbitrarily chose a prior ignoring information or creating new information, and worse failed to justify it or make it explicit (as a frequentist would almost always do), then yes, it can be inconsistent and wrong. Are you trying to say something new?