Frequentist, Bayesian, or Other?

For the discussion of math. Duh.

Moderators: gmalivuk, Moderators General, Prelates

jwwells
Posts: 180
Joined: Thu Sep 21, 2006 4:47 am UTC

Frequentist, Bayesian, or Other?

Postby jwwells » Sun Jul 11, 2010 6:58 am UTC

I've never really liked frequentist probability. It never made sense to me in any but the most specialized cases. But Bayesianism seems only a little better to me, and the betting approach feels downright psychological.

(This is not a serious discussion, except where it is. If you do not understand probability better than I do, you should not read this, because I will mislead you. If you do understand probability better than I do, please correct me so that I do not pollute impressionable young minds. Or just shake your head and sigh, that's okay, too.)

Frequentism:

PROS:

* Intuitive; statistically handy; doesn't require notions of prior probability; may even be objective.
* Dominant interpretation, as far as I can tell - lots of scientists seem to run with it, even if it bugs me so much that I want to turn into a horrible forum crank and post twenty pages of PROOFS THAT IT IS WRONG.
* Would appeal to most people who just want to get something useful done.

CONS:

* "There emerges the vision of the fair coin, the biased coin. This coin exists in some mental universe and all modern writers on probability theory have access to it. They toss it regularly and they speculate about what they 'observe.'"

- P. Davis and R. Hersh, quoted entirely out-of-context from here.

What are we looking at when we imagine an 'infinite set of trials?' What's the reference set of possible outcomes for these trials? If there's a fifty-fifty chance of rain tomorrow, does it mean that days similar to tomorrow would have rain half the time, or if tomorrow were somehow repeated 100 times, we'd see rain in 50 of those hypothetical trials on average? Or can we not talk about probability of rain on a specific day at all?

-

Bayesian probability:

Pros:

* Takes evidence into account in an adaptive way; doesn't require lots of trials; seems to fit everyday situations on some level
* Would appeal to Mr. Spock.
* All the cool kids are doing it, come on.

Cons:

* "There is a 95.23323% probability that you are Kira. I know this because I am a Bayesian."

- L in the anime Death Note (does not actually say this, but might as well do so.)

-

Betting approach:

Pros:

* Jibes well with human psychology; doesn't require priors.
* Would appeal to Captain Kirk. (May be a con for some)

Cons:

* If we all go crazy in the right way, the probability of the earth turning into a chicken overnight is 1.
* "This spam is promising me 10^100000000000000000000000000000000000000 dollars. Responding costs me practically nothing... and even if the probability of it being true is nil, could it really be less than 1 in 10^100000000000000000000000000000000000000? I don't believe in infinitesimals, so it sure sounds like a rational bet!"

(Responder is lured to foreign country and killed with disturbingly finite probability.)

-

Meh approach:

Pros:

* Good enough for most.

Cons:

* "Well, I mean, if I don't know either way, the odds of there being a woman president next election are 50/50. And if I don't know either way, the odds of there being a black president next election are 50/50. So the odds of there being a black woman president are 25%..."

* "No, seriously, I have this system where I double my bet every time I lose!"

-

Where do you stand, and why?

(For the record, I am a super-Reform Bayesian. Orthodox Bayesians are careful and rigorous. Reform Bayesians try to make reasonable assumptions, but stretch it a bit to make stuff like phylogenetic analysis possible so they can draw neat evolutionary trees. Super-Reform Bayesians say, "you know, I bet a Bayesian approach would be good here - hey, this program just gave me a Bayesian probability that is so cool it must be right")

User avatar
Dason
Posts: 1311
Joined: Wed Dec 02, 2009 7:06 am UTC
Location: ~/

Re: Frequentist, Bayesian, or Other?

Postby Dason » Sun Jul 11, 2010 10:53 pm UTC

I added some comments/questions in blue

jwwells wrote:Frequentism:

PROS:

* Intuitive; statistically handy; doesn't require notions of prior probability; may even be objective.
* Dominant interpretation, as far as I can tell - lots of scientists seem to run with it, even if it bugs me so much that I want to turn into a horrible forum crank and post twenty pages of PROOFS THAT IT IS WRONG.

You have a proof that the frequentist approach is wrong?


Bayesian probability:

Cons:

* "There is a 95.23323% probability that you are Kira. I know this because I am a Bayesian."

- L in the anime Death Note (does not actually say this, but might as well do so.)

This is all you could come up with for cons?

Betting approach:

Can you define what your "betting approach" is. What exactly do you mean by this?


(For the record, I am a super-Reform Bayesian. Orthodox Bayesians are careful and rigorous. Reform Bayesians try to make reasonable assumptions, but stretch it a bit to make stuff like phylogenetic analysis possible so they can draw neat evolutionary trees. Super-Reform Bayesians say, "you know, I bet a Bayesian approach would be good here - hey, this program just gave me a Bayesian probability that is so cool it must be right")
I guess I still don't understand what you mean by super-Reform Bayesian.


I'm a frequentist myself if you couldn't tell. I'm not a big fan of the bayesian approach to most things. I can see where it might be useful but I've never found a situation where I felt I needed to take a bayesian approach.
double epsilon = -.0000001;

User avatar
antonfire
Posts: 1772
Joined: Thu Apr 05, 2007 7:31 pm UTC

Re: Frequentist, Bayesian, or Other?

Postby antonfire » Sun Jul 11, 2010 11:47 pm UTC

How about when you were watching the weather and they said there's a 70% chance of rain tomorrow? Or do you just ignore statements like that because they're meaningless?
Jerry Bona wrote:The Axiom of Choice is obviously true; the Well Ordering Principle is obviously false; and who can tell about Zorn's Lemma?

User avatar
Velifer
Posts: 1132
Joined: Wed Dec 26, 2007 4:05 pm UTC
Location: 40ºN, 83ºW

Re: Frequentist, Bayesian, or Other?

Postby Velifer » Mon Jul 12, 2010 12:54 pm UTC

Bayarri, M.J. and J. O. Berger. 2004. The Interplay of Bayesian and Frequentist Analysis. Statistical Science v.19 no.1 58-80.
(and a link to a similar paper from them)

Can't we all just get along?
Time flies like an arrow, fruit flies have nothing to lose but their chains -Marx

User avatar
jestingrabbit
Factoids are just Datas that haven't grown up yet
Posts: 5967
Joined: Tue Nov 28, 2006 9:50 pm UTC
Location: Sydney

Re: Frequentist, Bayesian, or Other?

Postby jestingrabbit » Mon Jul 12, 2010 2:21 pm UTC

Velifer wrote:Can't we all just get along?


Probably not.
ameretrifle wrote:Magic space feudalism is therefore a viable idea.

User avatar
Charlie!
Posts: 2035
Joined: Sat Jan 12, 2008 8:20 pm UTC

Re: Frequentist, Bayesian, or Other?

Postby Charlie! » Mon Jul 12, 2010 11:03 pm UTC

I'm a bayesian when dealing with small numbers of events, and a frequentist when dealing with groups. Especially when the details of the experiment are well-known, frequentist logic provides big shortcuts at low loss in accuracy.
Some people tell me I laugh too much. To them I say, "ha ha ha!"

User avatar
Velifer
Posts: 1132
Joined: Wed Dec 26, 2007 4:05 pm UTC
Location: 40ºN, 83ºW

Re: Frequentist, Bayesian, or Other?

Postby Velifer » Tue Jul 13, 2010 1:17 pm UTC

Most people are intuitively Bayesian, but generally bad at stats. Look at how often people will make the gambler's fallacy (which I think of as incorrectly setting a prior). It's just how people intuit things. They bring all the information they have available to bear on a problem (including extraordinality biases like what happened to uncle Bob back in '68 and other informational errors). They certainly don't think of multiverses and strict counterfactuals. Ask an intro stats student to define a confidence interval, and nine times out of ten (p=.0132 CI90 0.82-1.1) you'll get the definition of a Bayesian credibility interval. They are intuitive. Frequentist confidence intervals require wacky mental gymnastics.

I use frequentist analysis when I can get away with it. It's simple, clean, quick, and accepted by other researchers. I use Bayesian methods when I need a different wrench.

<troll style:css-voice-set="Pompous English Ass">
...and besides, things like frequentist stats or the scientific method are just special cases of Bayesianism.
</troll>
Time flies like an arrow, fruit flies have nothing to lose but their chains -Marx

Dynotec
Posts: 89
Joined: Fri Dec 15, 2006 2:13 pm UTC
Contact:

Re: Frequentist, Bayesian, or Other?

Postby Dynotec » Tue Jul 20, 2010 7:29 pm UTC

I try to avoid all of the philosophical gymnastics and look at Bayesian Statistics as a generalized version of Maximum Likelihood Estimation with priors.

mouseposture
Posts: 42
Joined: Tue Sep 15, 2009 2:42 am UTC

Re: Frequentist, Bayesian, or Other?

Postby mouseposture » Wed Jul 21, 2010 1:20 am UTC

Velifer wrote: Frequentist confidence intervals require wacky mental gymnastics.


All this week, I've been worrying that the p value I had calculated correctly, could become retroactively incorrect later on, if I computed another p value -- but that this would either happen, or not, depending on my intention, or lack of same, to choose the lower of the two values.

User avatar
Dason
Posts: 1311
Joined: Wed Dec 02, 2009 7:06 am UTC
Location: ~/

Re: Frequentist, Bayesian, or Other?

Postby Dason » Wed Jul 21, 2010 2:55 am UTC

mouseposture wrote:
Velifer wrote: Frequentist confidence intervals require wacky mental gymnastics.


All this week, I've been worrying that the p value I had calculated correctly, could become retroactively incorrect later on, if I computed another p value -- but that this would either happen, or not, depending on my intention, or lack of same, to choose the lower of the two values.

I have no idea what you're trying to say there. But p-values don't ever become "retroactively incorrect". It's possible that you would conduct another experiment and then get a different p-value but that doesn't mean that the p-value you got before is somehow incorrect.
double epsilon = -.0000001;

SU3SU2U1
Posts: 396
Joined: Sun Nov 25, 2007 4:15 am UTC

Re: Frequentist, Bayesian, or Other?

Postby SU3SU2U1 » Wed Jul 21, 2010 4:08 am UTC

Anyone who does science is forced to be just a little Bayesian- all we really have is induction, how do you decide between competing theories without some Bayesian thought?

mouseposture
Posts: 42
Joined: Tue Sep 15, 2009 2:42 am UTC

Re: Frequentist, Bayesian, or Other?

Postby mouseposture » Wed Jul 21, 2010 10:25 pm UTC

Dason wrote:I have no idea what you're trying to say there.

You should have stopped there.

User avatar
Dason
Posts: 1311
Joined: Wed Dec 02, 2009 7:06 am UTC
Location: ~/

Re: Frequentist, Bayesian, or Other?

Postby Dason » Wed Jul 21, 2010 10:38 pm UTC

mouseposture wrote:
Dason wrote:I have no idea what you're trying to say there.

You should have stopped there.

A p-value is a number computed from data. It itself is a random variable. Unless you do the actual computation wrong it doesn't make sense for a p-value to be "incorrect". You can make an incorrect decision but the p-value itself is just a number. You may do another study later, get a different p-value, and then make a different decision but that doesn't make your previous calculation "retroactively incorrect".
double epsilon = -.0000001;

User avatar
Velifer
Posts: 1132
Joined: Wed Dec 26, 2007 4:05 pm UTC
Location: 40ºN, 83ºW

Re: Frequentist, Bayesian, or Other?

Postby Velifer » Thu Jul 22, 2010 1:32 pm UTC

Dason wrote:You may do another study later, get a different p-value, and then make a different decision but that doesn't make your previous calculation "retroactively incorrect".

Looks like a teachable moment...

In the case of clinical research, sometimes the drug is so harmful that the clinical trial needs to be prematurely stopped. Sometimes it's so incredibly beneficial that it's no longer ethical to withhold it from the control group. So during the trial, there are often a number of statistical analyses done to check for this. It's called "optional stopping."

A good frequentist should throw righteous fits about this, since this turns the single trial into a serialized design that should require different methods. A Bayesian will just shrug and wonder what the fuss is about, since nothing really changes except the statistician. The additional knowledge can even be useful. There's some philosophical quibbling about frequentist p-values violating the likelihood principle. Significance tests violate the principle, and assumptions made in different experimental designs will yield different significance values for the same sample.
Time flies like an arrow, fruit flies have nothing to lose but their chains -Marx


Return to “Mathematics”

Who is online

Users browsing this forum: No registered users and 11 guests