What-If 0065: "Twitter Timeline Height"

What if there was a forum for discussing these?

Moderators: Moderators General, Prelates, Magistrates

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26740
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby gmalivuk » Wed Oct 02, 2013 2:52 am UTC

Showsni wrote:Which is kind of my point; you end up with an obviously wrong answer because I'm not a proper random selection from the distribution. But then, doesn't the Doomsday thingy have exactly the same problem? We're using the number of total humans up to this date as our datum, which means we're using the humans alive today as our "random" choice. But this isn't a random selection from all the humans who will ever live, so it has the same error as thinking I might live to 520.
Hm, yeah. I see your point.

I guess like most controversial probability problems, it comes down to how exactly you're making your choice and what exactly you mean when you state probabilities.

On 5% of all the days of George Washington's life, it's true that he was less than 5% of the way through his life. For 5% of all humans, past present and future, it's true that they are among the first 5% ever to be born. On 5% of the days of George Washington's life, he would have been correct to believe that he'd live to be at least 20 times his current age. For 5% of all humans ever, it would be correct to believe that the total number of humans ever will be at least 20 times the number born before them. I think the difficulty lies in the fact that we can't help but have additional information in both cases. Washington at 50 obviously could be far more than 95% confident that he wouldn't live past 1000, and the last sterile vestiges of a post-nuclear-war humanity could be far more than 95% confident of being among the last humans ever born.

In the parlance of the German tank problem, this additional information should bias the estimator we use in the frequentist approach, and it should suggest something other than a uniform distribution for the Bayesian approach.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
willpellmn
Posts: 93
Joined: Wed Apr 21, 2010 11:05 am UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby willpellmn » Wed Oct 02, 2013 6:58 am UTC

gmalivuk wrote:Which is why you never cross the street or do anything else that might possibly kill you, right?


I would only fear to cross the street if cars were invisible. No car is so fast that I couldn't see and hear it coming and try to get out of the way.

The future may be unknowable, but it is not unpredictable.


Every predictive model I've ever seen (or at least every predictive model I've ever seen spelled out in formal terms; what I regard as "common sense" or "survival instinct" is technically a predictive model, but operates below the level of consciousness) seems to consist of nothing more than ridiculous intellectual wankery of the sort Randall is lampshading here. If you can reason based on today's population that the human race will probably die out sorta soonish, your reasoning sucks, because there's absolutely no sane way to predict such a thing based on just fiddling with a few raw numbers. You're just jumping through a few mathematical hoops and then congratulating yourself on how clever you are; it has nothing to do with the complex, messy, largely incomprehensible truths of reality, and those truths are what will determine whether humanity lives or dies. If we keep up the war and pollution, we could be dead in a century; if we build sustainable spaceships and make a vast galactic exodus, we could survive for billions of years. You can't just reduce all that data to "a 90% chance that history is mostly over already, for varying degrees of mostly". That's just complete nonsense.

If you truly believed it to be unpredictable, you wouldn't do anything you didn't thoroughly enjoy, because the only reason to do unenjoyable stuff is that you expect something better to happen as a result.


That is, as nearly as possible, exactly how I live my life. I can generally trust my employer, when I have one, to pay me every two weeks (because he's always done so in the past, for me and for large numbers of other employees, and is a big business which faces legal liability if it screws its staff members in too obvious a fashion), and so I don't mind working today for a payoff then. But I never try to calculate myself as having some percentage probability of not being fired within X numbers of months, because I know that a lot of factors, some under my control and some not, will determine how long I keep my job. I don't buy a car which will take five years to pay off, because that would be just asking to be fired and spend the next five years getting hassled by collections agencies about my unpaid bill. I don't do things which require a commitment that I am not extremely confident in my ability to fulfill, usually because I could do so today with my resources on hand, or very nearly so, combined with factors of how much I trust the business I'm working with and so forth. Like I said, I never gamble unless I'm comfortable with the losses, regardless of the supposed odds.

User avatar
Pfhorrest
Posts: 5399
Joined: Fri Oct 30, 2009 6:11 am UTC
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby Pfhorrest » Wed Oct 02, 2013 8:44 am UTC

Will, you talk about making choices based on past performance (of your employer), and based on your level of confidence (in your ability to fulfill commitments). Those are core concepts of statistics. Statistics just takes that kind of reasoning and applies a broad mathematical rigor to it, quantifying past performance to get a quantified level of confidence off of which to base a decision in a matter which your day-to-day intuition may not grasp.

I am similarly risk-averse as you, and there is a very good statistical reason for some people to be so risk-averse: the narrowness of the odds it makes rational sense to bet on is proportional to how long you can stand to keep losing.

Say there is some kind of activity which has a verifiable 51% chance of a positive outcome and a 49% chance of a negative outcome. The positive outcome is you win a thousand dollars, the negative outcome is that you lose a thousand dollars. For broke schmucks like me (and I would assume from your attitude probably you too), that's a horrible bet to take. You've got effectively a 50/50 chance of losing a thousand bucks, which can be a huge frickin' deal, and is not a loss you can suffer more than once, or a few times at most; and you've still got pretty high odds of losing a few times in a row before you win one at all. You can't afford to play long enough for the trend of you winning 2% of the time more to show up in your numbers.

But for a huge business with billions to gamble with, that's a great bet! You do a million deals a year, you lose about 490 thousand of them and thus $490 million (not a loss you or I could afford by any means!), but along the way somewhere in there you win on about 510 thousand other deals and win back $510 million, netting yourself a tidy $20 million net profit. It's a no brainer that you start doing whatever that risky activity is, when you can afford to eat the losses until the winning trend emerges.
Forrest Cameranesi, Geek of All Trades
"I am Sam. Sam I am. I do not like trolls, flames, or spam."
The Codex Quaerendae (my philosophy) - The Chronicles of Quelouva (my fiction)

rmsgrey
Posts: 3617
Joined: Wed Nov 16, 2011 6:35 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby rmsgrey » Wed Oct 02, 2013 11:49 am UTC

The core idea of the Doomsday Argument is that, given only the number of doohickeys manufactured to date, and no other information, you can say that we're probably somewhere in the middle of doohickey production rather than right at the start or right at the end.

The prediction it produces is a best-guess, given the single datum it accounts for. If we knew how to take proper account of the other data available, we'd be able to get a more reliable prediction - but in 19 out of 20 cases where all we have is a single production number chosen at random (assuming the numbers start at 1 and increase by 1 for each new item) the final production total will be within the bounds predicted by this method.

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26740
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby gmalivuk » Wed Oct 02, 2013 12:22 pm UTC

willpellmn wrote:
gmalivuk wrote:Which is why you never cross the street or do anything else that might possibly kill you, right?
I would only fear to cross the street if cars were invisible. No car is so fast that I couldn't see and hear it coming and try to get out of the way.
You predict, based on past experience. Which just amounts to a predictive model based on statistics without your having actually run any numbers.

Every predictive model I've ever seen (or at least every predictive model I've ever seen spelled out in formal terms; what I regard as "common sense" or "survival instinct" is technically a predictive model, but operates below the level of consciousness) seems to consist of nothing more than ridiculous intellectual wankery of the sort Randall is lampshading here. If you can reason based on today's population that the human race will probably die out sorta soonish, your reasoning sucks, because there's absolutely no sane way to predict such a thing based on just fiddling with a few raw numbers. You're just jumping through a few mathematical hoops and then congratulating yourself on how clever you are; it has nothing to do with the complex, messy, largely incomprehensible truths of reality, and those truths are what will determine whether humanity lives or dies. If we keep up the war and pollution, we could be dead in a century; if we build sustainable spaceships and make a vast galactic exodus, we could survive for billions of years. You can't just reduce all that data to "a 90% chance that history is mostly over already, for varying degrees of mostly". That's just complete nonsense.
You just made predictions there, when you said "if".

That is, as nearly as possible, exactly how I live my life. I can generally trust my employer, when I have one, to pay me every two weeks (because he's always done so in the past, for me and for large numbers of other employees, and is a big business which faces legal liability if it screws its staff members in too obvious a fashion), and so I don't mind working today for a payoff then.
Looks like a prediction reasoned from repeated past experience. Statistics without the mathematical rigor, in other words.

But I never try to calculate myself as having some percentage probability of not being fired within X numbers of months, because I know that a lot of factors, some under my control and some not, will determine how long I keep my job. I don't buy a car which will take five years to pay off, because that would be just asking to be fired and spend the next five years getting hassled by collections agencies about my unpaid bill. I don't do things which require a commitment that I am not extremely confident in my ability to fulfill, usually because I could do so today with my resources on hand, or very nearly so, combined with factors of how much I trust the business I'm working with and so forth. Like I said, I never gamble unless I'm comfortable with the losses, regardless of the supposed odds.
Yes you do. Walking across the street is gambling with your life, and you do it because you deem the odds sufficiently great against invisible oncoming cars.

Every time you say you trust something, this amounts to saying that thing is predictable enough for you to gamble with. Doing it without numbers just means you don't like math.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
cellocgw
Posts: 2046
Joined: Sat Jun 21, 2008 7:40 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby cellocgw » Wed Oct 02, 2013 12:58 pm UTC

To close out this thread (yeah, right):

90% of all posts about the Doomsday Thingie are incorrect. What are the odds that your post is correct?
https://app.box.com/witthoftresume
Former OTTer
Vote cellocgw for President 2020. #ScienceintheWhiteHouse http://cellocgw.wordpress.com
"The Planck length is 3.81779e-33 picas." -- keithl
" Earth weighs almost exactly π milliJupiters" -- what-if #146, note 7

Fateful
Posts: 2
Joined: Wed Oct 02, 2013 1:07 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Fateful » Wed Oct 02, 2013 1:41 pm UTC

Alright, I have included two arguments against the Doomsday Paradox, and one in favor. Please find them below:

The Doomsday Argument can Prove Any Number of Humans

Consider that the number 90% for the "middle" group was almost entirely arbitrary. As long as the "middle" group is 50% or more of the total population, guessing "middle" is still the best guess. I will show that by choosing an appropriate ratio for the "middle" group anywhere between 2N and infinite humans can be predicted.

Make the "middle" group size x. Then the percentage left over is no more than x (for the middle group) plus (1-x)/2, while the percentage already seen is no less than (1-x)/2.

Doing some of the same math seen in the "What If" post leads to a prediction of: Total = 2N/(1-x).

If you plot this function, it is easy to see that any value from 2N through infinity can be predicted using the appropriate x. Because any value can be produced, the argument is silly.


Fancier Statistics Gives an Infinite Expected Mean

This began as an attempt to resolve the above counter argument. We began by noticing that, given no information except the number of humans who have lived before us, there is a probability X that we are in the latter X portion. This leads to a probability X that there are N/(1-x) or less humans total. Reversing the formulas to solve for the probability that the total Nt is M or less gives a cumulative distribution function.

P(Nt <= M) = (M - N)/M for all M > N

Taking the derivative of the cumulative distribution gives the partial distribution:

p(Nt = M) = N/(x^2)

Then integral math can be used to compute the mean of the given partial distribution. That mean turns out to be: infinite! So a "rational" person given no other information would guess infinite people.

Means Aren't Good Measures

A counter argument points out that means aren't useful in this circumstance. However, other predictors aren't very helpful either. Most Likely Value is useless because it tends toward the bottom of the range, which always gives Nt = N. Looking for a confidence level x devolves into choosing the middle portion x, which means the predicted value depends on the desired level of confidence. There might be more exotic prediction methods, but my statistical toolkit is empty.


Maybe This Is A Reasonable Prediction?

So, I hope I have shown that this argument collapses into an unintelligible mess when examined with rigorous statistics. However, there is one advantage: the Doomsday Paradox does give a prediction.

If I want to make such a prediction, and all I get to know is how many people currently exist, the Doomsday Paradox gives me reasonable numbers. But therein lies the problem.


Why Don't I Just Make Up A Number?

As I have shown, I can adjust the ratios in the Doomsday Paradox to give me any number I want. So using the Doomsday Paradox to give me a number isn't much different from just making up a "reasonable" number. Why don't I just come out and say something along the lines of "there have been around 100 billion people so far, so 2 trillion seems like a reasonable total."

Honestly, the whole paradox just looks like fancy window dressing for making an educated guess.


Why Not Use Real Information To Make Our Guesses?

One final note: this whole paradox assumes that the only thing I know is how many people there have been so far. This assumption is not true, I actually know a great deal more than that. To make a quick comparison, look at the argument surrounding predicted ages. If the only thing I am allowed to know is how many years I have lived so far, massive age predictions of 500+ years look reasonable. But our intuition tells us that won't happen because it actually doesn't happen. We know more than just our current age.

This is also true of people. There is a wealth of information, and combining this information is pretty much guaranteed to give a better estimate. Why not use it?

Because we are physicists. The world is a sphere, and demographic data is a pain to work with.


Thanks for this "What If," it was a fun one!

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby ahammel » Wed Oct 02, 2013 2:08 pm UTC

cellocgw wrote:To close out this thread (yeah, right):

90% of all posts about the Doomsday Thingie are incorrect. What are the odds that your post is correct?
Frequentist: "It's a meaningless question. My post is either correct or it isn't, it doesn't make sense to talk about the odds that it's correct."
Bayesian: "Based on those figures, we can assign a prior probability that my post is correct. However, people who know what things like 'prior probability' mean probably write a greater than average number of the correct posts, so it's probably possible to revise our priors a bit with more data."
He/Him/His/Alex
God damn these electric sex pants!

Kit.
Posts: 1117
Joined: Thu Jun 16, 2011 5:14 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Kit. » Wed Oct 02, 2013 3:23 pm UTC

Flumble wrote:
eran_rathan wrote:
Flumble wrote:legalise hard drugs like opium and crocodile


Sir, I'm going to have to ask you to put down that crocodile, and keep your hands where I can see them.

Hahahah :P
My bad, I assumed desomorphine has the nickname crocodile in English.

Any reason to use homemade desomorphine if opium is legalized?

rjsteg
Posts: 18
Joined: Wed Dec 28, 2011 2:41 pm UTC

Best 'The Land Before Time'

Postby rjsteg » Wed Oct 02, 2013 3:50 pm UTC

Re: Tommy Wiseau's 'The Land Before Time XX' is widely regarded as the worst in the series.

I'm guessing that 'The Land Before Time XXX' is the best...but probably not for kids.

shinksma
Posts: 7
Joined: Fri Nov 30, 2012 5:53 pm UTC

Re: Best 'The Land Before Time'

Postby shinksma » Wed Oct 02, 2013 4:24 pm UTC

rjsteg wrote:Re: Tommy Wiseau's 'The Land Before Time XX' is widely regarded as the worst in the series.

I'm guessing that 'The Land Before Time XXX' is the best...but probably not for kids.

Kids don't enjoy Vin Diesel?

Oh, that other XXX-type film...

JeffR23 wrote:If the doomsday argument works today, it would have worked a thousand years ago. If it had worked a thousand years ago, we'd all be dead. We're not dead. Contradiction, so QED: the doomsday argument is silly.

Exactly, IMHO. There is some kind of point to be made about why the doomsday argument relies on a single outcome of a non-repeatable experiment ("how long does the human race survive?") and therefore is not a useful situation to have a statistical analysis done, but I'm not savvy enough to formulate a clever summary.

shinksma

Starchild
Posts: 2
Joined: Wed Oct 02, 2013 6:26 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Starchild » Wed Oct 02, 2013 6:36 pm UTC

Just wanted to point out that the Doomsday Argument doesn't necessarily predict doomsday. The starting point is from the origin of the human species, which naturally evolved from a lower form of life. It could be seen as a prediction of when humans will fully transcend to another form of existence and are no longer recognizably human. If technological progress continues in accordance with current models, frankly I would be surprised if humanity continues as is for another 800 years.

Also, the argument is predicated on the assumption that humanity will end at some point. If humanity continues to survive indefinitely into the future, then the set of "all humans who have/will ever live" is countably infinite and the "middle 90%" has no meaning, so the argument is completely invalid.

User avatar
Copper Bezel
Posts: 2426
Joined: Wed Oct 12, 2011 6:35 am UTC
Location: Web exclusive!

Re: What-If 0065: "Twitter Timeline Height"

Postby Copper Bezel » Wed Oct 02, 2013 7:28 pm UTC

It's still true that it provides the best available estimate based on that data, correct? Sometimes trivially true statements aren't so trivial (as long as they aren't taken out of context or abused, as almost any serious use of this particular estimate would be.) Intuitively, it seems not to be as pure a "random" sampling as the tank or the Land Before Time sequel, though. It's hard for me to wrap my head around applying the concept across time instead of across an extant population.
So much depends upon a red wheel barrow (>= XXII) but it is not going to be installed.

she / her / her

Crane
Posts: 74
Joined: Thu Mar 22, 2007 1:03 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Crane » Wed Oct 02, 2013 8:33 pm UTC

Humans will go extinct someday. Suppose that, after this happens, aliens somehow revive all humans who have ever lived. They line us up in order of birth and number us from 1 to N. Then they divide us divide them into three groups—the first 5%, the middle 90%, and the last 5%:
[... ] if for some reason every human answered "I'm in the middle group", 90% of them will (obviously) be right. This is true no matter how big N is.
Therefore, the argument goes, we should assume we're in the middle 90% of humans. Given that there have been a little over 100 billion humans so far, we should be able to assume with 95% probability that N is less than 2.2 trillion humans. If it's not, it means we're assuming we're in 5% of humans—and if all humans made that assumption, most of them would be wrong.
To put it more simply: Out of all people who will ever live, we should probably assume we're somewhere in the middle; after all, most people are.


Okay, my rebuttal:
The aliens divide the humans into three groups. The first 33%, the middle 33%, and the last 33%.
If for some reason every human answered "I'm in the middle group", 33% of them will be correct. This is true no matter how big N is.
The odds of them being in the last 33% or the first 33% are identical.
Therefore, there is no reason to make an assumption regarding which group one is in.

The sole basis for this argument seems to be defining the 'middle group' as being 90%, which is arbitrary.

If we decide to pick single percentile of humans in which we are, our odds of being correct are 1/100; we are equally likely to be in the first percentile of humanity or the last percentile.
Image

rmsgrey
Posts: 3617
Joined: Wed Nov 16, 2011 6:35 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby rmsgrey » Wed Oct 02, 2013 8:58 pm UTC

Crane wrote:
Humans will go extinct someday. Suppose that, after this happens, aliens somehow revive all humans who have ever lived. They line us up in order of birth and number us from 1 to N. Then they divide us divide them into three groups—the first 5%, the middle 90%, and the last 5%:
[... ] if for some reason every human answered "I'm in the middle group", 90% of them will (obviously) be right. This is true no matter how big N is.
Therefore, the argument goes, we should assume we're in the middle 90% of humans. Given that there have been a little over 100 billion humans so far, we should be able to assume with 95% probability that N is less than 2.2 trillion humans. If it's not, it means we're assuming we're in 5% of humans—and if all humans made that assumption, most of them would be wrong.
To put it more simply: Out of all people who will ever live, we should probably assume we're somewhere in the middle; after all, most people are.


Okay, my rebuttal:
The aliens divide the humans into three groups. The first 33%, the middle 33%, and the last 33%.
If for some reason every human answered "I'm in the middle group", 33% of them will be correct. This is true no matter how big N is.
The odds of them being in the last 33% or the first 33% are identical.
Therefore, there is no reason to make an assumption regarding which group one is in.

The sole basis for this argument seems to be defining the 'middle group' as being 90%, which is arbitrary.

If we decide to pick single percentile of humans in which we are, our odds of being correct are 1/100; we are equally likely to be in the first percentile of humanity or the last percentile.

In statistics, there's a general acceptance of 5% as being a significant number - so if you guess that you're in the middle 95% (or in the first 95% or in the last 95%) then you have a priori a 95% chance of being right. It's only if you have other data to work with that changing that estimate is justified...

If you look at being in the middle 1/3 then your conclusion would be that you are 1/3 certain rather than 90% certain or 95% certain of your figures on total human population over all time.

Starchild
Posts: 2
Joined: Wed Oct 02, 2013 6:26 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Starchild » Wed Oct 02, 2013 9:04 pm UTC

Just wanted to point out that the Doomsday Argument doesn't necessarily predict doomsday. The starting point is from the origin of the human species, which naturally evolved from a lower form of life. It could be seen as a prediction of when humans will fully transcend to another form of existence and are no longer recognizably human. If technological progress continues in accordance with current models, frankly I would be surprised if humanity continues as-is for another 800 years.

Also, the argument is predicated on the assumption that humanity will end at some point. If humanity continues to survive indefinitely into the future, then the set of "all humans who have/will ever live" is countably infinite and the "middle 90%" has no meaning, so the argument is completely invalid.

rmsgrey
Posts: 3617
Joined: Wed Nov 16, 2011 6:35 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby rmsgrey » Wed Oct 02, 2013 10:12 pm UTC

Starchild wrote:Also, the argument is predicated on the assumption that humanity will end at some point. If humanity continues to survive indefinitely into the future, then the set of "all humans who have/will ever live" is countably infinite and the "middle 90%" has no meaning, so the argument is completely invalid.


That would require either the universe to survive indefinitely into the future, or humanity to somehow escape the end of the (habitable) universe... See #1266

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26740
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby gmalivuk » Wed Oct 02, 2013 11:33 pm UTC

Crane wrote:
Humans will go extinct someday. Suppose that, after this happens, aliens somehow revive all humans who have ever lived. They line us up in order of birth and number us from 1 to N. Then they divide us divide them into three groups—the first 5%, the middle 90%, and the last 5%:
[... ] if for some reason every human answered "I'm in the middle group", 90% of them will (obviously) be right. This is true no matter how big N is.
Therefore, the argument goes, we should assume we're in the middle 90% of humans. Given that there have been a little over 100 billion humans so far, we should be able to assume with 95% probability that N is less than 2.2 trillion humans. If it's not, it means we're assuming we're in 5% of humans—and if all humans made that assumption, most of them would be wrong.
To put it more simply: Out of all people who will ever live, we should probably assume we're somewhere in the middle; after all, most people are.


Okay, my rebuttal:
The aliens divide the humans into three groups. The first 33%, the middle 33%, and the last 33%.
If for some reason every human answered "I'm in the middle group", 33% of them will be correct. This is true no matter how big N is.
The odds of them being in the last 33% or the first 33% are identical.
Therefore, there is no reason to make an assumption regarding which group one is in.

The sole basis for this argument seems to be defining the 'middle group' as being 90%, which is arbitrary.

If we decide to pick single percentile of humans in which we are, our odds of being correct are 1/100; we are equally likely to be in the first percentile of humanity or the last percentile.
None of this is actually a rebuttal of anything, though. The version in the what-if seems different because we often think of 90% or 95% probabilities as a lot, but the logic works fine regardless:

There is a 95% chance that we are not in the first 5% of humanity. There is a 2/3 chance that we are not in the first 1/3 of humanity. There is a 50% chance that we are (and an equal chance that we aren't) in the first 50% of humanity.

If you check the sequence position of every human ever, the mathematical average will be exactly at the 50% mark. This is therefore the expected value is 50%. That's really all it means to assume a random person is near the middle: the expected percentile for a random person is 50.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Simon_Jester23
Posts: 4
Joined: Thu Oct 03, 2013 12:33 am UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Simon_Jester23 » Thu Oct 03, 2013 12:56 am UTC

I think the problem with the Doomsday Argument is that saying "assume you are a random sample (size=1) of humanity" is itself a pretty significant assumption.

To take the German Tank Problem as an example, imagine you're a small nation invaded by the Wehrmacht. Your army collapses under the assault of hundreds of German panzers- by heroic efforts you knock out one of the monsters, and find that it has serial number #10.

This is not a good time to say "Relax, guys, statistically speaking they only have about twenty of the things!"

The statistical analysis is subject to outside evidence that weights the list of possible answers in favor of (or against) certain outcomes. In the case of the Doomsday Argument, we've careened back and forth through several different possibilities.
________________________

Cavemen living 20,000 years ago could use the Doomsday Argument quite reasonably- and if anything, they would probably overestimate the longevity of the human species. Even though they would (wrongly) assume they were among the middle 90% of all humans to live,* they would more than cancel that out by (wrongly) assuming that the total world population would remain constant at something on the close order of one million.

*OK, OK, technically living 20000 years ago might put them in the 10th percentile of all humans to be born in chronological order or something. You get the idea though.
________________________

Meanwhile, a lot of people born in 1945 expected to die in a nuclear war during their own lifetimes growing up (to quote from Kahn's On Thermonuclear War, a '50s psychologist would often get answers as follows from ten year old children:

"What will you be when you grow up?"

"An engineer, unless some fool pushes a button, in which case I'll be nothing."

Such a person might well assume (wrongly) that they were in the last 5% of all humans to be born. Which might lead us to think the "middle 90%" assumption is correct... until we imagine the perspective of a human living through the Toba eruption, who would probably think they were among the last 5% of all humans while being completely wrong- very much the opposite, they were among the first 5%!*

*Or 10%. Not sure off the top of my head...
____________________

Anyway. The practical upshot of all this is that our picture of whether or not we can make the assumption of mediocrity depends on details. We assume the Sun is an unremarkable star and are correct... but when we assume that atoms are unremarkable random samples of all the substance of the universe we are wrong. 95% of the universe consists of 'dark' substances whose nature and composition we know little or nothing of... except that they aren't the same kind of matter as that which we directly interact with and perceive.

We are self-aware of being unremarkable examples of our kind, but our kind itself may not be so unremarkable, due to the weak anthropic principle being in play.

Perhaps this avid speculation is itself a symptom of our being among the relatively early members of humanity, because a "typical" human is an arcology-dweller in the Nth century who is largely indifferent to such things, or lives such a sheltered life that they simply expects it to go on forever.

On the other hand, our fixation on this might be because of all the things that might wipe us out in the near future... which itself tends to suggest that maybe we're toward the tail end of the human distribution. People didn't worry much about doomsday actually happening to them until 1945, after all.

I don't buy into the idea that we can default back to the assumption of mediocrity, as opposed to the null hypothesis, when faced with such a confusion of evidence.

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby ahammel » Thu Oct 03, 2013 1:18 am UTC

Does the fact that "the first human" is not well defined complicate matters any?
He/Him/His/Alex
God damn these electric sex pants!

User avatar
Eternal Density
Posts: 5574
Joined: Thu Oct 02, 2008 12:37 am UTC
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby Eternal Density » Thu Oct 03, 2013 1:46 am UTC

Related to the German Tank Problem, I found this quote:
"Today I interviewed a trader, who'd forecast earnings for a food company by ordering a weekly muffin in order to count the invoice number."
Anthony Goldbloom


ahammel wrote:Does the fact that "the first human" is not well defined complicate matters any?

Seems pretty well defined in my worldview ;)
Play the game of Time! castle.chirpingmustard.com Hotdog Vending Supplier But what is this?
In the Marvel vs. DC film-making war, we're all winners.

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26740
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby gmalivuk » Thu Oct 03, 2013 1:51 am UTC

ahammel wrote:Does the fact that "the first human" is not well defined complicate matters any?
No, because the broad statistical point would remain equally valid regardless of when you started or finished counting things as "human". Whichever way you cut it, 95% of that group will have been born later than the first 5%.

Of course, "equally valid" applies to "not at all valid" as well as it does to "totally valid". You can cut down the total estimate by an order of magnitude if you're instead interested in counting, say, humans living in a world where the Internet exists. (That highlights one of the possible objections to the argument, which is that we may have good reason to assume we *are* in fact in the first 5%.)
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Eutychus
Posts: 446
Joined: Mon Jan 25, 2010 6:01 am UTC
Location: France

Re: What-If 0065: "Twitter Timeline Height"

Postby Eutychus » Thu Oct 03, 2013 5:13 am UTC

willpellmn:I quite liked your reasoning (I too am risk-averse) until I got to this:
I don't buy a car which will take five years to pay off, because that would be just asking to be fired and spend the next five years getting hassled by collections agencies about my unpaid bill.
[/quote][/quote]Haven't you just replaced statistics with superstition?

The way I understand it (and I got an E in Maths A-level, a long time ago) statistics are useful to construct a rational view of reality based on existing data, but they are not prescriptive; they don't dictate what will happen. You, personally, can always be part of the 0.0000001%. Whether you are or not and why are where metaphysics comes into life.
Be very careful about rectilinear assumptions. Raptors could be hiding there - ucim

Eutychus
Posts: 446
Joined: Mon Jan 25, 2010 6:01 am UTC
Location: France

Re: What-If 0065: "Twitter Timeline Height"

Postby Eutychus » Thu Oct 03, 2013 5:15 am UTC

Eutychus wrote:willpellmn:I quite liked your reasoning (I too am risk-averse) until I got to this:
I don't buy a car which will take five years to pay off, because that would be just asking to be fired and spend the next five years getting hassled by collections agencies about my unpaid bill.
Haven't you just replaced statistics with superstition?

The way I understand it (and I got an E in Maths A-level, a long time ago) statistics are useful to construct a rational view of reality based on existing data, but they are not prescriptive; they don't dictate what will happen. You, personally, can always be part of the 0.0000001%. Whether you are or not and why are where metaphysics comes into life.
Be very careful about rectilinear assumptions. Raptors could be hiding there - ucim

Eutychus
Posts: 446
Joined: Mon Jan 25, 2010 6:01 am UTC
Location: France

Re: What-If 0065: "Twitter Timeline Height"

Postby Eutychus » Thu Oct 03, 2013 5:15 am UTC

willpellmn:I quite liked your reasoning (I too am risk-averse) until I got to this:
I don't buy a car which will take five years to pay off, because that would be just asking to be fired and spend the next five years getting hassled by collections agencies about my unpaid bill.
Haven't you just replaced statistics with superstition?

The way I understand it (and I got an E in Maths A-level, a long time ago) statistics are useful to construct a rational view of reality based on existing data, but they are not prescriptive; they don't dictate what will happen. You, personally, can always be part of the 0.0000001%. Whether you are or not and why are where metaphysics comes into life.

(Apologies for triple post/UBB editing disaster above)
Be very careful about rectilinear assumptions. Raptors could be hiding there - ucim

User avatar
Uristqwerty
Posts: 27
Joined: Mon Nov 14, 2011 5:11 am UTC
Location: Canada

Re: What-If 0065: "Twitter Timeline Height"

Postby Uristqwerty » Thu Oct 03, 2013 5:58 am UTC

95% is a rather arbitrary number. Although it is a multiple of 5, and it seems fairly probable (19/20), there isn't any reason that 95% is any better in this specific context than 93%, 94%, 96%, or 97%.

The issue is, if I understand how the calculation works, each of those probabilities produces a very different prediction. And since the probability is arbitrary, that means that the result is too. Therefore it is a reasonable assumption that 95% was chosen as a balance between "seems probable", "seems believable", and "gives the result I want it to show".

eskarel
Posts: 7
Joined: Mon Oct 15, 2007 8:40 am UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby eskarel » Thu Oct 03, 2013 7:13 am UTC

The reason the doomsday theory is daft is because it's conflating two different ideas into one. If you randomly selected 1000 humans from the entirely of human existence, then, roughly speaking 95% of them wouldn't be in the first 5%. If however you were to take the first 1000 humans born ever, then all of them would be in the first 5%. The results of a sequential sample are not the same as the results of a random sample.

DarsVaeda
Posts: 13
Joined: Mon Jun 06, 2011 8:54 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby DarsVaeda » Thu Oct 03, 2013 8:58 am UTC

That theory shows nicely (some) math has stopped being reasonable and has started saying: I believe in god!

Kit.
Posts: 1117
Joined: Thu Jun 16, 2011 5:14 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Kit. » Thu Oct 03, 2013 11:38 am UTC

Can I predict the length of the OTT based on the fact that I joined it at page 47?

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26740
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby gmalivuk » Thu Oct 03, 2013 12:12 pm UTC

Uristqwerty wrote:95% is a rather arbitrary number. Although it is a multiple of 5, and it seems fairly probable (19/20), there isn't any reason that 95% is any better in this specific context than 93%, 94%, 96%, or 97%.

The issue is, if I understand how the calculation works, each of those probabilities produces a very different prediction. And since the probability is arbitrary, that means that the result is too. Therefore it is a reasonable assumption that 95% was chosen as a balance between "seems probable", "seems believable", and "gives the result I want it to show".
Then it looks like you don't understand how the calculation works.

Whatever probability you start with persists throughout the calculation. If there have been 100 billion people so far, the calculation goes, then we can be 50% sure there will be no more than 100 billion more, 95% sure there will be no more than 1.9 trillion more, and 99% sure there will be no more than 9.9 trillion more. The probability changes along with the estimate.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

speising
Posts: 2350
Joined: Mon Sep 03, 2012 4:54 pm UTC
Location: wien

Re: What-If 0065: "Twitter Timeline Height"

Postby speising » Thu Oct 03, 2013 3:25 pm UTC

Kit. wrote:Can I predict the length of the OTT based on the fact that I joined it at page 47?


not anymore. but, oh god, that means that it will probably still grow to 3044 pages!

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26740
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: What-If 0065: "Twitter Timeline Height"

Postby gmalivuk » Thu Oct 03, 2013 3:41 pm UTC

Well, only to the extent that a 50% chance is "probably".
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

Kit.
Posts: 1117
Joined: Thu Jun 16, 2011 5:14 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Kit. » Thu Oct 03, 2013 4:13 pm UTC

By the way, about the airplane on treadmill problem... not that stupid "v=2*v" problem, but a more physically correct one: what if the treadmill keeps its speed high enough to stop the plane from moving forward. Will the plane take off?

I used to think that the airplane will take off when the headwind generated by the treadmill's drag reaches the plane's "normal" take off speed, which would eventually happen with planes designed to take off from "normal" airstrips. I believed that the headwind would create a positive lift force, causing the positive feedback loop by reducing the chassis ground reaction force, and thus decreasing the energy dissipated by the chassis friction, up to the point where the only friction force applied to the airplane will be the headwind drag force; and the airplanes are designed to fly in such conditions. Now I'm not so sure.

Spoiler:
The headwind speed caused by the treadmill drag will be different at different heights, which could cause a negative lift force and increase the power dissipated in the chassis. If the power dissipated in the chassis reaches the power generated by the engines (minus the power consumed by air friction), we've got an equilibrium speed at which the plane won't fly.
Last edited by Kit. on Thu Oct 03, 2013 10:03 pm UTC, edited 1 time in total.

rmsgrey
Posts: 3617
Joined: Wed Nov 16, 2011 6:35 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby rmsgrey » Thu Oct 03, 2013 4:19 pm UTC

Simon_Jester23 wrote:On the other hand, our fixation on this might be because of all the things that might wipe us out in the near future... which itself tends to suggest that maybe we're toward the tail end of the human distribution. People didn't worry much about doomsday actually happening to them until 1945, after all.


I have heard that there was a sharp rise in the number of stone cathedrals being built in the 11th century compared to the 10th, at least in part because people expected the world to end in the year 1000 (give or take). Early Christians expected the world to end any day now (though, admittedly, they probably weren't a significant sample) - and the Jews were expecting a Messiah to turn up any decade...

Expecting the world to end is nothing new...

Simon_Jester23
Posts: 4
Joined: Thu Oct 03, 2013 12:33 am UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Simon_Jester23 » Thu Oct 03, 2013 8:14 pm UTC

On the one hand, that was a specific European Christian thing, "end times" mythology being a much larger part of Christianity than of most other religions. On the global scale it was little more than a blip.

On the other hand, you do have a seriously good point- but I don't think it strongly undermines my point.

I can come up with examples of people among the first humans who thought they would be the last (any isolated tribe which experiences a major natural disaster earlier than, say, 50000 years ago). I can come up with examples of people in the beginning/middle (as yet unsure which) who though they would be at the end. There will most likely be people at the end who think they are in the middle, or even the beginning.

The point is that there is no particular correlation between where you think you are on the list of all humans organized by birth date, and where you really are. Even if your argument for being in the middle is couched in statistical terms, it's still not reliable, because you don't have enough context to judge whether the statistical argument is valid. And context is crucial to good statistics, because without it you can't identify bias.

mcdigman
Posts: 77
Joined: Fri Aug 17, 2012 6:32 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby mcdigman » Sat Oct 05, 2013 7:23 pm UTC

Assuming that homo sapiens have existed for ~200,000 years, by the same logic, we can consider the probability that today is in the middle 90% of our species existence. If we choose a random year in the existence of homos sapiens, it is best to say it is in the middle 90%. That means that given that 2013 is a random year in the existence of the human species, at the very minimum, we should believe homo sapiens will exist for another 10,000 years, and at maximum, we're in for another 4,000,000 years of human existence, in our 90% interval. These have no overlap at all with the numbers from the population estimate at all - I think the best conclusion is that if you try to extrapolate by integrating probabilities over time, you can conclude if almost anything if you do it wrong just the right way.

User avatar
Copper Bezel
Posts: 2426
Joined: Wed Oct 12, 2011 6:35 am UTC
Location: Web exclusive!

Re: What-If 0065: "Twitter Timeline Height"

Postby Copper Bezel » Sat Oct 05, 2013 8:02 pm UTC

gmalivuk already mentioned that that's not a uniform distribution in the way that population is. There are a lot of tricks to show why this estimate doesn't work, but that's not one of them.

Edit: Should say that I don't know statistics, and it seems on the surface that existence per year should be a perfectly reasonable datum to record, but it seems not to be the case in fact.
So much depends upon a red wheel barrow (>= XXII) but it is not going to be installed.

she / her / her

Simon_Jester23
Posts: 4
Joined: Thu Oct 03, 2013 12:33 am UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby Simon_Jester23 » Sat Oct 05, 2013 8:59 pm UTC

Well, the trick is that you get very different results from applying the assumption of mediocrity to "this randomly chosen person is probably part of the middle of the set of all humans that exist" and saying "this randomly chosen year is probably part of the middle of the set of all years in which humans exist."

Both assumptions are equally valid, but they yield very inconsistent results- because human population has grown so rapidly that if we today are anywhere near the middle of the span of existence of the species, we are probably very close to the beginning of the list of all humans to be born.

Which to me undermines the whole assumption. How can you respect a statistical assumption that gives mutually contradictory predictions when used to analyze two different valid datasets from the same reality?



As to this having already been dropped, I must have missed why it was dropped. Why is assuming that this year is a randomly chosen sample of all the years in which humans exist any less valid than assuming that I am a randomly chosen sample of all the people who exist? Sure, there are more living humans during this year, and I am a priori more likely to be alive in this era than in any earlier one. But the mere existence of an increased number of observers looking at "this year" doesn't sound like it should prove anything.

User avatar
Copper Bezel
Posts: 2426
Joined: Wed Oct 12, 2011 6:35 am UTC
Location: Web exclusive!

Re: What-If 0065: "Twitter Timeline Height"

Postby Copper Bezel » Sat Oct 05, 2013 9:07 pm UTC

Like I said, intuitively that makes sense to me and I can't think of an argument against it. I thought that this was one, but now I realize that I may have misread it:

gmalivuk wrote:No, it only works for uniform distributions. That's why the what-if talks in terms of sequentially ordering humans before bringing dates into it, because we're uniformly distributed by number from first to last human (one human per number), but not uniform by year of birth.


He's talking about longevity distributions, but I thought he was making an offhand comment that the What If was using birth order instead of years because the latter wasn't a uniform distribution. But if simple order is a uniform distribution, then it seems relatively simple to graph damn near anything against itself. (Again, as you say, years in which humans have existed against time.)
So much depends upon a red wheel barrow (>= XXII) but it is not going to be installed.

she / her / her

mcdigman
Posts: 77
Joined: Fri Aug 17, 2012 6:32 pm UTC

Re: What-If 0065: "Twitter Timeline Height"

Postby mcdigman » Sat Oct 05, 2013 9:15 pm UTC

Copper Bezel wrote:gmalivuk already mentioned that that's not a uniform distribution in the way that population is. There are a lot of trickses to show why this estimate doesn't work, but that's not one of them.

Edit: Should say that I don't know statistics, and it seems on the surface that existence per year should be a perfectly reasonable datum to record, but it seems not to be the case in fact.


The point was that if you approach a problem like this wrong in the right way, you can conclude anything.

Maybe a better way to think about the problem can be achieved by analogy to the st. petersburg paradox. Suppose that, in every generation, every child either has two children or the entire human race goes extinct, with equal probability. Starting from 1 child, what is the average number of humans that will be born?

The answer is infinity: at every generation, the probability of being there halves but the number of children born if humanity doesn't go extinct doubles, such that the expected number of children born in every generation is constant: 1. If you keep doing this forever, the series 1+1+1+1+1... doesn't converge, so you expect on average to have infinitely many children.

Of course, the variance is also infinite, and the median is 2, so saying you expect infinitely many children is misleading, even if it is true.

My point here is that if there's any nonzero probability for any human to survive to the end of the universe (or an arbitrary point far in the future), then any attempt to put a finite value on the expected lifetime of the human race is going to be confounded by extremely high variance, so in the absence of strong prior reasons to believe otherwise, the best assumption is to assume you are in the first 5%.


Simon_Jester23 wrote:As to this having already been dropped, I must have missed why it was dropped. Why is assuming that this year is a randomly chosen sample of all the years in which humans exist any less valid than assuming that I am a randomly chosen sample of all the people who exist? Sure, there are more living humans during this year, and I am a priori more likely to be alive in this era than in any earlier one. But the mere existence of an increased number of observers looking at "this year" doesn't sound like it should prove anything.


They're both terrible assumptions. One reason is related to the anthropic principle: even though the human race has existed for 200,000 years, the chance that an observer with knowledge of statistics is considering that datapoint is in no way uniformally distributed over those 200,000 years, and could mean something very particular about the expected remaining lifetime of the human race, per http://xkcd.com/962/.


Return to “What If?”

Who is online

Users browsing this forum: No registered users and 3 guests