This was written on 20 November 2013 and if you
are over 50 and male could be the most important blog
you will read this year
I have
just finished reading The Signal and the
Noise: The Art and Science of Prediction by Nate Silver. The book discusses
a how a diverse set of forecasts ranging from politics, baseball and the
weather are prepared, the errors that are often made and how in many cases
‘expert predictions’ should be treated with many grains of salt.
However
a real strength of the book is the description of Bayesian reasoning in Chapter
8 which is a technique every manager should learn. A lot of management effort and time is spent
altering forecasts as new information is received. Unfortunately most of us just apply such
information intuitively. Bayes allows to
you make better predictions. This book made me really learn about Bayes
Theorem. I am the first to admit that
although I had listened to lectures about Bayes at both Cambridge and London
Business School I have never really sat down and learned it.
Simply
put one starts out with a prior probability.
One then takes the probabilities
of a new event being either true or false and then calculates a posterior
probability. Say x equals the prior probability, y equals a new event
probability that is true, and z equals a new event probability that is false.
The posterior probability is xy over xy plus (1-x)z. The secret to Bayes is
calculating both the true and false positives.
An example will make this much clearer.
Say for
example there has been an accident in a city involving a taxi cab.
* 85%
of the cabs in the city are white, and 15% are silver.
* A man
identified the cab involved in a hit and run as silver.
* The
court tested the witness' reliability, and the witness was able to correctly
identify the correct color 80% of the time, and failed 20% of the time.
What is
the probability the taxi cab was silver?
Here's
how we figure it out using Bayes theorem.
If the
cab was silver, a 15% chance, and correctly identified, an 80% chance, the
combined probability is .15 * .8 = .12, a 12% chance. These are true positives.
If the
cab was white, an 85% chance, and incorrectly identified, a 20% chance, the
combined probability is .85 * .2 = .17, a 17% chance. These are false positives
Since
the cab had to be either white or silver, the total probability of it being
identified as silver, whether right or wrong, is .12 + .17 = .29. In other
words, this witness could be expected to identify the cab as silver 29% of the
time whether he was right or wrong.
The
chances he was right are .12 out of .29, or 41% which I would suggest is much
lower that people would expect.
Now
recently I took a PSA test and my reading was above the supposed danger level. What is the probability I have prostate cancer?
The
chances of having prostate cancer at various ages are as follows:
For a
man in his 40s - 1 in 1000
For a
man in his 50s - 12 in 1000
For a
man in his 60s - 45 in 1000
For a
man in his 70s - 80 in 1000
I am 69
so my chances of prostate cancer would be 63 in 1000. However I have now had a
positive PSA test result.
Now
according to a medical website for every 100 men over age 50, with no symptoms,
who have the PSA test:
10 men
out of 100 tested will have a higher than normal level of PSA. These men must
then go through other tests and examinations. At the end of these tests:
• Three
of the ten men with a higher than normal PSA level will be found to have
prostate cancer.
• Seven
of the 10 men with a higher than normal PSA level will be found not to have
prostate cancer at the time of screening.
90 men
out of 100 tested will have a normal PSA level. Of these 90 men:
• 88 of
the men with a normal PSA level will not have prostate cancer.
• One
or two of the men with a normal PSA level will actually have prostate cancer,
undetected by the test.
The
probability that the PSA test gives a true positive for me is 0.063 x 88/90 or
0.0616 (xy in the formula; note two people out of 90 are missed.)
The
probability that the PSA test gives a false positive for me is 0.927 x .07 or
0.0656 ((1-x)z in the formula.
The sum
of the true and false positives is 0.1272 and so according to Bayes the
probability that I have prostate cancer is 48% which is much higher than I
originally thought and means that I will definitely go forward with a biopsy.
I never
would have come to this conclusion without reading Silver's book.
best silver taxi service in melbourne Silver Service taxi in Melbourne easy safe with cheap rates
ReplyDeleteHappy to read out this blog post where we can find amazing places to travel by limo services and I hired this from Hamptons car service which costs me affordable at my budget.
ReplyDelete