Saturday, March 8, 2014

Travels around my prostate #1

This was written on 20 November 2013 and if you are over 50 and male could be the most important blog you will read this year

I have just finished reading The Signal and the Noise: The Art and Science of Prediction by Nate Silver. The book discusses a how a diverse set of forecasts ranging from politics, baseball and the weather are prepared, the errors that are often made and how in many cases ‘expert predictions’ should be treated with many grains of salt.

However a real strength of the book is the description of Bayesian reasoning in Chapter 8 which is a technique every manager should learn.  A lot of management effort and time is spent altering forecasts as new information is received.  Unfortunately most of us just apply such information intuitively.  Bayes allows to you make better predictions. This book made me really learn about Bayes Theorem.  I am the first to admit that although I had listened to lectures about Bayes at both Cambridge and London Business School I have never really sat down and learned it.

Simply put one starts out with a prior probability.   One then takes the probabilities of a new event being either true or false and then calculates a posterior probability. Say x equals the prior probability, y equals a new event probability that is true, and z equals a new event probability that is false. The posterior probability is xy over xy plus (1-x)z. The secret to Bayes is calculating both the true and false positives.  An example will make this much clearer.

Say for example there has been an accident in a city involving a taxi cab.

* 85% of the cabs in the city are white, and 15% are silver.
* A man identified the cab involved in a hit and run as silver.
* The court tested the witness' reliability, and the witness was able to correctly identify the correct color 80% of the time, and failed 20% of the time.

What is the probability the taxi cab was silver?

Here's how we figure it out using Bayes theorem.

If the cab was silver, a 15% chance, and correctly identified, an 80% chance, the combined probability is .15 * .8 = .12, a 12% chance. These are true positives.
If the cab was white, an 85% chance, and incorrectly identified, a 20% chance, the combined probability is .85 * .2 = .17, a 17% chance. These are false positives
Since the cab had to be either white or silver, the total probability of it being identified as silver, whether right or wrong, is .12 + .17 = .29. In other words, this witness could be expected to identify the cab as silver 29% of the time whether he was right or wrong.

The chances he was right are .12 out of .29, or 41% which I would suggest is much lower that people would expect.

Now recently I took a PSA test and my reading was above the supposed danger level.  What is the probability I have prostate cancer?

The chances of having prostate cancer at various ages are as follows:
For a man in his 40s - 1 in 1000
For a man in his 50s - 12 in 1000
For a man in his 60s - 45 in 1000
For a man in his 70s - 80 in 1000

I am 69 so my chances of prostate cancer would be 63 in 1000. However I have now had a positive PSA test result.

Now according to a medical website for every 100 men over age 50, with no symptoms, who have the PSA test:

10 men out of 100 tested will have a higher than normal level of PSA. These men must then go through other tests and examinations. At the end of these tests:

• Three of the ten men with a higher than normal PSA level will be found to have prostate cancer.
• Seven of the 10 men with a higher than normal PSA level will be found not to have prostate cancer at the time of screening.

90 men out of 100 tested will have a normal PSA level. Of these 90 men:
• 88 of the men with a normal PSA level will not have prostate cancer.
• One or two of the men with a normal PSA level will actually have prostate cancer, undetected by the test.

The probability that the PSA test gives a true positive for me is 0.063 x 88/90 or 0.0616 (xy in the formula; note two people out of 90 are missed.)

The probability that the PSA test gives a false positive for me is 0.927 x .07 or 0.0656 ((1-x)z in the formula.

The sum of the true and false positives is 0.1272 and so according to Bayes the probability that I have prostate cancer is 48% which is much higher than I originally thought and means that I will definitely go forward with a biopsy.  

I never would have come to this conclusion without reading Silver's book. 

Chris Golis Australia's expert on practical emotional intelligence website: Linkedin: blog: mobile: +61-418-222219


  1. best silver taxi service in melbourne Silver Service taxi in Melbourne easy safe with cheap rates

  2. Happy to read out this blog post where we can find amazing places to travel by limo services and I hired this from Hamptons car service which costs me affordable at my budget.