Blog RSS Feed:

Blog - Investing Notes

December 17, 2007 - The Black Swan: The Impact of the Highly Improbable (Chapter-by-Chapter Review)

In his 2007 book The Black Swan: The Impact of the Highly Improbable, Nassim Taleb addresses human inability to process natural randomness, particularly combinations of low predictability and large impact. "It is easy to see that life is the cumulative effect of a handful of [largely unpredictable] significant shocks." This logic "makes what you don't know far more relevant than what you do know." Models that ignore this logic (such as those assuming Gaussian probability distributions for financial variables) inculcate mistakes that "can lead to severe consequences." Focusing principally on the perspective of an investor, here is a chapter-by-chapter review of some of the insights in this book:

Chapter 1 - The Apprenticeship of an Empirical Skeptic

• The human mind suffers from: (1) the illusion of understanding "a world that is more complicated (or random)" than understood; (2) a perception of historical organization/causality that is clear only afterwards; and, (3) the overvaluation of selected facts as filtered and interpreted by simplifying "experts."
• "History does not crawl, it jumps," going from one dislocation to another "with a few vibrations in between."
• People tend to herd on explanations that are far from reliable.

In summary, unpredictable "Black Swans" drive history, and the human mind is blind to them.

Our blog entry of 6/28/07 summarizes an approach to incorporating jumps (structural breaks) into a long-run analysis of stock returns (in a non-Talebian framework).

Chapter 2 - Yevgenia's Black Swan

• Experts cannot predict the socioeconomic future, but they can confidently explain the past.

Chapter 3 - The Speculator and the Prostitute

• Scalable variables (relatively unconstrained values) and nonscalable variables (highly constrained values) have different kinds of randomness.
  • Scalable variables (for example, wealth and stock returns) are not predictable via bell curves. Impact is concentrated in a few instances, so a single instance can substantially affect the statistics of even a large sample.
  • Nonscalable variables (for example, human height and weight) are predictable via bell curves (Gaussian statistics). Impact is not concentrated in a few instances, so no single instance substantially affects the statistics of a large sample.
• Black Swans are generally found (not found) among scalable (nonscalable) variables.

In summary, accidents (routines) rule for scalable (nonscalable) variables.

Our blog entry of 11/30/07 summarizes research on the impact of very rare daily return events in 15 international equity markets.

Chapter 4 - One Thousand and One Days, or How Not to Be a Sucker

• The timing and nature of Black Swans are inherently unpredictable. Induction from historical data is of little use for scalable variables.
• Positive (negative) Black Swans tend to develop slowly (quickly).
• Human thought processes resist accepting Black Swans.

In summary, it is very difficult to generalize from historical information in scalable environments.

Chapter 5 - Confirmation Shmonfirmation!

• People exhibit confirmation bias, a tendency to look for (ignore) evidence that supports (conflicts with) views already held.
• Seeking confirming evidence is a long and winding road to knowledge. Proving hypotheses wrong (falsifying) is far more powerful than finding evidence that they may be right. The fact that there are no Black Swans in a given historical interval does not prove that a variable is nonscalable. The fact that there are Black Swans proves that a variable is not nonscalable.

In summary, people are naturally inclined to jump to conclusions and focus on evidence confirming those conclusions.

For example, our blog entry of 1/5/07 summarizes two studies on the relationship between investor information gathering and portfolio performance.

Chapter 6 - The Narrative Fallacy

• People (especially historians) naturally construct stories (narratives) about historical events with unjustified assumptions of causality, thereby giving a false sense of understanding.
• People therefore think the world is less random than it really is. They are blind to Black Swans.
• Stories constructed around Black Swans that have recently occurred make people far overestimate the probability of recurrence. New Black Swans will be events that no one envisions.
• "The way to avoid the ills of the narrative fallacy is to favor experimentation over storytelling, experience over history, and clinical knowledge over theories."

In summary, susceptibility to storytelling about the behavior of scalable variables blinds people to the existence of Black Swans.

Chapter 7 - Living in the Antechamber of Hope

• Black Swan deniers seek a steady stream of modestly positive investment returns, hoping that no negative Black Swan will intervene (or oblivious to the possibility of Black Swans). Observers generally deem them "successful."
• Black Swan exploiters position for rare huge returns from positive and negative Black Swans, with their positions meantime bleeding slowly for unpredictable intervals while they wait. Observers generally deem them "unsuccessful."

In summary, Black Swan hunters delay gratification and must have considerable "personal and intellectual stamina."

Chapter 8 - Giacomo Casanova's Unfailing Luck: The Problem of Silent Evidence

• Silent evidence (survivorship bias) conceals randomness, especially Black Swans.
• Professions involving scalable variables have a lot of casualties and an extreme survivorship bias.
• Many long-term outperformers in scalable professions (such as investment advisors) began with an extreme lucky streak. Those that started with bad luck lost money and disappeared.

In summary, humans are made to be superficial, ignoring silent evidence and thereby distorting the importance of Black Swans.

Chapter 9 - The Ludic Fallacy, or the Uncertainty of the Nerd

• The uncertainty encountered in real life is more fundamental than the sterilized, domesticated uncertainty present in games.
• Black Swans are events that we do not envision and for which we cannot assign probabilities.

In summary, games are a poor analogy for real life with scalable variables.

Chapter 10 - The Scandal of Prediction

• Human beings are consistently overconfident about how much they know.
• People tend to underestimate (overestimate) severely the probability of a future (recently experienced) Black Swan.
• Professional forecasters are often more affected than others by these biases. The processing of additional input tends to make them more overconfident but not more accurate.
• In Black Swan-prone environments, "experts" (such as stockbrokers and economists) exhibit no more forecasting ability than simple extrapolations. They tend to herd and make lots of excuses for inaccuracies.
• "...[B]e a fox, with an open mind." (See our blog entry of 4/18/06 for a summary of Philip Tetlock's Expert Political Judgment: How Good is It? How Can We Know?, which Nassim Taleb cites here.)

In summary, there are strong, inherent, counterintuitive limitations on forecasting scalable variables.

Blog Synthesis: The Wisdom of Analysts, Experts and Gurus cites a range of relevant research. Guru Grades finds that stock market forecasters in aggregate exhibit little or no foresight.

Chapter 11 - How to Look for Bird Poop

• Discovery and diffusion of new things depends mostly on luck, severely limiting predictability.
• Even for systems with specified initial conditions and rules, complexity practically precludes accurate prediction.
• Common sense is maladapted to prediction for scalable variables and misplaces reliance on incapable "experts."

In summary, there are multiple, insurmountable barriers to accurate forecasting of scalable variables.

Chapter 12 - Epistemocracy, a Dream

• People do not naturally learn from the errors in their past predictions.
• The human mind is incapable of mixing chance with extrapolation of the past to forecast a fuzzy future.
• People tend to overestimate the personal impacts of both pleasant and unpleasant events, regardless of past experience with such events.
• There is no difference for practitioners between inherent randomness and incomplete information. Both are "unknowledge."

In summary, human beings are not well-equipped to learn from history.

Chapter 13 - Appelles the Painter, or What Do You Do if You Cannot Predict?

• "Do not listen to economic forecasters..."
• Put 85-90% of investments into "extremely safe instruments, like Treasury bills." Put the remaining 10-15% into "extremely speculative bets, as leveraged as possible (like options), preferably venture capital-style portfolios." This approach limits but does not eliminate (potentially positive) exposure to Black Swans.
• Some modest tricks for identifying "situations where favorable consequences are much larger than unfavorable ones:"
  • Distinguish between positive and negative potential Black Swan impacts.
  • Do not seek precision in anticipating Black Swans.
  • Seize anything that looks like a Black Swan opportunity. There are few such chances.
  • Set no store in government forecasts.
  • Waste no energy arguing with forecasters, stock analysts or economists.

In summary, people should rely on their own judgment to assume medium, positively-skewed risk to Black Swans.

See, however, our blog entry of 9/10/07, which summarizes research suggesting that selling insurance (equity options) to those who fear Black Swans is highly profitable. This research suggests that people are overly fearful of, rather than blind to, Black Swans.

Chapter 14 - From Mediocristan to Extremistan, and Back

• "Everything is transitory." The long tails of scalable variable probability distributions destabilize entrenched winners and elevate new ones.
• Globalization creates interlocking fragility, concentrating control of wealth in fewer hands and enabling devastating worldwide Black Swans. There will be "fewer but more severe crises."

In summary, scalable variables (and Black Swans) are ascendant.

Our blog entry of 9/24/07 summarizes a recent example of such fragility.

Chapter 15 - The Bell Curve, That Great Intellectual Fraud

• The existence of Black Swans for scalable variables negates the applicability of a Gaussian framework. Standard deviation, correlation and regression have little or no significance for such variables.
• Love of the certainty offered by Gaussian statistics sustains its misapplication to scalable variables.

In summary, overuse of Gaussian statistics reflects a fundamental flaw in the way people look at the world.

Chapter 16 - The Aesthetics of Randomness

• Power laws, such as those that describe fractals, generate distributions that fit empirical scalable variable data.
• Finding the right power law for a specific scalable variable involves guesswork that generally underestimates the impact of Black Swans (picks tails that are not fat enough).
• Power law distributions demand far more data to "build confidence" than does the Gaussian distribution.
• "History does not reveal its mind to us -- we need to guess what's inside of it."
• "Fractal randomness does not yield precise answers... Mandelbrot domesticated some Black Swans, but not all of them..."

In summary, the highly uncertain framework of power laws best describes socioeconomic behaviors.

Our blog entry of 1/7/06 summarizes a formal investigation of the applicability of power laws to asset markets.

Chapter 17 - Locke's Madmen, or Bell Curves in the Wrong Places

• "All our statistical tools are obsolete [or] meaningless."
• "...[I]t is contagion that determines the fate of a theory in social science."
• "A theory is like medicine (or government): often useless, sometimes necessary, always self-serving, and on occasion lethal. So it needs to be used with care, moderation, and close adult supervision."

In summary, work from the bottom up, starting with data and focusing on premises rather than theory.

Chapter 18 - The Uncertainty of the Phony

• Unlike the uncertainties of games and quantum physics, the uncertainties of socioeconomic and weather variables do not on average cancel. We cannot predict these latter variables.

Chapter 19 - Half and Half, or How to Get Even with the Black Swan

• "Worry less about small failures, more about large, potentially terminal ones."
• "Worry less about advertised and sensational risks, more about the more vicious hidden ones."
• Aggressively seek exposure to positive Black Swans (when an error in some model would be beneficial). Very conservatively avoid exposure to negative Black Swans (when an error in some model would be damaging).

In summary, a speculator's energy is most correctly focused on playing the right side of the potential effects of highly unpredictable extreme events.

In overall summary, this book is a generally accessible challenge to the widespread use of Gaussian statistics as tools of prediction in socioeconomics (encompassing investing). With strong emphasis on intractable uncertainty, it is necessarily parsimonious and vague regarding advice to investors.

We wonder whether Black Swans are most likely to be found where/when conventional risk management seems most intense...where experts are in consensus...where government is seeking to control.

For reviews of a few other books, see Blog Synthesis: Reviews of Books and Web Sites, including our blog entry of 9/26/05 for some notes on Nassim Taleb's Fooled by Randomness.