The Perils of Overconfidence

Never be afraid to fail. Failure is only a stepping stone to improvement. Never be overconfident because that will block your improvement. - Tony Jaa

What is overconfidence?

As per the definition- the quality of being too confident; excessive confidence is called the overconfidence. The definition says it’s a quality but in real life this quality is dragging us from our own improvement and is lacking us from the real success.

Most of the time we are too much confident(Overconfident) that we are right in our decision, in our way of thinking or in our way of work. We don’t even think from other person point of view(Empathy).

Due to this kind of overconfident, we don’t get the wisdom of the world, we don’t expand our mental or thinking ability, we stop thinking in a broad way. We just behave like the frog who knows only till the periphery of the well. Out of the well, he thinks there is no world.

Daniel Kahneman, the psychologist and writer said:

We're generally overconfident in our opinions and our impressions and judgments. - Daniel Kahneman

Normally we are overconfident while giving opinions and making judgement, it is the default human behavior.

Maria Konnikova, the writer of the book Mastermind-how to think like Sherlock Holmes writes on overconfidence in the book-

how do we make sure we don’t fall victim to overly confident thinking, thinking that forgets to challenge itself on a regular basis? No method is foolproof. In fact, thinking it foolproof is the very thing that might trip us up. Because our habits have become invisible to us, because we are no longer learning actively and it doesn’t seem nearly as hard to think well as it once did, we tend to forget how difficult the process once was. We take for granted the very thing we should value. We think we’ve got it all under control, that our habits are still mindful, our brains still active, our minds still constantly learning and challenged—especially since we’ve worked so hard to get there—but we have instead replaced one, albeit far better, set of habits with another. In doing so we run the risk of falling prey to those two great layers of success: complacency and overconfidence. These are powerful enemies indeed.

Maria continues overconfidence as –

Confidence in ourselves and in our skills, allows us to push our limits and achieve more than we otherwise would, to try even those borderline cases where a less confident person would bow out. A bit of excess confidence doesn’t hurt; a little bit of above-average sensation can go a long way toward our psychological well-being and even our effectiveness at problem solving. When we’re more confident, we take on tougher problems than we otherwise might. We push ourselves beyond our comfort zone.

Confidence is good for our growth, it allows us to work hard and grow more but overconfidence affect us psychologically, it makes us to feel that we only do it and make us rude and arrogant.

But there can be such a thing as being too certain of yourself: overconfidence, when confidence trumps accuracy. We become more confident of our abilities, or of our abilities as compared with others’, than we should be, given the circumstances and the reality. The illusion of validity grows ever stronger, the temptation to do things as you do ever more tempting. This surplus of belief in ourselves can lead to unpleasant results—like being so incredibly wrong about a case when you are usually so incredibly right, thinking a daughter is a husband, or a loving mother, a blackmailed wife.

Studies have shown that with experience, overconfidence increases instead of decreases.

The more you know and the better you are in reality, the more likely you are to overestimate your own ability—and underestimate the force of events beyond your control.

In one study, CEOs were shown to become more overconfident as they gained mergers and acquisitions experience: their estimates of a deal’s value become overly optimistic (something not seen in earlier deals).

In another, in contributions to pension plans, overconfidence correlated with age and education, such that the most overconfident contributors were highly educated males nearing retirement. In research from the University of Vienna, individuals were found to be, in general, not overconfident in their risky asset trades in an experimental market—until, that is, they obtained considerable experience with the market in question. Then levels of overconfidence rose apace.

What’s more, analysts who have been more accurate at predicting earnings in the prior four quarters have been shown to be less accurate in subsequent earnings predictions, and professional traders tend to have a higher degree of overconfidence than students. In fact, one of the best predictors of overconfidence is power, which tends to come with time and experience.

Maria suggests in her book

Overconfidence causes blindness, and blindness in turn causes blunders. We become so enamored of our own skill that we discredit information that experience would otherwise tell us shouldn’t be discredited—even information as glaring as Watson telling us that our theories are “all surmise,” as he does in this case—and we proceed as before. We are blinded for a moment to everything we know about not theorizing before the facts, not getting ahead of ourselves, prying deeper and observing more carefully, and we get carried away by the simplicity of our intuition.

Overconfidence replaces dynamic, active investigation with passive assumptions about our ability or the seeming familiarity of our situation. It shifts our assessment of what leads to success from the conditional to the essential. I am skilled enough that I can beat the environment as easily as I have been doing. Everything is due to my ability, nothing due to the fact that the surroundings just so happened to provide a good background for my skill to shine. And so I will not adjust my behavior. Holmes

Overconfident traders have been shown to perform worse than their less confident peers. They trade more and suffer lower returns. Overconfident CEOs have been shown to overvalue their companies and delay IPOs, with negative effects. They are also more likely to conduct mergers in general, and unfavorable mergers in particular. Overconfident managers have been shown to hurt their firms’ returns. And overconfident detectives have been shown to blemish their otherwise pristine record through an excess of self-congratulation.

What Charlie Munger said on overconfidence:

Similarly, the hedge fund known as ‘Long-Term Capital Management’ recently collapsed, through overconfidence in its highly leveraged methods, despite I.Q.’s of its principals that must have averaged 160. Smart, hard-working people aren’t exempted from professional disasters from overconfidence. Often, they just go aground in the more difficult voyages they choose, relying on their self-appraisals that they have superior talents and methods.

How you can detect the overconfidence?

Maria suggests some solution points to identify the overconfidence, let’s explore:

The best remedy for overconfidence is knowing when it is most likely to strike.

Spotting overconfidence, or the elements that lead to it, in others is one thing; identifying it in ourselves is something else entirely, and far more difficult.

Four sets of circumstances tend to predominate.

1.    overconfidence is most common when facing difficulty:

For instance, when we must make a judgment on a case where there’s no way of knowing all the facts. This is called the hard-easy effect. We tend to be underconfident on easy problems and overconfident on difficult ones. That means that we underestimate our ability to do well when all signs point to success, and we overestimate it when the signs become much less favorable, failing to adjust enough for the change in external circumstances.

Repeatedly, researchers have found that as the difficulty of the judgment increases, the mismatch between confidence and accuracy (overconfidence) increases dramatically.

One domain where the hard-easy effect is prevalent is in the making of future predictions - Consider the stock market. It’s impossible to predict the movement of a stock. Sure, you might have experience and even expertise - but you are nevertheless trying to predict the future.

people who at times have outsized success also have outsized failures? The more successful you are, the more likely you are to attribute everything to your ability—and not to the luck of the draw, which, in all future predictions, is an essential part of the equation. (It’s true of all gambling and betting, really, but the stock market makes it somewhat easier to think you have an inside, experiential edge.)

2.    overconfidence increases with familiarity

If I’m doing something for the first time, I will likely be cautious. But if I do it many times over, I am increasingly likely to trust in my ability and become complacent, even if the landscape should change. And when we are dealing with familiar tasks, we feel somehow safer, thinking that we don’t have the same need for caution as we would when trying something new or that we haven’t seen before.

In a classic example, Ellen Langer found that people were more likely to succumb to the illusion of control (a side of overconfidence whereby you think you control the environment than you actually do) if they played a lottery that was familiar versus one that was unknown.

It’s like the habit formation that I have explained in this post.

Each time we repeat something, we become better acquainted with it and our actions become more and more automatic, so we are less likely to put adequate thought or consideration into what we’re doing.

3.    overconfidence increases with information

If I know more about something, I am more likely to think I can handle it, even if the additional information doesn’t actually add to my knowledge in a significant way.

This is the exact effect we observed with the clinicians who were making judgments on a case, the more information they had about the patient’s background, the more confident they were in the accuracy of the diagnosis, yet the less warranted was that confidence.

4.    overconfidence increases with action

As we actively engage, we become more confident in what we are doing.

In one classic study, Langer found that individuals who flipped a coin themselves, in contrast to watching someone else flip it, were more confident in being able to predict heads or tails accurately, even though, objectively, the probabilities remained unchanged.

Another instance found when, individuals who chose their own lottery ticket were more confident in a lucky outcome than they were if a lottery ticket was chosen for them.

Let’s take the case of traders once again. The more they trade, the more confident they tend to become in their ability to make good trades. As a result, they often overtrade, and in so doing undermine their prior performance.

You should always remind yourself about our fallibility and ability to deceive ourselves into a very confident blunder.

Let me end the post with the excellent quote by Charles Bukowski

The problem with the world is that the intelligent people are full of doubts, while the stupid ones are full of confidence.

Keep reading, Keep learning
-Mahesh