A year ago, I watched a lecture by Lex Fridman on YouTube that discussed the Dunning-Kruger effect, and it’s a video I find myself returning to every now and then.
According to Wikipedia, the Dunning-Kruger effect is stated as follows:
The Dunning–Kruger effect is a cognitive bias in which people with limited competence in a particular domain overestimate their abilities
Basically, this effect describes how people new to a field or domain try to comprehend their expertise after some initial learning. Their confidence skyrockets. They overestimate their skill level.
I myself have been a part of this curve in many domains in my life(and am currently in the “valley of despair” in a few of them). The primary thing to learn from this effect is that it is not bad to feel good about one’s ability in a particular skill but to have a real benchmark and be humble in each competence level(assuming competence to be discrete). I myself try to be humble about everything I know and everything I don’t.
The problem isn’t falling into the valley of despair, but that most of the people don’t even continue their competence journey once they are confronted with this reality. The real value to take away is to be persistent with your learning and never assume yourself to be the top of the world because there is always someone in some corner of the world who is performing far better than you.
It is usually a good practice to stay around people (offline or online) who are further along the competence hierarchy, so that you stay grounded.
unfortunately, I haven’t been able to find the link to the specific youtube video which has been referred to here. If you happen to find it, please email me at atharva{at}atharvasingh.com