Do You Overvalue Expertise?

For almost 20 years I have been working towards becoming an expert in HR, learning, and organizational development.  My narrow interest started in school when I decided to major in psychology, and it only got narrow when I got a Masters in organizational psychology.  The field is such a passion of mine that I am constantly reading books, watching TED talks, and taking courses to increase the depth of my knowledge.

And in many ways that expertise paid off, as I’ve ran a global learning organization for Fortune 500 company, published articles in HR & Training journals, and even several industry awards.  In many ways, this could be a simple tale of how narrow focus and dedication is the right way to build a career, but it would be incomplete. 

Before I was interested in psychology, I was fascinated by math.  In addition to my Masters, I also have an MBA that specialized in entrepreneurship.  For every psychology book I read, I also read something from a wide variety of topics (theoretical physics, biographies, economics, philosophy, historical non-fiction, etc…).  And surprisingly these varied interests and experiences are what truly led to my success.

Success required more than just expertise

Combining psychology and math allowed me to think differently from other HR people.  Integrating my MBA knowledge into running a training department allowed me to solve business wide problems, not just provide training.  And some of my most innovative things I every created stemmed from merging my in-depth expertise with lessons I learned from other fields.  For example, the award-winning management program I helped create was based on inspiration I got from Toyota’s LEAN principles, a book on Charles Darwin, and a training course on Agile project management.

Having both a depth and a breadth of knowledge is what has helped me succeed in my career, but it was not something I fully thought about until I read Range: Why Generalists Triumph in a Specialized World by David Epstein.  Over my summer holiday I picked it up, and couldn’t put it down.  Especially since it was challenging some of my firmly held beliefs even when it came to being a better father.  I fully recommend this book to people thinking about their own career and development, and especially parents thinking about how best they can help their kids grow.

Amazon link to Range by David Epstein: https://www.amazon.com/dp/0735214484/ref=cm_sw_em_r_mt_dp_U_lpcBDb3NXXCPF

Depth vs Breadth

Range challenges a very common belief; that breadth of knowledge is just as important as depth of expertise, especially when it comes to innovation.  And it shouldn’t be surprising, because that’s how creative and innovative ideas are explained to us.  We hear about Edison who worked through hundreds and hundreds of lighting experiments before he got a working light bulb.  We hear about doctors and researchers dedicating their careers to curing a disease.  We even hear about artists living solely for their art.  But innovation doesn’t come from expertise alone.

In fact, expertise can be a hindering factor to innovative ideas.  One large research study looked at patents and what kinds of teams contributed to the creation of the new idea.  What they found was that expertise mattered more in low uncertainty fields, where the progress was more linear, and where patents were slightly more useful than current practices.  But in high uncertainty fields, where the researchers didn’t even know if they were asking the right questions, teams of generalists created more patents.  In simpler terms, experts might help you build a better mousetrap, while generalists might solve your garbage problems that were attracting the mice in the first place.

Breadth is also important in the artistic fields.  A study of comic book authors found that years of in-depth expertise had no connection to the authors success.  Instead author success was best predicted by how many different types of comics the person had worked on. Epstein writes “Individual creators started out with lower innovativeness than teams—they were less likely to produce a smash hit—but as their experience broadened they actually surpassed teams: an individual creator who had worked in four or more genres was more innovative than a team whose members had collective experience across the same number of genres.”

Experts are great at explaining the past, but crap at predicting the future

In addition to helping you be more innovative, broadening your knowledge will also help you make better decisions, especially when it comes to predicting the future.  In fact, your expertise might actually make you worse at predicting outcomes in your field of expertise.  In a study that lasted over 20 years, hundreds of experts were asked rate the probability of an event in their field of expertise.  For example, political scientists were asked to rate the probability of the USSR collapsing.  Over 82,361 probability estimates were gathered from experts, and results were hilariously captured by Epstein.

“The average expert was a horrific forecaster. Their areas of specialty, years of experience, academic degrees, and even (for some) access to classified information made no difference. They were bad at short-term forecasting, bad at long-term forecasting, and bad at forecasting in every domain. When experts declared that some future event was impossible or nearly impossible, it nonetheless occurred 15 percent of the time. When they declared a sure thing, it failed to transpire more than one-quarter of the time.”

They even coined a special term for the worst kind of experts, Hedgehog Experts.  These are the kinds of experts who focus solely on one topic, and most likely not even the full topic.  They take their narrow focus area and slice it up even more narrowly.  Instead of being an expert on the Cold War, they become an expert on grain production in the USSR during the Cold War.  The study found that an increase in specialization actually made predictions worse.

Those same researchers also conducted another multi-year study where teams were allowed to compete in prediction making.  They found that the best teams were ones with broad expertise and a willingness to question their own knowledge.  These teams even outperformed experienced intelligence analysts who had access to classified data.  Think about that for a second.  Teams comprised of a complete mixture of different backgrounds made 30% more successful predictions without having access to classified materials that the ‘true’ experts had. 

So should you be a specialized expert or a broad generalist?

So, what does this mean for you?  It means you need to assess what kind of industry you are in.  If you work in an industry where the goal is to innovate by stepping stones and to perform at a high level in repetitive tasks, then the path of an expert is for you.  But if you are in an environment that evolves and allows for some risk, then you need to broaden your knowledge.  That being said, if you have the capacity, then do both.  Maintain your in-depth expertise, while also broadening your horizons.

It also means that you should avoid putting a lot of faith in a panel of deep experts.  We want to believe that the panel of macroeconomists would be able to predict an upcoming recession, but research doesn’t back that up.  Instead we should be getting perspectives from many different vantage points, and actually listening to someone even if they don’t have the right credentials.

Finally, I think that we all need to value the generalists a little more.  Too often we (myself included) overvalue the person who has years of expertise and in-depth knowledge.  I know when I was hiring for my team, I wanted the deep experts.  But now I am trying to value generalists.  The generalist might not have the years of expertise, but they bring a different perspective to the table.  And often that different perspective is what sparks that “A-Ha” moment.