Ever since I saw Malcolm Gladwell’s TED talk a couple of years ago, I’ve been convinced that Gladwell is the most overrated “intellectual” writing books today.  Of course, it only makes sense that such an honor should be bestowed to somebody whose catalogue of works seeks a little too hard to define intellectualism.  But he does a great job of being Oprah to people who want their warm fuzzies and dose of euphoric superiority from a guy who does everything he can to look like he lives in that whimsical labyrinth of books we all wish we owned.

I haven’t read Gladwell’s new book, What the Dog Saw: And Other Adventures because I’ve been forced to spend far too much time explaining the fallacies in Gladwell’s work to people who don’t understand fallacies well, or statistics — particularly Bayes’ Theorem.  I’ve spent just as much time wondering if Gladwell’s mistakes are intentional or unintentional.  Does he really have his finger on the pulse of pseudo-intellectualism the way that Oprah understands the climate of new-ageism?  Or is he a kind of example of the intellectual Peter Principle — completely unable to rise above his flaws, though amazingly able to enunciate his bad logic so that other people with ceilings to their logic skills can fully believe his conclusions?

Apparently Steven Pinker took his shot at Gladwell in this New York Times review.  Pinker hits Gladwell’s failings so poignantly that I can’t help but to feel a bit of sorrow for Galdwell, who must certainly recognize some, if not most of the truth in Pinker’s critique: that Gladwell tackles topics of statistical and mathematical nature without a rigorous understanding of the statistics and mathematics.  Such lack of understanding leads Gladwell to some terrible conclusions that he showcases as “counterintuitive” in a Freakonomics kind of way.  Unfortunately, as Pinker points out, Gladwell also cobbles these stories together in kind of “populist” motif (Pinker’s word) that flatters (in my words) Gladwell’s audience: people who like to believe that they are superior to their earned assessments.  Ahem, almost everyone.

What spurred me to write this post was not Pinker’s critique of Gladwell.  Actually, I discovered the review only after following a link to Gladwell’s retort from Marginal Revolution.   I only searched, found, and read Pinker’s review because Gladwell did not link to it in his retort.  Not to mention that the retort itself seemed suspect.  I’ve read several of Pinker’s books and Gladwell’s retort is the only place I’ve ever seen Pinker look sloppy.  I suspected that Gladwell crafted a post that would accomplish several goals at once:

(i) discourage Gladwell’s biggest fans to turn a blind eye to an important critique without actually reading the critique (hence the lack of link to Pinker’s review),

(ii) use the straw man of Steve Sailer as a popular punching bag, and

(iii) provide Gladwell with an heir of superiority in manners if nothing else.  Gladwell writes as if he’s being polite even though it’s clear that he’s neutering the substance of the critique against him and his work.

Since Gladwell took the time to craft such a reponse — one that I consider either terribly misguided in its ignorance (does Gladwell really believe in his understanding of statistics?!) or else both dishonest and offensive to the author of the review, Steven Pinker — I will provide at least part of the rebuttal that I believe to be the continuation of Pinker’s point.

Gladwell (along with a couple of econ guys?) wants to measure NFL quarterback performance on a per-play basis.  What he fails to do is extract other key variables such as (a) when those plays occur during the career of the quarterback, (b) the interplay between team finances and quarterback performance history, and (c) the recent history of success of the quarterback position on that team.  I’m sure I could do better if I bothered to think through every one of the important variables at play.

(a) makes a substantial difference because only the most highly drafted quarterbacks start when they are young and have taken the fewest snaps in professional practices.  Highly drafted Peyton Manning threw a lot of interceptions during his rookie year.  Low round draft picks surely would as well — if any coaches were stupid enough to start rookies that other teams were happy to pass on several times over at the most important position in football.  These interceptions affect Manning’s per-play effectiveness, but they don’t count in to the per-play stats of the star QB for the New England Patriots, Tom Brady, who started his career in backup of Drew Bledsoe.

(b) is a bit harder to understand at first, and even harder to find a way around statistically.  Sometimes the decision to play a player is anchored by that player’s contract.  If a team’s QB is horrible, but has a huge contract due to being a top ten draft pick, then the team has almost no choice but to start that QB — at least until his n-year contract comes to a close or some other fool team takes the bait in trade.  Call a play played by such a QB a “forced bad play”.  Such forced bad plays clearly result in subpar outcomes.  But forced bad plays simply do not exist for players outside the top ten draft picks, so they act as a glaring statistical weight that should be corrected.  After correction, I seriously doubt that Gladwell’s conclusion stands up.

(c) was pointed out in a number of comments in Gladwell’s own blog using a different set of words.  Many people pointed out that top QBs are nearly always drafted by terrible teams (and started early — see (a) and then (b)).  I suspect that Gladwell ignores this point for mostly the same reason that politicians ignore correct critiques against their own character — because many people really do [quickly] forget a critique that is ignored by the target.  Persuasion master class aside, (c) punches a very big hole in Gladwell’s analysis.

Unless Gladwell and his team of middling social scientists found a way to correct for (c) that they’re not telling us, then they’re quite simply data-mining and reporting about it.  Sure, top ten drafted QBs get out-performed per play than those picked in other ranges of the draft.  But that…doesn’t…mean…that per play statistics are meaningful metrics.

In fact, I would conclude that overall per play metrics are simply not meaningful (unless other variables like (a), (b), and (c) were corrected for) at all!  And I would base my conclusion on the stated statistical inversion itself.  A statistical measure that flies in the face of both expert assessment and earned honors (like the Pro Bowls Pinker mentions) is simply not the metric to place your trust in.

Take a beep breath and revert back to common sense — before your counterintuitive-is-cool button was pressed: do you really believe that there is absolutely no advantage to all the expertise leveraged to make draft decisions that are collectively worth hundreds of millions of dollars per year not to mention scrutinized by hundreds of millions of people?  Even worse — that an illiterate nomad who has never been within 5000 miles of an NFL football stadium could (and would) do better (in the long run) at drafting QBs by flipping coins, casting lots, or some other choice of randomizing action?

No?  Good.  Then either you have a superior understanding of Bayes’ Theorem than Gladwell has or your BS detector is functioning too well and he can’t talk you out of giving it your attention.

Edit: I’ve fallen for one of the traps I suspect Gladwell intended in his retort.  I started focusing on the football story.  All I can say is that I’ve found few of Gladwell’s stories that I could rip apart any less easily.  Gladwell’s mistake on this particular topic is not the point.  Gladwell’s poor use of or terrible understanding of statistics is the point — particularly if his mistakes are either intentional, or a strong rationalization of his intellectual capabilities.

Comments are closed.