Brian and the Patent Law Meeting
Picture a man — call him Brian — forty-three years old, an accountant in Nashville. You’ve probably met a version of him, or been him for an afternoon of your own life. In the fall of 2021, Brian decided he knew enough about patent law to challenge his employer’s intellectual property policy. He’d read four articles about it. He’d watched a YouTube video. He was absolutely certain his employer was wrong and he was right. So he said so, directly, in a meeting with the company’s general counsel — a woman with nineteen years of intellectual property law experience and two relevant cases argued before federal circuit courts.
Brian was wrong, the way you’ve been wrong before too. He was wrong in a way that took about twelve minutes to demonstrate, with specific statutory citations, and at that point his certainty evaporated with a speed that should have been informative to him and wasn’t. By the next week, Brian had a similarly authoritative position on the company’s data security protocols. Here’s the thing you need to understand about Brian, and about the version of this that lives in you too: he isn’t stupid. He’s a product of a specific cultural failure, and you’ve probably absorbed some version of that same failure into yourself. He grew up in an era that told him information access equals knowledge, that everyone’s opinion deserves equal weight, and that credential and experience are barriers to authentic engagement rather than evidence of actual expertise.
Brian isn’t alone, and if you’re honest with yourself, you’ve had a Brian moment of your own. Statistically, he’s the median adult in any information-rich democracy you could name — which, if you’re listening to this, is probably the one you live in. Tom Nichols is a professor at the Naval War College. He’s the author of The Death of Expertise: The Campaign Against Established Knowledge and Why It Matters, published by Oxford University Press in 2017, and he documented the Brian pattern with depressing thoroughness. You will recognize it in these pages, and you should. The problem, in Nichols’s telling, isn’t that people are ignorant. Human beings have always been ignorant of most things. That’s fine. That’s inevitable. The problem is the collapse of the distinction between ignorance and knowledge — the widespread conviction that having access to information is equivalent to having processed and understood it well enough to justify confident action.
This episode is for you, about what I’m going to call the Epistemic Humility Protocol. Let me be clear about what that is not. It is not a prescription for self-doubt, and it is not a demand that you defer to authority every time someone with a credential disagrees with you. It’s a structured framework for knowing what you know, knowing what you don’t know, and making better decisions in the gap between those two things. It’s about rebuilding the cognitive architecture that the information environment around you has been systematically dismantling, so you can operate effectively in a world where the signal-to-noise ratio keeps collapsing. Stay with me through this one. It gets specific, and the specifics are what you can actually use.
The Dunning-Kruger Effect
Here’s the part the popular account leaves out, and it matters to you more than the part everyone quotes, because it might describe you specifically. Highly competent people, maybe you among them, tend to underestimate their relative competence. Partly that’s because they assume things that come easily to them come easily to everyone else too. And partly it’s because the more you actually know about a domain, the more you become aware of everything you still don’t know. The expert is more likely to be appropriately humble about their own expertise than the novice is, because the expert can see the edges of what they know and understands there’s territory beyond those edges. The novice can’t see the edges at all. That’s why the most confident voice in a room, on a complex topic, is frequently the least expert voice in that room. And it’s why dismissing an expert’s opinion in favor of your own confident assessment is backwards more often than you’d like to admit. The person with the most experience in a domain is usually the person most likely to have a realistic sense of their own uncertainty.
Now, I want to stop you before you take this too far in your own head. This is not an argument telling you to defer to authority uncritically. Dunning and Kruger’s research doesn’t establish that experts are always right about the thing you’re asking them about. Experts are frequently wrong, sometimes in ways that their own expertise makes them more resistant to recognizing, not less. Philip Tetlock’s research on expert political forecasting, published in his 2005 book Expert Political Judgment and extended in his 2015 book Superforecasting, documents with uncomfortable precision how poorly domain experts perform at prediction relative to how confident they are. The forecasting accuracy of the average eminent political scientist is barely better than chance. The fox beats the hedgehog here. The person with wide but shallow knowledge, who’s genuinely uncertain, often outperforms the narrow specialist who’s very confident. The lesson for you isn’t that expertise is worthless. It’s that expertise plus overconfidence is dangerous, and that calibrated uncertainty, knowing how confident you should be about a specific claim in a specific domain, is the actual skill the Dunning-Kruger research is pointing you toward.
Tom Nichols and the Structural Analysis
Nichols’s argument in The Death of Expertise isn’t really about individuals like Brian, or about you specifically. It’s structural. He’s describing the institutional and technological changes that produced the cultural conditions where a guy like Brian becomes a predictable outcome rather than an outlier. Three structural factors sit at the center of his analysis, and it’s worth walking through each one, because you’re living inside all three right now.
- The internet and the democratization of information access. Information access is not knowledge. Reading about a topic isn’t the same as understanding it, and understanding a topic isn’t the same as being able to apply it competently under real conditions. That distinction is obvious the moment you’ve tried to do something based on online research you’d never actually done before, and yet it’s been systematically obscured by a culture that treats information access as transformative in ways it demonstrably isn’t. You can read everything available online about surgery. That doesn’t make you a surgeon. You can read every article ever written about macroeconomics. That doesn’t mean you can accurately forecast a recession. The information is real. The processing required to turn information into usable knowledge is also real, and it takes time, practice, and feedback, none of which reading on your own provides.
- The collapse of institutional gatekeeping. The traditional mechanisms that used to distinguish credentialed knowledge from lay opinion, peer review, editorial oversight, professional licensing, formal education requirements, have been undermined by the same democratizing forces that made information universally accessible to you. That undermining isn’t entirely a bad thing. Gatekeeping institutions have historically been used to suppress legitimate challenges to established consensus, and decentralized information production has enabled some genuine advances. But the collapse of effective gatekeeping has also removed the mechanisms that used to filter out misinformation and maintain the distinction between expertise and confident ignorance.
- Higher education’s shift from challenging students to serving them. Nichols’s argument here is controversial in the academic circles it comes from. But it’s this: a customer-service model of higher education has reduced how often you get forced to confront your own ignorance and confusion, and that confrontation is a prerequisite for developing real competence. If nobody ever told you, clearly, “you are wrong about this in a specific and important way,” you never developed the metacognitive experience of being confidently wrong. That experience is the primary mechanism through which your intellectual humility actually gets learned. You develop epistemic humility by being wrong in a domain where you were confident. If your education protected you from that experience, you arrive at adulthood without the feedback-based training real epistemic humility requires. You may be living with that gap right now without knowing it.
Notice something about all three of those factors: none of them require you to be lazy or foolish. They’re just the water you swim in. Which is exactly why the protocol you’re about to learn has to be deliberate. Nobody drifts into good calibration. You build it on purpose, against the current.
Jennifer: The Self-Diagnosed Expert
Let me make this concrete with an illustration, and I want you to actually place yourself inside it, because you likely have your own version of it somewhere in your own health history. I’ll call her Jennifer, a composite built from a pattern that shows up constantly, not a specific person I’ve spoken with. Picture her at thirty-six, a marketing manager in Portland, diagnosed with an autoimmune condition at thirty-three. The diagnosis came from a board-certified rheumatologist with subspecialty training in autoimmune disease. The recommended treatment protocol was standard of care, well studied, with a substantial evidence base behind it. Jennifer read about her condition online for two weeks and decided she knew better. She put together a dietary protocol of her own design. It came from seven different wellness websites. It came from a podcast hosted by someone whose primary credential was having recovered from a different condition entirely. And it came from a book by a chiropractor who’d coined his own diagnostic category, one that doesn’t appear in any standard medical taxonomy you’d recognize.
Two years later, in this composite, her disease has progressed to a point where the standard treatment protocol is no longer sufficient. She needs a more aggressive intervention now, one she wouldn’t have needed had she started at diagnosis, and that gap is the one you want to avoid in your own life. She doesn’t see herself as someone who made a mistake based on insufficient expertise. She sees herself as someone who did her research. She’s not lying to you or to herself. She genuinely believes it. The internet gave her access to enormous amounts of information about autoimmune disease. She processed that information through a cognitive framework that couldn’t distinguish between a peer-reviewed clinical trial and a wellness blogger’s personal account, because she lacked the domain training that would let her make that distinction. She was confidently wrong in a way that had real, permanent health consequences. And she still hasn’t recognized it as what it was, because the culture around her doesn’t have an adequate framework for the difference between researching something and actually knowing it.
This pattern reproduces itself across medical, legal, financial, and policy domains millions of times a year, and you have almost certainly done a smaller version of it yourself, in a domain you’d rather not name out loud right now. The specific tragedy in Jennifer’s case is that the information she actually needed, the clinical trial evidence, the treatment guidelines, the specialist’s assessment of her specific presentation, was available to her the whole time. Her rheumatologist had it. She dismissed the rheumatologist as someone with conventional thinking who hadn’t considered the alternatives. That’s exactly the cognitive move Nichols documents as central to the death of expertise. It reframes expertise as a form of bias instead of a form of knowledge. Watch for that move in your own thinking, the next time you catch yourself dismissing someone because their view is “conventional.” It’s the tell that you’re about to do what Jennifer did.
Daniel Kahneman and the Noise Problem
Daniel Kahneman, the psychologist and Nobel laureate, published Thinking, Fast and Slow in 2011. You’ve probably heard the title even if you haven’t read it, and it’s widely read but somewhat less widely understood. His later book, Noise: A Flaw in Human Judgment, came out in 2021, co-authored with Olivier Sibony and Cass Sunstein. It gets at a problem that matters directly to you. Not the bias problem, which is systematic error in a predictable direction. The noise problem, which is random variability in judgment that should be consistent and isn’t.
Noise is the finding that two experts you’d consult in the same field, given identical information, will frequently arrive at different conclusions. Not because one is right and one is systematically wrong, but because judgment under uncertainty is more variable than experts admit and more variable than you probably assume. The variation between expert judgments is enormous in domains including medicine, law, sentencing, hiring, insurance underwriting, and financial forecasting. That doesn’t mean expertise is worthless to you. It means expertise is less precise than the expert typically lets on, and that expert overconfidence is just as real as novice overconfidence. It just shows up in different, subtler ways.
So what do you actually do when two credentialed people disagree with each other? This is worth answering directly, because you’re going to run into it. The first move is to figure out whether you’re looking at genuine expert disagreement or manufactured controversy. Genuine expert disagreement, where qualified specialists who’ve engaged seriously with the evidence hold different positions, is common on frontier questions, and it should produce real uncertainty in you, proportional to how wide the disagreement actually is. Manufactured controversy is different. That’s where the appearance of expert disagreement gets created by amplifying a small minority of credentialed dissenters against a strong consensus. That shouldn’t move your confidence the same amount. The existence of one physicist who disputes climate science doesn’t mean climate science is uncertain the way a genuinely contested question is uncertain. You need to look at the distribution of expert opinion, not just the fact that dissent exists somewhere, if you want to calibrate accurately.
The practical implication for you is calibrated skepticism, not blanket dismissal and not blanket deference. Expert consensus on well-studied questions, vaccine safety, anthropogenic climate change, the effectiveness of established medical protocols, reflects genuine accumulated knowledge, and you should take it seriously. Expert opinion at the frontier of knowledge is different. So is expert opinion on complex predictions with multiple interacting variables, or in domains where feedback loops are long and learning is slow. Weight those differently: more than your own confident intuition, less than the experts themselves usually recommend. The Epistemic Humility Protocol, which we’re about to get into, is built to help you make that distinction accurately instead of collapsing into either extreme.
The Epistemic Humility Protocol: Five Steps
So here’s the protocol you’re going to use. Five steps. Learn them in order, because the order matters for you. Each one sets up the next.
- Domain mapping. Before you form a strong opinion or make a significant decision, explicitly identify which domain the question belongs to and what your actual expertise in that domain is. Not what you’ve read. What you’ve done, studied formally, practiced with feedback on, and been corrected in. The distinction between “I’ve read about this” and “I have relevant expertise” is the first and most important move you can make, and it’s the one Brian skipped entirely.
- Source hierarchy. Not all sources are equal, and pretending otherwise isn’t democratic, it’s lazy. Build yourself a clear hierarchy. Primary research literature goes at the top. Then systematic reviews and meta-analyses. Then clinical or professional guidelines from relevant organizations. Then expert consensus from qualified specialists. Then informed lay synthesis from credible sources. Your own personal experience and intuition sit at the base. Weight what you read according to where it sits on that ladder. Reading a summary of a study is not the same as having read the study, which is not the same as being able to evaluate its methodology yourself.
- Confidence calibration. Before you act on a belief, rate your confidence in it explicitly, and ask yourself what evidence would change that confidence. If you can’t identify the evidence that would change your position, you’re not reasoning. You’re rationalizing. The calibrated thinker has a specific, articulable answer to the question, what would convince me I’m wrong about this.
- Expert consultation, genuinely undertaken. When your domain mapping shows you’re in territory where you lack expertise, go consult actual experts. Not because they’re always right, but because their domain knowledge, including their knowledge of the limits of current understanding, is almost always more accurate than whatever you’d extrapolate from your own general reading. Ask them not just for their conclusion but for their reasoning. Ask what would change their assessment. Ask how confident they actually are. The quality of your questions is where the value lives, not the consultation itself.
- Decision logging. Keep a brief record of significant decisions you make on your own judgment, in domains outside your expertise, including your confidence level at the time and your reasoning. Go back and review those records periodically and check your accuracy. The feedback loop between your confident assessment and the actual outcome is the primary mechanism through which your calibration improves. Most people never close that loop. The few who do become measurably more accurate over time, and there’s no reason you can’t be one of them.
Here’s what step four looked like when Brian actually used it, and it’s worth you picturing yourself doing the same thing. Weeks later, back in front of the same general counsel, he didn’t come in with a rebuttal, and neither should you when your own moment like this arrives. He asked her a single question she said no one had asked her in fifteen years of practice.
“What are the two or three things about this issue that someone with my background would most likely get wrong?”
She paused. Then she answered. Picture yourself asking your own version of that question to someone whose judgment you actually trust. It became the most useful conversation either of them had ever had about intellectual property, because for the first time, one of them actually knew what they didn’t know. Five steps. None of them are complicated on their own. What’s hard is doing all five, in order, on the questions that actually matter to you, instead of only on the easy ones where you were never that confident to begin with.
Philip Tetlock and the Superforecasters
The superforecaster profile has specific characteristics worth adopting yourself, and you can start using every one of them this week. They update their beliefs frequently, in small increments, as new evidence comes in, instead of holding out for one big dramatic revision that carries the emotional cost of admitting you were significantly wrong. They express beliefs in probabilistic terms instead of flat assertions. Not “this will happen.” Instead, “I estimate a sixty-eight percent probability this happens, with this confidence interval.” Try saying that about your own next big decision. They actively seek out disconfirming evidence rather than seeking confirmation of what they already believe. They break complex problems down into smaller sub-problems where the relevant evidence is clearer. And they’re explicitly calibrated: they track the relationship between how confident they say they are and how often they’re actually right, and they use that feedback to get better.
Here’s the finding that should genuinely surprise you, and maybe change how you pick who to listen to: Tetlock found that domain expertise was a poor predictor of forecasting accuracy. Political scientists were no better at predicting political outcomes than educated laypeople using the superforecaster method. What predicted accuracy wasn’t domain knowledge. It was epistemic process, the specific habits of seeking evidence, acknowledging uncertainty, reasoning probabilistically, and continuously updating that make up the superforecaster profile. That doesn’t mean domain expertise is useless to you. It means epistemic process is the primary driver of accurate judgment under uncertainty, and domain expertise without good epistemic process produces overconfident, poorly calibrated predictions that can be worse than a humble, disciplined non-expert’s guess.
Michael: The Investor Who Knew He Couldn’t Know
Here’s another composite worth sitting with, because you may recognize yourself in it if you’ve ever managed your own money. Call him Michael, forty-eight, a software engineer in San Jose, who started investing seriously in his late thirties, convinced he could identify undervalued stocks through his own research. He spent significant time every week reading company reports, analyst commentary, market news. He was confident. He also lost twenty-two percent of his portfolio in eighteen months, badly underperforming the index. Then he did something most confident investors never do: he actually compared his performance to the relevant benchmark, admitted the gap honestly, and asked the question Brian never asked, what am I missing.
What he was missing, and what you might be missing right now if you’re picking your own stocks in your own account, was the efficient market hypothesis and its empirical support. That’s research dating back to Eugene Fama’s work at the University of Chicago in the 1970s, sustained by decades of subsequent evidence. It shows that stock prices in liquid markets absorb available public information fast, and efficiently. An individual investor working only from public information, meaning you, can’t systematically beat the market after costs. Michael’s research wasn’t giving him information the market didn’t already have. It was giving him the illusion of expertise, while generating trading costs that pulled his net return below the index he could have owned with no research at all.
Michael’s response to that discovery is the protocol applied in real life, and it’s the response you’d want from yourself. He assessed his own domain competence honestly, not a professional, no insider access, no proprietary analytical tools. He consulted the research literature instead of market commentary, which is a different and much higher-quality source than what he’d been using. He updated his behavior accordingly and moved to index funds. And he stopped the behavior, active stock picking, that his own honest assessment told him he wasn’t equipped to do well. He isn’t a less sophisticated investor than he was at thirty-eight. He’s a considerably more sophisticated one, precisely because he’s now accurately calibrated about what he doesn’t know. That’s the whole shift you’re being asked to make. It looks like this.
Cass Sunstein and the Group Mind Problem
Cass Sunstein is a legal scholar at Harvard Law School. He’s the author of multiple books on group decision-making, behavioral economics, and institutional design, and his work explains something you’ve felt in yourself but maybe never named. He documents something he calls group polarization. It’s the finding that group deliberation tends to push you toward a more extreme version of the position you already held before the discussion started. Groups of people who are each individually somewhat confident their view is correct tend, after talking it over among themselves, to become more confident rather than less, even when nobody has introduced any new information at all.
This interacts with the death of expertise in a way that should concern you directly, because you’re probably a member of at least one group like this yourself right now, today. Online communities organized around shared beliefs, medical conditions, political views, investment strategies, parenting approaches, systematically generate the conditions for group polarization. The shared belief gets discussed among the people who already share it. The discussion produces a more extreme version of the belief. Dissenting views are absent or suppressed. The resulting certainty ends up higher than any individual member’s starting certainty, even though no information has been added that would justify that increase. The group has become more confident purely by talking to itself.
In our composite, Jennifer found her alternative treatment protocol partly through a Facebook group for people with her condition, the same kind of group you’re probably in right now for something you personally care about. The group’s members shared stories that confirmed the value of the approach, challenged the credibility of conventional medicine, and gradually raised each member’s confidence in the alternative framework. No new clinical evidence was ever generated. The shared testimonials produced a subjective feeling of validation that generated genuinely high confidence in an approach whose actual evidence base was thin. That’s group polarization in action, and in her case it cost her health.
The source-hierarchy step of the protocol exists specifically to counteract this in you. By anchoring your evidence in primary research rather than community consensus, you reduce the group polarization effect that makes community validation feel like evidence when it isn’t. A Facebook group’s consensus tells you what the Facebook group believes. A clinical trial tells you what happened to specific people under specific conditions. Those are two different kinds of information, and treating them as equivalent is precisely the failure Sunstein’s work documents as one of the primary mechanisms of collective irrationality. Notice which one you reach for first. That habit is worth examining.
Thomas: The Contractor Who Built the Wall Wrong
One more composite for you, and this one matters if you’ve ever assumed skill in one area transfers automatically to another. Call him Thomas McBride, fifty-one, a general contractor in Denver, who expanded his business into commercial renovation based on his own assessment that his residential construction expertise transferred directly. He had thirty years of residential experience. He was genuinely excellent at it. He didn’t consult a commercial construction specialist. He didn’t investigate the regulatory and code differences between residential and commercial work in his jurisdiction. He was confident, because his experience was real and his competence in the adjacent domain was genuine.
Think about what that means for you specifically, if you’ve ever assumed one skill of yours covers a neighboring one you’ve never actually tested. The commercial project required removing a load-bearing wall, and his residential experience hadn’t prepared him to identify it correctly in a steel-frame commercial structure, where the structural logic is completely different from wood-frame residential construction. The building inspector caught the error before it turned catastrophic. The project was halted, it required a structural engineer’s remediation plan, and it cost him roughly forty thousand dollars and his commercial license. He hadn’t been incompetent, exactly. He’d been incompetent in a domain he’d correctly assessed as adjacent to one where he was genuinely expert, without doing the work of finding out whether that adjacency was close enough to warrant his confidence.
The domain-mapping step of the protocol, the same one you just learned, would have asked him to explicitly identify the differences between residential and commercial construction as distinct competency domains, instead of treating commercial as just residential-plus-scale. The question isn’t only do I have expertise. It’s is my expertise in this specific domain, or in something similar enough that it transfers. Those are different questions, and the second one is considerably more demanding to answer honestly. Experts make Thomas’s error regularly, treating expertise in an adjacent domain as sufficient qualification for a new one, without mapping the gap between the two. The Dunning-Kruger effect doesn’t stop applying once you become an expert. It just moves to the edge of your competence, where you’re least likely to notice you’ve arrived.
What Epistemic Humility Is Not
It is not an argument that credentials always equal expertise, and you shouldn’t treat them as if they do. Credentials are a proxy for relevant training and feedback-based learning, but the proxy is imperfect. Credentialed professionals can be wrong. They can be operating on outdated information. They can be subject to systematic biases produced by their own training environment, and they can be defending institutional positions that serve their professional interests rather than the truth. The source hierarchy in the protocol weights credentialed expertise above lay opinion, not because credentials guarantee accuracy, but because the training and feedback credentials typically represent produces better calibration than the absence of that training and feedback. It’s a probabilistic bet you’re making, not an absolute rule you’re obeying.
It’s also not an argument against you personally challenging expert consensus when you have reason to. Scientific progress happens precisely through challenges to established consensus, and the history of medicine, physics, and economics you’ve inherited is full of legitimate challenges that were initially dismissed by credentialed authorities and later proven correct. So is it arrogant for experts to wave away every challenge that comes from outside their field? Sometimes, yes. Some of the most important scientific advances came from outside established disciplinary boundaries, and experts have systematic blind spots from their training that outsiders sometimes correctly spot. But the question that actually matters isn’t whether the challenger has credentials. It’s whether the challenge engages seriously with the evidence and reasoning underneath the established position. A challenge that says experts have been wrong before, so this consensus is suspect, isn’t a substantive challenge. A challenge that identifies specific methodological problems with specific studies, proposes an alternative mechanism, and generates testable predictions is a substantive one. If you dismiss the first kind, you’re not being arrogant. If you dismiss the second kind without engaging it, you’ve earned the criticism.
And this is not an argument for permanent uncertainty about everything. On plenty of questions, the uncertainty is minimal, and you know some of them already: the safety of childhood vaccines, the mechanism of evolution, the earth’s approximate age, the effectiveness of specific medical interventions backed by substantial trial evidence. On those, the right response from you isn’t performed agnosticism. It’s acknowledging that these questions have been effectively settled by accumulated evidence. Epistemic humility isn’t something you apply uniformly across every question you encounter. You calibrate it to the actual state of the evidence. High confidence where the evidence is strong and converging. Explicit uncertainty where it’s limited or conflicting. Honest acknowledgment from you when a question sits outside your domain of competence to evaluate at all.
The Social Media Acceleration
The death of expertise isn’t new to you, or to anyone. Nichols traces its roots back to the democratization of opinion that came with the printing press and the Enlightenment’s challenge to institutional authority. What’s new, and what’s changing the way it affects you specifically, is the speed at which confident misinformation now spreads, and the structural incentives social media platforms have built to accelerate that spread instead of slowing it down.
The core mechanism isn’t subtle. Engagement-optimized algorithms reward content that generates an emotional response in you. Confident, simple assertions that confirm what you already believe generate more engagement than detailed, qualified analysis that acknowledges complexity and uncertainty. The result is that the most confident, least detailed, most emotionally activating content gets systematically amplified in your feed. The most epistemically responsible content, the kind that says this is complicated, here’s what we actually know with confidence, here’s what’s genuinely uncertain, gets buried instead. It gets buried by the same systems you probably use as your primary source of information right now.
What that means for you, practically, is that epistemic responsibility now requires active counter-programming against your own information environment, not passive consumption of it. That environment is specifically built to produce the Brian pattern in you: high confidence, low epistemic rigor, and a strong pull toward the comfortable over the accurate. Resisting it takes deliberate effort on your part. The source-hierarchy step of the protocol is a counter-programming exercise, plainly stated. It redirects you away from the high-engagement, low-rigor content the algorithm prefers, and toward the primary sources and expert synthesis the algorithm doesn’t reward but that actually informs good judgment.
This isn’t a call for you to abstain from information entirely. Social media and online content can genuinely help you identify questions worth investigating, understand what concerns and arguments are circulating in a community, and find leads toward better sources. What it can’t do for you, and what using it as a primary source prevents, is distinguish between confident assertion and supported evidence, between a compelling narrative and rigorous methodology, between emotional resonance and accuracy. You have to draw that line yourself, deliberately, because the platform won’t draw it for you, and it has strong financial incentives to keep it blurred.
The Medical Domain
It’s also the domain where the legitimate criticisms of expertise get genuinely complicated for you, and you should hold both things at once in your own head. Medicine has a real history of overconfident expert consensus that turned out to be damaging. Hormone replacement therapy was endorsed at a scale that exposed millions of women to elevated cardiovascular and cancer risk. Helicobacter pylori was resisted for years as the actual cause of peptic ulcers. Pain medication was underdosed in cancer patients out of addiction fears that weren’t calibrated to that specific population. These are real failures of expert consensus. They’re part of why patients like you have learned skepticism, and that skepticism isn’t irrational.
But the right response to those failures isn’t Jennifer’s response, dismissing expert consensus in favor of a self-assembled alternative built from non-peer-reviewed sources. It’s Tetlock’s response. You apply the same evidence standards to the alternative claims that should have been applied to the consensus claims in the first place. You go seeking out the feedback mechanisms, the treatment outcome data, the published clinical evidence, that would actually distinguish between the alternatives for you. You do that instead of just preferring one because it fits your emotional or ideological leaning. The question in front of you isn’t should I trust the medical establishment. It’s what does the best available evidence actually show about this specific intervention for this specific condition, and how confident should I be in that evidence given its quality and source. Those are different questions. The second one is much harder to answer. It’s also the one that leads you somewhere useful.
The Legal Domain
Brian’s meeting with the general counsel reproduces itself across the legal domain constantly, with the same pattern and similar consequences, and it might be closer to your own life than you think. The legal system is a domain where your confidence is particularly dangerous to you. Legal reasoning is deeply technical, context-dependent, and contingent on specific statutory language, regulatory interpretation, and case law. You can’t access any of that from a lay summary you found online.
There’s a phenomenon worth naming directly: I-know-my-rights. It’s the widespread conviction that reading a summary of a legal principle is the same as understanding how it applies to your specific situation. Under current law. In your specific jurisdiction. It’s one of the most consequential everyday expressions of the death of expertise, and you’ve probably indulged it at least once. Employment law, contracts, landlord-tenant disputes, intellectual property, family law. These are all domains where you could easily make a decision based on confidently misapplying a general legal principle to your specific situation. And the general principle you’re leaning on might simply not apply the way you think it does.
Applied to your own legal situations, the protocol gets specific fast. The domain-mapping step asks you to identify not just whether legal knowledge is relevant, but which branch of law, in which jurisdiction, under which specific statutory regime. Legal knowledge doesn’t transfer cleanly across jurisdictions the way you might assume it does. The employment law that applies in California is materially different from what applies in Texas or in Germany. The principle you read about in a summary might be accurate for federal law and completely inapplicable in your state. The case that established the principle you’re relying on might have been overturned or distinguished since, in ways that eliminate its relevance to your situation entirely. The confidence appropriate to “I have a general understanding of this legal principle” is substantially lower than what most people apply when they actually act on that understanding. Yours probably is too. Check it before you act, not after.
The Financial Domain
The financial domain has a feature that sets it apart from medicine and law, and it matters directly to your wallet, your actual money: it’s a domain where other sophisticated actors benefit directly from your overconfidence. In medicine, your overconfidence in your own diagnosis primarily harms you. In financial markets, your overconfidence in your own stock-picking ability gets systematically exploited by professional counterparties with better information, better models, and faster execution than you’ll ever have.
Michael’s experience with stock picking is the norm, not the exception, and it’s probably your experience too if you’re honest about your own returns, because the research backs this up consistently. Actively managed retail portfolios underperform their relevant benchmarks after costs, and the degree of underperformance correlates directly with trading activity. The more confident you are, the more you trade, and the more you trade, the worse your performance relative to the market gets. Overconfidence in financial judgment is, quite literally, a mechanism through which wealth transfers from overconfident retail investors like you to professional market participants who are better positioned to take the other side of your overconfident trades.
This is also where Tetlock’s superforecaster findings have the most immediate practical application for you. Research on professional fund manager performance shows that the vast majority of actively managed funds underperform their benchmark over time. And past outperformance is a poor predictor of future outperformance. That means even the professional managers who beat the market in the past were largely doing it through luck rather than skill. If professional investors with research teams, proprietary data, and decades of experience can’t reliably beat a passive index, you, reading some financial commentary on your phone, are not going to do it either. The response the protocol points you toward, index investing, minimal trading, an honest acknowledgment that you’re not positioned to generate alpha, produces dramatically better outcomes for you than the alternative. That’s true for almost everyone who’s ever tried the alternative.
The Intellectual Virtue at the Heart of the Protocol
Epistemic humility is, at bottom, an intellectual virtue in you. It’s a stable disposition, like your physical fitness or your professional skill, and you develop it through deliberate practice. You don’t get it by simply deciding you already have it. You can’t decide to be epistemically humble and thereby become so, any more than you can decide to be physically strong and thereby become so. You build the virtue through the repeated practice of the specific habits that make it up: domain mapping, source hierarchy, confidence calibration, expert consultation undertaken in good faith, decision logging with honest updating.
What makes this genuinely hard for you is that it runs directly against several powerful psychological drives you didn’t choose and can’t simply switch off. The drive for cognitive closure, your preference for clear, certain answers over acknowledged uncertainty, is well documented, and it gets stronger under stress, time pressure, and high stakes, which is exactly when you need calibration the most. The drive for social conformity, to hold the beliefs your community values, is equally well documented, and it directly conflicts with your willingness to update toward uncomfortable evidence. The drive for self-consistency, to defend the positions you’ve publicly held, makes admitting you were wrong feel like losing face even when it’s actually a gain in accuracy.
Here’s where it starts young, and if you’re raising kids, this applies to them too. The most effective way to build good epistemic judgment is honest feedback in environments where being wrong has real but manageable consequences. Children who get to act on confident incorrect beliefs and experience the natural consequences, not catastrophic ones, but real ones, develop epistemic calibration through direct experience. Children who get consistently protected from being wrong arrive at adulthood differently. Well-meaning adults either agree with their incorrect positions, or shield them from the consequences of acting on them. And those children arrive at adulthood with the Brian pattern already installed in them: confident, uninformed, resistant to correction. Ask yourself honestly which version you were raised as, and which version you’re raising. The discomfort of being wrong and knowing it isn’t a trauma you need to spare anyone. It’s the primary mechanism through which intellectual development actually happens, in you and in anyone you’re raising.
These drives are real in you. You don’t overcome them by simply deciding to be more humble. You manage them through the specific habits of the protocol. You create explicit decision records that make acknowledging your errors a private practice rather than a public humiliation. You source information from outside your social community to reduce the conformity pressure on you. And you reframe “updating my position” as evidence of intellectual rigor in you, not evidence of weakness or inconsistency. That reframe is available to you because it’s accurate. The person who updates their position based on evidence is demonstrating exactly the epistemic quality that produces better decisions over time. The person who doesn’t is demonstrating the opposite. Brian eventually learned to see updating as strength rather than surrender. That’s the shift the protocol is built to produce in you. It takes time. It takes honest practice. It’s worth doing anyway.
Building an Information Diet That Produces Calibration
Redesigning your information diet doesn’t require you to read academic journals exclusively. It requires a few specific structural changes on your part. First, for the domains where you regularly need to make consequential decisions, medical, financial, legal, professional, identify one or two high-quality primary or synthesis sources and make those your default reference, instead of your social feed. The Cochrane Collaboration for medical evidence. A peer-reviewed journal in your own professional field. The IMF World Economic Outlook for macroeconomic information. These sources are less entertaining than the social media content covering the same topics. They’re more accurate. That tradeoff is clear, and it’s worth making for yourself.
Second, deliberately include sources that contradict what you already believe in your regular reading. Not so you’ll necessarily be persuaded, you might not be, and that’s sometimes the right outcome. But so you understand the strongest version of the opposing case, which you need in order to know whether your own position is actually well-founded or just comfortable. The best argument for the position you hold is worth your time. The best argument against it is essential. Without both, you’re not actually reasoning. You’re just selecting what already feels good to you.
Third, cut down on the high-volume, low-quality information you’re consuming right now. The news cycle produces enormous quantities of information about current events at a level of analysis that’s almost never sufficient for you to form a confident opinion on a complex topic. Following a breaking story through twenty-four hours of coverage doesn’t give you twenty-four hours of new information. It gives you one or two new facts wrapped in hours of speculation, repetition, and emotional amplification. For most fast-moving stories, the right move for you is to wait for the synthesis, the careful retrospective account that comes after the initial noise settles, rather than consume the noise live and form a confident opinion in the moment. The impatience that makes this hard is itself an epistemic failure: the conviction that you need to know right now, when accuracy actually requires waiting for the signal to separate from the noise.
The Practical Test: How Accurate Are You, Actually?
Now for the step most people skip, the one you’ll be most tempted to skip too, and the one that matters most for you: decision logging. Here’s a question you might be asking right now. Doesn’t all of this just paralyze you? If you have to rate your confidence and interrogate your sources on every decision, don’t you end up unable to act at all? No. The protocol isn’t about eliminating your confidence. It’s about calibrating it. Most decisions, including most of the important ones, need to get made with incomplete information, and that’s fine. The goal isn’t certainty before you act. It’s appropriate confidence given the evidence you actually have. The decision-logging step is where that gets built. By tracking your decisions and their outcomes, you build an accurate model of which kinds of decisions you tend to make well. You also learn which you tend to make poorly. From there, you can calibrate your confidence in your own judgment, domain by domain, based on feedback instead of general self-assessment.
Every forecasting researcher who’s tried to measure human calibration has found the same pattern in the people they study, and it’s very likely true of you too: confidence levels run systematically higher than accuracy rates. Meteorologists, who get regular, rapid feedback on how accurate their forecasts actually are, are among the most calibrated predictors on the planet. When a meteorologist says seventy percent chance of rain, it rains about seventy percent of the time they say that, because their feedback loop is tight enough to enforce that calibration. Political pundits, who rarely get clear feedback on how accurate their confident predictions were, are among the least calibrated people you’ll ever listen to. Their stated confidence bears almost no relationship to their actual accuracy.
The decision log is how you build a feedback loop into the domains of your life that don’t naturally provide one. Write down your significant confident predictions and decisions: I’m confident this business decision will produce X outcome. I believe this intervention will be more effective than the conventional approach. I expect this investment to outperform the market by Y percent. Then, once the outcome is known, compare it to what you predicted. Not casually. Precisely. This comparison will be uncomfortable for you. It will reveal that your confidence runs systematically higher than your accuracy in specific domains. That revelation is the single most useful piece of information the protocol can hand you, because it tells you exactly where your calibration needs the most work.
Tetlock’s superforecasters treat this as a discipline. They keep formal records of their predictions and their accuracy. They score themselves using Brier scores, a calibration metric that rewards both accuracy and appropriate uncertainty. They review their own records regularly to catch systematic biases in their predictions. You don’t need that level of rigor for ordinary life decisions. What you do need is the basic habit of honestly acknowledging your prediction outcomes and using them to update your model of your own judgment, instead of explaining the misses away.
“The single best predictor of good judgment is the willingness to be wrong, track the wrongness, and update accordingly.” — Philip Tetlock, paraphrase from Superforecasting.
When Expertise Is the Problem
- Treat experts whose consensus position is financially supported by parties with a strong interest in the conclusion as needing a higher evidentiary bar than experts with no such conflict.
- Be skeptical in proportion to the certainty being claimed, when that claim comes from a domain where feedback loops are long and the evidence base is thin.
- Give extra scrutiny to claims made by a very narrow group of specialists, without meaningful engagement from adjacent fields that would actually be positioned to evaluate the reasoning.
Honestly applying this protocol means you have to acknowledge, for yourself, the cases where expert consensus has failed, sometimes catastrophically, and where being skeptical of credentialed authority was genuinely the right call. The opioid crisis is one. Oxycontin was marketed by Purdue Pharma with the active assistance of credentialed medical researchers and with regulatory approval, on the basis of evidence that significantly understated the addiction risk. People who trusted the expert consensus and took the medication were harmed by it. People who were appropriately skeptical of industry-funded research and avoided the medication were, in retrospect, correct to be.
The question you need answered is how to spot these cases in advance, not after the fact. A few markers are reliable enough to use:
None of these markers work like an algorithm you can just run for yourself. They’re heuristics that require your own judgment to apply well, which means applying them requires domain knowledge in itself. The meta-level humility here is acknowledging that evaluating expert consensus accurately is a skill on its own, one that improves with practice, and one most people haven’t actually developed. Which is, once again, an argument for the protocol applied with real sophistication, rather than for either blanket deference or blanket dismissal from you. The middle path is harder to walk. It’s also the correct one. Walk it with the care it actually requires.
Brian, by his own account, is better now. He still forms opinions quickly. That’s just his nature, and it’s probably yours too in some domain. But he’s learned to label them clearly in his own mind: this is my initial opinion, formed without adequate domain knowledge, and it should be treated as a hypothesis, not a conclusion. That labeling doesn’t eliminate the opinion. It tracks it accurately, which is all the protocol is actually asking of you. You’re going to form opinions quickly. You’re human. The protocol doesn’t ask you to stop. It asks you to know, when you do it, what kind of opinion it actually is.
The Identity Problem
The deepest obstacle to epistemic humility in you isn’t cognitive. It’s identity. When a belief becomes part of who you are, when being anti-vaccine, or pro-this-diet, or skeptical-of-that-institution becomes part of your self-concept and your social identity, updating that belief starts to feel like self-destruction to you, not intellectual growth. This isn’t irrationality in the simple sense. It’s a rational response to a genuine social cost: communities organized around shared beliefs frequently punish members who publicly update away from them.
Nichols identifies this as one of the main mechanisms that makes the death of expertise self-reinforcing in you and in everyone around you. Once your epistemic positions become tribal markers, the social cost of updating them, the implicit exclusion from the community that shares them, becomes a powerful barrier to you honestly engaging with contrary evidence. Someone who’s publicly and repeatedly insisted vaccines cause autism doesn’t face merely the abstract cost of admitting they were wrong. They face the concrete cost of losing the community organized around that shared belief. That cost is real, and writing it off as mere stubbornness misses the genuine social dynamics you’re actually up against.
Notice how much this shows up in your own political opinions specifically, because that’s probably where it hits you hardest of anywhere in your life. Political positions are deeply identity-integrated in ways that make updating based on evidence feel like self-betrayal, which is exactly why the protocol is hardest to apply there and most needed there. Domain mapping still applies: most people’s political expertise is limited to consuming mediated accounts of political reality rather than systematically engaging with political science, policy research, or primary government data. Source hierarchy still applies: political commentary from partisan outlets should be weighted differently than peer-reviewed political science research. And confidence calibration still applies: given the genuine complexity of most policy questions, the appropriate confidence level in a specific policy position, for most people including you, is lower than what they typically hold. Applying this to your own political positions, not just the ones you already disagree with, is the genuinely hard part. It’s also the part that determines whether any of this actually changes anything for you.
The protocol addresses the identity problem through the private nature of the decision-logging step. You’re not being asked to publicly recant anything. You’re being asked to honestly track your accuracy in private, which lets the updating process begin without the social cost of public acknowledgment. If a public shift comes, it comes later, after the private evidence is sufficient to warrant it. This sequencing, private updating before public acknowledgment, is how intellectually honest people actually change their minds, and building the protocol around it, instead of demanding immediate public correctness from you, makes it dramatically more likely you’ll actually practice it.
Institutional Trust and When It’s Warranted
The death of expertise has been accelerated by genuine institutional failures, and they’ve reduced how much trust you personally can warrant giving to the expert institutions in your own life. Scientific replication crises in psychology and nutrition research, regulatory capture in pharmaceutical and financial sectors, journalistic failures of fact-checking, and documented cases of outright scientific fraud have all handed you legitimate evidence that institutional credentials don’t guarantee reliable knowledge. The question isn’t whether these failures are real. They are. The question is what your appropriate response to them should be.
The wrong response, the one you’ll be tempted toward, is what Sunstein calls system neglect: dismissing an entire institutional framework based on its failures, without giving proportionate weight to its successes. The medical establishment has produced the vaccines, antibiotics, surgical techniques, and diagnostic tools that have added thirty years to human life expectancy over the last century. The same scientific establishment that produced replication crises also produced the double-blind, placebo-controlled trial that makes those crises recognizable as failures in the first place, and that gives you the standard to evaluate claims against. The institutions are flawed and valuable at the same time. Your appropriate response is calibrated trust: high where the institutional track record and the quality of the feedback mechanisms warrant it, lower where conflicts of interest, inadequate feedback, or documented failures warrant your skepticism. Not blanket trust. Not blanket rejection. Calibrated trust. That’s what the protocol produces in you when you apply it consistently.
The long-term stakes here are bigger than you individually, and Nichols is forceful about this. The collapse of epistemic standards isn’t a neutral development. It has real political and social consequences that reach you whether you’re paying attention or not. Democratic governance requires citizens who can form accurate beliefs about complex policy questions, evaluate the claims of competing experts with reasonable sophistication, and make decisions that reflect something close to reality instead of confident misinformation. A society where the epistemic standards for forming and acting on beliefs have collapsed can’t function as a democracy, because democracy’s whole premise, that citizens can make informed choices, requires that citizens actually have the epistemic infrastructure to be informed. The protocol you’re learning here isn’t just a personal development tool for you. It’s a civic obligation you carry. Know what you know. Know what you don’t. Act from that distinction honestly. Everything else depends on enough people doing this.
Practical Daily Habits of Calibrated Thinkers
The protocol is useful to you as a framework, and it will actually change your life as a set of daily habits. Here are the specific habits that calibrated thinkers, the superforecasters, the expert practitioners with genuine epistemic humility, the decision-makers who consistently outperform their peers, demonstrate over and over.
- They hedge their language when they’re genuinely uncertain. “I think,” “my current understanding is,” “the evidence I’ve seen suggests,” not as rhetorical soft-pedaling, but as an accurate representation of their actual epistemic state. You’ll notice confident people tend to drop hedges even when hedges are warranted, because confidence gets socially rewarded and hedged language signals lower status. The calibrated thinker is willing to pay the social cost of accuracy. So should you.
- They separate their conclusions from their evidence. A fully calibrated statement isn’t just “X is true.” It’s closer to this: “my evidence for X is Y, my confidence in Y given my domain expertise is Z, and therefore my confidence in X is W.” Try building one of your own statements that way right now, on your own paper, about your own decision. Most people never explicitly separate these elements in their own thinking, which means they can’t identify where their reasoning went wrong once they’re eventually proven incorrect. If you separate conclusion from evidence, you can update precisely. If the evidence turns out wrong, you update the conclusion. If the evidence was right but your interpretation of it was wrong, you update the interpretation. Those are different updates, and making them precisely requires you to have had the pieces separated to begin with.
- They actively seek disconfirming evidence as a practice, not by accident. Once a week, take your most confidently held position in a domain where you’ve made consequential decisions, and spend twenty minutes specifically looking for the strongest available argument that you’re wrong. Not necessarily to change your mind. To test whether your position survives honest engagement with its best challenge. Positions that survive the test deserve your confidence in them. Positions that collapse under it needed updating anyway.
- They build consultation into consequential decisions. They have a process that includes explicit consultation with people whose expertise in the relevant domain exceeds their own, not as a vote to tally, but as a source of domain knowledge they genuinely lack. That consultation is only productive if it’s genuine, if you’re actually open to changing your view based on what you hear, rather than fishing for validation of a position you’d already formed. The difference between genuine consultation and seeking confirmation is the difference between actually using this protocol and performing it. Performance gets you nothing. Genuine practice gets you everything the protocol promises.
Thomas now consults a structural engineer before taking on any commercial renovation project, which is exactly what you should be doing in your own equivalent domain. Not because he’s stopped trusting his own judgment in residential construction, where it’s genuinely sound. But because he learned, at forty thousand dollars’ cost, exactly where his expertise ends and someone else’s begins. That line is the most valuable piece of professional knowledge he’s ever acquired. This protocol is the systematic practice of drawing that line accurately for yourself, every time you need to draw it, before your own forty-thousand-dollar lesson has to teach it to you.
The Long View
The case for this protocol isn’t only about you individually, even though everything so far has been aimed at you individually. It’s social too, and it’s worth you zooming out to see why. Nichols’s deepest concern in The Death of Expertise isn’t that individuals make worse decisions than they could. It’s that the collective epistemic degradation of a society produces institutional failures that no individual, including you, can fully protect themselves from. When the public can’t distinguish between the consensus of climate scientists and the claims of fossil-fuel-funded think tanks, climate policy fails. When voters can’t evaluate the empirical claims made in political advertising, democratic accountability fails. When patients can’t distinguish clinical evidence from wellness industry marketing, public health fails. You practicing epistemic humility isn’t just better for you individually. It’s a small, real contribution to the social epistemic infrastructure that all of these institutional functions actually depend on.
That sounds abstract, but it isn’t. Brian in a meeting with the general counsel is a small thing. Brian teaching his kids to ask “how do you know that?” and “what evidence would change your mind?” is a multigenerational contribution to the epistemic culture around him. The protocol practiced by you individually compounds into a culture practiced socially, which is the only level where the problems Nichols documents can actually get addressed. Start with yourself. That’s genuinely where this begins. The ripple effect is real, even if you can’t see it from where you’re standing. Begin now, with whatever question you’re most confidently wrong about, and ask it honestly. That’s enough to start. Do it every week, on something that actually matters to you. The calibration follows. The better decisions follow. And over time, the culture around you follows too, one accurately labeled hypothesis at a time.
You should also know where to go from here if this landed for you. The deeper framework for decision-making under uncertainty lives in our mindset tools section, and the relationship between expertise, humility, and effective action sits at the center of the resilience principles the whole show is built around. If you want the philosophy behind why we build it this way, that’s on the about page.
Closing: Know What You Know
The experts you talk to aren’t always right about your specific situation. You aren’t always right either, and you know it. The difference between you and them isn’t that they’re infallible. They’re not. It’s that they’ve typically invested years building the domain knowledge and the feedback loops that make their judgments more reliable than yours in their specific domains. Respecting that difference isn’t submission on your part. It’s accuracy, and accuracy serves you, not them. And accuracy, as Brian and Jennifer and Thomas all eventually discovered in their own ways, is the most useful thing you can bring to any consequential decision you face. Know what you know. Know what you don’t know. Act from that honest foundation. Everything downstream of that honesty gets better for you. Everything upstream of it is guesswork dressed up as certainty.
Kahneman showed you the noise in your own judgment. Tetlock showed you the fix for it. Nichols showed you the stakes for you. Dunning and Kruger showed you the mechanism inside you. Sunstein showed you the social amplification around you. The synthesis is in your hands now. Apply it honestly for yourself, apply it consistently in your own life, and measure your own results. Those results will tell you everything you need to know about how well-calibrated you personally were before you started this. That feedback is the beginning of wisdom for you. It’s available to you, and to anyone willing to do the honest work of finding out where they were wrong. Are you willing? Then you start today. The work is waiting for you, only you. Your calibration will follow.
Here’s how it actually ended for him, and it’s worth hearing before you go apply any of this yourself. Brian eventually went back to the general counsel and had the conversation he should have had from the start.
“I’ve been reading about this, and I want to understand where my understanding is incomplete before I form a strong view.”
She spent forty-five minutes with him. It’s the kind of forty-five minutes you could have with someone in your own life, if you asked the right question of the right person. He learned things in that conversation he wouldn’t have learned from another year of reading article summaries on his own. His position on the intellectual property policy changed, not because he deferred to her automatically, but because he engaged honestly with someone who actually knew the subject. That’s what this protocol produces in you when you actually practice it. Not passivity. Genuine learning, which turns out to be the one thing the internet can’t give you, but a good conversation with someone who actually knows what they’re talking about can give you.
One more thing before you go, and I want you to hold onto this one specifically. The most dangerous person in any room you walk into isn’t the one who knows the least. It’s the one who knows the least and knows it the least. This protocol makes you the most dangerous person in the room in the right sense: the one who knows exactly what they know, exactly what they don’t, and exactly what the difference is worth. That person is unstoppable in the domains they genuinely own, and honest enough in the domains they don’t to avoid the mistakes that undo everything else. Be that person. You have the protocol now. The rest is practice. Find the people who know what they’re talking about. Ask them the right questions. Know what you do not know. That is the whole thing, and it’s yours to start today.
Editorial StandardsCorrectionsMedical DisclaimerAbout Our ContentAffiliate DisclosureSite Map