Berkson's Paradox: How Selection Bias Creates False Associations

In 1946, at the Mayo Clinic in Rochester, Minnesota, a physician and statistician named Joseph Berkson discovered a puzzling phenomenon: according to hospital data, patients with diabetes appeared less likely to develop cholecystitis. Could diabetes somehow protect the gallbladder? Berkson's answer was: No, this is an illusion created by the data source itself. This discovery revealed one of the most insidious and dangerous traps in statistics — later named "Berkson's Paradox" in his honor.

I. What Is Berkson's Paradox?

Definition of the Paradox

Berkson's Paradox refers to the phenomenon where the method of sample selection creates a spurious negative correlation between two variables that are actually independent or positively correlated.^[1] Put more plainly: when we observe only a "filtered" sample, we may perceive associations that do not actually exist.

This phenomenon is also known as "Collider Bias" or "Selection Bias." In the terminology of causal inference, when we condition on a "collider" variable, we create a spurious association between originally unrelated variables.^[2]

Distinction from Simpson's Paradox

Berkson's Paradox is often confused with Simpson's Paradox, but their mechanisms are fundamentally different:

• Simpson's Paradox: When aggregating data, confounding variables cause trends to reverse. The issue is "how to correctly combine data."
• Berkson's Paradox: The sample selection mechanism creates spurious associations between otherwise unrelated variables. The issue is "where the data comes from."

Using causal diagrams to illustrate: Simpson's Paradox involves confounders (common causes), while Berkson's Paradox involves colliders (common effects). In Simpson's Paradox, we should control for the confounder; in Berkson's Paradox, we should not condition on the collider — but often our sample has already been conditioned.^[3]

Mathematical Explanation: The Trap of Conditional Probability

Let us use mathematics to understand Berkson's Paradox. Suppose there are two independent diseases A and B in the general population:

P(A) = p_A, P(B) = p_B, and P(A ∩ B) = p_A · p_B (independence)

Now suppose we only observe hospitalized patients. A person is hospitalized if they have disease A, disease B, or both. Let the hospitalization condition be H = A ∪ B.

Among hospitalized patients, the probability that a person has disease B given that they have disease A is:

P(B | A, H) = P(A ∩ B | H) / P(A | H)

Through calculation, it can be shown that:^[4]

P(B | A, H) < P(B | H)

This means: among hospitalized patients, those with disease A are actually less likely to have disease B — even though in the general population, A and B are completely independent! This negative correlation is purely an illusion created by the selection mechanism.

The intuitive explanation is: if a person is already hospitalized because of disease A, they "don't need" disease B to explain why they are in the hospital. Conversely, if they don't have A, they are more likely hospitalized because of B. This "explaining away" effect is the core of Berkson's Paradox.^[5]

II. Classic Medical Cases

Diabetes and Cholecystitis: Berkson's Original Discovery

In his classic 1946 paper "Limitations of the Application of Fourfold Table Analysis to Hospital Data," Berkson systematically described this problem for the first time.^[6]

He analyzed hospitalization data from the Mayo Clinic and found a puzzling pattern:

Group	Cholecystitis Incidence
Diabetic patients	Lower
Non-diabetic patients	Higher

If interpreted naively, this data would lead us to conclude that diabetes has a protective effect on the gallbladder. But Berkson pointed out that this "finding" was entirely an artifact of selection bias.

The Mechanism of the Bias

The problem was that the study sample consisted of hospitalized patients, not the general population. A person was included in the study on the condition of being "hospitalized," and reasons for hospitalization could be diabetes, cholecystitis, or both.

• Among hospitalized patients, if a person has diabetes, they already have a "reason" for being hospitalized and don't need cholecystitis to explain their presence.
• Conversely, if a hospitalized patient doesn't have diabetes, they are more likely hospitalized because of another condition (such as cholecystitis).
• Therefore, in this selected sample, diabetes and cholecystitis appear negatively correlated — but this is entirely an artifact of sample selection.

Berkson's key insight was: A hospital is not a microcosm of society. Research conclusions based on hospital data may be completely non-generalizable to the general population.

Respiratory Diseases and Fractures

Similar biases have been found in other medical studies. If we studied the relationship between respiratory diseases and fractures in a hospital, we might find a negative correlation: patients with respiratory diseases have fewer fractures.

This does not mean that asthma protects you from breaking a leg! Rather, it is because:

• Most people have neither respiratory diseases nor fractures, and they don't appear in the hospital sample.
• People who enter the hospital sample must have at least one problem.
• If they are hospitalized for respiratory disease, they don't "need" a fracture to explain their hospitalization; and vice versa.

The Obesity Paradox: A Contemporary Controversy

In recent years, a puzzling finding has emerged in epidemiology: among certain groups of chronic disease patients (such as those with heart disease or kidney disease), obese individuals actually have lower mortality rates — this is known as the "Obesity Paradox."^[7]

Many researchers suspect Berkson's Paradox is at work here. The logic is as follows:

• Both obesity and other risk factors can lead to heart disease.
• An obese heart disease patient may not "need" many other risk factors to develop the condition.
• A normal-weight heart disease patient may need more other risk factors (such as genetics, smoking) to develop the condition.
• Therefore, within the selected group of heart disease patients, obese individuals may have fewer overall risk factors.

A study published in the Journal of the American Medical Association in 2016 showed that when selection bias was controlled for, the obesity paradox largely disappeared.^[8] This case reminds us that even in top medical journals, Berkson's Paradox may be lurking.

III. Berkson's Paradox in Everyday Life

"Why Are Attractive People Often Less Considerate?" — The Dating Market Paradox

This is perhaps the most relatable example of Berkson's Paradox. Many people complain: "Why do good-looking people tend to have bad personalities?" or "Why are talented people often difficult to deal with?"

Suppose whether a person is worth dating depends on two independent traits: physical attractiveness (A) and likable personality (B). In the general population, these two may be independent — attractive people are just as likely to have good or bad temperaments as anyone else.

However, when we "filter" dating prospects:

• We typically only date people who have at least one strong point (physically attractive or likable personality).
• In this filtered group, if someone is very attractive, they already meet the dating "threshold" and don't need a great personality.
• If a dating prospect isn't particularly attractive, the fact that they're in the dating pool means they likely have an especially good personality.

Result: among the people we actually date, appearance and personality show a negative correlation — but this is caused by the selection mechanism, not a real pattern in the population.^[9]

This example illustrates that our observations about the world are often limited by the samples we encounter. If you always feel that "attractive people are difficult," perhaps the problem isn't with the world but with the fact that you're encountering a filtered sample.

Hollywood Movies: Why Do High-Budget Films Get Worse Reviews?

Critics and audiences often complain: why do films with hundreds of millions in investment often get worse reviews than low-budget independent films? Does more money mean worse quality?

This is Berkson's Paradox again. Films that make it to theatrical release must pass some "quality threshold" — either the script is good enough, or the stars are big enough, or the special effects are dazzling enough, or the production budget is high enough.

• A high-budget film can secure release on budget alone, without needing a particularly good script.
• A low-budget film that makes it to release typically needs an exceptionally good script or other outstanding qualities.

Therefore, among "released films" — a filtered group — cost and quality show a negative correlation. But if we could see all films ever produced (including low-budget duds that never made it to theaters), this correlation would disappear.

Academia: The Strange Citation Distribution in Top Journals

Why does the citation distribution of papers published in top journals sometimes look unusual? Why can some "low-impact" topics make it into elite journals?

For a paper to be published in a top journal, it needs to meet some "excellence threshold" — either the research topic is extremely important, or the methodology is innovative, or the data is unique, or the analysis is particularly sophisticated.

• If a paper's topic is already a trending issue (which would naturally attract many citations), the bar for methodological innovation may be lower.
• If a paper's topic is relatively niche, the fact that it made it into a top journal usually means the methodology or analysis is especially brilliant.

Result: in top journals, trending topics and methodological innovation show a negative correlation. This is not because researchers studying popular topics are less innovative, but rather a product of the selection mechanism.^[10]

Social Media: Why Do Influencers' Lives Look So Perfect?

Scrolling through Instagram or social media, you might notice that influencers all seem to live perfect lives: beautiful and wealthy, interesting and happy. Does this really reflect reality?

Becoming an "influencer" is itself a powerful selection mechanism. For someone to gain a large following on social media, they need to meet some "attractiveness threshold" — appearance, wealth, lifestyle, humor — at least one must be particularly outstanding.

In the general population, these traits may be independent. But among "influencers" — a highly filtered group:

• If an influencer has an ordinary appearance, their popularity likely stems from exceptionally interesting content or a particularly attractive lifestyle.
• If an influencer is exceptionally good-looking, they don't need other aspects to be equally outstanding to gain followers.

But overall, the people who enter your field of vision are all those who "exceeded the threshold" in some dimension. What you see is a filtered sample of the top, not the true distribution of the population. This partly explains why browsing social media can cause anxiety and dissatisfaction.

IV. The Statistical Explanation Behind "Great Achievers from Humble Origins"

"Great achievers from humble origins" is an age-old saying suggesting that people from impoverished backgrounds are more likely to achieve greatness. Similar observations include: "Among successful people, those from difficult backgrounds are disproportionately represented" or "Many top CEOs grew up in poor families."

Is this true? Or is Berkson's Paradox at work?

Selection Bias: Seeing Only the "Successful" Sample

Suppose success depends on two factors: innate advantages (P, such as family background and social resources) and personal ability (T, such as intelligence, perseverance, and luck). A person "succeeds" when P + T exceeds a certain threshold.

• People from privileged backgrounds (high P) need only average personal ability to succeed.
• People from disadvantaged backgrounds (low P) need exceptionally high personal ability to succeed.

Now, when we observe only the "successful" group:

• Successful individuals from wealthy families have a wide distribution of ability — ranging from average to exceptional.
• Successful individuals from poor families are almost all extraordinarily capable — because only exceptional ability can compensate for a disadvantaged background.

Therefore, within the filtered group of "successful people," family background and personal ability show a negative correlation. Successful people from poor backgrounds have higher average ability, creating the impression of "great achievers from humble origins."

But this does not mean poor families are "more likely" to produce successful people! If we looked at the entire population, children from wealthy families still have higher average achievement. We simply don't see the majority of people from poor backgrounds with average ability who never succeeded.

The Amplifying Effect of Media Narratives

This bias is further amplified by the media. "Rich kid inherits the family business" is not a compelling story, but "poor kid rises to the top through sheer determination" is an inspirational classic. The media's tendency to report underdog stories further reinforces the impression of "great achievers from humble origins."

The real lesson is: when we observe a filtered group, the patterns we see may be entirely a product of the filtering mechanism, not a causal relationship.

V. Historical Background: Joseph Berkson the Man

From Mathematics to Medicine

Joseph Berkson (1899-1982) was a unique scholar — trained in both medicine and statistics.^[11]

Berkson was born in New York City and earned a physics degree from the City College of New York before pursuing a master's in physics at Columbia University. However, his interests gradually shifted toward medicine, and he ultimately earned his medical doctorate from Johns Hopkins University.

In 1931, Berkson joined the Mayo Clinic and became the head of its Division of Biometry and Medical Statistics. He held this position for over 40 years, until his retirement in 1973. The Mayo Clinic was one of the world's largest medical institutions at the time, with abundant patient data that provided ideal material for Berkson's statistical research.

The Birth of the 1946 Paper

While analyzing data from the Mayo Clinic, Berkson discovered many seemingly contradictory associations. Researchers at the time frequently used "fourfold tables" (2x2 contingency tables) to analyze relationships between diseases, but Berkson realized this approach had a fundamental problem.

In 1946, he published his groundbreaking paper in the Biometrics Bulletin.^[6] In this paper, Berkson not only described the bias phenomenon but also provided a mathematical proof of its existence along with concrete numerical examples.

Notably, Berkson did not discover this problem in an abstract mathematical setting — he encountered inexplicable anomalous patterns in actual clinical data analysis and then traced the issue back to selection bias. This "discovering problems through practice" research approach remains an important model for scientific discovery today.

Subsequent Developments and Impact

The impact of Berkson's Paradox extends far beyond medical statistics. With the development of causal inference theory, particularly the systematic study of Directed Acyclic Graphs (DAGs) by Judea Pearl and his colleagues, Berkson's Paradox was understood as a special case of "collider bias."^[3]

In the causal diagram framework, when a variable is the "common effect" (collider) of two other variables, conditioning on this variable creates a spurious association between the originally independent variables. This theoretical framework helps researchers identify and avoid various forms of selection bias.

Today, Berkson's Paradox is a required topic in epidemiological methodology courses and a fundamental concept that all medical researchers must understand.^[12]

VI. How to Avoid Berkson's Paradox

1. Ask "Where Does the Sample Come From?"

This is the most important question. Before accepting any statistical conclusion, first ask:

• How was this study's sample selected?
• What kind of people enter this sample? What kind are excluded?
• Is the condition for entering the sample related to the variables being studied?

If the sample selection mechanism is related to the study variables, be alert to the possibility of Berkson's Paradox.

2. The Importance of Random Sampling

Berkson's Paradox occurs because the sample is not randomly drawn from the target population. Ideally, research should use random sampling to ensure the sample is representative of the target population.

Of course, random sampling is not always feasible — we cannot randomly assign who gets hospitalized. In such cases, researchers must explicitly acknowledge the sample's limitations and avoid over-generalizing conclusions.

3. Draw Causal Diagrams

The causal diagrams championed by Judea Pearl are a powerful tool for identifying Berkson's Paradox.^[13] Before analyzing data, draw what you believe to be the causal relationship diagram:

• Identify all collider variables
• Check whether you have inadvertently conditioned on a collider
• If so, consider how this might affect your conclusions

4. Use Sensitivity Analysis

When selection bias cannot be fully eliminated, conduct a sensitivity analysis: assuming the bias exists, how large would it need to be to change my conclusion? If only a small bias is needed to overturn the conclusion, the conclusion is not robust.^[14]

5. Think Critically About Data Sources

Ultimately, avoiding Berkson's Paradox requires critical thinking. Don't blindly trust data — consider how the data was generated, what its limitations are, and what biases may exist.

Remember: Data does not speak for itself; data must be interpreted within the correct framework.

Conclusion: Seeing the Unseen

What Berkson's Paradox teaches us goes far beyond a statistical technique. It is a profound lesson in epistemology — our understanding of the world is always limited by the samples we can observe.

In hospitals, we see patients; in dating markets, we meet people willing to date; on lists of successful people, we read about those who succeeded. What we cannot see is often more important than what we can.

The healthy people who never fell ill, the people who never entered the dating market, the ordinary people who never succeeded — their existence shapes the patterns we observe, even though they never appeared in our field of vision.

Berkson's 1946 insight reminds us to always ask: "Who is missing from this data?" The answer to this question is often the key to understanding what the data truly means.

In this age of data explosion, we are more easily misled by surface patterns than ever before. Understanding Berkson's Paradox not only makes us better data analysts but also more cautious observers — of the world, and of the limits of our own cognition.