Of course, while it’s all very well to piously state that correlation doesn’t imply causation, it does leave us with a conundrum: under what conditions, exactly, can we use experimental data to deduce a causal relationship between two or more variables?
The standard scientific answer to this question is that (with some caveats) we can infer causality from a well designed randomized controlled experiment. Unfortunately, while this answer is satisfying in principle and sometimes useful in practice, it’s often impractical or impossible to do a randomized controlled experiment. And so we’re left with the question of whether there are other procedures we can use to infer causality from experimental data. And, given that we can find more general procedures for inferring causal relationships, what does causality mean, anyway, for how we reason about a system?
It might seem that the answers to such fundamental questions would have been settled long ago. In fact, they turn out to be surprisingly subtle questions. Over the past few decades, a group of scientists have developed a theory of causal inference intended to address these and other related questions. This theory can be thought of as an algebra or language for reasoning about cause and effect. Many elements of the theory have been laid out in a famous book by one of the main contributors to the theory, Judea Pearl. Although the theory of causal inference is not yet fully formed, and is still undergoing development, what has already been accomplished is interesting and worth understanding.
In this post I will describe one small but important part of the theory of causal inference, a causal calculus developed by Pearl. This causal calculus is a set of three simple but powerful algebraic rules which can be used to make inferences about causal relationships. In particular, I’ll explain how the causal calculus can sometimes (but not always!) be used to infer causation from a set of data, even when a randomized controlled experiment is not possible. Also in the post, I’ll describe some of the limits of the causal calculus, and some of my own speculations and questions.
The post is a little technically detailed at points. However, the first three sections of the post are non-technical, and I hope will be of broad interest. Throughout the post I’ve included occasional “Problems for the author”, where I describe problems I’d like to solve, or things I’d like to understand better. Feel free to ignore these if you find them distracting, but I hope they’ll give you some sense of what I find interesting about the subject. Incidentally, I’m sure many of these problems have already been solved by others; I’m not claiming that these are all open research problems, although perhaps some are. They’re simply things I’d like to understand better. Also in the post I’ve included some exercises for the reader, and some slightly harder problems for the reader. You may find it informative to work through these exercises and problems.
Before diving in, one final caveat: I am not an expert on causal inference, nor on statistics. The reason I wrote this post was to help me internalize the ideas of the causal calculus. Occasionally, one finds a presentation of a technical subject which is beautifully clear and illuminating, a presentation where the author has seen right through the subject, and is able to convey that crystalized understanding to others. That’s a great aspirational goal, but I don’t yet have that understanding of causal inference, and these notes don’t meet that standard. Nonetheless, I hope others will find my notes useful, and that experts will speak up to correct any errors or misapprehensions on my part.
Let me start by explaining two example problems to illustrate some of the difficulties we run into when making inferences about causality. The first is known as Simpson’s paradox. To explain Simpson’s paradox I’ll use a concrete example based on the passage of the Civil Rights Act in the United States in 1964.
In the US House of Representatives, 61 percent of Democrats voted for the Civil Rights Act, while a much higher percentage, 80 percent, of Republicans voted for the Act. You might think that we could conclude from this that being Republican, rather than Democrat, was an important factor in causing someone to vote for the Civil Rights Act. However, the picture changes if we include an additional factor in the analysis, namely, whether a legislator came from a Northern or Southern state. If we include that extra factor, the situation completely reverses, in both the North and the South. Here’s how it breaks down:
North: Democrat (94 percent), Republican (85 percent)
South: Democrat (7 percent), Republican (0 percent)
Yes, you read that right: in both the North and the South, a larger fraction of Democrats than Republicans voted for the Act, despite the fact that overall a larger fraction of Republicans than Democrats voted for the Act.
You might wonder how this can possibly be true. I’ll quickly state the raw voting numbers, so you can check that the arithmetic works out, and then I’ll explain why it’s true. You can skip the numbers if you trust my arithmetic.
North: Democrat (145/154, 94 percent), Republican (138/162, 85 percent)
South: Democrat (7/94, 7 percent), Republican (0/10, 0 percent)
Overall: Democrat (152/248, 61 percent), Republican (138/172, 80 percent)
One way of understanding what’s going on is to note that a far greater proportion of Democrat (as opposed to Republican) legislators were from the South. In fact, at the time the House had 94 Democrats, and only 10 Republicans. Because of this enormous difference, the very low fraction (7 percent) of southern Democrats voting for the Act dragged down the Democrats’ overall percentage much more than did the even lower fraction (0 percent) of southern Republicans who voted for the Act.
(The numbers above are for the House of Congress. The numbers were different in the Senate, but the same overall phenomenon occurred. I’ve taken the numbers from Wikipedia’s article about Simpson’s paradox, and there are more details there.)
If we take a naive causal point of view, this result looks like a paradox. As I said above, the overall voting pattern seems to suggest that being Republican, rather than Democrat, was an important causal factor in voting for the Civil Rights Act. Yet if we look at the individual statistics in both the North and the South, then we’d come to the exact opposite conclusion. To state the same result more abstractly, Simpson’s paradox is the fact that the correlation between two variables can actually be reversed when additional factors are considered. So two variables which appear correlated can become anticorrelated when another factor is taken into account.
You might wonder if results like those we saw in voting on the Civil Rights Act are simply an unusual fluke. But, in fact, this is not that uncommon. Wikipedia’s page on Simpson’s paradox lists many important and similar real-world examples ranging from understanding whether there is gender-bias in university admissions to which treatment works best for kidney stones. In each case, understanding the causal relationships turns out to be much more complex than one might at first think.
To help address problems like the two example problems just discussed, Pearl introduced a causal calculus. In the remainder of this post, I will explain the rules of the causal calculus, and use them to analyse the smoking-cancer connection. We’ll see that even without doing a randomized controlled experiment it’s possible (with the aid of some reasonable assumptions) to infer what the outcome of a randomized controlled experiment would have been, using only relatively easily accessible experimental data, data that doesn’t require experimental intervention to force people to smoke or not, but which can be obtained from purely observational studies.
To state the rules of the causal calculus, we’ll need several background ideas. I’ll explain those ideas over the next three sections of this post. The ideas are causal models (covered in this section), causal conditional probabilities, and d-separation, respectively. It’s a lot to swallow, but the ideas are powerful, and worth taking the time to understand. With these notions under our belts, we’ll able to understand the rules of the causal calculus
Read the blog post for the rest, with diagrams.
How bad will climate change be? Not very.
No, this isn’t a denialist screed. Human greenhouse emissions will warm the planet, raise the seas and derange the weather, and the resulting heat, flood and drought will be cataclysmic.
Cataclysmic—but not apocalyptic. While the climate upheaval will be large, the consequences for human well-being will be small. Looked at in the broader context of economic development, climate change will barely slow our progress in the effort to raise living standards.
To see why, consider a 2016 Newsweek headline that announced “Climate change could cause half a million deaths in 2050 due to reduced food availability.” The story described a Lancet study, “Global and regional health effects of future food production under climate change,”  that made dire forecasts: by 2050 the effects of climate change on agriculture will shrink the amount of food people eat, especially fruits and vegetables, enough to cause 529,000 deaths each year from malnutrition and related diseases. The report added grim specifics to the familiar picture of a world made hot, hungry, and barren by the coming greenhouse apocalypse.
But buried beneath the gloomy headlines was a curious detail: the study also predicts that in 2050 the world will be better fed than ever before. The “reduced food availability” is only relative to a 2050 baseline when food will be more abundant than now thanks to advances in agricultural productivity that will dwarf the effects of climate change. Those advances on their own will raise per-capita food availability to 3,107 kilocalories per day; climate change could shave that to 3,008 kilocalories, but that’s still substantially higher than the benchmarked 2010 level of 2,817 kilocalories—and for a much larger global population. Per-capita fruit and vegetable consumption, the study estimated, will rise by 6.1 percent and meat consumption by 5.4 percent. The poorest countries will benefit most, with food availability rising 14 percent in Africa and Southeast Asia. Even after subtracting the 529,000 lives theoretically lost to climate change, the study estimates that improved diets will save a net 1,348,000 lives per year in 2050.
Tomorrow, Sunday, April 22, is Earth Day 2018 In the May 2000 issue of Reason Magazine, award-winning science correspondent Ronald Bailey wrote an excellent article titled “Earth Day, Then and Now” to provide some historical perspective on the 30th anniversary of Earth Day. In that article, Bailey noted that around the time of the first Earth Day […]
There is no clear correlation whatsoever between gun ownership rate and gun homicide rate. Not within the USA. Not regionally. Not internationally. Not among peaceful societies. Not among violent ones. Gun ownership doesn’t make us safer. It doesn’t make us less safe. The correlation simply isn’t there. It is blatantly not-there. It is so tremendously not-there that the “not-there-ness” of it alone should be a huge news story.
And anyone with access to the internet and a basic knowledge of Microsoft Excel can check for themselves. Here’s how you do it.
First, go to the Wikipedia page on firearm death rates in the United States. If you don’t like referencing Wikipedia, then instead go to this study from the journal Injury Prevention, which was widely sourced by media on both the left and right after it came out, based on a survey of 4000 respondents. Then go to this table published by the FBI, detailing overall homicide rates, as well as gun homicide rates, by state. Copy and paste the data into Excel, and plot one versus the other on a scatter diagram. Alternately, do the whole thing on the back of a napkin. It’s not hard. Here’s what you get:
This looks less like data and more like someone shot a piece of graph paper with #8 birdshot.
If the data were correlated, we should be able to develop a best fit relationship to some mathematical trend function, and calculate an “R^2 Value,” which is a mathematical way of describing how well a trendline predicts a set of data. R^2 Values vary between 0 and 1, with 1 being a perfect fit to the data, and 0 being no fit. The R^2 Value for the linear trendline on this plot is 0.0031. Total garbage. No other function fits it either.
I embellished a little with the plot, coloring the data points to correspond with whether a state is “red,” “blue,” or “swing,” according to the Romney-Obama era in which political demarcations were a little more even and a little more sensical. That should give the reader a vague sense of what the gun laws in each state are like. As you can see, there is not only no correlation whatsoever with gun ownership rate, there’s also no correlation whatsoever with state level politics.
But hey, we are a relatively unique situation on the planet, given our high ownership rates and high number of guns owned per capita, so surely there’s some supporting data linking gun ownership with gun homicide elsewhere, right?
So off we go to Wikipedia again, to their page listing countries by firearm related death rates. If Wikipedia gives you the willies, you’re going to have a harder time compiling this table on your own, because every line in it is linked to a different source. Many of them, however, come from http://www.gunpolicy.org. Their research is supported by UNSCAR, the UN Trust Facility Supporting Cooperation on Arms Regulation, so it is probably pretty reasonable data. They unfortunately do not have gun ownership rates, but do have “guns owned per 100 inhabitants,” which is a similar set we can compare against. And we drop that into Excel, or use the back of our napkin again, and now we are surely going to see how gun ownership drives gun homicide.
Well that’s disappointing.
Remember we are looking for an R^2 value close to 1, or hopefully at least up around 0.7. The value on this one is 0.0107. Garbage.
So let’s briefly recap. Gun Murder Rate is not correlated with firearm ownership rate in the United States, on a state by state basis. Firearm Homicide Rate is not correlated with guns per capita globally. It’s not correlated with guns per capita among peaceful countries, nor among violent countries, nor among European countries. So what in the heck is going on in the media, where we are constantly berated with signaling indicating that “more guns = more murder?”
One: They’re sneaking suicide in with the data, and then obfuscating that inclusion with rhetoric.
This is the biggest trick I see in the media, and very few people seem to pick up on it. Suicide, numerically speaking, is around twice the problem homicide is, both in overall rate and in rate by gun. Two thirds of gun deaths are suicides in the USA. And suicide rates are correlated with gun ownership rates in the USA, because suicide is much easier, and much more final, when done with a gun. If you’re going to kill yourself anyway, and you happen to have a gun in the house, then you choose that method out of convenience. Beyond that, there’s some correlation between overall suicide and gun ownership, owing to the fact that a failed suicide doesn’t show up as a suicide in the numbers, and suicides with guns rarely fail.
Two: They’re cooking the homicide data.
The most comprehensive example of this is probably this study from the American Journal of Public Health. It’s widely cited, and was very comprehensive in its analytical approach, and was built by people I admire and whom I admit are smarter than me. But to understand how they ended up with their conclusions, and whether those conclusions actually mean what the pundits say they mean, we have to look at what they actually did and what they actually concluded.
First off, they didn’t use actual gun ownership rates. They used fractional suicide-by-gun rates as a proxy for gun ownership. This is apparently a very common technique by gun policy researchers, but the results of that analysis ended up being very different from the ownership data in the Injury Prevention journal in my first graph of the article. The AJPH study had Hawaii at 25.8% gun ownership rate, compared to 45% in IP, and had Mississippi at 76.8% gun ownership rate, compared to 42.8% in IP. Could it be that suicidal people in Hawaii prefer different suicide methods than in Mississippi, and that might impact their proxy? I don’t know, but it would seem to me that the very use of a proxy at all puts the study on a very sketchy foundation. If we can’t know the ownership rate directly, then how can we check that the ratio of gun suicides properly maps over to the ownership rate? Further, the fact that the rates are so different in the two studies makes me curious about the sample size and sampling methods of the IP study. We can be absolutely certain that at least one of these studies, if not both of them, are wrong on the ownership rate data set. We know this purely because the data sets differ. They can’t both be right. They might both be wrong.
In the second article, we unpack “gun death” statistics and look carefully at suicide.
In the third article, we debunk the “gun homicide epidemic” myth.
In the fourth article, we expand upon why there is no link between gun ownership and gun homicide rate, and why gun buybacks and other gun ownership reduction strategies cannot work.
In the fifth article, we discuss why everyone should basically just ignore school shootings.
When I was in college, I happened across an article listing taboo topics in psychological research. These were “third rail” topics, that would put anyone investigating them in deep yogurt. One of those topics was “Race and IQ”.
It’s still a “third rail”.
In April of 2017, I published a podcast with Charles Murray, coauthor of the controversial (and endlessly misrepresented) book The Bell Curve. These are the most provocative claims in the book:
- Human “general intelligence” is a scientifically valid concept.
- IQ tests do a pretty good job of measuring it.
- A person’s IQ is highly predictive of his/her success in life.
- Mean IQ differs across populations (blacks < whites < Asians).
- It isn’t known to what degree differences in IQ are genetically determined, but it seems safe to say that genes play a role (and also safe to say that environment does too).
At the time Murray wrote The Bell Curve, these claims were not scientifically controversial—though taken together, they proved devastating to his reputation among nonscientists. That remains the case today. When I spoke with Murray last year, he had just been de-platformed at Middlebury College, a quarter century after his book was first published, and his host had been physically assaulted while leaving the hall. So I decided to invite him on my podcast to discuss the episode, along with the mischaracterizations of his research that gave rise to it.
Needless to say, I knew that having a friendly conversation with Murray might draw some fire my way. But that was, in part, the point. Given the viciousness with which he continues to be scapegoated—and, indeed, my own careful avoidance of him up to that moment—I felt a moral imperative to provide him some cover.
In the aftermath of our conversation, many people have sought to paint me as a racist—but few have tried quite so hard as Ezra Klein, Editor-at-Large of Vox. In response to my podcast, Klein published a disingenuous hit piece that pretended to represent the scientific consensus on human intelligence while vilifying me as, at best, Murray’s dupe. More likely, readers unfamiliar with my work came away believing that I’m a racist pseudoscientist in my own right.
John Tierney and John Stossel look at who is making war on science.
The right doesn’t like a few things, but their impact on how science is done is minimal. The left is much more active in shutting down science, and scientists, they don’t like.
This piece at Reason Mag is the text version.
Bookworm looks at some of the research on the effects of sex hormones.
I think there are true transgender individuals, but they represent the extreme tail end of the normal distribution curves. But I’ve been in conversations with people who cite these extreme examples and “reason” therefrom that gender is a meaningless social construct. So it’s an error to claim that there are two sexes. It’s probably also an error to claim that the sexes differ, and probably an error to believe that fiddling with a person’s levels of sex hormones would have any negative impact. (How this can be reconciled with the necessity of such fiddling in the first place is yet to be explained.)