A central question in health economics is “How valuable is medical care?” For some types of care, it isn’t a hard question: if your appendix is about to burst, the benefit of surgery is blindingly obvious and we can calculate a monetary value using estimates of the value of life. (Yes, these exist. No, I will not be discussing the concept right now.)
But for other types of care, it’s more complicated. How valuable is it to start having mammograms at 40 rather than 50? How much does it matter if statins were prescribed to slightly more people? In some cases, a huge number of people are being screened or treated to detect a very small number of problems. And those problems might have been found even without the screening.
This comes up in prenatal care. How valuable is the marginal prenatal appointment, the marginal ultrasound? If doctors saw low-risk women less frequently in their last months of pregnancy or did fewer tests, would it matter very much? These are questions for medical professionals but also for patients.
When I was 38 weeks pregnant, I found myself waiting for an appointment that was running 90 minutes late and wondering: How important is this appointment, really? Should I just leave so I can make my meeting? On the flip side, we may wonder if we should be pushing for more testing or screening than the default.
It is extremely difficult to assess the value of care because choices about the amount of care that individuals get depends on their other characteristics. If you looked, for example, at the relationship between number of ultrasounds in pregnancy and infant complications, you would almost certainly find that a greater number of ultrasounds is associated with higher infant complications. But this isn’t because ultrasounds do not help; it’s because when complications are expected, doctors do more ultrasounds.
Answering these questions in data requires being more clever about analyses. In empirical economics, one increasingly popular tool to do this is called “regression discontinuity” (I wrote about it briefly in the context of SIDS and back sleeping a few months ago). The idea behind a regression discontinuity analysis is relatively simple and perhaps best illustrated with a non-health example.
Imagine there was a high school where admission was based only on your score on a single test: above the cutoff, you get in, and below, you do not. Researchers are interested in learning the impact of going to this high school. It would be a problem to just compare kids who went to the high school with those who didn’t, because a lot of things might be different about them. But because of the cutoff, you have a bit of an experiment. Let’s say the cutoff is 87 points. The kids who got 86 points and the kids who got 87 points are probably quite similar, except that those with the 87 points got in and the 86 did not. If you followed those two groups over time and compared their outcomes, you’d be a lot more confident attributing differences to the high school attendance.
In regression discontinuity designs, we usually like to make graphs. In my imaginary high school example, you could conceive a graph like the one below, where I look at the impact on wages. You’ll see that I’ve illustrated a jump in wages at a score of 87. In this analysis, we’d attribute that jump to the testing high school.
It turns out that this approach is really useful in analyzing health questions, because there are a lot of somewhat arbitrary cutoff rules in the area of health. We diagnose people with diabetes at a cutoff of 6.5 on a particular blood test, for example. A level of 6.4 leads to a different diagnosis than 6.5. This lets researchers analyze impacts of diabetes diagnosis on behavior and health outcomes.
There are many cutoff rules in prenatal care. One of my favorite papers in economics, which you can see here, uses this approach to analyze the impact of more medical spending on infant survival. The authors of the paper (Doug Almond, Joe Doyle, Amanda Kowalski, and Heidi Williams) recognized the common use of a cutoff value of 1,500 grams to define infants as “very low birth weight.” That is: when doctors classify infant weight, an infant who weighs 1,499 grams (about 3.3 pounds) is classified as “very low birth weight,” whereas an infant weighing 1,501 grams is not classified this way.
It should be clear that there is only an extremely limited difference between these two infants. The cutoff of 1,500 grams was never intended to imply something meaningful about infant health. And yet, for various reasons, infants who are just above this cutoff — just outside the very-low-birth-weight range — get less care. The paper finds that they incur hospital charges of about $10,000 less (care for infants in this range is very expensive; the difference is about $95,000 versus $85,000). Most strikingly, though, the authors find that this care seems to impact mortality.
I’ve put their main graph below. What it shows is the one-year mortality risk (sadly, these are very-high-risk infants, especially in this earlier time period, 1983 through 2003) graphed against birth-weight bins. What you can see is that although in general higher-birth-weight infants are more likely to survive, this relationship reverses around the cutoff. Infants who are just above the cutoff — those who didn’t get the extra care that seems to be associated with the very-low-birth-weight classification — actually have higher mortality than those just below the cutoff.
The primary conclusion of this paper is that these added expenditures do have significant value in terms of infant survival.
This is a big lead-up to the reason I thought to write this post, which is to highlight a new paper that came out on Friday in JAMA Health Forum, with senior author Jessica Cohen.
The paper is interested in the question of the value of more prenatal care in improving infant and mother outcomes. It is a question with obvious policy implications, given that the U.S. lags behind most developed countries in these metrics, but it’s hard to answer, for the reasons I noted above. Basically, more screening and testing is usually driven by more concerns.
The paper uses a simple regression discontinuity strategy, focused around the fact that women who are over 35 are classified as of “advanced maternal age,” or AMA. Women who are expected to be 35 at the date of delivery get this classification; those who are 34 do not. We have a general sense that being in the AMA category brings more screening and testing, but that isn’t because there is something inherently riskier about giving birth at 35 years and 1 day versus 34 years and 364 days. The added scrutiny may reflect provider confusion on this point or, more likely, insurance rules.
In any case: just as the other paper uses the birth-weight cutoff, this paper uses the age cutoff. The researchers show first that women over 35 are more likely to have more advanced prenatal care. The most striking graph is on detailed ultrasound scans — I’ve included it below. It is pretty clear from this picture that when you tick over that age-35 cutoff (the x-axis of this graph is denoted in days around the 35th birthday), you get more scans.
The goal of the paper, though, is to ask whether there are impacts on mothers and infants. I’ve put the rest of the authors’ results graphs below. On one hand, they find no evidence that severe maternal morbidity is significantly decreased by having more screening. However, they do find a significant decline in perinatal mortality — this is defined as fetal death within a month before expected delivery, or death within the first seven days of life. The very likely interpretation of this is a reduction in stillbirth as a result of the added monitoring. The reduction is about 0.4 percentage points, which feels sizable to me.
Overall, this finding suggests that the added monitoring and screening is having an important impact, at least in this population. Interventions like ultrasounds or added prenatal visits and non-stress tests cost something, but actually the cost is relatively small when put alongside reductions in perinatal death.
The final question in a paper like this is what we might do differently in terms of policy. Most directly, this finding may suggest that there is value to doing some additional monitoring among women who are near but not at the cutoff. In other words, maybe there should be more ultrasounds or screening among 34-year-old women, or perhaps have a more nuanced discussion of who gets the monitoring, one that is not based entirely on age.
It’s more complicated to make statements about how we should treat all pregnant people based on this type of data. These results are “local” to the age cutoff that is used in the study. That is to say: they tell us about what happens if you give more prenatal care to people who are around age 35. The effect may not be the same as what you’d get if you did the same with 25-year-olds (or 45-year-olds). This is a general issue with this type of analysis; in a technical sense, we are concerned about the “external validity” of the findings.
This concludes our econometrics lesson for the week! Just remember: the next time you find yourself frustrated with some seemingly arbitrary cutoff, whether in health or in another area, tell yourself it’s a golden opportunity for an enterprising economist and their regression discontinuity toolbox.