Emily Oster

8 min Read Emily Oster

Emily Oster

Prenatal Tests and False Positives

Bayes’ rule is still very, very important

Emily Oster

8 min Read

Last week, the New York Times published a fantastic investigative piece on false positives in prenatal testing. One of the authors was the incomparable Sarah Kliff, and it is an absolutely awesome combination of on-the-ground reporting with patients, research into the companies that provide testing, and data visualization.

The conclusion of the piece was in some ways very alarming — these tests were not nearly as protective as they were sold to be.

Many people emailed me to ask about it, and the piece is so good that I wondered if there was really anything for me to add. But then I realized that there were no equations and no mention of Bayes’ rule. So I’m swooping in with more statistical lingo and to explain, just a bit more, why the conclusions shouldn’t have been surprising.

Background on prenatal testing 

Before getting into those equations, it’s worth stepping back to give a little background on the focus of the article, which is non-invasive prenatal testing.

When I was pregnant with Penelope, back in 2010, screening for chromosomal abnormalities (e.g. Down syndrome, other trisomies) came in two types. First, an ultrasound screening, which provided some information but with a lot of missed diagnoses. Second, invasive testing (either placental sampling earlier in pregnancy or amniocentesis later), which carried some risk of miscarriage but was more accurate.

By the time I was pregnant with Finn in 2014, a new option was available: non-invasive prenatal testing, using cell-free fetal DNA technology. These tests make use of the fact that some fetal DNA circulates in the maternal bloodstream. Treated correctly, a sample of blood from the pregnant person can be used to detect abnormalities in the fetus. To somewhat simplify, the approach is to look for evidence of DNA that wouldn’t otherwise be in the mother.

For example: let’s say you wanted to know the baby’s sex. The pregnant person typically has two X chromosomes. A male baby will have an XY, and a female will have XX. If you sample mom’s blood and you find evidence of circulating Y chromosomes in the cell-free DNA, this indicates a male fetus (since the mom’s own cell-free DNA wouldn’t contain a Y chromosome).

In their early conception, these tests were used to detect infant sex and the three primary trisomies (Down syndrome, trisomy 13, and trisomy 18). Detecting a trisomy means, effectively, looking for an imbalance in the presence of these chromosomes in the cell-free DNA. Assuming the pregnant person has two copies of each chromosome, if you observe an excess of chromosome 21 in the cell-free DNA, it suggests this must be due to an excess of chromosome 21 in the fetus, which would suggest Down syndrome.

These tests are very accurate for determining sex and Down syndrome in particular. Notably, they are substantially more accurate — both in terms of fewer false positives and false negatives — than the non-invasive ultrasound options that preceded them. However, they are still screening tests and not diagnostic. To be certain about these conditions, it is necessary to follow up a positive test with some invasive testing that is able to sequence fetal DNA.

The tests expanded

As a method for screening for major trisomies, there is some agreement about the value of these NIPT tests, as they are called. However, and this is the topic of the New York Times article, companies have started using these approaches to test for much, much rarer conditions. And therein lies the problem.

The conditions in question are mostly what are known as microdeletions. These are syndromes or disabilities that are a result of a small missing DNA piece in one chromosome. An example is the 22q11.2 deletion — a small missing piece of chromosome 22 that can lead to a developmental disorder called DiGeorge syndrome. This occurs in perhaps 1 in 4,000 births.

There are many kinds of these microdeletions, with varying prevalence, though all are quite rare. The claim, made frequently by NIPT-providing companies, is that the tests can detect microdeletions in the same way they detect Down syndrome or sex chromosomes. In a sense, they can. But in another sense, they are limited.

The Bayes’ analysis

To see the main issue, consider the test from a company called Harmony for this 22q11.2 microdeletion. Harmony provides some details about its test performance in this document.

According to the company’s analyses, the test detected 75% of cases with this deletion, and it saw only a 0.5% false positive rate. That is, of the cases without the microdeletion, only 0.5% of them showed a positive test result. As noted above, other sources put the overall risk of this microdeletion at about 1 in 4,000.

Let’s think about what that means if you do get a positive result.

To be concrete, imagine we have 80,000 people being tested. We expect, based on the underlying risk, that 20 of them are carrying a fetus with this microdeletion. When the 80,000 individuals are NIPT tested, 75% of those 20 cases (or 15, in expectation) will show up as positive tests. In addition, of the 79,980 people being tested who do not have a fetus with a microdeletion, 0.5% of them will get a false positive. That’s around 400 people.

Altogether, there are 415 positive tests: the 15 true positives, and 400 false positives. So if you get a positive test result, the chance that the fetus is actually affected is about 3.6%.

This is precisely the point that is made in the NYT story — that with these tests, which seem so accurate, in the vast majority of cases, even after a positive test, the fetus is in fact not affected.

The whole calculation is a straightforward application of Bayes’ rule, which I did a longer discussion of here. Intuitively, though, it somehow feels wrong. On its face, this sounds like a really good test! It detects 75% of cases, with only a 0.5% false positive rate. That seems like it should be helpful. And the fact is that it is really helpful, and it is hugely informative. Before the test, the risk was 1 in 4,000. After a positive test, it’s 4 in 100. This risk is a different order of magnitude — you’ve learned so much. You just haven’t learned everything.

The reason you can simultaneously have an excellent test and still this residual uncertainty is that the condition is very rare. This means that even a small false positive rate is a large number of false positive cases. The lower the baseline risk, the more significant this issue is.

Where’s the fire?

All of this is clear from an analysis of the published materials. If you read the find print and did the calculation, in principle the information was there. It’s not that the companies *lied* about their accuracy, at least not in terms of the numbers.

So what’s the issue driving the New York Times coverage? Primarily, it’s the companies overselling the accuracy of their tests, and the (completely understandable) patient reaction. In most cases, the literature from providers pays lip service to the idea that patients should undergo confirmative diagnostic testing before considering pregnancy termination or other measures. These confirmatory tests include either a CVS test or an amniocentesis; both are more invasive, so they aren’t likely to be the first step for many families, but they can provide certainty.

The companies say the need for this confirmation may be necessary. But in the same breath, the literature promotes the incredible accuracy of the tests.

The Times story uncovers cases of patients who underwent significant stress as a result of these false positive results, and even identifies cases in which patients terminated a pregnancy before confirmatory testing. That should not happen. Companies should not overstate the accuracy of their results. And doctors should be incredibly careful in how they present these tests and results to patients. Notably, I’d argue that it’s crucial to be clear with patients up-front about what they should expect with a positive result. Some of the biggest problems come when patients hear “positive” with little context; it can be difficult to grasp the nuances of false positives in a heightened emotional state.

From a patient standpoint, I think there is an important question about whether these tests are a good idea. On one hand, they do provide some information. On the other, the conditions they test for are very rare and in many cases somewhat poorly understood in terms of their impact. As a person who loves data, I err toward more information being better. But it’s only better if you understand and use it correctly.

A line graph with pink, yellow, and blue dots representing life's ups and downs.

Feb 21 2023

3 min read

Wins, Woes, and Autism

Your stories for the week

Emily Oster
A pregnant person is supported by a partner in a pool of water for a water birth.

Feb 23 2023

7 min read

Labor Positions

And what I might have done differently

Emily Oster

Jan 30 2023

9 min read

Prenatal Testing Deep Dive

A review of the options for trisomy detection

Emily Oster
A pregnant person has her blood pressure taken at a doctor's appointment.

Jan 20 2023

3 min read

Are Doctors Causing My Hypertension?

Ask ParentData

Emily Oster

Instagram

left right
Happy Father’s Day to the Fathers and Father figures in our ParentData community! 

Tag a Dad who this holiday may be tricky for. We’re sending you love. 💛

Happy Father’s Day to the Fathers and Father figures in our ParentData community!

Tag a Dad who this holiday may be tricky for. We’re sending you love. 💛
...

“Whilst googling things like ‘new dad sad’ and ‘why am I crying new dad,’ I came across an article written by a doctor who had trouble connecting with his second child. I read the symptoms and felt an odd sense of relief.” Today we’re bringing back an essay by Kevin Maguire of @newfatherhood about his experience with paternal postpartum depression. We need to demystify these issues in order to change things for the better. Comment “Link” for a DM to read his full essay.

#parentdata #postpartum #postpartumdepression #paternalmentalhealth #newparents #emilyoster

“Whilst googling things like ‘new dad sad’ and ‘why am I crying new dad,’ I came across an article written by a doctor who had trouble connecting with his second child. I read the symptoms and felt an odd sense of relief.” Today we’re bringing back an essay by Kevin Maguire of @newfatherhood about his experience with paternal postpartum depression. We need to demystify these issues in order to change things for the better. Comment “Link” for a DM to read his full essay.

#parentdata #postpartum #postpartumdepression #paternalmentalhealth #newparents #emilyoster
...

What does the data say about children who look more like one parent? Do they also inherit more character traits and mannerisms from that parent? Let’s talk about it 🔎

#emilyoster #parentdata #parentingcommunity #lookslikedaddy #lookslikemommy

What does the data say about children who look more like one parent? Do they also inherit more character traits and mannerisms from that parent? Let’s talk about it 🔎

#emilyoster #parentdata #parentingcommunity #lookslikedaddy #lookslikemommy
...

SLEEP DATA 💤 We asked you all about your kids’ sleep—and got nearly 15,000 survey responses to better understand kids’ sleep patterns. Comment “Link” for an article that breaks down our findings!

This graph shows sleeping location by age. You’ll notice that for the first three months, most kids are in their own sleeping location in a parent’s room. Then, over the first year, this switches toward their own room. As kids age, sharing a room with a sibling becomes more common. 

Head to the newsletter for more and stay tuned for part two next week on naps! 🌙

#parentdata #emilyoster #childsleep #babysleep #parentingcommunity

SLEEP DATA 💤 We asked you all about your kids’ sleep—and got nearly 15,000 survey responses to better understand kids’ sleep patterns. Comment “Link” for an article that breaks down our findings!

This graph shows sleeping location by age. You’ll notice that for the first three months, most kids are in their own sleeping location in a parent’s room. Then, over the first year, this switches toward their own room. As kids age, sharing a room with a sibling becomes more common.

Head to the newsletter for more and stay tuned for part two next week on naps! 🌙

#parentdata #emilyoster #childsleep #babysleep #parentingcommunity
...

Weekends are good for extra cups of ☕️ and listening to podcasts. I asked our team how they pod—most people said on walks or during chores. What about you?

Comment “Link” to subscribe to ParentData with Emily Oster, joined by some excellent guests.

#parentdata #parentdatapodcast #parentingpodcast #parentingtips #emilyoster

Weekends are good for extra cups of ☕️ and listening to podcasts. I asked our team how they pod—most people said on walks or during chores. What about you?

Comment “Link” to subscribe to ParentData with Emily Oster, joined by some excellent guests.

#parentdata #parentdatapodcast #parentingpodcast #parentingtips #emilyoster
...

Humility. That’s why. That’s the whole reason.

#emilyoster #secondbaby #parentingjokes #parentinghumor

Humility. That’s why. That’s the whole reason.

#emilyoster #secondbaby #parentingjokes #parentinghumor
...

Bug season is upon us. Besides annoyance, this can bring up safety concerns, particularly with ticks. They are carriers of diseases, most notably Lyme disease. So what’s the best course of action?

Prevention is key! I suggest:
⭐ Regular tick checks
⭐ Using bug sprays with DEET 
⭐ Wearing long sleeves and pants in the woods

Some parents worry about DEET, but repellants with up to 30% DEET are recommended by both the CDC and AAP. The data says you’re in the clear, so go for it. Enjoy your summer!

#parentdata #emilyoster #tickseason #bugbites #bugspray

Bug season is upon us. Besides annoyance, this can bring up safety concerns, particularly with ticks. They are carriers of diseases, most notably Lyme disease. So what’s the best course of action?

Prevention is key! I suggest:
⭐ Regular tick checks
⭐ Using bug sprays with DEET
⭐ Wearing long sleeves and pants in the woods

Some parents worry about DEET, but repellants with up to 30% DEET are recommended by both the CDC and AAP. The data says you’re in the clear, so go for it. Enjoy your summer!

#parentdata #emilyoster #tickseason #bugbites #bugspray
...

The list of what not to do while pregnant feels longer than a CVS receipt. At ParentData, we want to empower you to make the right decisions for you. 

What an amazing group of women, and an honor to speak at the #MomsFirstSummit debunking parenting myths. 

What are some pregnancy rules you chose to bend after being empowered by data?

#emilyoster #parentdata #pregnancyproblems #pregnancymyths

The list of what not to do while pregnant feels longer than a CVS receipt. At ParentData, we want to empower you to make the right decisions for you.

What an amazing group of women, and an honor to speak at the #MomsFirstSummit debunking parenting myths.

What are some pregnancy rules you chose to bend after being empowered by data?

#emilyoster #parentdata #pregnancyproblems #pregnancymyths
...

Looking for Memorial Day Weekend plans? Might be the perfect time to give potty training a shot. Potty training is notoriously difficult, and we unfortunately don’t have a lot of evidence-based guidance on what works best. So I asked the ParentData community to fill out a survey and share their knowledge — about 6,000 people responded.

👉Comment “Link” for a DM to an article that summarizes all of the best potty training advice we collected. 

Remember, you are not alone in the potty training struggle! It can be incredibly challenging, so please give yourself some grace.

#emilyoster #parentdata #pottytraining #pottytrainingtips #toddlertips

Looking for Memorial Day Weekend plans? Might be the perfect time to give potty training a shot. Potty training is notoriously difficult, and we unfortunately don’t have a lot of evidence-based guidance on what works best. So I asked the ParentData community to fill out a survey and share their knowledge — about 6,000 people responded.

👉Comment “Link” for a DM to an article that summarizes all of the best potty training advice we collected.

Remember, you are not alone in the potty training struggle! It can be incredibly challenging, so please give yourself some grace.

#emilyoster #parentdata #pottytraining #pottytrainingtips #toddlertips
...

We’re hiring an Associate Editor at ParentData! More details at my link in bio. Please share with the great writers and data-loving people in your network. 📊💻

We’re hiring an Associate Editor at ParentData! More details at my link in bio. Please share with the great writers and data-loving people in your network. 📊💻 ...

Do you brand things a certain way to get your kid to accept it? Like calling carrots “rabbit popsicles”? Or telling them to put on their “super speed socks” in the morning? Share your rebrands in the comments below! You never know who you might be helping out 👇

#emilyoster #funnytweets #relatabletweets #parentingjokes #kidssaythedarndestthings

Do you brand things a certain way to get your kid to accept it? Like calling carrots “rabbit popsicles”? Or telling them to put on their “super speed socks” in the morning? Share your rebrands in the comments below! You never know who you might be helping out 👇

#emilyoster #funnytweets #relatabletweets #parentingjokes #kidssaythedarndestthings
...

Have you ever panic-googled a parenting question when everyone else is asleep? If so, you’re not alone. 

Today is the first episode of a new biweekly series on my podcast: Late-Night Panic Google. On these mini-episodes, you’ll hear from some familiar names about the questions keeping them up at night, and how data can help. First up: @claireholt!

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#parentdata #emilyoster #claireholt #parentingstruggles #parentingtips #latenightpanicgoogle

Have you ever panic-googled a parenting question when everyone else is asleep? If so, you’re not alone.

Today is the first episode of a new biweekly series on my podcast: Late-Night Panic Google. On these mini-episodes, you’ll hear from some familiar names about the questions keeping them up at night, and how data can help. First up: @claireholt!

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#parentdata #emilyoster #claireholt #parentingstruggles #parentingtips #latenightpanicgoogle
...

Sun safety is a must for all ages, especially babies! Here are my tips for keeping your littlest ones protected in the sunshine:
☀️ Most importantly, limit their time out in hot weather. (They get hotter than you do!)
☀️ Keep them in the shade as much as possible when you’re out.
☀️ Long-sleeve but lightweight clothing is your friend, especially on the beach, where even in the shade you can get sunlight reflecting off different surfaces.
☀️ If you want to add a little sunscreen on their hands and feet? Go for it! But be mindful as baby skin tends to more prone to irritation.

Comment “Link” for a DM to an article on the data around sun and heat exposure for babies.

#sunsafety #babysunscreen #babyhealth #parentdata #emilyoster

Sun safety is a must for all ages, especially babies! Here are my tips for keeping your littlest ones protected in the sunshine:
☀️ Most importantly, limit their time out in hot weather. (They get hotter than you do!)
☀️ Keep them in the shade as much as possible when you’re out.
☀️ Long-sleeve but lightweight clothing is your friend, especially on the beach, where even in the shade you can get sunlight reflecting off different surfaces.
☀️ If you want to add a little sunscreen on their hands and feet? Go for it! But be mindful as baby skin tends to more prone to irritation.

Comment “Link” for a DM to an article on the data around sun and heat exposure for babies.

#sunsafety #babysunscreen #babyhealth #parentdata #emilyoster
...