Selection in Practice: The Value of Randomness

Emily Oster

7 min Read Emily Oster

Emily Oster

Selection in Practice: The Value of Randomness

In which I get angry about statistics

Emily Oster

7 min Read

Lately, we all seem to be wearing a lot of hats; I certainly feel like have five or six too many on at the moment. But even away from the world of COVID, I tend to divide my professional life into two arenas. The first is my book-author-pregnancy-and-baby-lady-hat, from whence this newsletter primarily comes. And, second, there’s my research.

With that second hat on, in the past few years I’ve been thinking a lot about statistical methods, about selection, and about sampling. One of my most recent publications is about the problems of learning about a general population of people from a selected sample of volunteers. That paper is pretty technical (it’s written with an actual econometrician, which I am not), and feels a far cry from much of what I do here.

And yet: sometimes the two arenas merge, and lately I have been thinking a lot about how these problems of sampling arise in the context of the virus. And, in particular, in the question of how we can understand the level of current virus exposure.

Exposure Rates: The Problem

There are a lot of unanswered questions in COVID-19 — how far it travels in the air, how best to treat it, why some groups and people are so much more affected than others.

I believe among the very biggest questions is simply how widespread the virus is — how many people have already been infected? This is an extremely important question, but it’s also very hard. Why?

A lot of our predictions about the path of the virus (and the world, the economy, etc) over the next few months rely on epidemic modeling. Many of these models are forms of an “S-I-R” model — “Susceptible-Infected-Recovered” — which chart out dynamics as a population moves from entirely virus-susceptible to infected and finally into recovered.

The results from these models vary a lot; the basic structure of the models is mostly similar, but depending on what numbers you insert in them, they give wildly different answers! We’ve seen that with movements in predictions about hospitalizations and deaths over time. To make the models better — both to figure out which ones are right, and to improve the best ones — we need to fit them to data.

But that means actually knowing what share of people are susceptible, infected or recovered at any given time. Without that information, we are basically just guessing.

You may think: surely we know that! Don’t we see information on infections and hospitalizations and deaths over time? I feel like I’ve seen a lot of graphs about this.

Well, yes. But in the context of COVID-19, that’s not close to enough. Many infections with COVID-19 are very mild and non-specific. A large share of people — perhaps half or even 75% — who are infected have no symptoms. Even people who are symptomatic are still often not tested.

This means for every case we see, there are at least some we do not see. How many is really unclear. Some people think there are 10 missing cases for every one we see; others think it’s just one or two.

The implications of these two views are hugely different. If 1% of the population has already been infected, then 99% of people are still susceptible. On the other hand, if 20% have already been infected, well, that’s a different story.

Among our #1 priorities should be to learn about this number. And here is where I’ve been contemplating the problems of selection.

The Problem of Sampling

The best way to learn about the share of the population who have been exposed to the virus is to either test everyone (best case, probably infeasible in the US) or to test a random sample of people. This testing could be for active current infection or testing for past infection using antibodies. (This antibody testing has started to come online in the last couple of weeks and promises to be even more useful than active infection testing.)

Regardless of which type of testing we are doing, it is crucial to have a random sample of people. There are a few examples of this — a very few. Iceland did some random population testing recently, which showed about 1% of the general population had active infection (half of them asymptomatic). There is one town in Italy which tested everyone early in the epidemic (3% active infection, about half asymptomatic). Antibody testing (which includes past infections) in among a random sample in Germany showed 15% had either been actively or in the past infected.

Second best to a random sample may be universal testing among a known population. We had a recent examples of this among, actually, pregnant women in NY. A publication earlier this week in the New England Journal of Medicine showed active COVID-19 infection among almost 15% of women admitted for delivery. (A very large share of these infections were asymptomatic. I’m still unpacking what this might mean for those of you who are pregnant; more on that next week).

This isn’t as a good as a random sample since pregnant women are different in many ways (gender, age, exposure to medical care) than the general population. Still, it has value in part because we can understand the sources of bias.

Most people agree that random or universal testing is the best approach. But it’s also very hard to execute. Identifying a random sample of people and testing them is much, much more challenging than testing what we’d call a “convenience sample” — people who it is easy to find and access. And given the difference in difficulty, you might be tempted to think, well, some data is better than no data. I’ll do the easier thing and at least learn something.

This thinking is really problematic, though. Put simply: if we do not understand the biases in our sampling, the resulting data is garbage. One recent frustrating example of this is a large NIH study which aims to do antibody testing among 10,000 volunteers. Volunteers are being solicited in various ways, like over Twitter and with other public postings. People are asked to email the NIH to enroll, at which point they may be sent a home test kit.

Dr. Fauci has suggested that this will give us a “clearer picture of the true magnitude of the COVID-19 pandemic in the United States.” But it will not! It will give a clear picture of the magnitude among people who, say, scroll Twitter for opportunities to be in studies like this. Are these people more or less likely to have had COVID-19? I have no idea. Maybe you pull more people who know they’ve been exposed (higher prevalence), or maybe you pull people who are more careful about exposure (lower prevalence). Maybe it’s a weird mix of both. We simply do not know. We’ll get some number out of this and it will be completely uninterpretable.

This is worse than nothing, since people will think that they’ve learned something.

I have similar problems with testing blood donors as a measure of prevalence. Yes, it’s convenient. But it’s not going to tell us anything broadly useful.

What to do? I’m afraid that despite how hard it is, we simply have no choice but to do better sampling when we test. As someone who is trying to get some random testing off the ground in various populations, I can attest to the many, many challenges of doing so. But it is worthwhile. We need to do this.

So if someone shows up at your door and tells you you’ve been randomly selected for testing, please, please consent.

Covid-19 rapid antigen tests arranged in a pattern on a yellow background.

Feb 20 2023

12 min read

COVID-19: Where to Go from Here

A long-term view of the virus

Emily Oster
Covid-19 rapid antigen tests arranged in a pattern on a yellow background.

Oct 20 2022

9 min read

Should You Get the Bivalent Booster?

The latest on the risks and benefits of COVID vaccines boosters for older adults, pregnant people, and kids

Emily Oster
A line graph with pink, yellow, and blue dots representing life's ups and downs.

Aug 16 2022

3 min read

Wins, Woes, and Doing It Again

We have our first story from a dad! And it’s a good one. 10/10 —Girl Dad with Confidence Growing by Read more

Emily Oster
Covid-19 rapid antigen tests arranged in a pattern on a yellow background.

Aug 15 2022

8 min read

Updated CDC Guidelines for School and Child Care

NO QUARANTINES!!!

Emily Oster

Instagram

left right
For children or adults with severe food allergies, they can be incredibly scary and restrictive. We may imagine that it’s easy to deal with a peanut allergy by, say, not eating peanut butter sandwiches. But for someone with a severe version of this allergy, they may never be able to go to a restaurant, for fear of a severe reaction to something in the air. Right now, there’s only one approved treatment for severe allergies like this and it’s limited to peanuts.

This is why the new medication Xolair is very exciting. It promises a second possible treatment avenue and one that works for other allergens. A new trail analyzed data from 177 children with severe food allergies. Two-thirds of the treatment group were able to tolerate the specified endpoint, versus just 7% of the placebo group. This is a very large treatment effect, and the authors found similarly large impacts on other allergens. 

There are some caveats: This treatment won’t work for everyone. (One-third of participants did not respond to it.) Additionally, this treatment is an injection given every two to four weeks, indefinitely. This may make it less palatable to children. 

Overall, even with caveats, this is life-changing news for many families!

#xolair #foodallergies #allergies #peanutallergy #emilyoster #parentdata

For children or adults with severe food allergies, they can be incredibly scary and restrictive. We may imagine that it’s easy to deal with a peanut allergy by, say, not eating peanut butter sandwiches. But for someone with a severe version of this allergy, they may never be able to go to a restaurant, for fear of a severe reaction to something in the air. Right now, there’s only one approved treatment for severe allergies like this and it’s limited to peanuts.

This is why the new medication Xolair is very exciting. It promises a second possible treatment avenue and one that works for other allergens. A new trail analyzed data from 177 children with severe food allergies. Two-thirds of the treatment group were able to tolerate the specified endpoint, versus just 7% of the placebo group. This is a very large treatment effect, and the authors found similarly large impacts on other allergens.

There are some caveats: This treatment won’t work for everyone. (One-third of participants did not respond to it.) Additionally, this treatment is an injection given every two to four weeks, indefinitely. This may make it less palatable to children.

Overall, even with caveats, this is life-changing news for many families!

#xolair #foodallergies #allergies #peanutallergy #emilyoster #parentdata
...

If you have a fever during pregnancy, you should take Tylenol, both because it will make you feel better and because of concerns about fever in pregnancy (although these are also overstated).

The evidence that suggests risks to Tylenol focuses largely on more extensive exposure — say, taking it for more than 28 days during pregnancy. There is no credible evidence, even correlational, to suggest that taking it occasionally for a fever or headache would be an issue.

People take Tylenol for a reason. For many people, the choice may be between debilitating weekly migraines and regular Tylenol usage. The impacts studies suggest are very small. In making this decision, we should weigh the real, known benefit against the suggestion of this possible risk. Perhaps not everyone will come out at the same place on this, but it is crucial we give people the tools to make the choice for themselves.

#emilyoster #parentdata #tylenol #pregnancy #pregnancytips

If you have a fever during pregnancy, you should take Tylenol, both because it will make you feel better and because of concerns about fever in pregnancy (although these are also overstated).

The evidence that suggests risks to Tylenol focuses largely on more extensive exposure — say, taking it for more than 28 days during pregnancy. There is no credible evidence, even correlational, to suggest that taking it occasionally for a fever or headache would be an issue.

People take Tylenol for a reason. For many people, the choice may be between debilitating weekly migraines and regular Tylenol usage. The impacts studies suggest are very small. In making this decision, we should weigh the real, known benefit against the suggestion of this possible risk. Perhaps not everyone will come out at the same place on this, but it is crucial we give people the tools to make the choice for themselves.

#emilyoster #parentdata #tylenol #pregnancy #pregnancytips
...

Parenting trends are like Cabbage Patch Kids: they’re usually only popular because a bunch of people are using them! Most of the time, these trends are not based on new scientific research, and even if they are, that new research doesn’t reflect all of what we’ve studied before.

In the future, before hopping onto the latest trend, check the data first. Unlike Cabbage Patch Kids, parenting trends can add a lot of unnecessary stress and challenges to your plate. What’s a recent trend that you’ve been wondering about?

#parentdata #emilyoster #parentingtips #parentingadvice #parentinghacks

Parenting trends are like Cabbage Patch Kids: they’re usually only popular because a bunch of people are using them! Most of the time, these trends are not based on new scientific research, and even if they are, that new research doesn’t reflect all of what we’ve studied before.

In the future, before hopping onto the latest trend, check the data first. Unlike Cabbage Patch Kids, parenting trends can add a lot of unnecessary stress and challenges to your plate. What’s a recent trend that you’ve been wondering about?

#parentdata #emilyoster #parentingtips #parentingadvice #parentinghacks
...

As of this week, 1 million copies of my books have been sold. This feels humbling and, frankly, unbelievable. I’m so thankful to those of you who’ve read and passed along your recommendations of the books.

When I wrote Expecting Better, I had no plan for all of this — I wrote that book because I felt compelled to write it, because it was the book I wanted to read. As I’ve come out with more books, and now ParentData, I am closer to seeing what I hope we can all create. That is: a world where everyone has access to reliable data, based on causal evidence, to make informed, confident decisions that work for their families.

I’m so grateful you’re all here as a part of this, and I want to thank you! If you’ve been waiting for the right moment to sign up for full access to ParentData, this is it. ⭐️ Comment “Link” for a DM with a discount code for 20% off of a new monthly or annual subscription to ParentData! 

Thank you again for being the best community of readers and internet-friends on the planet. I am so lucky to have you all here.

#parentdata #emilyoster #expectingbetter #cribsheet #familyfirm #parentingcommunity

As of this week, 1 million copies of my books have been sold. This feels humbling and, frankly, unbelievable. I’m so thankful to those of you who’ve read and passed along your recommendations of the books.

When I wrote Expecting Better, I had no plan for all of this — I wrote that book because I felt compelled to write it, because it was the book I wanted to read. As I’ve come out with more books, and now ParentData, I am closer to seeing what I hope we can all create. That is: a world where everyone has access to reliable data, based on causal evidence, to make informed, confident decisions that work for their families.

I’m so grateful you’re all here as a part of this, and I want to thank you! If you’ve been waiting for the right moment to sign up for full access to ParentData, this is it. ⭐️ Comment “Link” for a DM with a discount code for 20% off of a new monthly or annual subscription to ParentData!

Thank you again for being the best community of readers and internet-friends on the planet. I am so lucky to have you all here.

#parentdata #emilyoster #expectingbetter #cribsheet #familyfirm #parentingcommunity
...

Just eat your Cheerios and move on.

Just eat your Cheerios and move on. ...

The AAP’s guidelines recommend sleeping in the same room as your baby “ideally for the first six months.” However, the risk of SIDS is dramatically lower after four months, and the evidence in favor of the protective effect of room sharing is quite weak (both overall and even more so after four months). There is also growing evidence that infants who sleep in their own room by four months sleep better at four months, better at nine months, and even better at 30 months.

With this in mind, it’s worth asking why this recommendation continues at all — or at least why the AAP doesn’t push it back to four months. They say decreased arousals from sleep are linked to SIDS, which could mean that babies sleeping in their own room is risky. But this link is extremely indirect, and they do not show direct evidence to support it.

According to the data we have, parents should sleep in the same room as a baby for as long as it works for them! Sharing a room with a child may have negative impacts on both child and adult sleep. We should give families more help in navigating these trade-offs and making the decisions that work best for them.

#emilyoster #parentdata #roomsharing #sids #parentingguide

The AAP’s guidelines recommend sleeping in the same room as your baby “ideally for the first six months.” However, the risk of SIDS is dramatically lower after four months, and the evidence in favor of the protective effect of room sharing is quite weak (both overall and even more so after four months). There is also growing evidence that infants who sleep in their own room by four months sleep better at four months, better at nine months, and even better at 30 months.

With this in mind, it’s worth asking why this recommendation continues at all — or at least why the AAP doesn’t push it back to four months. They say decreased arousals from sleep are linked to SIDS, which could mean that babies sleeping in their own room is risky. But this link is extremely indirect, and they do not show direct evidence to support it.

According to the data we have, parents should sleep in the same room as a baby for as long as it works for them! Sharing a room with a child may have negative impacts on both child and adult sleep. We should give families more help in navigating these trade-offs and making the decisions that work best for them.

#emilyoster #parentdata #roomsharing #sids #parentingguide
...

It was an absolute pleasure to be featured on the @tamronhallshow! We talked about all things data-driven parenting and, in this clip, what I call the plague of secret parenting. To balance having a career and having a family, we can’t hide the fact that we’re parents. If mothers and fathers at the top can speak more openly about child-care obligations, it will help us all set a new precedent.

Watch the full segment at the link in my bio 🔗

#tamronhall #tamronhallshow #emilyoster #parentingsupport #workingparents

It was an absolute pleasure to be featured on the @tamronhallshow! We talked about all things data-driven parenting and, in this clip, what I call the plague of secret parenting. To balance having a career and having a family, we can’t hide the fact that we’re parents. If mothers and fathers at the top can speak more openly about child-care obligations, it will help us all set a new precedent.

Watch the full segment at the link in my bio 🔗

#tamronhall #tamronhallshow #emilyoster #parentingsupport #workingparents
...

Invisible labor. It’s the work — in our households especially — that has to happen but that no one sees. It’s making the doctor’s appointment, ensuring birthday cards are purchased, remembering the milk.

My guest on this episode, @everodsky, has come up with a solution here, or at least a way for us to recognize the problem and make our own solutions. I’ve wanted to speak with Eve for ages, since I read her book Fair Play. We had a great conversation about the division of household labor, one I think you’ll get a lot out of!

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#emilyoster #parentdata #parentdatapodcast #parentingpodcast #householdtips #fairplay #invisiblelabor

Invisible labor. It’s the work — in our households especially — that has to happen but that no one sees. It’s making the doctor’s appointment, ensuring birthday cards are purchased, remembering the milk.

My guest on this episode, @everodsky, has come up with a solution here, or at least a way for us to recognize the problem and make our own solutions. I’ve wanted to speak with Eve for ages, since I read her book Fair Play. We had a great conversation about the division of household labor, one I think you’ll get a lot out of!

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#emilyoster #parentdata #parentdatapodcast #parentingpodcast #householdtips #fairplay #invisiblelabor
...

Prenatal vitamins 💊 If there is any product that seems designed to prey on our fears, it’s this one. You’re newly pregnant and you want to do it right. Everyone agrees you need prenatal vitamins, so you get them. But do you want to be that person who just… buys the generic prenatal vitamins?

Good news: fancier vitamins are not better.  Folic acid is the most important prenatal ingredient. Iron (with vitamin C) and DHA are also nice to have. Other included ingredients have only weak or no evidence to support their use. (If you do not consume animal products, add B12, plus a few others depending on your diet.)

Vitamins are just vitamins. Any prenatal vitamin that contains these is enough. 

Comment “Link” for a DM to an article with everything you need to know about prenatal vitamins.

#emilyoster #parentdata #prenatalvitamins #pregnancydiet #pregnancytips

Prenatal vitamins 💊 If there is any product that seems designed to prey on our fears, it’s this one. You’re newly pregnant and you want to do it right. Everyone agrees you need prenatal vitamins, so you get them. But do you want to be that person who just… buys the generic prenatal vitamins?

Good news: fancier vitamins are not better. Folic acid is the most important prenatal ingredient. Iron (with vitamin C) and DHA are also nice to have. Other included ingredients have only weak or no evidence to support their use. (If you do not consume animal products, add B12, plus a few others depending on your diet.)

Vitamins are just vitamins. Any prenatal vitamin that contains these is enough.

Comment “Link” for a DM to an article with everything you need to know about prenatal vitamins.

#emilyoster #parentdata #prenatalvitamins #pregnancydiet #pregnancytips
...

When it comes to introducing your newborn to the world, timing matters. It’s a good idea to minimize germ exposure in the first 6-8 weeks; after that, it’s inevitable and, very likely, a good idea! This doesn’t mean you need to be trapped inside. The most significant exposure risks are from seeing other people at home — family, etc. These interactions are not infinitely risky, but they do pose more risk than a walk or a trip to the grocery store, since they involve closer interaction. Think simple and make sure everyone is washing their hands before holding the baby. 💛

#parentdata #emilyoster #newborncare #parentingadvice #parentingtips

When it comes to introducing your newborn to the world, timing matters. It’s a good idea to minimize germ exposure in the first 6-8 weeks; after that, it’s inevitable and, very likely, a good idea! This doesn’t mean you need to be trapped inside. The most significant exposure risks are from seeing other people at home — family, etc. These interactions are not infinitely risky, but they do pose more risk than a walk or a trip to the grocery store, since they involve closer interaction. Think simple and make sure everyone is washing their hands before holding the baby. 💛

#parentdata #emilyoster #newborncare #parentingadvice #parentingtips
...

The first edition of Hot Flash is out now! Comment “Link” for a DM to learn more about the late-reproductive stage.

There are times when we expect hormonal shifts. Our reproductive lives are bookended by puberty and menopause. We discuss those changes often because they are definitive and dramatic — a first period is something many of us remember clearly. But between ages 13 and 53, our hormones are changing in more subtle ways. During the late-reproductive stage (in your 40s), you can expect a lot of changes in your menstrual cycle, including the length and symptoms you experience throughout. It’s an important time in our lives that is often overlooked!

🔥 Hot Flash from ParentData is a weekly newsletter on navigating your health and hormones in the post-reproductive years. Written by Dr. Gillian Goddard, Hot Flash provides all of the information you need to have a productive, evidence-based conversation about hormonal health with your doctor.

#emilyoster #parentdata #hotflash #perimenopause #womenshealth

The first edition of Hot Flash is out now! Comment “Link” for a DM to learn more about the late-reproductive stage.

There are times when we expect hormonal shifts. Our reproductive lives are bookended by puberty and menopause. We discuss those changes often because they are definitive and dramatic — a first period is something many of us remember clearly. But between ages 13 and 53, our hormones are changing in more subtle ways. During the late-reproductive stage (in your 40s), you can expect a lot of changes in your menstrual cycle, including the length and symptoms you experience throughout. It’s an important time in our lives that is often overlooked!

🔥 Hot Flash from ParentData is a weekly newsletter on navigating your health and hormones in the post-reproductive years. Written by Dr. Gillian Goddard, Hot Flash provides all of the information you need to have a productive, evidence-based conversation about hormonal health with your doctor.

#emilyoster #parentdata #hotflash #perimenopause #womenshealth
...

There are plenty of reels telling you how to parent. Plenty of panic headlines saying that “studies show” what’s best for your kid. Even good data, from a trusted source, can send us into a spiral of comparison. But I want you to remember that no one knows your kid better than you. It’s important to absorb the research, but only you will know the approach that works best for you and your child. 💙

Now tell me in the comments: what’s a parenting move you’ve made recently that feels right to you?

#parentingcommunity #parentingsupport #parentingquotes #emilyoster #parentdata

There are plenty of reels telling you how to parent. Plenty of panic headlines saying that “studies show” what’s best for your kid. Even good data, from a trusted source, can send us into a spiral of comparison. But I want you to remember that no one knows your kid better than you. It’s important to absorb the research, but only you will know the approach that works best for you and your child. 💙

Now tell me in the comments: what’s a parenting move you’ve made recently that feels right to you?

#parentingcommunity #parentingsupport #parentingquotes #emilyoster #parentdata
...

Let’s talk about sex (after) baby! Today on the podcast, I was lucky enough to speak with @enagoski about her new book on sexual connection in long-term relationships. Especially after having kids, this is something many people struggle with. Emily tells us to stop worrying about what’s “normal” and focus on pleasure in its many forms.

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#parentdata #parentdatapodcast #emilyoster #emilynagoski #comeasyouare #cometogether #longtermrelationship #intimacy #relationships

Let’s talk about sex (after) baby! Today on the podcast, I was lucky enough to speak with @enagoski about her new book on sexual connection in long-term relationships. Especially after having kids, this is something many people struggle with. Emily tells us to stop worrying about what’s “normal” and focus on pleasure in its many forms.

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#parentdata #parentdatapodcast #emilyoster #emilynagoski #comeasyouare #cometogether #longtermrelationship #intimacy #relationships
...

Ever wondered if you can safely use leftover baby formula? 🍼 The CDC says to throw out unused formula immediately because of the risk of bacterial growth. However, research suggests that bacterial concentrations do not appreciably increase after 3, 12, or even 24 hours at refrigerator temperatures. Good news! This means there’s not a strong data-based reason to throw out formula right away if you store it in the fridge.

Comment “Link” for a DM to an article on another common formula question: should you throw away old formula powder?

#emilyoster #parentdata #babyformula #babyfeeding #parentingstruggles

Ever wondered if you can safely use leftover baby formula? 🍼 The CDC says to throw out unused formula immediately because of the risk of bacterial growth. However, research suggests that bacterial concentrations do not appreciably increase after 3, 12, or even 24 hours at refrigerator temperatures. Good news! This means there’s not a strong data-based reason to throw out formula right away if you store it in the fridge.

Comment “Link” for a DM to an article on another common formula question: should you throw away old formula powder?

#emilyoster #parentdata #babyformula #babyfeeding #parentingstruggles
...

What’s the most important piece of advice for new parents? Here’s one answer, but I want to hear from you! Share your suggestions in the comments ⬇️

#emilyoster #parentdata #parentingtips #parentingadvice #newparents #parentingcommunity

What’s the most important piece of advice for new parents? Here’s one answer, but I want to hear from you! Share your suggestions in the comments ⬇️

#emilyoster #parentdata #parentingtips #parentingadvice #newparents #parentingcommunity
...

What's in the bag of a Vagina Economist? 👀 Someone please tell me this looks familiar to you.

What`s in the bag of a Vagina Economist? 👀 Someone please tell me this looks familiar to you. ...