Case Rates, Positivity and Tracking the Pandemic

Emily Oster

9 min Read Emily Oster

Emily Oster

Case Rates, Positivity and Tracking the Pandemic

Emily Oster

9 min Read

In the last couple of years I’ve spent a huge share of my professional time writing.  Books, opinion pieces, this newsletter, and so on.  But when I introduce myself, my job, I still list “Economist who studies health economics and statistical methods” first. It’s my “real job”.  And, today, this newsletter comes to you with that hat on.  We’re going to talk data. Specifically, COVID case data, positivity data and testing.

Graphs, tables and, yes, equations coming.

The TL;DR here is that I think there’s a lot of confusion in how we track the pandemic and how we compare across locations, and it may lead us to poor conclusions.

(If you want to nerd out with me, the data and code for this is here.  And, no, I do not want to hear about how you could have done it better with R, thanks.  I’m old, I’m committed to Stata, and I can’t hear you la la la).

The Basic Issue: Case Rates, Positivity Rates & An Identity

Broadly, there is a lot of interest in tracking the COVID-19 pandemic over time. I don’t think I need to convince you of that, and I’ll talk a bit more below about where this tracking impacts policy.  And it’s easy to find trackers online. Johns Hopkins has a good one, as does the New York Times.  For the most part, they rely on data from USA Facts, which is what I’ll use here.

Our goal, really, is to track the infection rate.  That is: in the population overall, what share of people have COVID-19 right now.  We do not know this. If we really wanted to be able to accurately track the evolution of the pandemic we’d want to randomly test a segment of the population over time in a consistent manner.  I made this point, as did many, many others, early in the pandemic.  We never did this, which is unfortunate but has left us with imperfect options.

In practice, there are two primary variables we use in tracking: case rates (positive cases per population, usually quoted per 100,000 people) and positivity rates (share of tests which are positive).  Both are flawed. Case rate is an undercount, because not all cases are detected; positivity is an over-count, because sick people are more likely to be tested.

This post is about how to understand their relationship and what they mean.

We can start with the data that goes into these calculations.  There are three relevant pieces of information: (1) total population (I’ll work with state-level data, so this is state population), (2) number of tests performed and (3) number of positive tests.  Given these data, we can calculate case rates and positivity rates and think about how they relate. The simple calculations are below.

The positivity rate is positive tests divided by total tests.  The case rate is positive test divided by population.

If you put these together, you can see the relationship between the case rate and the positivity rate.

This result is what we’d refer to as an “identity”; it’s just a feature of how these variables are calculated.  But what it tells us is that case rate and positivity rate are related, but the number of tests also matters. Put differently, you can get the same case rate with wildly different positivity rates, depending on the number of tests being done.

Why does this matter?  Who cares?!?!

I’d argue it matters because although we sometimes talk about these two things as both measuring the pandemic, they actually measure somewhat different things. And while they do tend to move together, there is definitely not a perfect correlation.

You can see this in the graph below, where I illustrate the relationship — at the state-month level — between case rates and positivity rates. The graph slopes up but there is a lot of variation.

I’ve marked some example points.  For example, the case rate in Arizona in January was very similar to South Dakota in November; both had rates around 130 in 100,000.  However, their positivity was wildly different: 50% in South Dakota versus 16% in Arizona. Alaska and Idaho both had rates around 72 per 100,000 at some point in the fall, but this reflected a positivity of 3.8% in Alaska and 54% In Idaho.

These differences are largest when the case rates are higher, but they persist at low rates. If we look at the state-months with case rates between 10 and 20 per 100,000 people (basically fairly low), we see positivity rates which range from 1.2% (Massachusetts in October) to 18% (Idaho, September).

Given the intimate relationship with testing, these differences must be driven by testing volume.  Another way to see this is in the table below which shows the case rate, positivity rates and testing volume for January (so far) for states with the highest and lowest case rates. There is huge overlap in positivity rates even at these tail end of case rates.  Iowa has a case rate about a third as large as Rhode Island, but the positivity rate is about 6 times higher.  This reflects the fact that Rhode Island did 17 times as many tests per capita than Iowa during this period.

Who is doing it right?

It has become common to use case rates as our primary pandemic measure, and this analysis above suggests they might be misleading in the sense that the same case rate may reflect different underlying infection rates depending on how much testing is going on. Looking at the data above, one natural conclusion is that if two places have the same case rate but one has a higher positivity rate, the one with the higher positivity rate probably actually has a higher infection rate.

But a skeptical person might argue: well, maybe the case rates are right, and places like Rhode Island and Alaska are just testing a lot of low risk negative people for no reason. Maybe Idaho is doing a good job, only using their tests on positive people. Maybe Alaska in November (case rate 72, positivity rate 3.8%) actually has the same underlying infection rate as Idaho in December (case rate 72, positivity rate 54%). Maybe it’s just that Idaho is a very efficient tester.

This is very unlikely to be true. For one thing, it simply stretches credulity to think that Idaho would have some magical solution to finding positive cases. The rates imply they are testing only 0.8 negative people for each positive person, versus 35 in Alaska.

A second thing that makes this unlikely is the relationship with hospitalizations. If similar case rates reflected similar underlying infection rates, we’d expect them to translate into similar hospitalizations. In fact, what we see is in places which are doing more testing — say, the state-months in the highest 20% in terms of per capita testing rates — each detected case has about a 2.8% chance of showing up as a hospitalization 4 weeks later. In those in the bottom 20%, this figure is 4%. Assuming that the actual risk of hospitalization is similar, this suggests we are missing cases in the lower testing locations.

None of this is to say that case rates are not reflecting infection rates at all. If infection rates are very low — say, in a place like New Zealand — even with a lot of testing you wont turn up many cases.  But in our US context, it’s likely the case that case rates do not tell the whole story.

Why Does This Matter?

This is a bit of a cautionary tale about case rates.  When we look at, say, the New York Times rankings of states, it’s probably not quite right to assume those with higher case rates necessarily have higher infection rates.  But does this actually matter for anything?

I would argue yes.  For example: we are often basing reopening decisions on case rate numbers. California has said, for example, that schools can open if case rates drop below 28 per 100,000. But case rates in this range (say, 24 to 28) in the data reflect ranges of positivity between 2.3% and 17%.  Moreover, the link with testing volume makes this manipulable. If you want fewer cases, testing less can deliver that.

This is another reason to worry.  A focus on case rates de-incentivizes testing. I realize I’m a Rhode Island booster in general but one of the things that the state has done really, really well is testing.  Last weekend I took my kids by the zoo after violin lesson for a COVID test; drive up, out in 5 minutes, text result in an hour. But ramping up testing has contributed to higher case rates.  From a policy maker standpoint, there is a temptation to lower testing rates to seem like things are improving.

From a purely practical standpoint, if you’re hoping to predict hospitalizations, this matters because depending on your testing rate you’d predict variation in the hospitalization risk.

All this adds together to say: how we measure the pandemic does matter, and this particular case-focused approach has some limitations.

Could We Do Better?

Can we improve on the current reporting?  It’s worth noting first that there is no way to back out infection rates reliably.  Case rates are what we call a lower bound — the infection rate cannot be below that.  But we have no good way to get an upper bound, or a good point estimate.  That’s why we all said back in April we needed random testing. But anyway.

There are clearly more sophisticated ways to engage in modeling here that could improve what we can learn. One option I toyed with for the purposes of this post was to look at the relationship between marginal increases in testing and positivity rate holding constant underlying prevalence (say, with state-week fixed effects), which could be applied to the distribution broadly and used to correct the rates.  (Don’t worry if this doesn’t make sense!)  In the end, I think virtually anything in this space is likely to be subject to a lot of concern about what assumptions go into it.  So I’m not sure there is a “right” solution (although smarter people than I may have ideas, which I would love to hear).

I think we can do better, though, by recognizing these issues and focusing on a variety of variables rather than targeting just one. I would argue no reopening decisions should be based on a single metric — not case rates, not positivity rates, not any one number. Ideally we’d have an approach which combines a whole set of information to make an informed decision.

Either that, or random testing. I’m never giving up on that.

Covid-19 rapid antigen tests arranged in a pattern on a yellow background.

Feb 20 2023

12 min read

COVID-19: Where to Go from Here

A long-term view of the virus

Emily Oster
Covid-19 rapid antigen tests arranged in a pattern on a yellow background.

Oct 20 2022

9 min read

Should You Get the Bivalent Booster?

The latest on the risks and benefits of COVID vaccines boosters for older adults, pregnant people, and kids

Emily Oster
A line graph with pink, yellow, and blue dots representing life's ups and downs.

Aug 16 2022

3 min read

Wins, Woes, and Doing It Again

We have our first story from a dad! And it’s a good one. 10/10 —Girl Dad with Confidence Growing by Read more

Emily Oster
Covid-19 rapid antigen tests arranged in a pattern on a yellow background.

Aug 15 2022

8 min read

Updated CDC Guidelines for School and Child Care

NO QUARANTINES!!!

Emily Oster

Instagram

left right
The AAP’s guidelines recommend sleeping in the same room as your baby “ideally for the first six months.” However, the risk of SIDS is dramatically lower after four months, and the evidence in favor of the protective effect of room sharing is quite weak (both overall and even more so after four months). There is also growing evidence that infants who sleep in their own room by four months sleep better at four months, better at nine months, and even better at 30 months.

With this in mind, it’s worth asking why this recommendation continues at all — or at least why the AAP doesn’t push it back to four months. They say decreased arousals from sleep are linked to SIDS, which could mean that babies sleeping in their own room is risky. But this link is extremely indirect, and they do not show direct evidence to support it.

According to the data we have, parents should sleep in the same room as a baby for as long as it works for them! Sharing a room with a child may have negative impacts on both child and adult sleep. We should give families more help in navigating these trade-offs and making the decisions that work best for them.

#emilyoster #parentdata #roomsharing #sids #parentingguide

The AAP’s guidelines recommend sleeping in the same room as your baby “ideally for the first six months.” However, the risk of SIDS is dramatically lower after four months, and the evidence in favor of the protective effect of room sharing is quite weak (both overall and even more so after four months). There is also growing evidence that infants who sleep in their own room by four months sleep better at four months, better at nine months, and even better at 30 months.

With this in mind, it’s worth asking why this recommendation continues at all — or at least why the AAP doesn’t push it back to four months. They say decreased arousals from sleep are linked to SIDS, which could mean that babies sleeping in their own room is risky. But this link is extremely indirect, and they do not show direct evidence to support it.

According to the data we have, parents should sleep in the same room as a baby for as long as it works for them! Sharing a room with a child may have negative impacts on both child and adult sleep. We should give families more help in navigating these trade-offs and making the decisions that work best for them.

#emilyoster #parentdata #roomsharing #sids #parentingguide
...

It was an absolute pleasure to be featured on the @tamronhallshow! We talked about all things data-driven parenting and, in this clip, what I call the plague of secret parenting. To balance having a career and having a family, we can’t hide the fact that we’re parents. If mothers and fathers at the top can speak more openly about child-care obligations, it will help us all set a new precedent.

Watch the full segment at the link in my bio 🔗

#tamronhall #tamronhallshow #emilyoster #parentingsupport #workingparents

It was an absolute pleasure to be featured on the @tamronhallshow! We talked about all things data-driven parenting and, in this clip, what I call the plague of secret parenting. To balance having a career and having a family, we can’t hide the fact that we’re parents. If mothers and fathers at the top can speak more openly about child-care obligations, it will help us all set a new precedent.

Watch the full segment at the link in my bio 🔗

#tamronhall #tamronhallshow #emilyoster #parentingsupport #workingparents
...

Invisible labor. It’s the work — in our households especially — that has to happen but that no one sees. It’s making the doctor’s appointment, ensuring birthday cards are purchased, remembering the milk.

My guest on this episode, @everodsky, has come up with a solution here, or at least a way for us to recognize the problem and make our own solutions. I’ve wanted to speak with Eve for ages, since I read her book Fair Play. We had a great conversation about the division of household labor, one I think you’ll get a lot out of!

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#emilyoster #parentdata #parentdatapodcast #parentingpodcast #householdtips #fairplay #invisiblelabor

Invisible labor. It’s the work — in our households especially — that has to happen but that no one sees. It’s making the doctor’s appointment, ensuring birthday cards are purchased, remembering the milk.

My guest on this episode, @everodsky, has come up with a solution here, or at least a way for us to recognize the problem and make our own solutions. I’ve wanted to speak with Eve for ages, since I read her book Fair Play. We had a great conversation about the division of household labor, one I think you’ll get a lot out of!

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#emilyoster #parentdata #parentdatapodcast #parentingpodcast #householdtips #fairplay #invisiblelabor
...

Prenatal vitamins 💊 If there is any product that seems designed to prey on our fears, it’s this one. You’re newly pregnant and you want to do it right. Everyone agrees you need prenatal vitamins, so you get them. But do you want to be that person who just… buys the generic prenatal vitamins?

Good news: fancier vitamins are not better.  Folic acid is the most important prenatal ingredient. Iron (with vitamin C) and DHA are also nice to have. Other included ingredients have only weak or no evidence to support their use. (If you do not consume animal products, add B12, plus a few others depending on your diet.)

Vitamins are just vitamins. Any prenatal vitamin that contains these is enough. 

Comment “Link” for a DM to an article with everything you need to know about prenatal vitamins.

#emilyoster #parentdata #prenatalvitamins #pregnancydiet #pregnancytips

Prenatal vitamins 💊 If there is any product that seems designed to prey on our fears, it’s this one. You’re newly pregnant and you want to do it right. Everyone agrees you need prenatal vitamins, so you get them. But do you want to be that person who just… buys the generic prenatal vitamins?

Good news: fancier vitamins are not better. Folic acid is the most important prenatal ingredient. Iron (with vitamin C) and DHA are also nice to have. Other included ingredients have only weak or no evidence to support their use. (If you do not consume animal products, add B12, plus a few others depending on your diet.)

Vitamins are just vitamins. Any prenatal vitamin that contains these is enough.

Comment “Link” for a DM to an article with everything you need to know about prenatal vitamins.

#emilyoster #parentdata #prenatalvitamins #pregnancydiet #pregnancytips
...

When it comes to introducing your newborn to the world, timing matters. It’s a good idea to minimize germ exposure in the first 6-8 weeks; after that, it’s inevitable and, very likely, a good idea! This doesn’t mean you need to be trapped inside. The most significant exposure risks are from seeing other people at home — family, etc. These interactions are not infinitely risky, but they do pose more risk than a walk or a trip to the grocery store, since they involve closer interaction. Think simple and make sure everyone is washing their hands before holding the baby. 💛

#parentdata #emilyoster #newborncare #parentingadvice #parentingtips

When it comes to introducing your newborn to the world, timing matters. It’s a good idea to minimize germ exposure in the first 6-8 weeks; after that, it’s inevitable and, very likely, a good idea! This doesn’t mean you need to be trapped inside. The most significant exposure risks are from seeing other people at home — family, etc. These interactions are not infinitely risky, but they do pose more risk than a walk or a trip to the grocery store, since they involve closer interaction. Think simple and make sure everyone is washing their hands before holding the baby. 💛

#parentdata #emilyoster #newborncare #parentingadvice #parentingtips
...

The first edition of Hot Flash is out now! Comment “Link” for a DM to learn more about the late-reproductive stage.

There are times when we expect hormonal shifts. Our reproductive lives are bookended by puberty and menopause. We discuss those changes often because they are definitive and dramatic — a first period is something many of us remember clearly. But between ages 13 and 53, our hormones are changing in more subtle ways. During the late-reproductive stage (in your 40s), you can expect a lot of changes in your menstrual cycle, including the length and symptoms you experience throughout. It’s an important time in our lives that is often overlooked!

🔥 Hot Flash from ParentData is a weekly newsletter on navigating your health and hormones in the post-reproductive years. Written by Dr. Gillian Goddard, Hot Flash provides all of the information you need to have a productive, evidence-based conversation about hormonal health with your doctor.

#emilyoster #parentdata #hotflash #perimenopause #womenshealth

The first edition of Hot Flash is out now! Comment “Link” for a DM to learn more about the late-reproductive stage.

There are times when we expect hormonal shifts. Our reproductive lives are bookended by puberty and menopause. We discuss those changes often because they are definitive and dramatic — a first period is something many of us remember clearly. But between ages 13 and 53, our hormones are changing in more subtle ways. During the late-reproductive stage (in your 40s), you can expect a lot of changes in your menstrual cycle, including the length and symptoms you experience throughout. It’s an important time in our lives that is often overlooked!

🔥 Hot Flash from ParentData is a weekly newsletter on navigating your health and hormones in the post-reproductive years. Written by Dr. Gillian Goddard, Hot Flash provides all of the information you need to have a productive, evidence-based conversation about hormonal health with your doctor.

#emilyoster #parentdata #hotflash #perimenopause #womenshealth
...

There are plenty of reels telling you how to parent. Plenty of panic headlines saying that “studies show” what’s best for your kid. Even good data, from a trusted source, can send us into a spiral of comparison. But I want you to remember that no one knows your kid better than you. It’s important to absorb the research, but only you will know the approach that works best for you and your child. 💙

Now tell me in the comments: what’s a parenting move you’ve made recently that feels right to you?

#parentingcommunity #parentingsupport #parentingquotes #emilyoster #parentdata

There are plenty of reels telling you how to parent. Plenty of panic headlines saying that “studies show” what’s best for your kid. Even good data, from a trusted source, can send us into a spiral of comparison. But I want you to remember that no one knows your kid better than you. It’s important to absorb the research, but only you will know the approach that works best for you and your child. 💙

Now tell me in the comments: what’s a parenting move you’ve made recently that feels right to you?

#parentingcommunity #parentingsupport #parentingquotes #emilyoster #parentdata
...

Let’s talk about sex (after) baby! Today on the podcast, I was lucky enough to speak with @enagoski about her new book on sexual connection in long-term relationships. Especially after having kids, this is something many people struggle with. Emily tells us to stop worrying about what’s “normal” and focus on pleasure in its many forms.

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#parentdata #parentdatapodcast #emilyoster #emilynagoski #comeasyouare #cometogether #longtermrelationship #intimacy #relationships

Let’s talk about sex (after) baby! Today on the podcast, I was lucky enough to speak with @enagoski about her new book on sexual connection in long-term relationships. Especially after having kids, this is something many people struggle with. Emily tells us to stop worrying about what’s “normal” and focus on pleasure in its many forms.

Listen and subscribe to ParentData with Emily Oster in your favorite podcast app 🎧

#parentdata #parentdatapodcast #emilyoster #emilynagoski #comeasyouare #cometogether #longtermrelationship #intimacy #relationships
...

Ever wondered if you can safely use leftover baby formula? 🍼 The CDC says to throw out unused formula immediately because of the risk of bacterial growth. However, research suggests that bacterial concentrations do not appreciably increase after 3, 12, or even 24 hours at refrigerator temperatures. Good news! This means there’s not a strong data-based reason to throw out formula right away if you store it in the fridge.

Comment “Link” for a DM to an article on another common formula question: should you throw away old formula powder?

#emilyoster #parentdata #babyformula #babyfeeding #parentingstruggles

Ever wondered if you can safely use leftover baby formula? 🍼 The CDC says to throw out unused formula immediately because of the risk of bacterial growth. However, research suggests that bacterial concentrations do not appreciably increase after 3, 12, or even 24 hours at refrigerator temperatures. Good news! This means there’s not a strong data-based reason to throw out formula right away if you store it in the fridge.

Comment “Link” for a DM to an article on another common formula question: should you throw away old formula powder?

#emilyoster #parentdata #babyformula #babyfeeding #parentingstruggles
...

What’s the most important piece of advice for new parents? Here’s one answer, but I want to hear from you! Share your suggestions in the comments ⬇️

#emilyoster #parentdata #parentingtips #parentingadvice #newparents #parentingcommunity

What’s the most important piece of advice for new parents? Here’s one answer, but I want to hear from you! Share your suggestions in the comments ⬇️

#emilyoster #parentdata #parentingtips #parentingadvice #newparents #parentingcommunity
...

What's in the bag of a Vagina Economist? 👀 Someone please tell me this looks familiar to you.

What`s in the bag of a Vagina Economist? 👀 Someone please tell me this looks familiar to you. ...

Comment ”link” for a DM to learn more about tongue ties 🔗

Breastfeeding is often difficult, especially at the start. For babies with tongue ties, many infants (and their moms) struggle to get the hang of a good latch. This can lead to painful nipples and to inefficient feeding, and then low weight gain.

So what does the data say about the increasingly common practice of cutting tongue-ties in infants to improve breastfeeding success? Several weeks ago, @nytimes published a long and quite scary article on this topic.

After diving into the data, here is what I found. There is limited evidence that frenotomy procedures improve breastfeeding efficacy and the harms of the procedure are minimal. Many women do report that it alleviates pain and helps them with breastfeeding. However, it should not be a first-line treatment for breastfeeding problems.

#parentdata #emilyoster #tonguetie #tonguetiebabies #breastfeedingsupport

Comment ”link” for a DM to learn more about tongue ties 🔗

Breastfeeding is often difficult, especially at the start. For babies with tongue ties, many infants (and their moms) struggle to get the hang of a good latch. This can lead to painful nipples and to inefficient feeding, and then low weight gain.

So what does the data say about the increasingly common practice of cutting tongue-ties in infants to improve breastfeeding success? Several weeks ago, @nytimes published a long and quite scary article on this topic.

After diving into the data, here is what I found. There is limited evidence that frenotomy procedures improve breastfeeding efficacy and the harms of the procedure are minimal. Many women do report that it alleviates pain and helps them with breastfeeding. However, it should not be a first-line treatment for breastfeeding problems.

#parentdata #emilyoster #tonguetie #tonguetiebabies #breastfeedingsupport
...

Tag a friend who needs to hear this 💛 For many choices in parenting, there is no one right answer. We can use research and data to make informed decisions, but ultimately, it won’t tell you what to do. Only you can decide what will be best for your kids and your family.

I’m here to remind you to take a deep breath and trust yourself. I’ll be here to support you along the way. 

Thank you to everyone who submitted videos, including:
@sarah.consoli
@jess_lynn627
@nicolevandenwills
@thedrblair
@ncbenedict29
@haleycimini
@iamkellysnodgrass
@calesse_smith
@garnet__gordon
@jencoopgaiser87
@danigirl18c
@jamielundergreen
@carly_comber
@thecelebratingmama
@emilyannbynum
@eeliz413

#emilyoster #parentdata #parentingadvice #parentingsupport #parentingquotes

Tag a friend who needs to hear this 💛 For many choices in parenting, there is no one right answer. We can use research and data to make informed decisions, but ultimately, it won’t tell you what to do. Only you can decide what will be best for your kids and your family.

I’m here to remind you to take a deep breath and trust yourself. I’ll be here to support you along the way.

Thank you to everyone who submitted videos, including:
@sarah.consoli
@jess_lynn627
@nicolevandenwills
@thedrblair
@ncbenedict29
@haleycimini
@iamkellysnodgrass
@calesse_smith
@garnet__gordon
@jencoopgaiser87
@danigirl18c
@jamielundergreen
@carly_comber
@thecelebratingmama
@emilyannbynum
@eeliz413

#emilyoster #parentdata #parentingadvice #parentingsupport #parentingquotes
...

Congratulations on making it through another year of panic headlines! We’ve had some doozies this year, like aspartame causing cancer and the perils of white noise, but these headlines are very often based on poor data. Correlation does not equal causation. There will certainly be more panic headlines in 2024, but ParentData is here to debunk them for you.

#emilyoster #parentdata #happynewyear2024 #panicheadline #datadriven

Congratulations on making it through another year of panic headlines! We’ve had some doozies this year, like aspartame causing cancer and the perils of white noise, but these headlines are very often based on poor data. Correlation does not equal causation. There will certainly be more panic headlines in 2024, but ParentData is here to debunk them for you.

#emilyoster #parentdata #happynewyear2024 #panicheadline #datadriven
...